Getting Started with Function Optimization in Python

 

Getting Started with Function Optimization in Python

Whether you are tuning machine learning hyperparameters, calculating the lowest production cost, or finding the equilibrium point in a physical system, optimization is one of the core pillars of scientific computing.

At its core, optimization is about finding an input that minimizes or maximizes an objective function.

Python’s scipy.optimize library provides high-performance tools to solve these problems without requiring you to manually compute complex derivatives. Let’s walk through how to define a function, find its minimum using SciPy, and visualize the result with Matplotlib.

1. The Problem: Minimizing a Quadratic Function

Consider a simple quadratic objective function:

$$f(x) = (x - 4)^2 + 3$$
Because this is a standard parabola opening upwards, we know algebraically that:

  • The vertex (lowest point) occurs where $x - 4 = 0$, giving an optimal input of $x = 4$.

  • The minimum value of the function at that point is $f(4) = 3$.

Let’s see how SciPy handles this automatically.

2. Implementation with scipy.optimize

First, define the objective function and pass it to scipy.optimize.minimize:

Python
import numpy as np
import matplotlib.pyplot as plt
from scipy import optimize

# 1. Define the objective function
def objective(x):
    return (x[0] - 4)**2 + 3

# 2. Run the optimizer with an initial guess (x0 = 0)
result = optimize.minimize(objective, x0=[0])

# 3. Inspect the results
print("Optimal x:", round(result.x[0], 4))
print("Minimum value:", round(result.fun, 4))
print("Success:", result.success)

Output:

Plaintext
Optimal x: 4.0
Minimum value: 3.0
Success: True

Breaking Down the Code:

  • x[0] indexing: scipy.optimize.minimize passes inputs as NumPy arrays or lists. Even for 1D problems, indexing with x[0] ensures compatibility with multi-variable solvers.

  • x0=[0]: Optimizers need a starting point to begin iterative search steps. Here, we start searching at $x = 0$.

  • The result object:

    • result.x: An array containing the optimal input values.

    • result.fun: The minimum value evaluated at result.x.

    • result.success: A boolean flag confirming whether the algorithm converged successfully.

3. Visualizing the Solution

Visualizing optimization paths and final minima helps verify behavior, especially when dealing with complex or non-convex surfaces.

Python
# Generate domain points for plotting
x = np.linspace(-2, 10, 300)
y = (x - 4)**2 + 3

# Plot the curve
plt.figure(figsize=(9, 4.5))
plt.plot(x, y, linewidth=2.5, label="Objective Function")

# Highlight the found minimum
plt.scatter(result.x[0], result.fun, color="tab:blue", s=100, zorder=3)
plt.annotate(
    "Minimum", 
    (result.x[0], result.fun),
    xytext=(5, 5), 
    textcoords="offset points"
)

# Styling
plt.title("Optimization: Finding the Minimum", fontsize=15, weight="bold")
plt.xlabel("x")
plt.ylabel("Objective")
plt.tight_layout()
plt.show()
The resulting plot displays a smooth parabolic curve with a marker cleanly seated at $(4, 3)$, providing visual confirmation that the solver reached the true global minimum.

Comments

Popular posts from this blog

I am from mechanical background.. how I can start with python? Tell me career opportunities also .

Root Finding in Python: Solving and Visualizing Equations with SciPy

Here's a step-by-step guide to help you install Visual Studio Code (VSCode) for Python on Windows