Getting Started with Function Optimization in Python
Getting Started with Function Optimization in Python
Whether you are tuning machine learning hyperparameters, calculating the lowest production cost, or finding the equilibrium point in a physical system, optimization is one of the core pillars of scientific computing.
At its core, optimization is about finding an input that minimizes or maximizes an objective function.
Python’s
scipy.optimize library provides high-performance tools to solve these problems without requiring you to manually compute complex derivatives. Let’s walk through how to define a function, find its minimum using SciPy, and visualize the result with Matplotlib.1. The Problem: Minimizing a Quadratic Function
Consider a simple quadratic objective function:
$$f(x) = (x - 4)^2 + 3$$
Because this is a standard parabola opening upwards, we know algebraically that:
- The vertex (lowest point) occurs where $x - 4 = 0$, giving an optimal input of $x = 4$.
- The minimum value of the function at that point is $f(4) = 3$.
Let’s see how SciPy handles this automatically.
2. Implementation with scipy.optimize
First, define the objective function and pass it to
scipy.optimize.minimize:Python
import numpy as np
import matplotlib.pyplot as plt
from scipy import optimize
# 1. Define the objective function
def objective(x):
return (x[0] - 4)**2 + 3
# 2. Run the optimizer with an initial guess (x0 = 0)
result = optimize.minimize(objective, x0=[0])
# 3. Inspect the results
print("Optimal x:", round(result.x[0], 4))
print("Minimum value:", round(result.fun, 4))
print("Success:", result.success)
Output:
Plaintext
Optimal x: 4.0
Minimum value: 3.0
Success: True
Breaking Down the Code:
x[0]indexing:scipy.optimize.minimizepasses inputs as NumPy arrays or lists. Even for 1D problems, indexing withx[0]ensures compatibility with multi-variable solvers.x0=[0]: Optimizers need a starting point to begin iterative search steps. Here, we start searching at $x = 0$.- The
resultobject:result.x: An array containing the optimal input values.result.fun: The minimum value evaluated atresult.x.result.success: A boolean flag confirming whether the algorithm converged successfully.
3. Visualizing the Solution
Visualizing optimization paths and final minima helps verify behavior, especially when dealing with complex or non-convex surfaces.
Python
# Generate domain points for plotting
x = np.linspace(-2, 10, 300)
y = (x - 4)**2 + 3
# Plot the curve
plt.figure(figsize=(9, 4.5))
plt.plot(x, y, linewidth=2.5, label="Objective Function")
# Highlight the found minimum
plt.scatter(result.x[0], result.fun, color="tab:blue", s=100, zorder=3)
plt.annotate(
"Minimum",
(result.x[0], result.fun),
xytext=(5, 5),
textcoords="offset points"
)
# Styling
plt.title("Optimization: Finding the Minimum", fontsize=15, weight="bold")
plt.xlabel("x")
plt.ylabel("Objective")
plt.tight_layout()
plt.show()
The resulting plot displays a smooth parabolic curve with a marker cleanly seated at $(4, 3)$, providing visual confirmation that the solver reached the true global minimum.
Comments
Post a Comment