Non-Convex Univariate Function Optimization

Data Science, AI and ML
#1

A convex function is one that does not resemble a basin, meaning that it may have more than one hill or valley.

This can make it more challenging to locate the global optima as the multiple hills and valleys can cause the search to get stuck and report a false or local optima instead.

We can define a non-convex univariate function as follows.

objective function

def objective(x):
return (x - 2.0) * x * (x + 2.0)**2.0
We can sample this function and create a line plot of input values to objective values.

The complete example is listed below.

plot a non-convex univariate function

from numpy import arange
from matplotlib import pyplot

objective function

def objective(x):
return (x - 2.0) * x * (x + 2.0)**2.0

define range

r_min, r_max = -3.0, 2.5

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]

plot inputs vs target

pyplot.plot(inputs, targets, ‘–’)
pyplot.show()
Running the example evaluates input values in our specified range using our target function and creates a plot of the function inputs to function outputs.

Note: in a real optimization problem, we would not be able to perform so many evaluations of the objective function so easily. This simple function is used for demonstration purposes so we can learn how to use the optimization algorithm.

Next, we can use the optimization algorithm to find the optima.

As before, we can call the minimize_scalar() function to optimize the function, then summarize the result and plot the optima on a line plot.

The complete example of optimization of an unconstrained non-convex univariate function is listed below.

optimize non-convex objective function

from numpy import arange
from scipy.optimize import minimize_scalar
from matplotlib import pyplot

objective function

def objective(x):
return (x - 2.0) * x * (x + 2.0)**2.0

minimize the function

result = minimize_scalar(objective, method=‘brent’)

summarize the result

opt_x, opt_y = result[‘x’], result[‘fun’]
print(‘Optimal Input x: %.6f’ % opt_x)
print(‘Optimal Output f(x): %.6f’ % opt_y)
print(‘Total Evaluations n: %d’ % result[‘nfev’])

define the range

r_min, r_max = -3.0, 2.5

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]

plot inputs vs target

pyplot.plot(inputs, targets, ‘–’)

plot the optima

pyplot.plot([opt_x], [opt_y], ‘s’, color=‘r’)

show the plot

pyplot.show()
Running the example first solves the optimization problem and reports the result.