First, we can define the objective function.
In this case, we will use a one-dimensional objective function, specifically x^2 shifted by a small amount away from zero. This is a convex function and was chosen because it is easy to understand and to calculate the first derivative.
objective(x) = (-5 + x)^2
Note that the line search is not limited to one-dimensional functions or convex functions.
The implementation of this function is listed below.
objective function
def objective(x):
return (-5.0 + x)**2.0
1
2
3
objective function
def objective(x):
return (-5.0 + x)**2.0
The first derivative for this function can be calculated analytically, as follows:
gradient(x) = 2 * (-5 + x)
The gradient for each input value just indicates the slope towards the optima at each point. The implementation of this function is listed below.
gradient for the objective function
def gradient(x):
return 2.0 * (-5.0 + x)
1
2
3
gradient for the objective function
def gradient(x):
return 2.0 * (-5.0 + x)
We can define an input range for x from -10 to 20 and calculate the objective value for each input.
…
define range
r_min, r_max = -10.0, 20.0
prepare inputs
inputs = arange(r_min, r_max, 0.1)
compute targets
targets = [objective(x) for x in inputs]
1
2
3
4
5
6
7
…
define range
r_min, r_max = -10.0, 20.0
prepare inputs
inputs = arange(r_min, r_max, 0.1)
compute targets
targets = [objective(x) for x in inputs]
We can then plot the input values versus the objective values to get an idea of the shape of the function.
…
plot inputs vs objective
pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5
…
plot inputs vs objective
pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Tying this together, the complete example is listed below.
plot a convex objective function
from numpy import arange
from matplotlib import pyplot
objective function
def objective(x):
return (-5.0 + x)**2.0
gradient for the objective function
def gradient(x):
return 2.0 * (-5.0 + x)
define range
r_min, r_max = -10.0, 20.0
prepare inputs
inputs = arange(r_min, r_max, 0.1)
compute targets
targets = [objective(x) for x in inputs]
plot inputs vs objective
pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
plot a convex objective function
from numpy import arange
from matplotlib import pyplot
objective function
def objective(x):
return (-5.0 + x)**2.0
gradient for the objective function
def gradient(x):
return 2.0 * (-5.0 + x)
define range
r_min, r_max = -10.0, 20.0
prepare inputs
inputs = arange(r_min, r_max, 0.1)
compute targets
targets = [objective(x) for x in inputs]
plot inputs vs objective
pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Running the example evaluates input values (x) in the range from -10 to 20 and creates a plot showing the familiar parabola U-shape.
The optima for the function appears to be at x=5.0 with an objective value of 0.0.
Line Plot of Convex Objective Function