Define the Objective Function in line search

First, we can define the objective function.

In this case, we will use a one-dimensional objective function, specifically x^2 shifted by a small amount away from zero. This is a convex function and was chosen because it is easy to understand and to calculate the first derivative.

objective(x) = (-5 + x)^2
Note that the line search is not limited to one-dimensional functions or convex functions.

The implementation of this function is listed below.

objective function

def objective(x):
return (-5.0 + x)**2.0
1
2
3

objective function

def objective(x):
return (-5.0 + x)**2.0
The first derivative for this function can be calculated analytically, as follows:

gradient(x) = 2 * (-5 + x)
The gradient for each input value just indicates the slope towards the optima at each point. The implementation of this function is listed below.

gradient for the objective function

def gradient(x):
return 2.0 * (-5.0 + x)
1
2
3

gradient for the objective function

def gradient(x):
return 2.0 * (-5.0 + x)
We can define an input range for x from -10 to 20 and calculate the objective value for each input.

define range

r_min, r_max = -10.0, 20.0

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]
1
2
3
4
5
6
7

define range

r_min, r_max = -10.0, 20.0

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]
We can then plot the input values versus the objective values to get an idea of the shape of the function.

plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5

plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Tying this together, the complete example is listed below.

plot a convex objective function

from numpy import arange
from matplotlib import pyplot

objective function

def objective(x):
return (-5.0 + x)**2.0

gradient for the objective function

def gradient(x):
return 2.0 * (-5.0 + x)

define range

r_min, r_max = -10.0, 20.0

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]

plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

plot a convex objective function

from numpy import arange
from matplotlib import pyplot

objective function

def objective(x):
return (-5.0 + x)**2.0

gradient for the objective function

def gradient(x):
return 2.0 * (-5.0 + x)

define range

r_min, r_max = -10.0, 20.0

prepare inputs

inputs = arange(r_min, r_max, 0.1)

compute targets

targets = [objective(x) for x in inputs]

plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Running the example evaluates input values (x) in the range from -10 to 20 and creates a plot showing the familiar parabola U-shape.

The optima for the function appears to be at x=5.0 with an objective value of 0.0.

Line Plot of Convex Objective Function