# Define the Objective Function in line search

First, we can define the objective function.

In this case, we will use a one-dimensional objective function, specifically x^2 shifted by a small amount away from zero. This is a convex function and was chosen because it is easy to understand and to calculate the first derivative.

objective(x) = (-5 + x)^2
Note that the line search is not limited to one-dimensional functions or convex functions.

The implementation of this function is listed below.

# objective function

def objective(x):
return (-5.0 + x)**2.0
1
2
3

# objective function

def objective(x):
return (-5.0 + x)**2.0
The first derivative for this function can be calculated analytically, as follows:

gradient(x) = 2 * (-5 + x)
The gradient for each input value just indicates the slope towards the optima at each point. The implementation of this function is listed below.

# gradient for the objective function

return 2.0 * (-5.0 + x)
1
2
3

# gradient for the objective function

return 2.0 * (-5.0 + x)
We can define an input range for x from -10 to 20 and calculate the objective value for each input.

# define range

r_min, r_max = -10.0, 20.0

# prepare inputs

inputs = arange(r_min, r_max, 0.1)

# compute targets

targets = [objective(x) for x in inputs]
1
2
3
4
5
6
7

# define range

r_min, r_max = -10.0, 20.0

# prepare inputs

inputs = arange(r_min, r_max, 0.1)

# compute targets

targets = [objective(x) for x in inputs]
We can then plot the input values versus the objective values to get an idea of the shape of the function.

# plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5

# plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Tying this together, the complete example is listed below.

# plot a convex objective function

from numpy import arange
from matplotlib import pyplot

# objective function

def objective(x):
return (-5.0 + x)**2.0

# gradient for the objective function

return 2.0 * (-5.0 + x)

# define range

r_min, r_max = -10.0, 20.0

# prepare inputs

inputs = arange(r_min, r_max, 0.1)

# compute targets

targets = [objective(x) for x in inputs]

# plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# plot a convex objective function

from numpy import arange
from matplotlib import pyplot

# objective function

def objective(x):
return (-5.0 + x)**2.0

# gradient for the objective function

return 2.0 * (-5.0 + x)

# define range

r_min, r_max = -10.0, 20.0

# prepare inputs

inputs = arange(r_min, r_max, 0.1)

# compute targets

targets = [objective(x) for x in inputs]

# plot inputs vs objective

pyplot.plot(inputs, targets, ‘-’, label=‘objective’)
pyplot.legend()
pyplot.show()
Running the example evaluates input values (x) in the range from -10 to 20 and creates a plot showing the familiar parabola U-shape.

The optima for the function appears to be at x=5.0 with an objective value of 0.0.

Line Plot of Convex Objective Function