A candidate solution is a single input to the objective function.
The form of a candidate solution depends on the specifics of the objective function. It may be a single floating point number, a vector of numbers, a matrix of numbers, or as complex as needed for the specific problem domain.
Most commonly, vectors of numbers. For a test problem, the vector represents the specific values of each input variable to the function (x = x1, x2, x3, …, xn). For a machine learning model, the vector may represent model coefficients or weights.
Mathematically speaking, optimization is the minimization or maximization of a function subject to constraints on its variables.
— Page 2, Numerical Optimization, 2006.
There may be constraints imposed by the problem domain or the objective function on the candidate solutions. This might include aspects such as:
- The number of variables (1, 20, 1,000,000, etc.)
- The data type of variables (integer, binary, real-valued, etc.)
- The range of accepted values (between 0 and 1, etc.)
Importantly, candidate solutions are discrete and there are many of them.
The universe of candidate solutions may be vast, too large to enumerate. Instead, the best we can do is sample candidate solutions in the search space. As a practitioner, we seek an optimization algorithm that makes the best use of the information available about the problem to effectively sample the search space and locate a good or best candidate solution.
- Search Space: Universe of candidate solutions defined by the number, type, and range of accepted inputs to the objective function.
Finally, candidate solutions can be rank-ordered based on their evaluation by the objective function, meaning that some are better than others.