The use of randomness in the algorithms often means that the techniques are referred to as “heuristic search” as they use a rough rule-of-thumb procedure that may or may not work to find the optima instead of a precise procedure.

Many stochastic algorithms are inspired by a biological or natural process and may be referred to as “metaheuristics” as a higher-order procedure providing the conditions for a specific search of the objective function. They are also referred to as “*black box*” optimization algorithms.

Metaheuristics is a rather unfortunate term often used to describe a major subfield, indeed the primary subfield, of stochastic optimization.

— Page 7, Essentials of Metaheuristics, 2011.

There are many stochastic optimization algorithms.

Some examples of stochastic optimization algorithms include:

- Iterated Local Search
- Stochastic Hill Climbing
- Stochastic Gradient Descent
- Tabu Search
- Greedy Randomized Adaptive Search Procedure

Some examples of stochastic optimization algorithms that are inspired by biological or physical processes include:

- Simulated Annealing
- Evolution Strategies
- Genetic Algorithm
- Differential Evolution
- Particle Swarm Optimization