A range of different optimization algorithms may be used, although two of the simplest and most common methods are random search and grid search.
Random Search. Define a search space as a bounded domain of hyperparameter values and randomly sample points in that domain.
Grid Search. Define a search space as a grid of hyperparameter values and evaluate every position in the grid.
Grid search is great for spot-checking combinations that are known to perform well generally. Random search is great for discovery and getting hyperparameter combinations that you would not have guessed intuitively, although it often requires more time to execute.
More advanced methods are sometimes used, such as Bayesian Optimization and Evolutionary Optimization.