A hyperparameter is a machine learning parameter whose value is chosen before a learning algorithm is trained.
Hyperparameters should not be confused with parameters . In machine learning, the label parameter is used to identify variables whose values are learned during training. The prefix hyper is used to identify higher-level parameters that control the learning process.
Every variable that an AI engineer or ML engineer chooses before model training begins can be referred to as a hyperparameter – as long as the value of the variable remains the same when training ends.
It’s important to choose the right hyperparameters before training begins because this type of variable has a direct impact on the performance of the resulting machine learning model. Examples of hyperparameters in machine learning include:
Number of epochs
Number of branches in a decision tree
Number of clusters in a clustering algorithm
Hyperparameters may also be referred to as meta parameters.
The process of choosing which hyperparameters to use is called hyperparameter tuning. The process of tuning may also be referred to as hyperparameter optimization (HPO).