What is hyperparameter tuning?

Data Science, AI and ML
1

Hyperparameter tuning is a process of selecting the best parameters of the model so that our prediction accuracy will be good. For example- When we use Random Forest Classifier to train the model, The question arises that what parameters of the Random Forest Classifier we use so that accuracy of the model will be high. The main parameters used by a Random Forest Classifier are:

  • criterion = the function used to evaluate the quality of a split.
  • max_depth = maximum number of levels allowed in each tree.
  • max_features = maximum number of features considered when splitting a node.
  • min_samples_leaf = minimum number of samples which can be stored in a tree leaf.
  • min_samples_split = minimum number of samples necessary in a node to cause node splitting.
  • n_estimators = number of trees in the ensemble.

To solve this issue we use different methods.

  • Manual Search.
  • Random Search.
  • Grid Search.
  • Automated Hyperparameter Tuning (Bayesian Optimization, Genetic Algorithms )
  • Artificial Neural Networks (ANNs) Tuning.
2

In machine learning, a hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters are derived via training. Hyperparameters are the variables which determines the network structure (E.g.: Number of Hidden Units) and also the variables which determine how the network is trained (E.g.: Learning Rate). Hyperparameters are set before training (before optimizing the weights and bias).

For example, in decision tree shown below. The hyperparameters are

  1. Number of leaf nodes
  2. Depth of tree
  3. Minimum sample required to split the node

main-qimg-b5d2b77b6e0d6ead881d2294332dcc59
main-qimg-b5d2b77b6e0d6ead881d2294332dcc59720×382 20.9 KB