Characteristics of Feature Selection Algorithms

[chayan-kathuria]

The purpose of a feature selection algorithms is to identify relevant features according to a definition of relevance. However, the notion of relevance in machine learning has not yet been rigorously defined on a common agreement. A primary definition of relevance is the notion of being relevant with respect to an objective.

There are several considerations in the literature to characterize feature selection algorithms. In view of these, it is possible to describe this characterization as a search problem in the hypothesis space as follows:

Search Organization: general strategy with which the space of hypothesis is explored.
Generation of Successors: mechanism by which possible variants (successor candidates) of the current hypothesis are proposed.
Evaluation Measure: function by which successor candidates are evaluated, allowing to compare different hypotheses to guide the search process.