What happens if ML model has too many features

[rajanikant-ghate]

“Too many” needs to be quantified with respect to the number of rows, and number classes to be determined (in case of classification). In general, if you recall from basic maths that, for solving ‘n’ variable equation, you need ‘n’ simultaneous equations.

So first constraint is that, the mathematics behind the algorithms should work. So simply exaggerating, a training set of 2 rows with binary class with 100 columns will definitely not be a good idea.

But realistically speaking, most of the times, the model parameters will converge. However, three major challenges will remain:

1. Model will become overly complex and hence might be overfitting.
2. Model will be computationally expensive.
3. Explaining the feature importance will be challenging