Does feature reducing improve prediction score?

Feature reduction techniques such as LDA and PCA are very powerful in reducing the number of features. However, that doesn’t improve the prediction score drastically. Then why still bother about learning them or implement them?

  • Computation power:
    Lower the number of features, lesser will be the computation time for training the model as well as for deploying the model. The real life scneario is not merely few thousand rows with 10 odd columns, but lakhs of rows with 100 plus columns.

  • Feature selection:
    Machine learning is not just used for deploying the models for real time prediction, but also to understand how the machine learnt, and what features it weighs highest for prediction. Having lean number of linearly independent features is a must before conducting feature selection.