What is the difference between PCA and LDA?

  • Principal Component Analysis or PCA for short is a commonly used unsupervised linear transformation technique. PCA reduces the number of dimensions by finding the maximum variance in high-dimensional data.
  • Linear Discriminant Analysis or LDA for short is a supervised method that takes class labels into account when reducing the number of dimensions. The goal of LDA is to find a feature subspace that best optimizes class separability.

Linear discriminant analysis is very similar to PCA both look for linear combinations of the features which best explain the data.

The main difference is that the Linear discriminant analysis is a supervised dimensionality reduction technique that also achieves the classification of the data simultaneously.

  • LDA focuses on finding a feature subspace that maximizes the separability between the groups.
  • While Principal component analysis is an unsupervised Dimensionality reduction technique, it ignores the class label.
  • PCA focuses on capturing the direction of maximum variation in the data set.