Do you think that imputing missing values with mean is a quick and efficient way for handling missing values?
Get your myths ready to be busted.
Imputing missing depends completely on the data and the domain you’re in.
Numerical/Categorical/Time Series ; different data need to be imputed differently.
Focusing on numerical, imputing with mean is a very commonly used way. But, it has more cons than pros.
It messes up the statistics, the distribution, the quantiles and doesn’t take correlation among features into account.
Solution? pip install impyute.
Impyute is a library specially designed for smart imputations. Two of its best techniques are: KNN and MICE.
KNN (K-Nearest Neighbors) finds the similar values of the nearest neighbors and imputes its average.
MICE (Multivariate Imputation by Chained Equations): What a heavy name!
Simply put, MICE considers the feature with missing values as a dependent variable, and the remaining features as the predictors.
From these multiple fitted models, MICE picks up the best ones and imputes using them.
Results? Way better than mean imputations.
The data statistics remain intact, model performance improves.
Your thoughts on imputations?
#machinelearning #datascience