Machine Learning Models need to be explainable.
Explainability means you can clearly see what the model is learning, which feature is impacting the output and how.
But why is this important?
If you build a simple regression model with 2 features, you can easily interpret the model by looking at its parameters after training.
Or if you make a small decision tree, you can visualize it and see how it is splitting and what features are impacting output.
But is it possible for random forests with 200 trees? Or neural networks?
ML models are not meant to be black boxes and there is a need for you to be able to explain their working to the client and be sure that it is not being biased or learning parameters that it should not.
There are many frameworks present exactly for this.
One of which is: SHAP (SHapley Additive exPlainations).
With SHAP you can explain models by visualizing various types of plots to understand how each feature is impacting the output and to what extent.
There are frameworks for neural networks as well like DeepLIFT and DeepSHAP.
The image below shows the impact of all the features on the left on the classes of the output using SHAP.
Did you know about SHAP?
#machinelearning #datascience #python
