LIME and SHAP are two popular methods used in machine learning for interpreting and explaining the predictions of complex models. Let me briefly explain each:
LIME (Local Interpretable Model-agnostic Explanations): LIME is a technique that explains the predictions of any machine learning classifier by approximating it locally with an interpretable model. It works by perturbing the input data and seeing how the predictions change. This helps understand which features are most important for a particular prediction.
Key points:
- Local: Explains individual predictions
- Model-agnostic: Can be used with any machine learning model
- Interpretable: Provides explanations in an easy-to-understand format
LIME was introduced in the paper Why should I trust you?
How lime works, step by step
- Perturbation: LIME generates a new dataset consisting of perturbed instances around the instance being explained. These perturbed instances are created by randomly turning on or off certain features of the original instance.
- Prediction: The complex model is then used to make predictions for these perturbed instances. This provides information about how the model behaves in the vicinity of the instance being explained.
- Weighting: The perturbed instances are weighted according to their proximity to the original instance using a kernel function (e.g., an exponential kernel). Instances closer to the original instance are given higher weights.
- Model Fitting: A simple, interpretable model (e.g., a linear model or a decision tree) is then fitted to the perturbed instances, trying to minimize the locally weighted loss function. The interpretable model is designed to approximate the complex model well in the local region around the instance being explained.
- Explanation: The coefficients or feature importances of the interpretable model are then used as explanations for the original instance. They indicate how much each feature contributes to the complex model’s prediction for that specific instance.
The tradeoff with explainability
When decision must be justified, rather models are used that are better interpretable by nature, such as decision trees or sparse feature linear models. This is the predictive performance - interpretability trade-off
LIME can help to explain any black-box classifier.
To produce the explanations, LIME feeds the black-box model with small variations of the original data sample and probes how the model’s predictions change. From these variations, it learns an interpretable model which approximates the black-box classifier in the vicinity of the original data sample. Locally, the interpretable model provides a faithful approximation of the black-box model, even though it is likely not a globally good approximator.
Compatibility
Both LIME and SHAP are often not very useful on Time series data due to the few features.
Resources
https://www.openlayer.com/blog/post/understanding-lime-in-5-steps