Local Interpretable Model-agnostic Explanations (LIME)

Local interpretable model-agnostic explanations (LIME) is a method for explaining why a machine-learning model made one specific prediction. Many accurate models — gradient-boosted trees, deep networks, large ensembles — are black boxes: they output a score but no readable reason. LIME, introduced by Marco Ribeiro, Sameer Singh, and Carlos Guestrin in their 2016 paper "Why Should I Trust You?", cracks open a single decision. It does so by building a simple surrogate model — usually a small linear model — that imitates the black box, but only in the immediate neighborhood of the one case you want to explain. The simple model's weights then read out as the reasons: which features pushed this particular prediction up, which pushed it down, and by how much, in the local region around that case.

The three words in the name describe the method exactly. Local means the explanation holds only near the one prediction, not across the whole model — the black box may be wildly complex globally, but near any single point it can often be approximated by something simple. Interpretable means the surrogate is a model a person can read, like a short weighted list of features. Model-agnostic means LIME treats the original model as a sealed box, probing it only by feeding in inputs and watching outputs, so it works on any model regardless of its internals. To build the local picture, LIME perturbs the input — nudging feature values to create many nearby variations — observes how the black box responds, and fits the weighted simple model to those responses.

LIME's main companion and rival is SHAP, which also explains individual predictions but grounds its feature contributions in cooperative game theory, giving values with stronger theoretical guarantees of consistency. The practical trade-off is familiar: LIME is fast, intuitive, and easy to apply to any model, while SHAP is often more stable and principled but heavier to compute. Both are local, per-prediction explainers, and both are model-agnostic in their general forms. LIME also differs sharply from global interpretability — a single linear or shallow-tree model that is transparent everywhere by design. LIME does not make the black box transparent overall; it gives a faithful-enough snapshot around one case at a time.

The honest limitation of LIME is instability. Because it relies on random perturbations and a local fit, running it twice on the same prediction can yield somewhat different explanations, and the result depends on choices like how the neighborhood is defined and how wide it is. The local linear approximation can also mislead where the true decision surface curves sharply near the point. So LIME explanations should be read as plausible, local, approximate accounts — useful for debugging a model, spotting a feature that should not be driving decisions, or giving a human a reason to scrutinize a borderline case — not as exact, guaranteed attributions. Treat a LIME explanation as a hypothesis about why the model decided, worth checking, rather than ground truth.

Use LIME when you have an accurate but opaque model and need a human-readable reason for individual predictions — to debug surprising scores, to check that the model is leaning on sensible features rather than leaks or proxies, or to give an analyst grounds to review a particular case. Apply it to the specific predictions that matter rather than expecting a global summary, and present the output as the local drivers of that one decision. Because explanations can vary run to run, check their stability — re-run a few times, or compare with SHAP — before acting on any single explanation, and be cautious near sharp decision boundaries where the local linear fit is weakest.

Avoid the common misreadings. LIME is local, so do not generalize one explanation into a claim about how the model behaves everywhere. It is model-agnostic and approximate, so its feature weights are an estimate of local influence, not exact causal contributions, and certainly not real-world causation. Do not present a LIME explanation as a guarantee, especially in high-stakes or regulated settings where stability and fidelity matter — pair it with more robust methods and human judgment there. And remember that an explanation of a wrong model is still wrong: LIME tells you why the model decided, which is only as trustworthy as the model itself. Used as a focused, local debugging and scrutiny tool, LIME makes black-box decisions legible enough to question.

Local interpretable model-agnostic explanations (LIME), from Ribeiro, Singh & Guestrin (2016), explains one prediction of any model with a simple local surrogate, making black-box decisions legible.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where local interpretable model-agnostic explanations (lime) is a core concern: