SHAP: Game-theory-based attribution
SHAP assigns each feature a contribution to the prediction using cooperative game theory — mathematically rigorous and theoretically defensible.
SHAP uses concepts from cooperative game theory to assign each feature a contribution to the model's prediction for a specific input — how much did each feature push the prediction up or down from the average? SHAP values have a mathematically rigorous property: they're the only attribution method satisfying certain fairness axioms (efficiency, symmetry, dummy, and additivity), making SHAP outputs more theoretically defensible than simpler attribution approaches. SHAP is computationally efficient for tree-based models (where exact SHAP values can be computed quickly) and approximated for other model types.
- Game-theory based — cooperative game theory attribution.
- Fairness axioms — efficiency, symmetry, dummy, additivity.
- Theoretically defensible — rigorous mathematical foundation.
- Tree efficient — exact values for tree-based models.
- Approximated — for neural networks and other model types.
LIME: Local approximation explanation
LIME explains predictions by fitting a simple interpretable model to the local neighborhood of a specific input — model-agnostic and widely applicable.
LIME explains a specific prediction by perturbing the input and fitting a simple, interpretable model (typically linear) to the model's behavior in the local neighborhood of that specific input — the simple local model's feature weights provide an explanation that's accurate near the prediction being explained, even if the global model is highly complex. LIME is model-agnostic (works with any model that can be queried for predictions), slightly less computationally efficient than tree SHAP, and produces different explanation artifacts (linear approximation weights rather than additive game-theoretic values).
- Local approximation — fits simple model to local neighborhood.
- Model-agnostic — works with any queryable model.
- Linear explanation — feature weights from simple model.
- Accurate locally — explains behavior near specific input.
- Complementary — different artifact than SHAP values.
SHAP vs. LIME: Which to use
Both have strengths — SHAP is more rigorous and efficient for tree models; LIME is more model-agnostic and widely applicable.
Both tools are widely used and each has practical strengths. SHAP produces more theoretically rigorous attributions and is particularly efficient for gradient-boosted tree models — the most common model type for enterprise tabular ML. LIME is more model-agnostic and can work with any black-box model through query-based perturbation. ClickMasters evaluates both for your specific model type and regulatory context, sometimes using both together since they can produce complementary explanations.
- SHAP — rigorous, efficient for tree models.
- LIME — model-agnostic, wide applicability.
- Complementary — sometimes both used together.
- Regulatory context — choice depends on requirements.
- Model type — gradient-boosted trees favor SHAP.
How ClickMasters implements explainability tools for enterprise clients
ClickMasters integrates SHAP and LIME as standard practice for any consequential classification or regression model — building explanation infrastructure that produces both individual-prediction explanations (for regulatory adverse action reasons, auditor review, or customer-facing explainability) and aggregate feature importance analysis (for model validation and bias audit purposes).
- Standard practice — for consequential models.
- Individual-prediction explanations — regulatory, audit, customer-facing.
- Aggregate feature importance — validation and bias audit.
- Human-readable templates — non-technical stakeholder presentation.
- Bias audit support — alongside dedicated fairness testing.
AI explainability tools pricing
| Project Type | Price Range | Timeline |
|---|---|---|
| SHAP/LIME integration for existing model | $20,000 – $40,000 | 4–8 weeks |
| Explainability included in new model development | Included in development scope | Varies |
| Regulatory-ready explanation reporting system | $30,000 – $65,000 | 8–14 weeks |
Common objections, answered directly
- "Our model is a black box — can SHAP actually explain it meaningfully?" — SHAP works with any model that produces predictions — for neural networks, SHAP uses approximation methods (DeepSHAP, KernelSHAP) that work with any model architecture.
- "How much will this cost?" — $20,000–$65,000 depending on whether this is integration alone or regulatory reporting. Fixed quote after Discovery.
- "Can SHAP/LIME satisfy adverse action explanation requirements for credit decisions?" — SHAP values can produce individual-prediction feature importance suitable for adverse action reason codes — ClickMasters recommends legal counsel for specific FCRA compliance determinations about your implementation.
- "Do SHAP and LIME always agree on which features are most important?" — They often agree on the most important features but can differ on the precise attribution values, particularly for models where the decision boundary is highly nonlinear — ClickMasters investigates meaningful disagreements rather than treating one method as definitively correct.
Limitations and risks of AI explainability tools
AI explainability tools have limitations. SHAP and LIME can disagree on attribution values. Explanation quality depends on model complexity. SHAP approximations for neural networks may be less accurate. LIME explanations are only locally valid. Computational cost increases with feature count and model complexity. Explanations can be misused as definitive causal analysis. SHAP values are model-specific — retraining changes explanations. Human interpretation of SHAP outputs requires training.
- Disagreement — SHAP and LIME can differ.
- Model complexity — affects explanation quality.
- Approximation accuracy — neural networks less exact.
- Local validity — LIME only accurate locally.
- Computational cost — increases with features and complexity.
- Misuse risk — not definitive causal analysis.
- Model-specific — retraining changes explanations.
- Interpretation training — required for stakeholders.
Best practices for explainability tool implementation
Implementing AI explainability tools effectively requires careful method selection, interpretation guidance, and integration with existing model governance. The following practices can help ensure your explainability infrastructure delivers regulatory and audit value.
Recommended approach:
- 01Select appropriate method — SHAP for tree models, LIME for model-agnostic.
- 02Validate explanations — test against domain expectations.
- 03Design for your audience — regulatory, audit, or customer-facing.
- 04Build human-readable templates — from SHAP/LIME outputs.
- 05Combine with fairness testing — don't rely solely on SHAP.
- 06Plan for model retraining — explanations update with models.
- 07Document interpretation guidelines — train stakeholders.
- 08Monitor explanation consistency — track changes over model versions.