SHAP (SHapley Additive exPlanations)

Overview

SHAP is a unified approach to explain machine learning model outputs using Shapley values from cooperative game theory. This skill provides comprehensive guidance for:

• Computing SHAP values for any model type
• Creating visualizations to understand feature importance
• Debugging and validating model behavior
• Analyzing fairness and bias
• Implementing explainable AI in production

SHAP works with all model types: tree-based models (XGBoost, LightGBM, CatBoost, Random Forest), deep learning models (TensorFlow, PyTorch, Keras), linear models, and black-box models.

When to Use This Skill

Trigger this skill when users ask about:

• "Explain which features are most important in my model"
• "Generate SHAP plots" (waterfall, beeswarm, bar, scatter, force, heatmap, etc.)
• "Why did my model make this prediction?"
• "Calculate SHAP values for my model"
• "Visualize feature importance using SHAP"
• "Debug my model's behavior" or "validate my model"
• "Check my model for bias" or "analyze fairness"
• "Compare feature importance across models"
• "Implement explainable AI" or "add explanations to my model"
• "Understand feature interactions"
• "Create model interpretation dashboard"

Quick Start Guide

Step 1: Select the Right Explainer

Decision Tree:

• Tree-based model? (XGBoost, LightGBM, CatBoost, Random Forest, Gradient Boosting)

- Use shap.TreeExplainer (fast, exact)

• Deep neural network? (TensorFlow, PyTorch, Keras, CNNs, RNNs, Transformers)

- Use shap.DeepExplainer or shap.GradientExplainer

• Linear model? (Linear/Logistic Regression, GLMs)

- Use shap.LinearExplainer (extremely fast)

• Any other model? (SVMs, custom functions, black-box models)

- Use shap.KernelExplainer (model-agnostic but slower)

• Unsure?

- Use shap.Explainer (automatically selects best algorithm)

See references/explainers.md for detailed information on all explainer types.

Step 2: Compute SHAP Values

import shap

# Example with tree-based model (XGBoost)
import xgboost as xgb

# Train model
model = xgb.XGBClassifier().fit(X_train, y_train)

# Create explainer
explainer = shap.TreeExplainer(model)

# Compute SHAP values
shap_values = explainer(X_test)

# The shap_values object contains:
# - values: SHAP values (feature attributions)
# - base_values: Expected model output (baseline)
# - data: Original feature values

Step 3: Visualize Results

For Global Understanding (entire dataset):

# Beeswarm plot - shows feature importance with value distributions
shap.plots.beeswarm(shap_values, max_display=15)

# Bar plot - clean summary of feature importance
shap.plots.bar(shap_values)

For Individual Predictions:

# Waterfall plot - detailed breakdown of single prediction
shap.plots.waterfall(shap_values[0])

# Force plot - additive force visualization
shap.plots.force(shap_values[0])

For Feature Relationships:

# Scatter plot - feature-prediction relationship
shap.plots.scatter(shap_values[:, "Feature_Name"])

# Colored by another feature to show interactions
shap.plots.scatter(shap_values[:, "Age"], color=shap_values[:, "Education"])

See references/plots.md for comprehensive guide on all plot types.

Core Workflows

This skill supports several common workflows. Choose the workflow that matches the current task.

Workflow 1: Basic Model Explanation

Goal: Understand what drives model predictions Steps:

• Train model and create appropriate explainer
• Compute SHAP values for test set
• Generate global importance plots (beeswarm or bar)
• Examine top feature relationships (scatter plots)
• Explain specific predictions (waterfall plots)

Example:

# Step 1-2: Setup
explainer = shap.TreeExplainer(model)
shap_values = explainer(X_test)

# Step 3: Global importance
shap.plots.beeswarm(shap_values)

# Step 4: Feature relationships
shap.plots.scatter(shap_values[:, "Most_Important_Feature"])

# Step 5: Individual explanation
shap.plots.waterfall(shap_values[0])

Workflow 2: Model Debugging

Goal: Identify and fix model issues Steps:

• Compute SHAP values
• Identify prediction errors
• Explain misclassified samples
• Check for unexpected feature importance (data leakage)
• Validate feature relationships make sense
• Check feature interactions

See references/workflows.md for detailed debugging workflow.

Workflow 3: Feature Engineering

Goal: Use SHAP insights to improve features Steps:

• Compute SHAP values for baseline model
• Identify nonlinear relationships (candidates for transformation)
• Identify feature interactions (candidates for interaction terms)
• Engineer new features
• Retrain and compare SHAP values
• Validate improvements

See references/workflows.md for detailed feature engineering workflow.

Workflow 4: Model Comparison

Goal: Compare multiple models to select best interpretable option Steps:

• Train multiple models
• Compute SHAP values for each
• Compare global feature importance
• Check consistency of feature rankings
• Analyze specific predictions across models
• Select based on accuracy, interpretability, and consistency

See references/workflows.md for detailed model comparison workflow.

Workflow 5: Fairness and Bias Analysis

Goal: Detect and analyze model bias across demographic groups Steps:

• Identify protected attributes (gender, race, age, etc.)
• Compute SHAP values
• Compare feature importance across groups
• Check protected attribute SHAP importance
• Identify proxy features
• Implement mitigation strategies if bias found

See references/workflows.md for detailed fairness analysis workflow.

Workflow 6: Production Deployment

Goal: Integrate SHAP explanations into production systems Steps:

• Train and save model
• Create and save explainer
• Build explanation service
• Create API endpoints for predictions with explanations
• Implement caching and optimization
• Monitor explanation quality

See references/workflows.md for detailed production deployment workflow.

Key Concepts

SHAP Values

Definition: SHAP values quantify each feature's contribution to a prediction, measured as the deviation from the expected model output (baseline). Properties:

• Additivity: SHAP values sum to difference between prediction and baseline
• Fairness: Based on Shapley values from game theory
• Consistency: If a feature becomes more important, its SHAP value increases

Interpretation:

• Positive SHAP value → Feature pushes prediction higher
• Negative SHAP value → Feature pushes prediction lower
• Magnitude → Strength of feature's impact
• Sum of SHAP values → Total prediction change from baseline

Example:

Baseline (expected value): 0.30
Feature contributions (SHAP values):
  Age: +0.15
  Income: +0.10
  Education: -0.05
Final prediction: 0.30 + 0.15 + 0.10 - 0.05 = 0.50

Background Data / Baseline

Purpose: Represents "typical" input to establish baseline expectations Selection:

• Random sample from training data (50-1000 samples)
• Or use kmeans to select representative samples
• For DeepExplainer/KernelExplainer: 100-1000 samples balances accuracy and speed

Impact: Baseline affects SHAP value magnitudes but not relative importance

Model Output Types

Critical Consideration: Understand what your model outputs

• Raw output: For regression or tree margins
• Probability: For classification probability
• Log-odds: For logistic regression (before sigmoid)

Example: XGBoost classifiers explain margin output (log-odds) by default. To explain probabilities, use model_output="probability" in TreeExplainer.

Common Patterns

Pattern 1: Complete Model Analysis

# 1. Setup
explainer = shap.TreeExplainer(model)
shap_values = explainer(X_test)

# 2. Global importance
shap.plots.beeswarm(shap_values)
shap.plots.bar(shap_values)

# 3. Top feature relationships
top_features = X_test.columns[np.abs(shap_values.values).mean(0).argsort()[-5:]]
for feature in top_features:
    shap.plots.scatter(shap_values[:, feature])

# 4. Example predictions
for i in range(5):
    shap.plots.waterfall(shap_values[i])

Pattern 2: Cohort Comparison

# Define cohorts
cohort1_mask = X_test['Group'] == 'A'
cohort2_mask = X_test['Group'] == 'B'

# Compare feature importance
shap.plots.bar({
    "Group A": shap_values[cohort1_mask],
    "Group B": shap_values[cohort2_mask]
})

Pattern 3: Debugging Errors

# Find errors
errors = model.predict(X_test) != y_test
error_indices = np.where(errors)[0]

# Explain errors
for idx in error_indices[:5]:
    print(f"Sample {idx}:")
    shap.plots.waterfall(shap_values[idx])

    # Investigate key features
    shap.plots.scatter(shap_values[:, "Suspicious_Feature"])

Performance Optimization

Speed Considerations

Explainer Speed (fastest to slowest):

• LinearExplainer - Nearly instantaneous
• TreeExplainer - Very fast
• DeepExplainer - Fast for neural networks
• GradientExplainer - Fast for neural networks
• KernelExplainer - Slow (use only when necessary)
• PermutationExplainer - Very slow but accurate

Optimization Strategies

For Large Datasets:

# Compute SHAP for subset
shap_values = explainer(X_test[:1000])

# Or use batching
batch_size = 100
all_shap_values = []
for i in range(0, len(X_test), batch_size):
    batch_shap = explainer(X_test[i:i+batch_size])
    all_shap_values.append(batch_shap)

For Visualizations:

# Sample subset for plots
shap.plots.beeswarm(shap_values[:1000])

# Adjust transparency for dense plots
shap.plots.scatter(shap_values[:, "Feature"], alpha=0.3)

For Production:

# Cache explainer
import joblib
joblib.dump(explainer, 'explainer.pkl')
explainer = joblib.load('explainer.pkl')

# Pre-compute for batch predictions
# Only compute top N features for API responses

Troubleshooting

Issue: Wrong explainer choice

Problem: Using KernelExplainer for tree models (slow and unnecessary) Solution: Always use TreeExplainer for tree-based models

Issue: Insufficient background data

Problem: DeepExplainer/KernelExplainer with too few background samples Solution: Use 100-1000 representative samples

Issue: Confusing units

Problem: Interpreting log-odds as probabilities Solution: Check model output type; understand whether values are probabilities, log-odds, or raw outputs

Issue: Plots don't display

Problem: Matplotlib backend issues Solution: Ensure backend is set correctly; use plt.show() if needed

Issue: Too many features cluttering plots

Problem: Default max_display=10 may be too many or too few Solution: Adjust max_display parameter or use feature clustering

Issue: Slow computation

Problem: Computing SHAP for very large datasets Solution: Sample subset, use batching, or ensure using specialized explainer (not KernelExplainer)

Integration with Other Tools

Jupyter Notebooks

• Interactive force plots work seamlessly
• Inline plot display with show=True (default)
• Combine with markdown for narrative explanations

MLflow / Experiment Tracking

```python

import mlflow

with mlflow.start_run():

# Train model

model = train_model(X_train, y_train)

# Compute SHAP

explainer = shap.TreeExplainer(model)

shap_values = explainer(X_test)

# Log plots

shap.plots.beeswarm(shap_values, show=False)

mlflow.log_figure(plt.gcf(), "shap_beeswarm.png")

plt.close()

# Log feature importance metrics

mean_abs_shap = np.abs(shap_values.values).mean(axis=0)

for feature, importance in zip(X_test.columns, mean_abs_shap):

mlflow.log_metric(f"shap_{feature}", impo