Skip to content

Explainers: Feature Importance by Group, Partial Dependence by Group, and Explanation Disagreement #1117

Closed

Description

@yakew7

SHAP Values and LIME explain one prediction. The questions an auditor asks next are about groups: does the model rely on different features for different people, and do two explanation methods agree. Add all three in one pull request.

explainers/feature-importance-by-group.md - Does the Model Rely on Different Features for Different Groups?

Permutation importance computed separately on each group's test rows for a real audit's model, with intervals, and a comparison of the rankings. Show a feature that matters much more for one group, relate it to aggregation bias and proxy use, and cover why a feature with equal importance can still have different effects.

Key citation to verify: Breiman (2001), "Random Forests" (Machine Learning), for permutation importance, and Fisher, Rudin, Dominici (2019), "All Models Are Wrong, but Many Are Useful" (JMLR) - confirm.

explainers/partial-dependence-by-group.md - What Is Partial Dependence by Group?

The model's average prediction as one feature varies, drawn separately per group, so a curve that is flat for one group and steep for another is visible. Compute it with scikit-learn for a real audit's gradient-boosting model and compare the curves, noting the assumption of independent features and the regions with no data for a group.

Key citation to verify: Friedman (2001), "Greedy Function Approximation: A Gradient Boosting Machine" (Annals of Statistics) - confirm.

explainers/explanation-disagreement.md - What Happens When Two Explanation Methods Disagree?

Compare feature rankings from permutation importance, a linear model's coefficients and (if installed) SHAP on the same model and data, with a rank-correlation per group, and show which disagreements matter for a fairness claim such as "the model does not use feature X". Cover why an explanation is evidence about a model, not a proof.

Key citation to verify: Krishna et al. (2022), "The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective" (arXiv:2202.01602) - confirm.

Related explainers: shap-values, lime, aggregation-bias, proxy-variables, counterfactual-explanation, disaggregated-evaluation.

See CONTRIBUTING.md: add explainers/<slug>.md plus a one-line entry in assets/explainers-data.json, then run make build-explainers; the build script generates the pages, sitemap, llms-full.txt and OG images, and llms.txt and the README table must list each explainer (scripts/check_explainer_count.py checks both). Match the structure of a close sibling explainer, use real code run against one of the repo's audits (faircode.benchmark.load_audit loads an audit exactly like the benchmark harness) and real, not invented, results, and if you quote a Fair Code result say whether it comes from results/ (current) or paper/results-frozen/ (the earlier reference snapshot), per CLAUDE.md. Verify every citation yourself before using it.

Done when: the 3 explainers, their README, CONTRIBUTING, ROADMAP, llms.txt and CHANGELOG entries, the website JSON entries and the regenerated generated files are merged, make lint passes, and each explainer's code is runnable from the repo root.

Activity

  1. added
    documentationImprovements or additions to documentation
    help wantedExtra attention is needed
    on Oct 21, 2026
  2. yakew7 commented on Oct 23, 2026

    @yakew7
    OwnerAuthor

    Fixed in commit 46d3ee3 (Add three explainers: Feature Importance by Group, Partial Dependence by Group, Explanation Disagreement (#1117)), now on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationhelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions