Skip to content

Explainers: Membership Inference by Group and Fairness Under Data Poisoning #1138

Closed as not planned

Description

@yakew7

Differential Privacy covers privacy's cost to small groups, but not how a model's leakage and its robustness differ by group. Add both explainers in one pull request.

explainers/membership-inference-by-group.md - Is a Model More Likely to Leak Some Groups' Training Data?

A simple loss-threshold membership inference attack run separately per group on a real audit with a flexible model (random forest or gradient boosting), reporting the attack's advantage per group with intervals, and relating the gap to the train-test accuracy gap per group. Say what the numbers do and do not show about real-world privacy risk.

Key citation to verify: Shokri et al. (2017), "Membership Inference Attacks Against Machine Learning Models" (IEEE S&P), and Chang, Shokri (2021), "On the Privacy Risks of Algorithmic Fairness" (EuroS&P) - confirm.

explainers/fairness-under-data-poisoning.md - How Does Poisoning a Few Rows Affect a Fairness Gap?

Flip or insert a small fraction of training rows targeted at one group on a real audit and measure how far the parity gap and per-group accuracy move, against the same number of random flips. Report how few rows are needed to move the gap by a fixed amount, and what simple defences (data sanitisation, robust loss) do.

Key citation to verify: Solans, Biggio, Castillo (2020), "Poisoning Attacks on Algorithmic Fairness" (ECML PKDD) - confirm.

Related explainers: differential-privacy, label-noise-by-group, distribution-shift, feedback-loop-bias, disaggregated-evaluation.

See CONTRIBUTING.md: add explainers/<slug>.md plus an entry in assets/explainers-data.json (with a description of 70 to 160 characters, and a seo_title if the title plus the site name would pass 65), then run make build-explainers; the build generates the pages, sitemap, llms-full.txt and OG images, and llms.txt and the README table must list each explainer (scripts/check_explainer_count.py checks both, and scripts/check_explainer_structure.py checks the sections). Match a close sibling explainer, give every explainer a Limitations section and at least two inbound links from older ones, use real code run against one of the repo's audits (faircode.benchmark.load_audit loads an audit exactly like the benchmark harness) and real, not invented, results, and if you quote a Fair Code result say whether it comes from results/ (current) or paper/results-frozen/ (the earlier reference snapshot), per CLAUDE.md. Verify every citation yourself before using it.

Done when: the 2 explainers, their README, CONTRIBUTING, ROADMAP, llms.txt and CHANGELOG entries, the website JSON entries and the regenerated generated files are merged, make lint passes, and each explainer's code is runnable from the repo root.

Activity

  1. added
    documentationImprovements or additions to documentation
    help wantedExtra attention is needed
    on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationhelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions