Add AAFP clinical recommendation scraper

[Abduhu1]

What this adds

This PR adds an AAFP scraper for Clinical Insights pages.

It discovers eligible pages from the AAFP sitemap or accepts a specific page through --url. Each clinical recommendation drawer is saved as a separate ScrapedDocument. This avoids mixing recommendations with different populations, sources, dates, and AAFP positions into one document.

The scraper preserves the recommendation content as Markdown and records useful metadata, including:

• Topic and category
• Recommendation title and subtitle
• Source links
• AAFP status and status date
• Topic URL and section anchor

It also reports errors when discovery or page parsing fails instead of silently returning an empty result.

Files changed

• datasets/amfv_datasets/scraping/aafp.py

  • Adds AAFP discovery, parsing, metadata extraction, and error handling.

• datasets/amfv_datasets/scraping/cli.py

  • Registers aafp as a scraper source.
  • Rejects --source all when used with a single --url.

• datasets/test/test_scraping_aafp.py

  • Adds offline tests using synthetic HTML, XML, and mocked HTTP requests.

• datasets/test/test_scraping_cli.py

  • Updates CLI source tests and adds coverage for the new validation rule.

No new dependencies were added.

How to test

uv sync --frozen

uv run --frozen pytest datasets/test/test_scraping_aafp.py -v

uv run --frozen pytest -q

uv run --frozen ruff check .

uv run --frozen ruff format --check .

uv run --frozen amfv-scrape --help

Optional live smoke test:

uv run --frozen amfv-scrape \
  --source aafp \
  --url "https://www.aafp.org/clinical-insights/cardiometabolic-health/hypertension" \
  --documents 2 \
  --no-progress \
  --output /tmp/aafp-smoke.jsonl

Results

• AAFP scraper tests: 74 passed
• Full test suite: 105 passed
• Ruff checks and formatting passed
• Direct URL and automatic discovery smoke tests produced valid documents

This PR contains scraper code and synthetic test fixtures only. It does not add scraped AAFP content.

[CLAassistant]

All committers have signed the CLA.