From b639bb94e7fb3fe2592146721886232ebe0b3366 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 15:59:26 +1000 Subject: [PATCH 1/2] =?UTF-8?q?Annotate=20the=20seven=20new=20upstream=20d?= =?UTF-8?q?ata=20refs=20=E2=80=94=20unblocks=20the=20strict=20build?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The audit-dashboard build has been red since the 2026-07-27 schedule. It is not broken code: build_audit.py scans the live main of the 8 lecture repos, and --strict exits 1 when any reference has neither a manifest nor an annotation. Three upstream additions landed refs this file had never seen, so every data-lectures PR since has failed on drift it did not cause (most recently #27). The gap, and where each ref came from: NEWQDATA.csv, and phillips_drifts_volatilities / phillips_lost_conquest / phillips_two_stories — added by QuantEcon/lecture-python.myst#1001 (Jul 26), the first four warnings on the Jul 27 schedule polars — added by QuantEcon/lecture-python-programming#408 (Jul 30), a Polars rewrite of pandas' FRED and yfinance sections sargent_surico — added by QuantEcon/lecture-python.myst#1020 (Aug 1) Series lists are read off the lecture sources rather than inferred. The six API entries are annotated incidental except polars, which mirrors pandas' lesson framing — the fetch itself is the teaching point there. NEWQDATA.csv is recorded verbatim: the lecture cites it as Cogley and Sargent's quarterly dataset and commits no builder beside it. The note flags it as a migration candidate — it is read from lecture-python.myst main by raw URL, and the lecture already documents every transformation and rebuilds the post-2000Q4 continuation from FRED. Verified against clones of all 8 repos: `build_audit.py scan --strict` goes from exit 1 with 7 warnings to exit 0 with none. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/audit_annotations.yml | 36 +++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/scripts/audit_annotations.yml b/scripts/audit_annotations.yml index c523de8..dbbdbd5 100644 --- a/scripts/audit_annotations.yml +++ b/scripts/audit_annotations.yml @@ -18,6 +18,16 @@ # toy invented in-lecture teaching table datasets: + NEWQDATA.csv: + description: Cogley–Sargent quarterly US data (inflation, unemployment, T-bill), 1948Q2–2000Q4 + provenance: verbatim + flags: [new-since-2026-07-15] + note: > + added by lecture-python.myst#1001; the lecture cites it as + CogleySargent2005's dataset and no builder is committed beside it. + A migration candidate — it is read from lecture-python.myst main by raw + URL, and the lecture documents every transformation and rebuilds the + post-2000Q4 continuation from FRED SCF_plus_mini.csv: description: SCF+ wealth/income survey extract provenance: constructed-committed @@ -187,9 +197,35 @@ api: lecture-python-programming:pandas:fredgraph.csv URL: series: UNRATE pedagogy: lesson + lecture-python-programming:polars:yfinance: + series: 11-ticker exercise list + pedagogy: lesson + note: mirror of pandas' yfinance exercise, rewritten with pl.DataFrame + lecture-python-programming:polars:fredgraph.csv URL: + series: UNRATE + pedagogy: lesson + note: mirror of pandas' FRED section — the lesson is pl.read_csv from a URL lecture-python.myst:kesten_processes:yfinance: series: ^IXIC pedagogy: incidental + lecture-python.myst:phillips_drifts_volatilities:fredgraph.csv URL: + series: CPIAUCSL, UNRATE, TB3MS + pedagogy: incidental + note: > + splices post-2000Q4 observations onto the Cogley-Sargent sample + (NEWQDATA.csv) to test whether the extra quarter-century changes the + findings. Snapshot-ready, but a snapshot would need periodic refresh — + the section's point is the *latest* vintage + lecture-python.myst:phillips_lost_conquest:pandas_datareader: + series: PCEPI, GDPC1, GDPPOT, FEDFUNDS + pedagogy: incidental + lecture-python.myst:phillips_two_stories:pandas_datareader: + series: CPIAUCNS, LNS14000028 + pedagogy: incidental + note: LNS14000028 is the unemployment rate for white men 20+ + lecture-python.myst:sargent_surico:pandas_datareader: + series: M2SL, GDPDEF, GDPC1, TB3MS + pedagogy: incidental lecture-python.myst:unemployment_linear:pandas_datareader: series: UNRATE pedagogy: incidental From 7e53d6cf34f407e110b03c3d893aa6a475529004 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 16:00:14 +1000 Subject: [PATCH 2/2] =?UTF-8?q?Drop=20the=20us=5Fadult=5Fheights=20annotat?= =?UTF-8?q?ion=20=E2=80=94=20the=20manifest=20owns=20it=20now?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This file's header states the rule: migrated datasets do not get `datasets:` entries, because the manifest is their source of truth. The other six manifest-backed datasets follow it; us_adult_heights.csv was left behind when #27 migrated it out of lecture-python-intro. Not cosmetic. `class` and `title` already came from the manifest, but `flags` and `note` are annotation-only, so the dashboard was rendering "NHANES-derived, extraction not scripted" for a dataset whose manifest records builder_status: committed with the builder in scripts/ — the exact claim #27 disproved by reproducing the file byte for byte. Strict scan stays green: the manifest already supplies class `constructed` and the real title. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/audit_annotations.yml | 5 ----- 1 file changed, 5 deletions(-) diff --git a/scripts/audit_annotations.yml b/scripts/audit_annotations.yml index dbbdbd5..d6d7f79 100644 --- a/scripts/audit_annotations.yml +++ b/scripts/audit_annotations.yml @@ -146,11 +146,6 @@ datasets: test_pwt.csv: description: Penn World Table 7.0 extract provenance: author-assembled - us_adult_heights.csv: - description: Heights of US adults, from NHANES (prob_dist motivating example) - provenance: constructed-lost - flags: [new-since-2026-07-15] - note: added by lecture-python-intro#790; NHANES-derived, extraction not scripted usa-gini-nwealth-tincome-lincome.csv: description: US Gini coefficients, wealth & income (long series) provenance: constructed-committed