Fix stale CSV header missing 'open' column in statistics_history.csv - #377
Open
Chessing234 wants to merge 2 commits into
Open
Fix stale CSV header missing 'open' column in statistics_history.csv#377Chessing234 wants to merge 2 commits into
Chessing234 wants to merge 2 commits into
Conversation
The 'open' field was added to the row schema in commit 06fdb21 but the CSV header (first line) was never updated to include it, since the header is only written when the file is first created. Every row since 2025-12-12 has 11 values while the header only names 10 columns, so csv.DictReader (used by scripts/plot_statistics_history.py) puts the 11th value under the None restkey instead of 'open', silently discarding the recorded value on every read.
Adding `open` to the CSV header makes csv.DictReader hand back
`open: None` for the 350 rows written before that column existed, and
`int(None)` raises. The existing `row.get("open", ...)` fallback cannot
help: DictReader creates a key for every column the header declares, so
the default only fires when the header itself lacks the column.
Read the numeric cells through a helper that treats missing and blank as
the default, so both the chart generation and the last-row comparison in
update_history keep working across a column addition.
Chessing234
force-pushed
the
fix/statistics-history-csv-header
branch
from
August 13, 2026 01:12
27bdd37 to
55d4c70
Compare
Contributor
Author
|
rebased onto main, and added the script side — adding
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
openfield was added to the row schema in commit 06fdb21 (scripts/plot_statistics_history.py FIELDNAMES) but the CSV header on line 1 of data/statistics_history.csv was never updated, since the header is only written byupdate_history()when the file doesn't yet exist. Every row appended since 2025-12-12 has 11 values while the header names only 10 columns.Verified with
csv.DictReader(what the repo's own scripts use to read this file): the 11th value on each affected row lands under theNonerestkey instead ofopen, sorow.get("open", ...)ingenerate_charts()never finds the recorded value and silently falls back to a computed default. This happens to match numerically right now (default = total_problems - total_solved), which is why it wasn't noticed, but the CSV is malformed for any other consumer (pandas, spreadsheets, etc.) and the real recorded 'open' values are being discarded on every read.Fix: add
opento the header row so it matches the 11 columns actually written. Confirmed with DictReader before/after that the 'open' key now parses correctly for all post-06fdb21 rows, and that older rows (pre-'open' schema) correctly getNonefor the missing field. Ranpython scripts/validate.py— passes.