Skip to content

data: re-deliver BAFU-2026 inventory with unit-keyed flow ids - #3

Merged
lbougan merged 1 commit into
mainfrom
import/bafu-2026-unit-keyed-flow-ids
Sep 14, 2026
Merged

lbougan merged 1 commit into
mainfrom
import/bafu-2026-unit-keyed-flow-ids

Conversation

@lbougan

@lbougan lbougan commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Why

On 2026-09-02 the biosphere flow id scheme in sentier-importers changed (PR #13): the unit joined the uuid5 hash, so ids are now uuid5(name|category|subcategory|unit) instead of uuid5(name|category|subcategory|""). sentier-vocab (#33) and sentier-mappings were re-minted with the new scheme, but the payload in this repo was still the 2026-08-07 generation. As a result, 0 of 2,515 distinct biosphere flow ids in the inventory matched any source code in the bafu-2026-v1__ef-3.1 bridge in sentier-mappings, so no biosphere exchange could be resolved to an EF 3.1 flow.

How it was regenerated

Regenerated from sentier-importers at f14d448 using the 22 registry sources bafu-<sector>-processes and bafu-<sector>-exchanges for the 11 sectors (01-agriculture through 10-building-services plus 99-obsolete), processes before exchanges for each sector, delivered into this checkout with:

uv run sentier-importers run <source> --deliver-local /home/laurenz/dds/sentier-inventory

No changes were made to the importer. Only the 11 exchanges.parquet files differ; every processes.parquet and metadata.json came out byte-identical to main (the source citation in each metadata description is intact).

Verification

  • Bridge match, distinct biosphere flow ids in exchanges.parquet versus source codes in sentier-mappings/data/bafu-2026-v1__ef-3.1/biosphere-*.json: before 0 / 2,515, after 2,558 / 2,679. The distinct count rose because flows that share name and category but differ in unit now receive separate ids, which is the purpose of the re-mint. The 121 remaining unmatched ids are flows the EF 3.1 bridge does not cover at all (noise, Thorium-232, gross calorific energy in biomass, ocean salt water, Uranium alpha, and similar), affecting 2.96% of biosphere exchange rows.
  • Process id stability: for all 11 sectors the set of process_id in the new processes.parquet is identical to main, and row counts and column sets of exchanges.parquet are unchanged. The only column that differs is flow; 100% of biosphere rows were re-keyed and 0 technosphere rows changed, so technosphere links are untouched.
  • scripts/validate.py (the CI schema gate) passes locally: OK, 11 sector folder(s) validate against the schema contracts.
  • File size: the largest change is +5.3 KB (07-construction/exchanges.parquet, 1,525,244 to 1,530,528 bytes); several files shrank slightly. No sharding needed.

🤖 Generated with Claude Code

@lbougan
lbougan merged commit 0ca2a80 into main Sep 14, 2026
2 checks passed
@lbougan
lbougan deleted the import/bafu-2026-unit-keyed-flow-ids branch September 14, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant