From 3c843fd74c633b5cfc5b3b7328b7ee8da0fcc9ab Mon Sep 17 00:00:00 2001 From: Laurenz Bougan Date: Mon, 12 Oct 2026 16:16:16 +0200 Subject: [PATCH] docs: sentier-inventory organised by source folder, sector as a column --- SPECS.md | 10 +- bafu/README.md | 7 +- bafu/data/fetch.md | 6 +- core/sectors/README.md | 4 +- core/sectors/communication-services.md | 10 +- core/sectors/consumer-discretionary.md | 10 +- core/sectors/consumer-staples.md | 12 +- core/sectors/crosswalk.md | 96 ++++++------ core/sectors/energy.md | 12 +- core/sectors/financials.md | 11 +- core/sectors/health-care.md | 10 +- core/sectors/industrials.md | 10 +- core/sectors/information-technology.md | 12 +- core/sectors/materials.md | 12 +- core/sectors/real-estate.md | 12 +- core/sectors/utilities.md | 11 +- ...ssess-a-building-product-in-real-estate.md | 13 +- .../assess-a-material-in-materials.md | 14 +- ...sess-a-transport-service-in-industrials.md | 12 +- ...gricultural-product-in-consumer-staples.md | 12 +- .../assess-an-electricity-mix-in-utilities.md | 12 +- .../assess-an-energy-carrier-in-energy.md | 12 +- .../use-cases/choose-a-background-database.md | 6 +- index.md | 20 +-- log.md | 1 + sentier/README.md | 4 +- sentier/commands.md | 6 +- sentier/modules/README.md | 4 +- sentier/modules/sentier-brightway/README.md | 13 +- .../api/sentier_brightway.inventory.md | 30 +++- sentier/modules/sentier-brightway/classes.md | 19 ++- sentier/modules/sentier-brightway/commands.md | 14 +- sentier/modules/sentier-brightway/datasets.md | 26 ++-- sentier/modules/sentier-brightway/layout.md | 10 +- sentier/modules/sentier-importers/README.md | 14 +- ...sentier_importers.sources.bafu.ecospold.md | 24 +-- ...orters.sources.bafu.inventory_exchanges.md | 10 +- ...orters.sources.bafu.inventory_processes.md | 14 +- sentier/modules/sentier-importers/datasets.md | 33 ++-- .../use-cases/run-an-importer-end-to-end.md | 18 ++- sentier/modules/sentier-inventory/README.md | 53 ++++--- sentier/modules/sentier-inventory/classes.md | 53 +++++-- sentier/modules/sentier-inventory/commands.md | 41 +++-- sentier/modules/sentier-inventory/datasets.md | 109 +++++++------ sentier/modules/sentier-inventory/gotchas.md | 69 ++++---- sentier/modules/sentier-inventory/layout.md | 77 +++++---- sentier/modules/sentier-inventory/status.md | 61 +++++--- .../sentier-inventory/use-cases/README.md | 13 +- .../read-a-sector-folder-with-pandas.md | 125 --------------- .../read-a-source-folder-with-pandas.md | 147 ++++++++++++++++++ .../validate-the-inventory-schema.md | 46 +++--- sources.md | 5 +- vocabulary.md | 9 +- 53 files changed, 805 insertions(+), 589 deletions(-) delete mode 100644 sentier/modules/sentier-inventory/use-cases/read-a-sector-folder-with-pandas.md create mode 100644 sentier/modules/sentier-inventory/use-cases/read-a-source-folder-with-pandas.md diff --git a/SPECS.md b/SPECS.md index 43412b6..573de93 100644 --- a/SPECS.md +++ b/SPECS.md @@ -112,7 +112,7 @@ lca-wiki/ │ ├── roadmap.md # open work, where contributors can help │ └── modules/ │ ├── sentier-brightway/ sentier-vocab/ sentier-importers/ -│ ├── sentier-inventory/ # documents BAFU-2026 as imported by Sentier (11 sector folders) and links to ../../bafu/ (D3) +│ ├── sentier-inventory/ # documents BAFU-2026 as imported by Sentier (one source folder, sector as a column) and links to ../../bafu/ (D3) │ ├── sentier-methods/ sentier-mappings/ │ └── sentier-agribalyse/ # public twin only; private adapter repositories are never documented ├── brightway/ @@ -157,7 +157,7 @@ Rules: dataset descriptors are reachable from the module page. Descriptors stay broad: never per datapoint. - `sentier/modules/sentier-inventory/datasets.md` describes BAFU-2026 as Sentier - ships it (11 sector folders, unit-keyed flow ids, parquet + YAML schema) and links + ships it (one folder per source release, sector as a column, unit-keyed flow ids, parquet + YAML schema) and links to `bafu/knowledge/` for the source database itself (D3). - `api/` files: an API reference generated by running pdoc by hand, one file per submodule, against the version recorded in `README.md`; every file stamped with @@ -216,8 +216,8 @@ Rules of precedence and marking: Sector pages use the **Global Industry Classification Standard (GICS)**: 11 sector pages; industry-group pages only when a use case needs one. `core/sectors/crosswalk.md` -maps each GICS sector to sentier-inventory folders (`01-agriculture … -10-building-services`), BAFU's own groups and ecoinvent's ISIC divisions (D11). Use +maps each GICS sector to sentier-inventory `sector` values (`agriculture` to +`building-services`), BAFU's own groups and ecoinvent's ISIC divisions (D11). Use cases and dataset descriptors reference sectors by GICS name. ## 9. Skill kit @@ -350,7 +350,7 @@ Brightway and Sentier channels. Not automated. | D8 | Codebase doc depth | Hand-written pages plus a generated pdoc markdown API reference per package | | D9 | Starter vocab labelling | Tagged EF/PEF (European Commission); ILCD bullet first where both define a term | | D10 | ecoinvent glossary | Open access per Laurenz: import with attribution and link | -| D11 | Sectors | GICS taxonomy plus `core/sectors/crosswalk.md` to sentier-inventory folders, BAFU groups, ISIC | +| D11 | Sectors | GICS taxonomy plus `core/sectors/crosswalk.md` to sentier-inventory `sector` values (folders until 2026-10-12), BAFU groups, ISIC | | D12 | Porting from the maintainers' internal wiki | Public-repo pages only, after a privacy pass | | D13 | Language | English only | | D14 | Local build folder | the maintainer's local checkout | diff --git a/bafu/README.md b/bafu/README.md index 5d75a8f..7f5be7b 100644 --- a/bafu/README.md +++ b/bafu/README.md @@ -3,7 +3,7 @@ title: bafu type: branch summary: "The BAFU LCA database as an external data source: what it is, how to get it, what it covers." audience: [P1, P2] -updated: 2026-09-21 +updated: 2026-10-12 sources: [bafu-2026-release, openlca-bafu-2026-notes, bafu-terms-of-use-2025, esu-bafu-database] related: [../sentier/modules/sentier-inventory/README.md, ../core/sectors/README.md] --- @@ -66,8 +66,9 @@ with no warranty (source: bafu-terms-of-use-2025). The full reading is in ## How the tooling in this wiki reaches it -The Sentier platform ships this database as the `bafu-2026` inventory package. The -package and the way its data is written is documented on its own module page, +The Sentier platform ships this database as the inventory source `bafu-2026`, release +`v1`, one folder `data/bafu-2026-v1/` with the sector of every process as a column. The +folder and the way its data is written is documented on its own module page, [../sentier/modules/sentier-inventory/](../sentier/modules/sentier-inventory/); the loader that installs it into a Brightway project or a folder of plain files is [../sentier/modules/sentier-brightway/](../sentier/modules/sentier-brightway/). Neither is diff --git a/bafu/data/fetch.md b/bafu/data/fetch.md index 11e733e..7bcf6c2 100644 --- a/bafu/data/fetch.md +++ b/bafu/data/fetch.md @@ -3,7 +3,7 @@ title: Fetching a BAFU release type: database summary: Where the files are, why the download cannot be fully automated, and how to verify and unpack them. audience: [P1, P3] -updated: 2026-09-21 +updated: 2026-10-12 sources: [bafu-2026-release, openlca-bafu-2026-notes, bafu-terms-of-use-2025] related: [../knowledge/access-and-licence.md, ../knowledge/format.md] --- @@ -83,8 +83,8 @@ hundreds of megabytes. Keep the downloads in a directory of your own, keep the - openLCA: import the `.zolca` file. - Brightway and anything else in this wiki's tooling: the Sentier platform ships this - database as the `bafu-2026` inventory package, and that route is documented on its own - module page, [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/), + database as the inventory source `bafu-2026`, one folder `data/bafu-2026-v1/` with the + sector as a column, and that route is documented on its own module page, [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/), with the loader at [../../sentier/modules/sentier-brightway/](../../sentier/modules/sentier-brightway/). - Your own importer: write it against the ecoSpold v1 archive, and read diff --git a/core/sectors/README.md b/core/sectors/README.md index e6005f3..1480204 100644 --- a/core/sectors/README.md +++ b/core/sectors/README.md @@ -3,7 +3,7 @@ title: core/sectors type: readme summary: The 11 GICS sector pages and the crosswalk to the other sector taxonomies in use. audience: [P1, P2] -updated: 2026-09-21 +updated: 2026-10-12 sources: [] --- @@ -18,7 +18,7 @@ in this wiki serve it. Eleven pages, one per sector of the Global Industry Classification Standard (GICS): energy, materials, industrials, consumer discretionary, consumer staples, health care, financials, information technology, communication services, utilities, real estate. -Plus `crosswalk.md`, which maps each GICS sector onto the sector folders of the Sentier +Plus `crosswalk.md`, which maps each GICS sector onto the sector values of the Sentier inventory, the BAFU groups and the ISIC divisions ecoinvent uses. Industry-group pages are added only when a use case needs one. diff --git a/core/sectors/communication-services.md b/core/sectors/communication-services.md index 3ecb39b..46fe7e0 100644 --- a/core/sectors/communication-services.md +++ b/core/sectors/communication-services.md @@ -3,8 +3,8 @@ title: Communication Services type: sector summary: "Telecommunication networks, media and interactive services: what an LCA of the GICS Communication Services sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -40,8 +40,8 @@ Foreground data is a network topology with per-element power draw and a traffic Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `02-electricity` for the operating phase, `09-electronics` for network and terminal - equipment, `07-construction` for masts, ducts and buildings + the `sector` values `electricity` for the operating phase, `electronics` for network + and terminal equipment, `construction` for masts, ducts and buildings (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `computers & network` and `electricity` (bafu-2026-release). @@ -84,7 +84,7 @@ and read this page for what the sector adds to them. The gap is on | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `02-electricity`, `09-electronics`, `07-construction` | +| Sentier inventory `sector` values | `electricity`, `electronics`, `construction` | | BAFU:2026 top-level process categories | computers & network, electricity | | ISIC Rev.4 sections | J (information and communication), R (arts, entertainment and recreation) | diff --git a/core/sectors/consumer-discretionary.md b/core/sectors/consumer-discretionary.md index 1aa2d09..66ee3b3 100644 --- a/core/sectors/consumer-discretionary.md +++ b/core/sectors/consumer-discretionary.md @@ -3,8 +3,8 @@ title: Consumer Discretionary type: sector summary: "Vehicles, durables, apparel and consumer services: what an LCA of the GICS Consumer Discretionary sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -45,8 +45,8 @@ Foreground data is a bill of materials, a manufacturing recipe and a use-phase behaviour model. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `08-materials` and `09-electronics` for product content, `04-transport` for the - vehicle and distribution side, `06-waste` for end of life + the `sector` values `materials` and `electronics` for product content, `transport` + for the vehicle and distribution side, `waste` for end of life (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `textiles`, `private consumption`, `electronics`, `flooring` and `transport systems` @@ -89,7 +89,7 @@ and read this page for what the sector adds to them. The gap is on | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `08-materials`, `09-electronics`, `04-transport`, `06-waste` | +| Sentier inventory `sector` values | `materials`, `electronics`, `transport`, `waste` | | BAFU:2026 top-level process categories | textiles, private consumption, electronics, flooring, transport systems | | ISIC Rev.4 sections | C (manufacturing), G (wholesale and retail trade), I (accommodation and food service), P (education), S (other service activities) | diff --git a/core/sectors/consumer-staples.md b/core/sectors/consumer-staples.md index 83f98ce..c21f90d 100644 --- a/core/sectors/consumer-staples.md +++ b/core/sectors/consumer-staples.md @@ -3,8 +3,8 @@ title: Consumer Staples type: sector summary: "Food, beverages, tobacco and household products: what an LCA of the GICS Consumer Staples sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -40,9 +40,9 @@ Foreground data is a farm or a factory inventory: yields, inputs, energy, and th emission model behind the field emissions. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - the `01-agriculture` folder, which holds the highest-precedence rank in the - inventory, with `03-chemicals` for fertiliser and crop protection and `06-waste` for - residues and packaging waste (sentier-inventory-readme). + the `sector` value `agriculture`, the only sector that serves one GICS sector alone, + with `chemicals` for fertiliser and crop protection and `waste` for residues and + packaging waste (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `agricultural`, `food industry` and `landfarming` (bafu-2026-release). - [../../ecoinvent/](../../ecoinvent/) - structure only, no amounts. @@ -85,7 +85,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `01-agriculture`, `03-chemicals`, `06-waste` | +| Sentier inventory `sector` values | `agriculture`, `chemicals`, `waste` | | BAFU:2026 top-level process categories | agricultural, food industry, landfarming | | ISIC Rev.4 sections | A (agriculture, forestry and fishing), C (manufacturing, food and beverages), G (wholesale and retail trade) | diff --git a/core/sectors/crosswalk.md b/core/sectors/crosswalk.md index 25fa98d..cbce24f 100644 --- a/core/sectors/crosswalk.md +++ b/core/sectors/crosswalk.md @@ -1,10 +1,10 @@ --- title: Sector crosswalk type: sector -summary: "Maps each GICS sector onto the Sentier inventory sector folders, the BAFU:2026 process categories and the ISIC Rev.4 sections." +summary: "Maps each GICS sector onto the Sentier inventory sector values, the BAFU:2026 process categories and the ISIC Rev.4 sections." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [energy.md, materials.md, industrials.md, consumer-discretionary.md, consumer-staples.md, health-care.md, financials.md, information-technology.md, communication-services.md, utilities.md, real-estate.md] --- @@ -16,7 +16,7 @@ how they line up, and where they do not. | Taxonomy | What it classifies | Levels | Source | |---|---|---|---| | GICS | companies, by the business they earn from | 11 sectors, 25 industry groups, 74 industries, 163 sub-industries | gics-2023 | -| Sentier inventory folders | inventory rows, by sector of the process | 11 folders, ranked by a numeric prefix | sentier-inventory-readme | +| Sentier inventory sectors | inventory rows, by sector of the process | 11 values of the `sector` column, in one source folder today | sentier-inventory-readme | | BAFU:2026 process categories | processes in the openLCA package, by subject | 59 top-level categories in the v1.1 package, nested further; see the note below on the count | bafu-2026-release | | ISIC Rev.4 | economic activities, by kind of activity | 21 sections A to U, then divisions | unsd-isic-rev4 | @@ -32,11 +32,15 @@ digit code. The structure in this wiki is the March 2023 one, after the review t carved Real Estate out of Financials and turned Telecommunication Services into Communication Services (gics-2023). -**Sentier inventory folders.** `data/-/`, one folder per sector, each with -`processes.parquet`, `exchanges.parquet` and `metadata.json`. The `NN` prefix sets -resolution precedence: the lower rank wins when records overlap. Inventory is organised -by sector and not by data source, and background databases that exist only as link -targets are not folders (sentier-inventory-readme). +**Sentier inventory sectors.** The `sector` column of `processes.parquet`, eleven +enumerated values, inside one folder per published release of one datasource, +`data/-/` (today only `data/bafu-2026-v1/`), each folder holding +`processes.parquet`, `exchanges.parquet` and `metadata.json`. Inventory is organised by +source with the sector as a row attribute; there is no rank and no precedence between +folders, and background databases that exist only as link targets are not folders. Until +2026-10-12 the sector was a numbered folder, `data/-/`, whose rank prefix set +precedence; that layout was dropped because sector files from several sources did not +stack (sentier-inventory-readme, sentier-inventory-pr-5). **BAFU:2026 process categories.** The openLCA package of the Swiss Federal Administration LCI database carries a category tree; its process branch has 59 top-level @@ -68,19 +72,19 @@ with environmentally extended input-output work. Read a row as "a study in this GICS sector usually draws on these". A cell saying "none" means no entry of that taxonomy corresponds, not that the sector is unimportant. -| GICS sector | Sentier inventory folders | BAFU:2026 top-level process categories | ISIC Rev.4 sections | +| GICS sector | Sentier inventory `sector` values | BAFU:2026 top-level process categories | ISIC Rev.4 sections | |---|---|---|---| -| [Energy](energy.md) | `05-energy`, `03-chemicals`, `04-transport` | `fuels`, `oil`, `natural gas`, `pipeline`, `biomass`, `underground deposit` | B, C, D, H | -| [Materials](materials.md) | `08-materials`, `03-chemicals`, `07-construction` | `metals`, `minerals`, `chemicals`, `plastics`, `glass`, `ceramics`, `wood`, `paper+ board`, `cardboard`, `construction materials`, `insulation materials` | B, C | -| [Industrials](industrials.md) | `04-transport`, `07-construction`, `08-materials`, `09-electronics`, `10-building-services` | `transport systems`, `mechanical`, `building processes`, `construction processes`, `compressed air`, `recycling` | C, E, F, H, N | -| [Consumer Discretionary](consumer-discretionary.md) | `08-materials`, `09-electronics`, `04-transport`, `06-waste` | `textiles`, `private consumption`, `electronics`, `flooring`, `transport systems` | C, G, I, P, S | -| [Consumer Staples](consumer-staples.md) | `01-agriculture`, `03-chemicals`, `06-waste` | `agricultural`, `food industry`, `landfarming` | A, C, G | -| [Health Care](health-care.md) | `03-chemicals`, `09-electronics`, `06-waste` | `chemicals`, `incineration` | C, Q | +| [Energy](energy.md) | `energy`, `chemicals`, `transport` | `fuels`, `oil`, `natural gas`, `pipeline`, `biomass`, `underground deposit` | B, C, D, H | +| [Materials](materials.md) | `materials`, `chemicals`, `construction` | `metals`, `minerals`, `chemicals`, `plastics`, `glass`, `ceramics`, `wood`, `paper+ board`, `cardboard`, `construction materials`, `insulation materials` | B, C | +| [Industrials](industrials.md) | `transport`, `construction`, `materials`, `electronics`, `building-services` | `transport systems`, `mechanical`, `building processes`, `construction processes`, `compressed air`, `recycling` | C, E, F, H, N | +| [Consumer Discretionary](consumer-discretionary.md) | `materials`, `electronics`, `transport`, `waste` | `textiles`, `private consumption`, `electronics`, `flooring`, `transport systems` | C, G, I, P, S | +| [Consumer Staples](consumer-staples.md) | `agriculture`, `chemicals`, `waste` | `agricultural`, `food industry`, `landfarming` | A, C, G | +| [Health Care](health-care.md) | `chemicals`, `electronics`, `waste` | `chemicals`, `incineration` | C, Q | | [Financials](financials.md) | none | none | K | -| [Information Technology](information-technology.md) | `09-electronics`, `02-electricity`, `03-chemicals`, `06-waste` | `electronics`, `computers & network`, `electronics waste` | C, J | -| [Communication Services](communication-services.md) | `02-electricity`, `09-electronics`, `07-construction` | `computers & network`, `electricity` | J, R | -| [Utilities](utilities.md) | `02-electricity`, `05-energy`, `06-waste` | `electricity`, `electricity by fuel`, `power plants`, `photovoltaic`, `wind power`, `impoundment`, `heat`, `heating`, `heat pumps`, `water`, `wastewater treatment`, `nuclear waste`, `incineration`, `landfill`, `waste`, `waste management` | D, E | -| [Real Estate](real-estate.md) | `07-construction`, `10-building-services`, `08-materials`, `02-electricity` | `building components`, `building processes`, `construction`, `construction materials`, `construction processes`, `construction waste`, `flooring`, `insulation materials`, `ventilation` | F, L | +| [Information Technology](information-technology.md) | `electronics`, `electricity`, `chemicals`, `waste` | `electronics`, `computers & network`, `electronics waste` | C, J | +| [Communication Services](communication-services.md) | `electricity`, `electronics`, `construction` | `computers & network`, `electricity` | J, R | +| [Utilities](utilities.md) | `electricity`, `energy`, `waste` | `electricity`, `electricity by fuel`, `power plants`, `photovoltaic`, `wind power`, `impoundment`, `heat`, `heating`, `heat pumps`, `water`, `wastewater treatment`, `nuclear waste`, `incineration`, `landfill`, `waste`, `waste management` | D, E | +| [Real Estate](real-estate.md) | `construction`, `building-services`, `materials`, `electricity` | `building components`, `building processes`, `construction`, `construction materials`, `construction processes`, `construction waste`, `flooring`, `insulation materials`, `ventilation` | F, L | The ISIC section letters stand for: A agriculture, forestry and fishing; B mining and quarrying; C manufacturing; D electricity, gas, steam and air conditioning supply; @@ -92,27 +96,27 @@ activities; N administrative and support service activities; P education; Q huma and social work activities; R arts, entertainment and recreation; S other service activities (unsd-isic-rev4). -## Sentier inventory folders to GICS +## Sentier inventory sectors to GICS The reverse direction, which is what a reader holding a parquet file needs. -| Folder | GICS sectors it mainly serves | +| `sector` value | GICS sectors it mainly serves | |---|---| -| `01-agriculture` | Consumer Staples | -| `02-electricity` | Utilities, Information Technology, Communication Services, Real Estate | -| `03-chemicals` | Materials, Consumer Staples, Health Care, Energy, Information Technology | -| `04-transport` | Industrials, Consumer Discretionary, Energy | -| `05-energy` | Energy, Utilities | -| `06-waste` | Utilities, Consumer Discretionary, Health Care, Information Technology | -| `07-construction` | Real Estate, Industrials, Materials, Communication Services | -| `08-materials` | Materials, Consumer Discretionary, Industrials, Real Estate | -| `09-electronics` | Information Technology, Consumer Discretionary, Communication Services, Health Care | -| `10-building-services` | Real Estate, Industrials | -| `99-obsolete` | none; it holds legacy link targets rather than a sector (sentier-inventory-readme) | - -No folder corresponds to Financials, and no folder is exclusive to one GICS sector -except `01-agriculture`. Where two folders could supply the same record, the lower rank -prefix wins (sentier-inventory-readme). +| `agriculture` | Consumer Staples | +| `electricity` | Utilities, Information Technology, Communication Services, Real Estate | +| `chemicals` | Materials, Consumer Staples, Health Care, Energy, Information Technology | +| `transport` | Industrials, Consumer Discretionary, Energy | +| `energy` | Energy, Utilities | +| `waste` | Utilities, Consumer Discretionary, Health Care, Information Technology | +| `construction` | Real Estate, Industrials, Materials, Communication Services | +| `materials` | Materials, Consumer Discretionary, Industrials, Real Estate | +| `electronics` | Information Technology, Consumer Discretionary, Communication Services, Health Care | +| `building-services` | Real Estate, Industrials | +| `obsolete` | none; it marks legacy link targets rather than a sector (sentier-inventory-readme) | + +No sector value corresponds to Financials, and none is exclusive to one GICS sector +except `agriculture`. A sector is a filter on the column, not a folder, so there is no +precedence to resolve between sectors (sentier-inventory-readme). ## BAFU:2026 top-level process categories to GICS @@ -177,10 +181,10 @@ the first-choice sector page and the sector pages repeat the full lists. | `wind power` | Utilities | | `wood` | Materials | | `Others` | none; a residual bucket | -| `energy, obsolete` | none; legacy, maps to `99-obsolete` | -| `material, obsolete` | none; legacy, maps to `99-obsolete` | -| `processing, obsolete` | none; legacy, maps to `99-obsolete` | -| `waste treatment, obsolete` | none; legacy, maps to `99-obsolete` | +| `energy, obsolete` | none; legacy, maps to `obsolete` | +| `material, obsolete` | none; legacy, maps to `obsolete` | +| `processing, obsolete` | none; legacy, maps to `obsolete` | +| `waste treatment, obsolete` | none; legacy, maps to `obsolete` | ## What does not map @@ -188,10 +192,10 @@ the first-choice sector page and the sector pages repeat the full lists. impact is a financed portfolio, which is an attribution of other sectors rather than an inventory; see [financials.md](financials.md). - **Commercial and professional services** inside Industrials, and **media** inside - Communication Services, have activities in ISIC but no dedicated inventory folder or + Communication Services, have activities in ISIC but no dedicated inventory sector or BAFU category. -- **`99-obsolete`** and the four BAFU categories ending in `obsolete` are version - artefacts, not sectors. They exist so that older references still resolve. `Others` is +- **The `obsolete` sector value** and the four BAFU categories ending in `obsolete` are + version artefacts, not sectors. They exist so that older references still resolve. `Others` is a residual bucket rather than a legacy one. - **ISIC sections M, O, T and U** (professional and scientific activities, public administration, household activities, extraterritorial bodies) have no GICS sector at @@ -200,7 +204,7 @@ the first-choice sector page and the sector pages repeat the full lists. ## How to extend this page Add a sector page and its row here in the same change, as -[README.md](README.md) requires. When a new Sentier inventory folder appears, add it to -the reverse table and to every GICS row it serves; when a BAFU release changes its +[README.md](README.md) requires. When a new Sentier inventory `sector` value appears in +the schema enumeration, add it to the reverse table and to every GICS row it serves; when a BAFU release changes its category tree, re-read the package and update the third table with a new access date on the source row in [../../sources.md](../../sources.md). Keep "none" where it is true. diff --git a/core/sectors/energy.md b/core/sectors/energy.md index e0ee421..daf046f 100644 --- a/core/sectors/energy.md +++ b/core/sectors/energy.md @@ -3,8 +3,8 @@ title: Energy type: sector summary: "Fuel chains from extraction to combustion: what an LCA of the GICS Energy sector needs and which of this wiki's data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -44,9 +44,9 @@ conversion step plus a measured or reported emission profile; the rest is backgr This wiki covers the background in three places. - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - the `05-energy` folder holds the energy carriers, the largest process count of any - sector folder, with `03-chemicals` and `04-transport` supplying the process inputs - and the delivery steps (sentier-inventory-readme). + the `sector` value `energy` holds the energy carriers, the largest process count of + any sector, with `chemicals` and `transport` supplying the process inputs and the + delivery steps (sentier-inventory-readme). - [../../bafu/](../../bafu/) - the BAFU:2026 database, whose top-level process categories include `fuels`, `oil`, `natural gas`, `pipeline`, `biomass` and `underground deposit` (bafu-2026-release). @@ -96,7 +96,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `05-energy`, `03-chemicals`, `04-transport` | +| Sentier inventory `sector` values | `energy`, `chemicals`, `transport` | | BAFU:2026 top-level process categories | fuels, oil, natural gas, pipeline, biomass, underground deposit | | ISIC Rev.4 sections | B (mining and quarrying), C (manufacturing, refined petroleum), D (electricity, gas, steam), H (transportation and storage) | diff --git a/core/sectors/financials.md b/core/sectors/financials.md index 8e8469e..22e9b7a 100644 --- a/core/sectors/financials.md +++ b/core/sectors/financials.md @@ -3,8 +3,8 @@ title: Financials type: sector summary: "Banks, financial services and insurance: why a product LCA rarely fits the GICS Financials sector, and what this wiki does and does not cover." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -45,9 +45,8 @@ sector pages: office and data-centre energy, IT equipment, business travel, pape building services. - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - no sector folder corresponds to Financials; the folders are - `01-agriculture` through `10-building-services` plus `99-obsolete`, all of them - physical (sentier-inventory-readme). + no `sector` value corresponds to Financials; the values are `agriculture` through + `building-services` plus `obsolete`, all of them physical (sentier-inventory-readme). - [../../bafu/](../../bafu/) - no top-level process category corresponds to financial activity (bafu-2026-release). - [../../ecoinvent/](../../ecoinvent/) - structure only, no amounts. @@ -87,7 +86,7 @@ and read this page for what the sector adds to them. The gap is on | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | none | +| Sentier inventory `sector` values | none | | BAFU:2026 top-level process categories | none | | ISIC Rev.4 sections | K (financial and insurance activities) | diff --git a/core/sectors/health-care.md b/core/sectors/health-care.md index c99ee53..4306071 100644 --- a/core/sectors/health-care.md +++ b/core/sectors/health-care.md @@ -3,8 +3,8 @@ title: Health Care type: sector summary: "Pharmaceuticals, devices and care delivery: what an LCA of the GICS Health Care sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -39,8 +39,8 @@ Foreground data is a synthesis route or a device bill of materials, plus a facil energy model. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `03-chemicals` for synthesis inputs, `09-electronics` for instrumented devices, - `06-waste` for clinical waste routes (sentier-inventory-readme). + the `sector` values `chemicals` for synthesis inputs, `electronics` for instrumented + devices, `waste` for clinical waste routes (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `chemicals` and `incineration` (bafu-2026-release). - [../../ecoinvent/](../../ecoinvent/) - structure only, no amounts. @@ -84,7 +84,7 @@ and read this page for what the sector adds to them. The gap is on | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `03-chemicals`, `09-electronics`, `06-waste` | +| Sentier inventory `sector` values | `chemicals`, `electronics`, `waste` | | BAFU:2026 top-level process categories | chemicals, incineration | | ISIC Rev.4 sections | C (manufacturing, pharmaceuticals and medical devices), Q (human health and social work activities) | diff --git a/core/sectors/industrials.md b/core/sectors/industrials.md index 7b6079c..f5ad505 100644 --- a/core/sectors/industrials.md +++ b/core/sectors/industrials.md @@ -3,8 +3,8 @@ title: Industrials type: sector summary: "Capital goods, commercial services and transport: what an LCA of the GICS Industrials sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -46,8 +46,8 @@ Foreground data is a bill of materials plus a use-phase energy model for capital and a fleet and operations model for transport. The background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `04-transport` for transport processes, `08-materials` and `09-electronics` for the - bill of materials, `07-construction` and `10-building-services` for infrastructure + the `sector` values `transport` for transport processes, `materials` and `electronics` + for the bill of materials, `construction` and `building-services` for infrastructure and installed equipment (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `transport systems`, `mechanical`, `building processes`, `construction processes`, `compressed air` and @@ -94,7 +94,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `04-transport`, `07-construction`, `08-materials`, `09-electronics`, `10-building-services` | +| Sentier inventory `sector` values | `transport`, `construction`, `materials`, `electronics`, `building-services` | | BAFU:2026 top-level process categories | transport systems, mechanical, building processes, construction processes, compressed air, recycling | | ISIC Rev.4 sections | C (manufacturing), E (water supply, sewerage, waste management), F (construction), H (transportation and storage), N (administrative and support services) | diff --git a/core/sectors/information-technology.md b/core/sectors/information-technology.md index 7649139..2133da3 100644 --- a/core/sectors/information-technology.md +++ b/core/sectors/information-technology.md @@ -3,8 +3,8 @@ title: Information Technology type: sector summary: "Semiconductors, hardware and software services: what an LCA of the GICS Information Technology sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -42,9 +42,9 @@ Foreground data is a bill of materials with die area and process node for hardwa a utilisation and allocation model for services. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `09-electronics`, which is the smallest sector folder by process count, with - `02-electricity` for the operating phase, `03-chemicals` for process gases and - `06-waste` for end of life (sentier-inventory-readme). + the `sector` value `electronics`, one of the smallest sectors by process count, with + `electricity` for the operating phase, `chemicals` for process gases and `waste` for + end of life (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `electronics`, `computers & network` and `electronics waste` (bafu-2026-release). - [../../ecoinvent/](../../ecoinvent/) - structure only, no amounts. @@ -86,7 +86,7 @@ and read this page for what the sector adds to them. The gap is on | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `09-electronics`, `02-electricity`, `03-chemicals`, `06-waste` | +| Sentier inventory `sector` values | `electronics`, `electricity`, `chemicals`, `waste` | | BAFU:2026 top-level process categories | electronics, computers & network, electronics waste | | ISIC Rev.4 sections | C (manufacturing, computer and electronic products), J (information and communication) | diff --git a/core/sectors/materials.md b/core/sectors/materials.md index 6cbb765..cee8933 100644 --- a/core/sectors/materials.md +++ b/core/sectors/materials.md @@ -3,8 +3,8 @@ title: Materials type: sector summary: "Bulk materials from ore and feedstock to the factory gate: what an LCA of the GICS Materials sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -40,9 +40,9 @@ Foreground data is the process recipe: inputs per unit of output, energy carrier direct emissions and the fate of by-products. Background data comes from: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - the `08-materials` folder, with `03-chemicals` for feedstocks and `07-construction` - for the construction-bound materials; sector folders carry a rank prefix and the - lower rank wins when records overlap (sentier-inventory-readme). + the `sector` value `materials`, with `chemicals` for feedstocks and `construction` for + the construction-bound materials; the sector is a column of the process table, so a + sector is a filter on one source folder (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `metals`, `minerals`, `chemicals`, `plastics`, `glass`, `ceramics`, `wood`, `paper+ board`, `cardboard`, `construction materials` and `insulation materials` (bafu-2026-release). @@ -88,7 +88,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `08-materials`, `03-chemicals`, `07-construction` | +| Sentier inventory `sector` values | `materials`, `chemicals`, `construction` | | BAFU:2026 top-level process categories | metals, minerals, chemicals, plastics, glass, ceramics, wood, paper+ board, cardboard, construction materials, insulation materials | | ISIC Rev.4 sections | B (mining and quarrying), C (manufacturing) | diff --git a/core/sectors/real-estate.md b/core/sectors/real-estate.md index 2bcd722..cb832b0 100644 --- a/core/sectors/real-estate.md +++ b/core/sectors/real-estate.md @@ -3,8 +3,8 @@ title: Real Estate type: sector summary: "Buildings and property over a reference study period: what an LCA of the GICS Real Estate sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -42,9 +42,9 @@ Foreground data is a quantity take-off of the building plus an energy model of i operation. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `07-construction`, which carries the largest exchange count of any sector folder, - with `10-building-services` for installed technical equipment, `08-materials` for - product content and `02-electricity` for operation (sentier-inventory-readme). + the `sector` value `construction`, which carries the largest exchange count of any + sector, with `building-services` for installed technical equipment, `materials` for + product content and `electricity` for operation (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `building components`, `building processes`, `construction`, `construction materials`, `construction processes`, `construction waste`, `flooring`, `insulation materials` @@ -92,7 +92,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `07-construction`, `10-building-services`, `08-materials`, `02-electricity` | +| Sentier inventory `sector` values | `construction`, `building-services`, `materials`, `electricity` | | BAFU:2026 top-level process categories | building components, building processes, construction, construction materials, construction processes, construction waste, flooring, insulation materials, ventilation | | ISIC Rev.4 sections | F (construction), L (real estate activities) | diff --git a/core/sectors/utilities.md b/core/sectors/utilities.md index 7835789..874ff01 100644 --- a/core/sectors/utilities.md +++ b/core/sectors/utilities.md @@ -3,8 +3,8 @@ title: Utilities type: sector summary: "Electricity, gas, water and heat supply: what an LCA of the GICS Utilities sector needs and which data sources cover it." audience: [P1, P2] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, ef-pef-method, sentier-inventory-readme, bafu-2026-release, unsd-isic-rev4, sentier-inventory-pr-5] related: [crosswalk.md, ../../bafu/, ../../sentier/modules/sentier-inventory/] --- @@ -38,8 +38,9 @@ Foreground data is a plant or a network model: generation by technology, losses, capacity factors, and the treatment train for water. Background: - [../../sentier/modules/sentier-inventory/](../../sentier/modules/sentier-inventory/) - - `02-electricity`, one of the largest sector folders and the Brightcon deliverable, - with `05-energy` for the fuels behind it and `06-waste` for wastewater and residues + the `sector` value `electricity`, one of the largest sectors and the Brightcon + deliverable, with `energy` for the fuels behind it and `waste` for wastewater and + residues (sentier-inventory-readme). - [../../bafu/](../../bafu/) - BAFU:2026 top-level categories `electricity`, `electricity by fuel`, `power plants`, `photovoltaic`, `wind power`, `impoundment`, @@ -87,7 +88,7 @@ data usually lacks. | Taxonomy | This sector maps onto | |---|---| -| Sentier inventory folders | `02-electricity`, `05-energy`, `06-waste` | +| Sentier inventory `sector` values | `electricity`, `energy`, `waste` | | BAFU:2026 top-level process categories | electricity, electricity by fuel, power plants, photovoltaic, wind power, impoundment, heat, heating, heat pumps, water, wastewater treatment, nuclear waste, incineration, landfill, waste, waste management | | ISIC Rev.4 sections | D (electricity, gas, steam and air conditioning supply), E (water supply; sewerage, waste management and remediation) | diff --git a/core/use-cases/assess-a-building-product-in-real-estate.md b/core/use-cases/assess-a-building-product-in-real-estate.md index 8d5e4e2..e14da7c 100644 --- a/core/use-cases/assess-a-building-product-in-real-estate.md +++ b/core/use-cases/assess-a-building-product-in-real-estate.md @@ -3,8 +3,8 @@ title: Assess a building product in Real Estate type: use-case summary: "Take a construction product from bill of materials to a declaration and then into a building study: modules A1 to D, the take-off, and the operational energy that competes with it." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, en-15804-2019, epd-pcr-2019-14, epd-international-pcr-library, ef-pef-method, bafu-2026-release, sentier-inventory-readme, ilcd-2010, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, en-15804-2019, epd-pcr-2019-14, epd-international-pcr-library, ef-pef-method, bafu-2026-release, sentier-inventory-readme, ilcd-2010, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/real-estate.md, ../standards/en-15804.md, prepare-an-epd.md, assess-a-material-in-materials.md, ../concepts/functional-unit.md] --- @@ -40,10 +40,9 @@ are the two assumptions that decide comparability (epd-international-pcr-library). No PEFCR in the Commission's current list covers buildings; the closest listed rule is Synthetic Turf, a landscaping product rather than a building one (ef-pef-method). -- **Tools and background data.** Sentier inventory folders `07-construction`, which - carries the largest exchange count of any sector folder, `10-building-services` for - installed technical equipment, `08-materials` for product content and `02-electricity` - for operation (sentier-inventory-readme); BAFU:2026 top-level categories `building +- **Tools and background data.** Sentier inventory `sector` values `construction`, which + carries the largest exchange count of any sector, `building-services` for installed + technical equipment, `materials` for product content and `electricity` for operation (sentier-inventory-readme); BAFU:2026 top-level categories `building components`, `building processes`, `construction`, `construction materials`, `construction processes`, `construction waste`, `flooring`, `insulation materials` and `ventilation`, whose new and updated construction datasets, including construction @@ -92,7 +91,7 @@ declarations, because the take-off mixes product-specific and generic data [../../sentier/modules/sentier-brightway/use-cases/](../../sentier/modules/sentier-brightway/use-cases/) and [../../brightway/modules/bw2data/use-cases/](../../brightway/modules/bw2data/use-cases/); - the inventory folders themselves are documented in + the inventory sectors themselves are documented in [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/) and the BAFU release in [../../bafu/use-cases/](../../bafu/use-cases/). diff --git a/core/use-cases/assess-a-material-in-materials.md b/core/use-cases/assess-a-material-in-materials.md index 0a61a9d..381eb04 100644 --- a/core/use-cases/assess-a-material-in-materials.md +++ b/core/use-cases/assess-a-material-in-materials.md @@ -3,8 +3,8 @@ title: Assess a material in Materials type: use-case summary: "Produce a cradle-to-gate result for a bulk material: the electricity mix that decides the ranking, the scrap and recycled-content rules, and the declaration that follows." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, epd-international-pcr-library, epd-pcr-2019-14, en-15804-2019, ef-pef-method, sentier-inventory-readme, bafu-2026-release, ecoinvent-kb-system-models, ilcd-2010, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, epd-international-pcr-library, epd-pcr-2019-14, en-15804-2019, ef-pef-method, sentier-inventory-readme, bafu-2026-release, ecoinvent-kb-system-models, ilcd-2010, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/materials.md, ../concepts/allocation-and-multifunctionality.md, prepare-an-epd.md, assess-an-electricity-mix-in-utilities.md, choose-a-background-database.md] --- @@ -40,10 +40,10 @@ those is an allocation decision that changes the number additionally sit under the EN 15804 core rules (en-15804-2019; [../standards/en-15804.md](../standards/en-15804.md)). No PEFCR in the Commission's current list covers bulk materials (ef-pef-method). -- **Tools and background data.** Sentier inventory folder `08-materials`, with - `03-chemicals` for feedstocks and `07-construction` for the construction-bound - materials; sector folders carry a rank prefix and the lower rank wins when records - overlap (sentier-inventory-readme). BAFU:2026 top-level categories `metals`, `minerals`, +- **Tools and background data.** Sentier inventory `sector` value `materials`, with + `chemicals` for feedstocks and `construction` for the construction-bound materials; + the sector is a column of the process table, filtered out of one source folder + (sentier-inventory-readme). BAFU:2026 top-level categories `metals`, `minerals`, `chemicals`, `plastics`, `glass`, `ceramics`, `wood`, `paper+ board`, `cardboard`, `construction materials` and `insulation materials` (bafu-2026-release). @@ -113,7 +113,7 @@ twice (lca-wiki-editorial). 8. **Install the background and calculate.** Loader: [../../sentier/modules/sentier-brightway/use-cases/](../../sentier/modules/sentier-brightway/use-cases/); - the `08-materials` folder and the rank rule in + the `materials` sector and the source folder it sits in, in [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/); BAFU release in [../../bafu/use-cases/](../../bafu/use-cases/); a licensed ecoinvent release through diff --git a/core/use-cases/assess-a-transport-service-in-industrials.md b/core/use-cases/assess-a-transport-service-in-industrials.md index b1a0ba2..453c978 100644 --- a/core/use-cases/assess-a-transport-service-in-industrials.md +++ b/core/use-cases/assess-a-transport-service-in-industrials.md @@ -3,8 +3,8 @@ title: Assess a transport service in Industrials type: use-case summary: "Model a tonne-kilometre or a passenger-kilometre: load factor, empty running, the split between vehicle and infrastructure, and the allocation on a mixed service." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, epd-international-pcr-library, ef-pef-method, ilcd-2010, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, epd-international-pcr-library, ef-pef-method, ilcd-2010, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/industrials.md, ../concepts/functional-unit.md, ../concepts/allocation-and-multifunctionality.md, assess-an-energy-carrier-in-energy.md, assess-an-electricity-mix-in-utilities.md] --- @@ -41,9 +41,9 @@ output it delivers before it is scrapped `Infrastructure & buildings` and `Services` (epd-international-pcr-library). Of the PEFCRs the Commission lists, Aviation, Drones & eVTOLs is in development and is the only one touching this sector (ef-pef-method). -- **Tools and background data.** Sentier inventory folder `04-transport` for transport - processes, with `08-materials` and `09-electronics` for the bill of materials and - `07-construction` and `10-building-services` for infrastructure and installed equipment +- **Tools and background data.** Sentier inventory `sector` value `transport` for + transport processes, with `materials` and `electronics` for the bill of materials and + `construction` and `building-services` for infrastructure and installed equipment (sentier-inventory-readme); BAFU:2026 top-level categories `transport systems`, `mechanical`, `building processes`, `construction processes`, `compressed air` and `recycling` (bafu-2026-release). @@ -104,7 +104,7 @@ has to take it on trust. 6. **Amortise the vehicle over its real service life and utilisation.** Manufacturers publish design lives, not observed ones, and the observed figure is what the study needs ([../sectors/industrials.md](../sectors/industrials.md)). The bill of materials - comes from `08-materials` and `09-electronics` + comes from the `materials` and `electronics` sectors (sentier-inventory-readme; [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/)). diff --git a/core/use-cases/assess-an-agricultural-product-in-consumer-staples.md b/core/use-cases/assess-an-agricultural-product-in-consumer-staples.md index 10ff90b..a8b3802 100644 --- a/core/use-cases/assess-an-agricultural-product-in-consumer-staples.md +++ b/core/use-cases/assess-an-agricultural-product-in-consumer-staples.md @@ -3,8 +3,8 @@ title: Assess an agricultural product in Consumer Staples type: use-case summary: "Model a food or beverage product where primary production decides the answer: field emissions, the crop and residue allocation, the milk and meat split, and the PEFCR that governs it." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, ef-pef-method, eu-recommendation-2021-2279, epd-international-pcr-library, sentier-inventory-readme, bafu-2026-release, ilcd-2010, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, ef-pef-method, eu-recommendation-2021-2279, epd-international-pcr-library, sentier-inventory-readme, bafu-2026-release, ilcd-2010, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/consumer-staples.md, ../concepts/allocation-and-multifunctionality.md, run-a-pef-study.md, ../concepts/functional-unit.md, document-data-quality-and-uncertainty.md] --- @@ -39,9 +39,9 @@ decision, which is precisely why the sector has category rules Food and Dairy Products, with Marine Fish in development (ef-pef-method). If a PEFCR exists, it should be used (eu-recommendation-2021-2279, section 2). The International EPD System's PCR library carries `Food & beverages` (epd-international-pcr-library). -- **Tools and background data.** Sentier inventory folder `01-agriculture`, which holds - the highest-precedence rank in the inventory, with `03-chemicals` for fertiliser and - crop protection and `06-waste` for residues and packaging waste +- **Tools and background data.** Sentier inventory `sector` value `agriculture`, the only + sector serving one GICS sector alone, with `chemicals` for fertiliser and crop + protection and `waste` for residues and packaging waste (sentier-inventory-readme); BAFU:2026 top-level categories `agricultural`, `food industry` and `landfarming` (bafu-2026-release). @@ -91,7 +91,7 @@ that was outside the system last year may be inside it this year. 5. **Install the background and link the farm gate to processing.** Loader: [../../sentier/modules/sentier-brightway/use-cases/](../../sentier/modules/sentier-brightway/use-cases/); - the `01-agriculture` folder and its rank precedence in + the `agriculture` sector and the source folder it sits in, in [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/); the BAFU release in [../../bafu/use-cases/](../../bafu/use-cases/). For Agribalyse-facing work in this sector, the public module is diff --git a/core/use-cases/assess-an-electricity-mix-in-utilities.md b/core/use-cases/assess-an-electricity-mix-in-utilities.md index 9cd2dd9..f8c9974 100644 --- a/core/use-cases/assess-an-electricity-mix-in-utilities.md +++ b/core/use-cases/assess-an-electricity-mix-in-utilities.md @@ -3,8 +3,8 @@ title: Assess an electricity mix in Utilities type: use-case summary: "Model a delivered kilowatt-hour: voltage level, grid boundary, imports, generation technologies and the temporal resolution a yearly average hides." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, bafu-2026-release, sentier-inventory-readme, epd-international-pcr-library, ef-pef-method, ilcd-2010, openlca-bafu-2026-notes, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, bafu-2026-release, sentier-inventory-readme, epd-international-pcr-library, ef-pef-method, ilcd-2010, openlca-bafu-2026-notes, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/utilities.md, ../concepts/functional-unit.md, ../concepts/uncertainty.md, choose-a-background-database.md, assess-an-energy-carrier-in-energy.md] --- @@ -38,9 +38,9 @@ mix built here is reused, so it has to carry its own metadata. water supply (ef-pef-method). The International EPD System's PCR library carries `Electricity, steam & fuels`, which is the programme-rule family for a grid or a generation declaration (epd-international-pcr-library). -- **Tools and background data.** Sentier inventory folders `02-electricity`, one of the - largest sector folders, with `05-energy` for the fuels behind it and `06-waste` for - wastewater and residues (sentier-inventory-readme); BAFU:2026 top-level categories +- **Tools and background data.** Sentier inventory `sector` values `electricity`, one of + the largest sectors, with `energy` for the fuels behind it and `waste` for wastewater + and residues (sentier-inventory-readme); BAFU:2026 top-level categories `electricity`, `electricity by fuel`, `power plants`, `photovoltaic`, `wind power`, `impoundment`, `heat`, `heating`, `heat pumps`, `water`, `wastewater treatment`, `nuclear waste`, `incineration`, `landfill`, `waste` and `waste management` @@ -93,7 +93,7 @@ as choice uncertainty rather than as stochastic spread 5. **Install the background and build the mix as a linked process.** Loader: [../../sentier/modules/sentier-brightway/use-cases/](../../sentier/modules/sentier-brightway/use-cases/); - the `02-electricity` folder and its neighbours are documented in + the `electricity` sector and its neighbours are documented in [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/); the BAFU release itself in [../../bafu/use-cases/](../../bafu/use-cases/). Holding the mix as a node with shares as exchanges is the job of diff --git a/core/use-cases/assess-an-energy-carrier-in-energy.md b/core/use-cases/assess-an-energy-carrier-in-energy.md index 333e5d7..4d8ee51 100644 --- a/core/use-cases/assess-an-energy-carrier-in-energy.md +++ b/core/use-cases/assess-an-energy-carrier-in-energy.md @@ -3,8 +3,8 @@ title: Assess an energy carrier in Energy type: use-case summary: "Model a fuel chain to a named point: the energy basis, the refinery allocation, fugitive and vented methane, and whether combustion is inside the boundary." audience: [P1] -updated: 2026-09-22 -sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, epd-international-pcr-library, ef-pef-method, ilcd-2010, lca-wiki-editorial] +updated: 2026-10-12 +sources: [gics-2023, sentier-inventory-readme, bafu-2026-release, epd-international-pcr-library, ef-pef-method, ilcd-2010, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../sectors/energy.md, ../concepts/allocation-and-multifunctionality.md, ../concepts/system-boundary.md, assess-an-electricity-mix-in-utilities.md, choose-a-background-database.md] --- @@ -38,9 +38,9 @@ content of fuel ([../sectors/energy.md](../sectors/energy.md)). finds its programme rules (epd-international-pcr-library). No PEFCR in the Commission's current list covers fuels or extraction, and rules outside that list are either outdated or private initiatives (ef-pef-method). -- **Tools and background data.** Sentier inventory folder `05-energy`, which holds the - energy carriers and the largest process count of any sector folder, with `03-chemicals` - and `04-transport` supplying the process inputs and the delivery steps +- **Tools and background data.** Sentier inventory `sector` value `energy`, which holds + the energy carriers and the largest process count of any sector, with `chemicals` and + `transport` supplying the process inputs and the delivery steps (sentier-inventory-readme); BAFU:2026 top-level categories `fuels`, `oil`, `natural gas`, `pipeline`, `biomass` and `underground deposit` (bafu-2026-release). @@ -101,7 +101,7 @@ cradle-to-grave study of an energy service carries the combustion emissions 7. **Install the background and build the chain.** Loader: [../../sentier/modules/sentier-brightway/use-cases/](../../sentier/modules/sentier-brightway/use-cases/); - the `05-energy` folder in + the `energy` sector in [../../sentier/modules/sentier-inventory/use-cases/](../../sentier/modules/sentier-inventory/use-cases/); the BAFU release in [../../bafu/use-cases/](../../bafu/use-cases/); a licensed ecoinvent release through diff --git a/core/use-cases/choose-a-background-database.md b/core/use-cases/choose-a-background-database.md index ca453fe..b61b418 100644 --- a/core/use-cases/choose-a-background-database.md +++ b/core/use-cases/choose-a-background-database.md @@ -3,8 +3,8 @@ title: Choose a background database type: use-case summary: "Pick the background for a study on licence, coverage, system model and format: BAFU, ecoinvent or EF-compliant data sets, and how the tooling in this wiki reaches each." audience: [P1] -updated: 2026-09-22 -sources: [ilcd-2010, bafu-2026-release, bafu-terms-of-use-2025, openlca-bafu-2026-notes, esu-bafu-database, ecoinvent-website, ecoinvent-licences, ecoinvent-kb-system-models, ecospold2-format, ef-compliant-data-guide-2.0, ef-data-guide-3.1-addendum, ef-transitional-guidance-2026, eplca-developer-ef, sentier-brightway-repo, sentier-methods-repo, sentier-inventory-readme, lca-wiki-editorial] +updated: 2026-10-12 +sources: [ilcd-2010, bafu-2026-release, bafu-terms-of-use-2025, openlca-bafu-2026-notes, esu-bafu-database, ecoinvent-website, ecoinvent-licences, ecoinvent-kb-system-models, ecospold2-format, ef-compliant-data-guide-2.0, ef-data-guide-3.1-addendum, ef-transitional-guidance-2026, eplca-developer-ef, sentier-brightway-repo, sentier-methods-repo, sentier-inventory-readme, lca-wiki-editorial, sentier-inventory-pr-5] verified: no related: [../concepts/foreground-and-background.md, ../concepts/data-quality.md, ../../bafu/README.md, ../../ecoinvent/README.md, map-flows-between-nomenclatures.md] --- @@ -138,7 +138,7 @@ Not a database but a compliance class: a data set is EF compliant when it meets guidance says what to do when it does not exist (ef-transitional-guidance-2026). 3. **Check coverage against your sector page.** Each GICS sector page names the BAFU - top-level categories and the Sentier inventory folders that serve it; see + top-level categories and the Sentier inventory sectors that serve it; see [../sectors/](../sectors/) and the crosswalk table in [../sectors/crosswalk.md](../sectors/crosswalk.md). diff --git a/index.md b/index.md index 854b8c8..1748c96 100644 --- a/index.md +++ b/index.md @@ -382,7 +382,7 @@ When you add, rename or remove a page, edit this file. - [Communication Services](core/sectors/communication-services.md) — Telecommunication networks, media and interactive services: what an LCA of the GICS Communication Services sector needs and which data sources cover it. - [Consumer Discretionary](core/sectors/consumer-discretionary.md) — Vehicles, durables, apparel and consumer services: what an LCA of the GICS Consumer Discretionary sector needs and which data sources cover it. - [Consumer Staples](core/sectors/consumer-staples.md) — Food, beverages, tobacco and household products: what an LCA of the GICS Consumer Staples sector needs and which data sources cover it. -- [Sector crosswalk](core/sectors/crosswalk.md) — Maps each GICS sector onto the Sentier inventory sector folders, the BAFU:2026 process categories and the ISIC Rev.4 sections. +- [Sector crosswalk](core/sectors/crosswalk.md) — Maps each GICS sector onto the Sentier inventory sector values, the BAFU:2026 process categories and the ISIC Rev.4 sections. - [Energy](core/sectors/energy.md) — Fuel chains from extraction to combustion: what an LCA of the GICS Energy sector needs and which of this wiki's data sources cover it. - [Financials](core/sectors/financials.md) — Banks, financial services and insurance: why a product LCA rarely fits the GICS Financials sector, and what this wiki does and does not cover. - [Health Care](core/sectors/health-care.md) — Pharmaceuticals, devices and care delivery: what an LCA of the GICS Health Care sector needs and which data sources cover it. @@ -525,19 +525,19 @@ When you add, rename or remove a page, edit this file. ### sentier/modules/sentier-inventory/ -- [sentier-inventory](sentier/modules/sentier-inventory/README.md) — The platform's life cycle inventory: processes and exchanges as parquet, one folder per sector, with a schema contract and a CI validator. -- [sentier-inventory schema contracts and validator](sentier/modules/sentier-inventory/classes.md) — The public surface of a data-only repository: three table contracts, one metadata schema, one validation script. -- [sentier-inventory commands](sentier/modules/sentier-inventory/commands.md) — The validator, the inspection one-liner, and what continuous integration runs. -- [sentier-inventory datasets](sentier/modules/sentier-inventory/datasets.md) — The BAFU-2026 inventory as the platform ships it: eleven sector folders, two tables per folder, flow identifiers that carry their unit. -- [sentier-inventory gotchas](sentier/modules/sentier-inventory/gotchas.md) — What surprises people: the obsolete folder, per-folder guarantees, absent optional columns and flow identifiers that match nothing else. -- [sentier-inventory layout](sentier/modules/sentier-inventory/layout.md) — The repository tree: eleven sector folders, four schema files, one validator. -- [sentier-inventory status](sentier/modules/sentier-inventory/status.md) — Where the inventory repository stands: one source database, eleven sectors, unit processes only. +- [sentier-inventory](sentier/modules/sentier-inventory/README.md) — The platform's life cycle inventory: processes and exchanges as parquet, one folder per source release with the sector as a column, a schema contract and a CI validator. +- [sentier-inventory schema contracts and validator](sentier/modules/sentier-inventory/classes.md) — The public surface of a data-only repository: two table contracts, one metadata schema, one validation script. +- [sentier-inventory commands](sentier/modules/sentier-inventory/commands.md) — The validator, the inspection one-liners, and what continuous integration runs. +- [sentier-inventory datasets](sentier/modules/sentier-inventory/datasets.md) — The BAFU-2026 inventory as the platform ships it: one source folder, two tables and a metadata file, the sector as a column, flow identifiers that carry their unit. +- [sentier-inventory gotchas](sentier/modules/sentier-inventory/gotchas.md) — What surprises people: a layout change pending merge, the obsolete sector, per-folder guarantees, absent optional columns and flow identifiers that match nothing else. +- [sentier-inventory layout](sentier/modules/sentier-inventory/layout.md) — The repository tree: one source folder, four schema files, one validator. +- [sentier-inventory status](sentier/modules/sentier-inventory/status.md) — Where the inventory repository stands: one source folder, schema 0.3.0 on an open pull request, unit processes only. ### sentier/modules/sentier-inventory/use-cases/ - [sentier-inventory use cases](sentier/modules/sentier-inventory/use-cases/README.md) — End-to-end things a practitioner does with the Sentier inventory data. -- [Read a sector folder with pandas](sentier/modules/sentier-inventory/use-cases/read-a-sector-folder-with-pandas.md) — Open one sector's processes and exchanges parquet in a dataframe, check them against the folder's declared row counts, and see the contracted columns. -- [Validate the inventory schema](sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md) — Run the repository's own validator over every sector folder, the same command continuous integration runs on every pull request. +- [Read a source folder with pandas](sentier/modules/sentier-inventory/use-cases/read-a-source-folder-with-pandas.md) — Open one source's processes and exchanges parquet in a dataframe, check them against the folder's declared row counts and sectors, filter one sector out of the column, and see the contracted columns. +- [Validate the inventory schema](sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md) — Run the repository's own validator over every source folder, the same command continuous integration runs on every pull request. ### sentier/modules/sentier-mappings/ diff --git a/log.md b/log.md index b020956..d9807ec 100644 --- a/log.md +++ b/log.md @@ -17,3 +17,4 @@ Append-only. One line per task, newest at the bottom, in the form ## [2026-09-22] lint | Phase 5: review pass over 52 pages (contradictions, stale claims, roadmaps, sourcing), cold-start persona review (8 questions, 10 improvements applied or recorded), vocabulary pass (58 terms added, 208 total; ISO paraphrased, EF quoted, singular/plural merged); index gained a persona-first header; api/ folders carry licence banners; SPECS.md scrubbed of maintainer workspace paths ## [2026-09-22] task | README rewritten: architecture tree at the top, concise sections ## [2026-09-22] task | README architecture tree switched to a Mermaid flowchart, as in sentier-brightway +## [2026-10-12] task | sentier-inventory organised by source folder (data/-/, today bafu-2026-v1), sector as a required column, schema 0.3.0: 51 pages updated across sentier/modules/sentier-inventory, sentier-brightway, sentier-importers, core/sectors, core/use-cases, bafu, index, vocabulary (Source folder added, Sector folder marked superseded), sources (three pull-request rows); read-a-sector-folder-with-pandas renamed to read-a-source-folder-with-pandas; both inventory use cases re-verified; change is on open pull requests #5, #29 and #3, pending merge diff --git a/sentier/README.md b/sentier/README.md index 5640a2f..c476484 100644 --- a/sentier/README.md +++ b/sentier/README.md @@ -3,7 +3,7 @@ title: sentier type: branch summary: "The Sentier platform branch: what it is, its modules, and how its data reaches Brightway." audience: [P1, P2, P3] -updated: 2026-09-21 +updated: 2026-10-12 sources: [sentier-dev-repo, sentier-dev-adr-1, sentier-brightway-repo, sentier-importers-repo] related: [platform.md, commands.md, modules/README.md] --- @@ -52,7 +52,7 @@ branch. (source: sentier-importers-repo, sentier-brightway-repo) | [modules/sentier-brightway/](modules/sentier-brightway/) | Application | The loader that installs the platform data into a Brightway project, or writes it as plain files. | | [modules/sentier-agribalyse/](modules/sentier-agribalyse/) | Application | The public Agribalyse-facing work of the platform. | | [modules/sentier-vocab/](modules/sentier-vocab/) | Data | The controlled vocabulary: one IRI per term, with the schemas that define each term type. | -| [modules/sentier-inventory/](modules/sentier-inventory/) | Data | The life cycle inventory, processes and exchanges as parquet, organised per sector. | +| [modules/sentier-inventory/](modules/sentier-inventory/) | Data | The life cycle inventory, processes and exchanges as parquet, one folder per source release with the sector as a column. | | [modules/sentier-methods/](modules/sentier-methods/) | Data | The impact assessment methods and their characterisation factor tables. | | [modules/sentier-mappings/](modules/sentier-mappings/) | Cross-cutting | The bridge packages that map one nomenclature onto another, as reviewable data. | diff --git a/sentier/commands.md b/sentier/commands.md index e0919b5..a6cbd2f 100644 --- a/sentier/commands.md +++ b/sentier/commands.md @@ -3,8 +3,8 @@ title: Sentier commands type: module-page summary: One table of every runnable command across the Sentier packages, with how each row was checked. audience: [P1, P2] -updated: 2026-09-22 -sources: [sentier-brightway-repo, sentier-importers-repo, sentier-vocab-repo, sentier-inventory-repo, sentier-methods-repo, sentier-mappings-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-importers-repo, sentier-vocab-repo, sentier-inventory-repo, sentier-inventory-pr-5, sentier-methods-repo, sentier-mappings-repo] related: [README.md, modules/sentier-brightway/commands.md, modules/sentier-importers/commands.md] --- @@ -45,7 +45,7 @@ sentier-methods-repo, sentier-mappings-repo). | `uv run python -m sentier_vocab coverage` | sentier-vocab | Regenerates the coverage matrix of the vocabulary. | README | | `uv run sentier-vocab ` | sentier-vocab | The same command line as `python -m sentier_vocab`, through the console script. | README | | `uv run bash scripts/generate.sh` | sentier-vocab | Fetches the external ontologies and runs the transitional importers. | README | -| `uv run --with "jsonschema[format],pyyaml,pyarrow" python scripts/validate.py` | sentier-inventory | Validates every sector folder against the schema contracts; what CI runs on each pull request. | README | +| `uv run --with "jsonschema[format],pyyaml,pyarrow" python scripts/validate.py` | sentier-inventory | Validates every source folder, `data/-/`, against the schema contracts; what CI runs on each pull request. | README | | `uv run --with pyarrow --with pyyaml --with 'jsonschema[format]' python scripts/validate.py` | sentier-methods | Validates every method data folder against the schemas; what CI runs on each pull request. | README | | `uv run --with-requirements scripts/requirements.txt python scripts/validate.py` | sentier-mappings | Validates every source-and-target pair against the package and metadata schemas. | README | | `uv run --with-requirements scripts/requirements.txt pytest -q tests` | sentier-mappings | Runs the validator's own test suite. | README | diff --git a/sentier/modules/README.md b/sentier/modules/README.md index 4c3ae55..da18e10 100644 --- a/sentier/modules/README.md +++ b/sentier/modules/README.md @@ -3,7 +3,7 @@ title: sentier/modules type: readme summary: One folder per Sentier package or data repository, each following the codebase template. audience: [P1, P2, P3] -updated: 2026-09-22 +updated: 2026-10-12 sources: [] --- @@ -20,7 +20,7 @@ documents one Sentier package or data repository. | [sentier-brightway/](sentier-brightway/) | Application | The loader that installs the platform data into a Brightway project, or writes it as plain files. | | [sentier-agribalyse/](sentier-agribalyse/) | Application | The public Agribalyse-facing work of the platform. | | [sentier-vocab/](sentier-vocab/) | Data | The controlled vocabulary of the platform. | -| [sentier-inventory/](sentier-inventory/) | Data | The life cycle inventory data, organised per sector. | +| [sentier-inventory/](sentier-inventory/) | Data | The life cycle inventory data, one folder per source release, the sector as a column. | | [sentier-methods/](sentier-methods/) | Data | The impact assessment methods and characterisation factors. | | [sentier-mappings/](sentier-mappings/) | Cross-cutting | The bridge packages that map one nomenclature onto another. | diff --git a/sentier/modules/sentier-brightway/README.md b/sentier/modules/sentier-brightway/README.md index ec7573b..2f8890f 100644 --- a/sentier/modules/sentier-brightway/README.md +++ b/sentier/modules/sentier-brightway/README.md @@ -3,8 +3,8 @@ title: sentier-brightway type: module summary: The loader that installs Sentier data into a Brightway project or a folder of plain files. audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-brightway-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-brightway-pr-3] related: [../../platform.md, ../../commands.md, ../../../bafu/README.md] --- @@ -25,6 +25,13 @@ The package is **not on PyPI**: a request to `https://pypi.org/pypi/sentier-brig on 2026-09-21 answered 404, so install it from git. The `CHANGELOG.md` heading for 0.1.0 still reads "unreleased". (source: sentier-brightway-repo) +**Re-read on 2026-10-12** at commit `3239a4e`, the head of branch `feat/source-folders` +proposed as pull request #3 and not yet merged. That commit follows the source-folder +layout of sentier-inventory schema 0.3.0: the loader opens the one folder +`data/-/` whose metadata names the requested source, keeps the +`sector` column on every process, and pins sentier-inventory at `cf54f55`. The pages +below say where it differs from 0.1.0 at `a9dba75`. (source: sentier-brightway-pr-3) + ## Install Install it into the same Python environment as Brightway or the Activity Browser: @@ -61,7 +68,7 @@ pinned commit of its own repository. Broad descriptors only; the details are in | Dataset | Version | Provenance | Where it is documented | |---|---|---|---| -| BAFU-2026 life cycle inventory | v1, 11,947 processes in 11 sector folders | sentier-inventory | [../../../bafu/](../../../bafu/), [../sentier-inventory/](../sentier-inventory/) | +| BAFU-2026 life cycle inventory | v1, 11,947 processes in one source folder, `data/bafu-2026-v1/`, the sector as a column | sentier-inventory | [../../../bafu/](../../../bafu/), [../sentier-inventory/](../sentier-inventory/) | | EF 3.1 elementary flows (biosphere) | EF 3.1 shards of the vocabulary | sentier-vocab | [../sentier-vocab/](../sentier-vocab/) | | EF 3.1 impact assessment methods | 25 impact categories, global factors | sentier-methods | [../sentier-methods/](../sentier-methods/) | | BAFU to EF 3.1 flow bridge | four ordered mapping packages | sentier-mappings | [../sentier-mappings/](../sentier-mappings/) | diff --git a/sentier/modules/sentier-brightway/api/sentier_brightway.inventory.md b/sentier/modules/sentier-brightway/api/sentier_brightway.inventory.md index 87d8496..27afa22 100644 --- a/sentier/modules/sentier-brightway/api/sentier_brightway.inventory.md +++ b/sentier/modules/sentier-brightway/api/sentier_brightway.inventory.md @@ -21,6 +21,9 @@ uvx --python 3.11 --from pydoc-markdown==4.8.2 pydoc-markdown \ ``` Generated file: do not edit by hand; regenerate with the command above. +Exception recorded here: on 2026-10-12 the docstrings and the `load_inventory` signature +below were updated by hand to commit `3239a4e` (pull request #3), which replaced the +sector-folder reader with a source-folder one; the rest of the file is as generated. The output of that command is then passed through one mechanical pass: links the wiki cannot resolve are flattened to their text, absolute file paths become ``, and names of private repositories become ``. @@ -31,7 +34,7 @@ the wiki cannot resolve are flattened to their text, absolute file paths become # sentier\_brightway.inventory -Read sentier-inventory sector folders into two flat frames. +Read one source folder of sentier-inventory (``data/-/``) into two frames. @@ -62,15 +65,32 @@ class Inventory() #### exchanges + + +#### source\_version + +``metadata.source_version`` of the folder read + + + +#### sectors + +``metadata.sectors`` of the folder read, sorted + #### load\_inventory ```python -def load_inventory(data_root: Path) -> Inventory +def load_inventory(data_root: Path, + source: str = DEFAULT_SOURCE, + source_version: str | None = None) -> Inventory ``` -Concatenate every ``data/-/`` folder of sentier-inventory. +The processes and exchanges of one inventory ``source`` release. -Sectors such as ``99-obsolete`` are intentionally included: their processes are -still link targets for exchanges recorded in other sectors. +sentier-inventory keeps one folder per published release, ``data/-/``, +with ``sector`` as a column on the processes. Folder selection reads only each folder's +``metadata.json`` (``source``, ``source_version``, ``sectors``), so other sources' parquet +is never opened. Without ``source_version`` the source must be present exactly once; with +it, that release is read. The ``sector`` column stays on the returned frame. diff --git a/sentier/modules/sentier-brightway/classes.md b/sentier/modules/sentier-brightway/classes.md index 5c57f58..7deee11 100644 --- a/sentier/modules/sentier-brightway/classes.md +++ b/sentier/modules/sentier-brightway/classes.md @@ -3,16 +3,17 @@ title: sentier-brightway classes and functions type: module-page summary: The public surface of sentier-brightway, in the order a user meets it. audience: [P1, P3] -updated: 2026-09-22 -sources: [sentier-brightway-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-brightway-pr-3] --- # sentier-brightway classes and functions Hand-written companion to the generated [api/](api/README.md) folder. The names below are the ones exported from `sentier_brightway/__init__.py` plus the few -module-level helpers a user calls directly, read at commit `a9dba75`. -(source: sentier-brightway-repo) +module-level helpers a user calls directly, read at commit `a9dba75`; the inventory +reader rows follow commit `3239a4e` (pull request #3, pending merge). +(source: sentier-brightway-repo, sentier-brightway-pr-3) ## The public API @@ -36,7 +37,8 @@ module-level helpers a user calls directly, read at commit `a9dba75`. | `fetch.resolve_data_root` | function | Picks the data root: an explicit path, the environment variable, or the verified download cache. | `assemble` | | `fetch.load_packaged_manifest` | function | Parses the pinned `sources.toml` into `Source` records. | `resolve_data_root`, the file-mode manifest | | `fetch.materialize` | function | Downloads every pinned file and checks its sha256 into the cache. | `resolve_data_root` | -| `inventory.load_inventory` | function | Reads the sector folders into a processes frame and an exchanges frame. | `assemble` | +| `inventory.load_inventory` | function | Reads the one source folder whose metadata names `source` (and `source_version` when given) into a processes frame, `sector` column kept, and an exchanges frame; errors list the source and version pairs present. | `assemble` | +| `inventory.scan_source_folders` / `inventory.SourceFolder` | function, frozen dataclass | Describe every `data/*/` folder of the inventory by its `metadata.json` alone, source, version and sectors, without opening parquet. | `load_inventory` | | `flows.load_ef_flows` / `flows.load_bafu_flows` | functions | Read the elementary-flow terms of one upstream source out of the vocabulary shards. | `assemble` | | `methods.load_methods` | function | Reads the 25 methods and their global characterisation factors. | `assemble` | | `bridge.load_bridge` | function | Reads the ordered mapping packages into one source-code to target-flow mapping. | `assemble` | @@ -67,6 +69,13 @@ module-level helpers a user calls directly, read at commit `a9dba75`. `import sentier_brightway` and the whole of file mode work without Brightway. - **`assemble` is the seam.** Anything that reads data returns a frame, anything that writes takes a `BuildResult`; tests exercise the middle without touching a project. +- **Every entry point takes a source.** At commit `3239a4e` `assemble`, `coverage`, + `import_bafu_db` and `import_bafu_files` take `source="bafu-2026"` and + `source_version=None`; `import_db` and `import_files` are the same functions under + source-neutral names, and the `import_bafu_*` names are kept as thin wrappers. The + inventory and residual databases are named after the source, never the version, so a + re-release replaces an install in place and a second source installs beside it. + (source: sentier-brightway-pr-3) - **The errors are typed**: `fetch.FetchError` and `fetch.IntegrityError` for the data, `files.ExistingOutputError` for a non-empty output folder, `writer.ExistingDatabaseError` for a project that already holds the databases. The command line turns them into a diff --git a/sentier/modules/sentier-brightway/commands.md b/sentier/modules/sentier-brightway/commands.md index f9eb9b9..06dc884 100644 --- a/sentier/modules/sentier-brightway/commands.md +++ b/sentier/modules/sentier-brightway/commands.md @@ -3,8 +3,8 @@ title: sentier-brightway commands type: module-page summary: The four command line subcommands and the Python entry points, checked against their own --help. audience: [P1] -updated: 2026-09-21 -sources: [sentier-brightway-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-brightway-pr-3] --- # sentier-brightway commands @@ -40,6 +40,11 @@ Every entry point, checked on 2026-09-21 against its own `--help` in a clone at default is a path on the author's machine, so pass it explicitly. - `--files DIR`: reuse an existing file-mode export instead of writing a new one into `/files` first. +- `--source ID` and `--source-version V`, on every command at commit `3239a4e` (pull + request #3, pending merge): the inventory source to install, which is the folder + `data/-/` of sentier-inventory and names the databases, default + `bafu-2026`; and the release of that source to read when the data root serves several, + otherwise the only one present. (source: sentier-brightway-pr-3) ### Examples @@ -72,6 +77,11 @@ Used interactively rather than from a shell; all four are exported from the pack | `datapackage.score(out_dir, process_code, method_id)` | Scores one process of a files export for one method id, without `bw2data`. | | `registry.load_registry(folder)` | Reads the five parquet tables of an export back into frames. | +At commit `3239a4e` the first four calls also take `source="bafu-2026"` and +`source_version=None`, the Python form of the two flags above, and `import_db` and +`import_files` are the same functions under source-neutral names. +(source: sentier-brightway-pr-3) + After an install, link your own activities to the installed inventory and biosphere and score them with stock `bw2calc`; in the Activity Browser, reload the project and the methods appear under the Sentier method prefix. (source: sentier-brightway-repo) diff --git a/sentier/modules/sentier-brightway/datasets.md b/sentier/modules/sentier-brightway/datasets.md index a087cca..d36805d 100644 --- a/sentier/modules/sentier-brightway/datasets.md +++ b/sentier/modules/sentier-brightway/datasets.md @@ -3,8 +3,8 @@ title: sentier-brightway datasets type: module-page summary: The four pinned datasets sentier-brightway fetches, and the two shapes it writes them in. audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-brightway-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-brightway-pr-3] related: [../../../bafu/README.md, ../sentier-methods/README.md] --- @@ -17,14 +17,20 @@ amounts and no characterisation factor values appear on this page. ## BAFU-2026 life cycle inventory -- **Version:** v1, 11,947 processes, pinned by commit in `sources.toml` -- **Sectors:** eleven folders, ranked and numbered: agriculture, electricity, chemicals, - transport, energy, waste, construction, materials, electronics, building services, and - an obsolete folder kept for completeness +- **Version:** v1, 11,947 processes, pinned by commit in `sources.toml`: at commit + `3239a4e` the pin is sentier-inventory `cf54f55`, three files under + `data/bafu-2026-v1/` instead of thirty-three sector files (source: sentier-brightway-pr-3) +- **Sectors:** eleven values of the `sector` column, read from the folder's metadata and + kept on every process node: agriculture, electricity, chemicals, transport, energy, + waste, construction, materials, electronics, building services, and `obsolete` for the + legacy link targets kept for completeness - **Provenance:** published by the Swiss federal administration and delivered into the - platform by the importers; distributed through the sentier-inventory repository -- **Schema:** two parquet tables per sector folder, processes and exchanges, plus a - metadata file; the columns are contracted in the inventory repository + platform by the importers; distributed through the sentier-inventory repository as the + source `bafu-2026`, release `v1` +- **Schema:** one folder per published release of one datasource, holding two parquet + tables, processes and exchanges, plus a metadata file naming the source, its version + and the sectors present; the loader picks the folder by that metadata, and the columns + are contracted in the inventory repository - **Methods:** none; this is inventory only - **Vocabulary:** process and flow terms are IRIs in the Sentier vocabulary - **Source branch:** [../../../bafu/](../../../bafu/) for the database itself, @@ -69,7 +75,7 @@ amounts and no characterisation factor values appear on this page. | Object | Content | |---|---| -| inventory database | the 11,947 processes | +| inventory database | the 11,947 processes, each tagged with its `source`, `source_version` and `sector` | | biosphere database | the EF 3.1 elementary flows, with inventory emissions relinked onto them | | residual database | the inventory flows with no EF 3.1 counterpart, exchanges intact, no factor | | methods | the 25 EF 3.1 impact categories, under the Sentier method prefix | diff --git a/sentier/modules/sentier-brightway/layout.md b/sentier/modules/sentier-brightway/layout.md index 80d6824..3d0909f 100644 --- a/sentier/modules/sentier-brightway/layout.md +++ b/sentier/modules/sentier-brightway/layout.md @@ -3,15 +3,17 @@ title: sentier-brightway layout type: module-page summary: How the sentier-brightway repository is organised, file by file. audience: [P2, P3] -updated: 2026-09-21 -sources: [sentier-brightway-repo] +updated: 2026-10-12 +sources: [sentier-brightway-repo, sentier-brightway-pr-3] --- # sentier-brightway layout The tree below is the clone of `github.com/sentier-dev/sentier-brightway` at commit `a9dba75` (2026-09-17), read on 2026-09-21. Only the files a reader would open are -listed. (source: sentier-brightway-repo) +listed; the `inventory.py` line follows commit `3239a4e` (pull request #3, pending merge), +which replaced the sector-folder reader with a source-folder one and added `--source-version` +to `cli.py`. (source: sentier-brightway-repo, sentier-brightway-pr-3) ``` src/sentier_brightway/ @@ -20,7 +22,7 @@ src/sentier_brightway/ constants.py every shared name: database names, method prefix, IRIs, citation fetch.py resolve the data root: local checkout, or verified download cache sources.toml the pinned manifest: repository, commit, file, sha256 - inventory.py read the inventory sector folders into two flat frames + inventory.py read one source folder of the inventory, data/-/, into two flat frames flows.py read elementary-flow terms from the vocabulary, split by source methods.py read the impact methods and their global characterisation factors bridge.py read the ordered mapping packages that relink source flows diff --git a/sentier/modules/sentier-importers/README.md b/sentier/modules/sentier-importers/README.md index 5495033..80f37c9 100644 --- a/sentier/modules/sentier-importers/README.md +++ b/sentier/modules/sentier-importers/README.md @@ -3,8 +3,8 @@ title: sentier-importers type: module summary: The pipelines that turn external LCA databases into the Sentier data format. audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-importers-repo] +updated: 2026-10-12 +sources: [sentier-importers-repo, sentier-importers-pr-29] related: [../../platform.md, ../../commands.md, ../sentier-brightway/README.md] --- @@ -25,6 +25,11 @@ dated 2026-09-17 · **as of:** 2026-09-21. The package is **not on PyPI**: a request to `https://pypi.org/pypi/sentier-importers/json` on 2026-09-21 answered 404, so work from a clone. (source: sentier-importers-repo) +**Re-read on 2026-10-12** at commit `a27910f`, the head of branch `feat/source-folders` +proposed as pull request #29 and not yet merged, for the way inventory tables are +delivered: one folder per source release in sentier-inventory, the sector as a column. +(source: sentier-importers-pr-29) + ## Install Python 3.10 or newer, and `uv`: @@ -49,7 +54,10 @@ platformdirs, pyarrow, pyyaml and rdflib. (source: sentier-importers-repo) own repository's checker instead. See [gotchas.md](gotchas.md). 4. **Delivers by pull request, never by push.** A run is a dry run unless delivery is asked for, and files then land in the target repository's data folder on a branch. - See [commands.md](commands.md). + For inventory targets the registry category is the source folder, + `data/-/` (`bafu-2026-v1`), and a delivery replaces that + folder whole: sources never share a folder, so nothing is merged. See + [commands.md](commands.md) and [datasets.md](datasets.md). ## Datasets diff --git a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.ecospold.md b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.ecospold.md index 92f8948..c1d713e 100644 --- a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.ecospold.md +++ b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.ecospold.md @@ -21,6 +21,10 @@ uvx --python 3.11 --from pydoc-markdown==4.8.2 pydoc-markdown \ ``` Generated file: do not edit by hand; regenerate with the command above. +Exception recorded here: on 2026-10-12 the `SECTORS` and `sector_for` docstrings were +updated by hand to commit `a27910f` (pull request #29), where the sector ids lost their +folder rank prefix, and `sector_slug`, removed in that commit, was replaced by the new +`SOURCE_FOLDER` constant; the rest of the file is as generated. The output of that command is then passed through one mechanical pass: links the wiki cannot resolve are flattened to their text, absolute file paths become ``, and names of private repositories become ``. @@ -58,7 +62,9 @@ regenerated data is bit-identical across users). #### SECTORS -Sector folder -> BAFU ``referenceFunction`` categories (covers all 59 observed). +Sector id -> BAFU ``referenceFunction`` categories (covers all 59 observed). +The ids are the ``processes.sector`` enum of sentier-inventory (schema 0.3.0); the +dict order is the former folder rank order and names the vocab per-sector files. @@ -68,20 +74,14 @@ Sector folder -> BAFU ``referenceFunction`` categories (covers all 59 observed). def sector_for(category: str) -> str ``` -Sector folder for a BAFU category; raises ``ParseError`` on unknown categories. +Sector id for a BAFU category; raises ``ParseError`` on unknown categories. - + -#### sector\_slug +#### SOURCE\_FOLDER -```python -def sector_slug(sector: str) -> str -``` - -Rank-free sector name (``01-agriculture`` -> ``agriculture``). - -Vocab payload files are content-named per sector without the inventory -folder's rank prefix. +sentier-inventory folder holding this release: ``data/-/``. +One folder per published release; a delivery rewrites it whole. diff --git a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_exchanges.md b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_exchanges.md index 313f8af..8bce53b 100644 --- a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_exchanges.md +++ b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_exchanges.md @@ -21,6 +21,9 @@ uvx --python 3.11 --from pydoc-markdown==4.8.2 pydoc-markdown \ ``` Generated file: do not edit by hand; regenerate with the command above. +Exception recorded here: on 2026-10-12 the docstrings below were updated by hand to +commit `a27910f` (pull request #29), one registry entry per release instead of per +sector folder; the rest of the file is as generated. The output of that command is then passed through one mechanical pass: links the wiki cannot resolve are flattened to their text, absolute file paths become ``, and names of private repositories become ``. @@ -31,7 +34,10 @@ the wiki cannot resolve are flattened to their text, absolute file paths become # sentier\_importers.sources.bafu.inventory\_exchanges -BAFU-2026 -> sentier-inventory ``exchanges.parquet``, one registry entry per sector. +BAFU-2026 -> sentier-inventory ``exchanges.parquet``, one registry entry per release. + +Every dataset's exchanges are emitted into the one source folder +(``ecospold.SOURCE_FOLDER``); the sector lives on ``processes.parquet`` only. Flow ids: the production flow is the process's own UUID; technosphere inputs resolve through the export-wide ``number -> uuid`` map (exact — 0 @@ -47,7 +53,7 @@ vocabulary stay joinable by construction. class BafuInventoryExchangesSource(Source) ``` -Emit the exchange table rows for one sector folder. +Emit the exchange table rows of the whole release. diff --git a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_processes.md b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_processes.md index baf6fb6..d7c89d9 100644 --- a/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_processes.md +++ b/sentier/modules/sentier-importers/api/sentier_importers.sources.bafu.inventory_processes.md @@ -21,6 +21,9 @@ uvx --python 3.11 --from pydoc-markdown==4.8.2 pydoc-markdown \ ``` Generated file: do not edit by hand; regenerate with the command above. +Exception recorded here: on 2026-10-12 the docstrings below were updated by hand to +commit `a27910f` (pull request #29), one registry entry per release instead of per +sector folder; the rest of the file is as generated. The output of that command is then passed through one mechanical pass: links the wiki cannot resolve are flattened to their text, absolute file paths become ``, and names of private repositories become ``. @@ -31,11 +34,12 @@ the wiki cannot resolve are flattened to their text, absolute file paths become # sentier\_importers.sources.bafu.inventory\_processes -BAFU-2026 -> sentier-inventory ``processes.parquet``, one registry entry per sector. +BAFU-2026 -> sentier-inventory ``processes.parquet``, one registry entry per release. -Each entry's ``category`` is the sector folder (``NN-``); ``transform`` -keeps only the datasets whose BAFU category routes there. Rows follow the -sentier-inventory ``schema/process.yaml`` contract exactly. +The entry's ``category`` is the source folder (``ecospold.SOURCE_FOLDER``, +``data/bafu-2026-v1/``); ``transform`` emits every dataset and writes the sector +its BAFU category routes to into the ``sector`` column. Rows follow the +sentier-inventory ``schema/process.yaml`` contract exactly, column order included. @@ -55,7 +59,7 @@ Provenance comment: BAFU category path, obsolete marker, source citation. class BafuInventoryProcessesSource(Source) ``` -Emit the process table rows for one sector folder. +Emit the process table rows of the whole release, sector as a column. diff --git a/sentier/modules/sentier-importers/datasets.md b/sentier/modules/sentier-importers/datasets.md index 2b35dd9..a4ef4b9 100644 --- a/sentier/modules/sentier-importers/datasets.md +++ b/sentier/modules/sentier-importers/datasets.md @@ -3,23 +3,26 @@ title: sentier-importers datasets type: module-page summary: What the registered sources read, what they write, and into which target repository. audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-importers-repo] +updated: 2026-10-12 +sources: [sentier-importers-repo, sentier-importers-pr-29] --- # sentier-importers datasets -The repository ships no dataset of its own. It ships 66 registered sources, read from -`registry.yaml` at commit `dac2e67` on 2026-09-21, which read external data and deliver -it into four target repositories. Descriptors stay broad: no amounts and no -characterisation factor values appear on this page. (source: sentier-importers-repo) +The repository ships no dataset of its own. It ships 46 registered sources, read from +`registry.yaml` at commit `a27910f` on 2026-10-12, the head of branch +`feat/source-folders` proposed as pull request #29 and not yet merged, which read +external data and deliver it into four target repositories. There were 66 at commit +`dac2e67` on 2026-09-21: the 22 per-sector inventory blocks of the Swiss federal family +became two. Descriptors stay broad: no amounts and no characterisation factor values +appear on this page. (source: sentier-importers-repo, sentier-importers-pr-29) ## Targets | Target repository | Sources delivering to it | Files land in | Validated by | |---|---|---|---| | sentier-vocab | 34 | the repository's data folder | its LinkML schema, at a pinned reference | -| sentier-inventory | 23 | the repository's data folder | the target repository's own checker | +| sentier-inventory | 3 | one folder per source release, `data/-/`, replaced whole on delivery | the target repository's own checker | | sentier-mappings | 7 | the repository's data folder | the randonneur package schema, in the source | | sentier-methods | 2 | the repository's data folder | the target repository's own checker | @@ -29,14 +32,20 @@ The published data is documented on the target module pages: ## Source families -### Swiss federal inventory (44 registered sources) +### Swiss federal inventory (24 registered sources) - **Reads:** the published exchange-format inventory, from local paths. - **Writes:** process and elementary-flow vocabulary terms; processes and exchanges as - parquet, one pair per sector folder; and the flow bridge onto the EF 3.1 nomenclature - as ordered mapping packages, with a coverage sidecar. -- **Sectors:** the eleven sector folders of the inventory, agriculture through building - services, plus an obsolete folder. + parquet, one pair in the inventory's source folder `data/bafu-2026-v1/`, from the two + registry blocks `bafu-processes` and `bafu-exchanges` with category `bafu-2026-v1`; + the folder's `metadata.json`, written by the local regeneration script from the + staged parquet and the publisher constants; and the flow bridge onto the EF 3.1 + nomenclature as ordered mapping packages, with a coverage sidecar. +- **Sectors:** every dataset is emitted, and its sector is written into the `sector` + column of the process table from the map of BAFU categories onto the inventory's + eleven sector ids, agriculture through building-services plus obsolete. A delivery + replaces the source folder whole; sources never share a folder. + (source: sentier-importers-pr-29) - **Source branch:** [../../../bafu/](../../../bafu/) ### Agribalyse and the EF 3.1 reference package (15 registered sources) diff --git a/sentier/modules/sentier-importers/use-cases/run-an-importer-end-to-end.md b/sentier/modules/sentier-importers/use-cases/run-an-importer-end-to-end.md index 6c8d8a3..e4bbb8b 100644 --- a/sentier/modules/sentier-importers/use-cases/run-an-importer-end-to-end.md +++ b/sentier/modules/sentier-importers/use-cases/run-an-importer-end-to-end.md @@ -3,8 +3,8 @@ title: Run an importer end to end type: use-case summary: "Run the reference plugin through the full staged pipeline offline, inspect the staged output, and see where a delivery would go." audience: [P1, P2] -updated: 2026-09-22 -sources: [sentier-importers-repo] +updated: 2026-10-12 +sources: [sentier-importers-repo, sentier-importers-pr-29] verified: yes verified_on: 2026-09-22 verified_env: "uv 0.11.14, Python 3.10.12, Ubuntu 22.04, sentier-importers 0.0.1 from a clone, throwaway cache and output directories, no network needed" @@ -65,7 +65,8 @@ new plugin is copied from, and it is the only source enabled by default. ## Expected output Step 1 prints one line per registered source: its name, the target repository and -category it writes into, and whether it is enabled. There are 66 of them and exactly one +category it writes into, and whether it is enabled. There are 46 of them at commit +`a27910f` (66 before the per-sector inventory blocks collapsed into two) and exactly one is enabled, which is why the run-everything smoke test stays fast. Step 2 prints the source name and the number of valid rows, three for the reference @@ -77,11 +78,13 @@ each normalised into the shape the plugin's `transform` declares. It is a toy sh an [LCI](../../../../vocabulary.md#lci) shape, on purpose. Step 5 puts the same file into the target repository's data folder in your checkout, so -`git diff` there shows exactly what a pull request would propose. +`git diff` there shows exactly what a pull request would propose. For the inventory +target the category is the source folder, and a real delivery such as the two `bafu-*` +inventory sources replaces `data/bafu-2026-v1/` whole. (source: sentier-importers-pr-29) ## Pitfalls -- **Most sources will not run for you.** 65 of the 66 are disabled, and several of the +- **Most sources will not run for you.** 45 of the 46 are disabled, and several of the disabled ones read local paths holding licensed or large inputs. Naming one of those explicitly fails on a missing file, not on a bug. - **A run is a dry run.** Nothing leaves the staging folder without `--deliver`, and @@ -102,7 +105,7 @@ Step 5 puts the same file into the target repository's data folder in your check [schema](../../../../vocabulary.md#schema), [nomenclature](../../../../vocabulary.md#nomenclature), [randonneur package](../../../../vocabulary.md#randonneur-package), -[sector folder](../../../../vocabulary.md#sector-folder). +[source folder](../../../../vocabulary.md#source-folder). ## Verification @@ -112,4 +115,5 @@ network reachable from the plugin. Steps 1 to 4 were executed as written and too 3.9 s, 3.3 s and under a second. Step 1 listed 66 sources with one enabled; step 2 reported three valid rows; step 3 wrote one JSON file holding three records with the fields `id`, `label` and `region`. Step 5 was not run, because it writes into a checkout -of another repository. +of another repository. Step 1 was run again on 2026-10-12 at commit `a27910f` and listed +46 sources with one enabled; the other steps were not repeated. diff --git a/sentier/modules/sentier-inventory/README.md b/sentier/modules/sentier-inventory/README.md index ea8258d..d904f00 100644 --- a/sentier/modules/sentier-inventory/README.md +++ b/sentier/modules/sentier-inventory/README.md @@ -1,10 +1,10 @@ --- title: sentier-inventory type: module -summary: "The platform's life cycle inventory: processes and exchanges as parquet, one folder per sector, with a schema contract and a CI validator." +summary: "The platform's life cycle inventory: processes and exchanges as parquet, one folder per source release with the sector as a column, a schema contract and a CI validator." audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] related: [../../../bafu/README.md, ../sentier-mappings/README.md, ../../roadmap.md] --- @@ -12,16 +12,18 @@ related: [../../../bafu/README.md, ../sentier-mappings/README.md, ../../roadmap. sentier-inventory is a data repository, not a package. It holds standardised life cycle inventory: one row per process and one row per exchange, stored as parquet, organised -into numbered sector folders. There is no Python package and no loader; the only code in +into one folder per published release of one datasource, with the sector of every +process held as a column. There is no Python package and no loader; the only code in the repository is the validator that continuous integration runs on every pull request. Data arrives as pull requests from the importers, is read by the loader that installs it into an LCA tool, and its cross-source flow identifiers are resolved through the mapping repository. (source: sentier-inventory-repo) -**Version documented:** schema version `0.1.0`, as declared in every sector's +**Version documented:** schema version `0.3.0`, as declared in the source folder's `metadata.json` · **obtained from:** a clone of -`github.com/sentier-dev/sentier-inventory` at commit `6fced18`, dated 2026-09-17 · -**as of:** 2026-09-21. +`github.com/sentier-dev/sentier-inventory` at commit `cf54f55`, dated 2026-10-12, the +head of branch `feat/source-folders` proposed as pull request #5 and not yet merged · +**as of:** 2026-10-12. (source: sentier-inventory-pr-5) ## Install @@ -32,7 +34,7 @@ git clone https://github.com/sentier-dev/sentier-inventory.git ``` ```bash -uv run --with pandas,pyarrow python -c "import pandas as pd; print(pd.read_parquet('data/01-agriculture/processes.parquet'))" +uv run --with pandas,pyarrow python -c "import pandas as pd; print(pd.read_parquet('data/bafu-2026-v1/processes.parquet'))" ``` To install the same data into a Brightway project instead of reading it by hand, use @@ -41,33 +43,38 @@ pinned commit. (source: sentier-inventory-repo) ## What it does -1. **Carries the inventory, organised by sector.** Eleven folders, ranked by a numeric - prefix that also sets resolution precedence. See [datasets.md](datasets.md). +1. **Carries the inventory, organised by source.** One folder per published release of + one datasource, `data/-/`, today only `data/bafu-2026-v1/`. + Every process row carries `source`, `source_version` and `sector`, so a consumer picks + a folder by its metadata and filters rows by sector. There is no rank and no + precedence between folders. See [datasets.md](datasets.md). 2. **Contracts the columns.** Plain YAML per parquet table, plus a JSON Schema for the - per-sector metadata file. See [classes.md](classes.md). + per-source metadata file. See [classes.md](classes.md). 3. **Validates on every pull request.** One self-contained script checks columns, types, - enumerations, declared row counts, primary key uniqueness and the exchange to process - foreign key. See [commands.md](commands.md). + enumerations, declared row counts, primary key uniqueness, the exchange to process + foreign key, one source and one version per folder equal to the metadata and to the + folder name, and the declared sectors against the column. See + [commands.md](commands.md). ## Datasets The repository ships one dataset today: the Swiss federal life cycle inventory database -as the platform harmonises it, spread across eleven sector folders. Broad descriptors -only; the detail is in [datasets.md](datasets.md). +as the platform harmonises it, in one source folder. Broad descriptors only; the detail +is in [datasets.md](datasets.md). | What | Shape | Scale | |---|---|---| -| Processes | `data/-/processes.parquet`, one row per process | 11,947 rows in total | -| Exchanges | `data/-/exchanges.parquet`, one row per technosphere or biosphere edge | about 420,000 rows in total | -| Sector metadata | `data/-/metadata.json` | sector id, title, rank, schema version, row counts and the provenance line | +| Processes | `data/-/processes.parquet`, one row per process, its sector in a column | 11,947 rows | +| Exchanges | `data/-/exchanges.parquet`, one row per technosphere or biosphere edge | 420,063 rows | +| Source metadata | `data/-/metadata.json` | source, source version, title, publisher, licence, citation, schema version, sectors present, row counts | | Schema contract | `schema/*.yaml` and `schema/metadata.schema.json` | four files | -Provenance: every sector's metadata names the same source, the life cycle inventory -database of the Swiss federal administration, BAFU:2026. The database itself is -documented on its own branch, [../../../bafu/](../../../bafu/); this repository +Provenance: the one folder's metadata names its source, the life cycle inventory +database of the Swiss federal administration, BAFU:2026, release v1. The database itself +is documented on its own branch, [../../../bafu/](../../../bafu/); this repository documents only what the platform does with it. Licence: MIT for the repository, with the -citation requirement of the source database carried in the sector metadata. -(source: sentier-inventory-repo) +publisher's terms of use and the citation requirement of the source database carried in +the source metadata. (source: sentier-inventory-pr-5) ## Pages diff --git a/sentier/modules/sentier-inventory/classes.md b/sentier/modules/sentier-inventory/classes.md index 3e46ad3..31964f9 100644 --- a/sentier/modules/sentier-inventory/classes.md +++ b/sentier/modules/sentier-inventory/classes.md @@ -1,17 +1,18 @@ --- title: sentier-inventory schema contracts and validator type: module-page -summary: "The public surface of a data-only repository: three table contracts, one metadata schema, one validation script." +summary: "The public surface of a data-only repository: two table contracts, one metadata schema, one validation script." audience: [P1, P3] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] --- # sentier-inventory schema contracts and validator There is no Python package here, so there are no classes to document. What a consumer depends on is the shape of the tables and the guarantees the validator enforces. Read -from the repository at commit `6fced18`. (source: sentier-inventory-repo) +from the repository at commit `cf54f55`, schema version `0.3.0`, the head of pull +request #5 (not yet merged). (source: sentier-inventory-pr-5) ## The process table @@ -27,13 +28,22 @@ is described inline on the process row; the exchanges live in the other table. | `reference_amount` | double | yes | amount of the reference flow, usually one | | `location` | string | yes | geography code | | `process_type` | enumeration | yes | `unit`, `system` or `lci_result` | +| `sector` | enumeration | yes | the sector the process belongs to, one of the eleven values in `schema/common.yaml`; replaces the former folder level | +| `source` | string | yes | datasource id, lower-kebab `-` such as `bafu-2026`; one value per folder, equal to the metadata | +| `source_version` | string | yes | the publisher's release of that datasource, such as `v1`; one value per folder, equal to the metadata | | `technology` | string | no | free text | | `valid_from` | date | no | date from which the record is valid | | `comment` | string | no | free text | +The `sector` enumeration holds `agriculture`, `electricity`, `chemicals`, `transport`, +`energy`, `waste`, `construction`, `materials`, `electronics`, `building-services` and +`obsolete`. The map from a publisher's own categories onto these values lives with the +importers, not here. (source: sentier-inventory-pr-5) + ## The exchange table `exchanges.parquet`, one row per exchange, joined to the process table on `process_id`. +Provenance is on the parent process row, not repeated here. | Column | Type | Required | What it holds | |---|---|---|---| @@ -48,28 +58,35 @@ is described inline on the process row; the exchanges live in the other table. | `uncertainty_type` | int | no | Brightway uncertainty distribution code, nullable | | `loc`, `scale`, `minimum`, `maximum` | double | no | uncertainty parameters, nullable | -## The sector metadata +## The source metadata -`metadata.json` is validated against `schema/metadata.schema.json`. It requires -`sector`, `title`, `rank` and `schema_version`, and in practice also carries -`description`, which holds the provenance line, and `row_counts`, one integer per -parquet file. Additional properties are allowed. +`metadata.json` describes one source folder and is validated against +`schema/metadata.schema.json`. It requires `source`, `source_version`, `title`, +`publisher`, `license`, `citation`, `schema_version`, `sectors` and `row_counts`, and +may carry `source_url`, `provenance` (the vocabulary's Source IRI of the datasource) +and `retrieved`. `license` is the licence name, URL or terms-of-use summary as the +publisher states it; `citation` is the sentence the publisher requires; `sectors` is the +sorted list of distinct `sector` values present; `row_counts` holds one integer per +parquet file. Additional properties are allowed. The former `sector`, `rank`, +`sources` and `description` keys are gone. (source: sentier-inventory-pr-5) ## What the validator guarantees -`scripts/validate.py` walks every `data/-/` folder and fails the build on -any of the following: +`scripts/validate.py` walks every `data/-/` folder that holds a +`metadata.json` and fails the build on any of the following: | Check | What it means for a consumer | |---|---| -| metadata validates against its JSON Schema | every folder has a sector id, a title, a rank and a schema version | +| metadata validates against its JSON Schema | every folder names its source, version, publisher, licence and citation | | declared `row_counts` match the parquet | the counts in metadata can be trusted without opening the file | | no unknown columns | a reader can rely on the contract being the whole column list | | required columns present and non-null | the required columns never need a null check | | column types match the contract | an all-null optional column passes whatever its physical type, so type-check an optional column yourself | -| enumeration columns hold only allowed values | direction, flow type and process type are safe to switch on | +| enumeration columns hold only allowed values | direction, flow type, process type and sector are safe to switch on | | `process_id` unique per folder | the primary key is a key, within one folder | | every exchange `process_id` resolves in the same folder | no dangling exchanges, and no cross-folder joins are needed | +| one `source` and one `source_version` per folder, equal to the metadata, and the folder named `-` | a folder is one release of one source, and the folder name, the metadata and every row agree | +| distinct `sector` values equal `sectors`, in both directions | a consumer can pick a folder by its metadata alone and know which sectors it will find | The validator exits non-zero and lists every violation it found rather than stopping at the first one. @@ -77,9 +94,13 @@ the first one. ## Notes - Uniqueness and referential integrity are checked **per folder**, not across the - repository. Nothing in the contract promises that a process identifier is unique - across sectors, although the rank rule implies a conflict resolves in favour of the - lower-numbered folder. + repository. Since every folder holds exactly one source and sources never share a + file, nothing in the contract needs to say what happens when two folders overlap: a + consumer reads one folder, and links across sources go through + [../sentier-mappings/](../sentier-mappings/). +- `source` is redundant with the folder name on purpose: a parquet file read outside the + repository still knows where it came from, and the loader in + [../sentier-brightway/](../sentier-brightway/) filters on it as a consistency guard. - Optional columns may be absent from the file entirely. Read defensively. - The contract is plain YAML on purpose. LinkML is used only in [../sentier-vocab/](../sentier-vocab/); the data-layer repositories describe columns diff --git a/sentier/modules/sentier-inventory/commands.md b/sentier/modules/sentier-inventory/commands.md index b08a656..322c33f 100644 --- a/sentier/modules/sentier-inventory/commands.md +++ b/sentier/modules/sentier-inventory/commands.md @@ -1,45 +1,56 @@ --- title: sentier-inventory commands type: module-page -summary: The validator, the inspection one-liner, and what continuous integration runs. +summary: The validator, the inspection one-liners, and what continuous integration runs. audience: [P1] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] --- # sentier-inventory commands There is no command line interface and no package. Two commands are worth knowing, both -taken from the repository README and its workflow file, checked on 2026-09-21 against a -clone at commit `6fced18`. Run them from the repository root. -(source: sentier-inventory-repo) +taken from the repository README and its workflow file, checked on 2026-10-12 against a +clone at commit `cf54f55` (pull request #5, not yet merged). Run them from the repository +root. (source: sentier-inventory-pr-5) | Command | What it does | Key options | |---|---|---| -| `uv run --with "jsonschema[format],pyyaml,pyarrow" python scripts/validate.py` | validate every sector folder against the schema contracts; exits non-zero listing every violation | none; the script takes no arguments | -| `uv run --with pandas,pyarrow python -c "import pandas as pd; print(pd.read_parquet('data/01-agriculture/processes.parquet'))"` | read one delivered table | change the path for another sector or for `exchanges.parquet` | +| `uv run --with "jsonschema[format],pyyaml,pyarrow" python scripts/validate.py` | validate every source folder against the schema contracts; exits non-zero listing every violation | none; the script takes no arguments | +| `uv run --with pandas,pyarrow python -c "import pandas as pd; print(pd.read_parquet('data/bafu-2026-v1/processes.parquet'))"` | read one delivered table | change the path for another source folder or for `exchanges.parquet` | `scripts/validate.py --help` is not available: the script has no argument parser, so there are no flags to verify. Its docstring is the specification and says it is run from the repository root as `python scripts/validate.py`. The workflow installs `jsonschema[format]`, `pyyaml` and `pyarrow` with pip and runs exactly that command on every pull request to the default branch, on every push to any other branch, and on -manual dispatch. (source: sentier-inventory-repo) +manual dispatch. On success it prints one line saying how many source folders validate, +one today. (source: sentier-inventory-pr-5) -## Reading a sector without pandas +## Reading a source folder without pandas ```bash -uv run --with pyarrow python -c "import pyarrow.parquet as pq; t = pq.read_table('data/02-electricity/exchanges.parquet'); print(t.num_rows, t.column_names)" +uv run --with pyarrow python -c "import pyarrow.parquet as pq; t = pq.read_table('data/bafu-2026-v1/exchanges.parquet'); print(t.num_rows, t.column_names)" ``` -## Checking what a sector claims +## Reading one sector + +The sector is a column, so a sector is a filter rather than a path. Parquet predicate +pushdown reads only the matching row groups: + +```bash +uv run --with pandas,pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/bafu-2026-v1/processes.parquet', filters=[('sector', '==', 'electricity')]); print(p.shape)" +``` + +## Checking what a source folder claims ```bash -cat data/05-energy/metadata.json +cat data/bafu-2026-v1/metadata.json ``` -The declared row counts are validated against the files, so reading the metadata is a -cheap way to size a sector before loading it. +The declared row counts and the `sectors` list are validated against the files, so +reading the metadata is a cheap way to size a source and see which sectors it holds +before loading it. ## Notebook entry points diff --git a/sentier/modules/sentier-inventory/datasets.md b/sentier/modules/sentier-inventory/datasets.md index 5022fab..ca37425 100644 --- a/sentier/modules/sentier-inventory/datasets.md +++ b/sentier/modules/sentier-inventory/datasets.md @@ -1,10 +1,10 @@ --- title: sentier-inventory datasets type: module-page -summary: "The BAFU-2026 inventory as the platform ships it: eleven sector folders, two tables per folder, flow identifiers that carry their unit." +summary: "The BAFU-2026 inventory as the platform ships it: one source folder, two tables and a metadata file, the sector as a column, flow identifiers that carry their unit." audience: [P1, P2] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] related: [../../../bafu/README.md, ../sentier-mappings/datasets.md] --- @@ -12,25 +12,27 @@ related: [../../../bafu/README.md, ../sentier-mappings/datasets.md] One dataset, described broadly. No amounts appear on this page: the descriptors say what a table holds and how it is keyed, never what any exchange is worth. Counts are as of -commit `6fced18`, 2026-09-17. (source: sentier-inventory-repo) +commit `cf54f55`, 2026-10-12, the head of pull request #5 (not yet merged). +(source: sentier-inventory-pr-5) ## BAFU-2026 life cycle inventory -- **Version:** schema version `0.1.0`; the source database is named `BAFU:2026` in every - sector's metadata, and the mapping repository refers to the same release as - `bafu-2026-v1`. -- **Sectors:** eleven folders, ranked. Agriculture, electricity, chemicals, transport, - energy carriers, waste management, construction, materials, electronics and building - services, plus a rank-99 obsolete folder holding legacy link targets so that older - references still resolve. +- **Version:** schema version `0.3.0`; the datasource id is `bafu-2026`, the publisher's + release `v1`, so the folder is `data/bafu-2026-v1/`, and the mapping repository refers + to the same release as `bafu-2026-v1` in its pair name. +- **Sectors:** eleven values of the `sector` column on the process table: agriculture, + electricity, chemicals, transport, energy, waste, construction, materials, electronics + and building services, plus `obsolete` for legacy link targets kept so that older + references still resolve. The sector is a row attribute, not a folder. - **Provenance:** the life cycle inventory database of the Swiss federal administration, - BAFU:2026, carried as the `description` of every sector's `metadata.json`. The - database itself, its methodology, its licence and how to obtain it are documented on - [../../../bafu/](../../../bafu/). Which upstream source supplied which rows is - deliberately not recorded here: the inventory is organised by sector, not by source, - and the import log lives with the importers. -- **Schema:** two parquet tables and a metadata file per folder, contracted in - `schema/`. See [classes.md](classes.md) for the columns. + BAFU:2026, named in the folder's `metadata.json` together with the publisher, the + publisher's terms of use, the required citation sentence and the vocabulary's Source + IRI of the datasource. The database itself, its methodology, its licence and how to + obtain it are documented on [../../../bafu/](../../../bafu/). Every row also carries + `source` and `source_version`; which upstream file supplied which row is not recorded + here, that import log lives with the importers. +- **Schema:** two parquet tables and a metadata file in the source folder, contracted + in `schema/`. See [classes.md](classes.md) for the columns. - **Methods:** none. This is inventory only; impact assessment lives in [../sentier-methods/](../sentier-methods/). - **Vocabulary:** process and flow identifiers are the platform's own. Alignment to the @@ -41,33 +43,36 @@ commit `6fced18`, 2026-09-17. (source: sentier-inventory-repo) ### Scale, per sector -Row counts as declared in each sector's `metadata.json`, which the validator checks -against the files. - -| Rank | Sector | Processes | Exchanges | -|---|---|---|---| -| 01 | agriculture | 210 | 7,634 | -| 02 | electricity | 2,033 | 39,133 | -| 03 | chemicals | 510 | 35,566 | -| 04 | transport | 2,017 | 91,780 | -| 05 | energy carriers | 3,188 | 46,489 | -| 06 | waste management | 848 | 25,777 | -| 07 | construction | 927 | 97,366 | -| 08 | materials | 867 | 32,386 | -| 09 | electronics | 300 | 4,553 | -| 10 | building services | 268 | 11,887 | -| 99 | obsolete | 779 | 27,492 | - -That is 11,947 processes and about 420,000 exchanges. Every process in the release is a -unit process: the `process_type` column holds only `unit` today, although the contract -also allows system processes and aggregated inventory results. +Row counts per value of the `sector` column, read from the parquet. The folder's +`metadata.json` declares the totals and the list of sectors present, and the validator +checks both against the files. + +| Sector | Processes | Exchanges | +|---|---|---| +| agriculture | 210 | 7,634 | +| electricity | 2,033 | 39,133 | +| chemicals | 510 | 35,566 | +| transport | 2,017 | 91,780 | +| energy | 3,188 | 46,489 | +| waste | 848 | 25,777 | +| construction | 927 | 97,366 | +| materials | 867 | 32,386 | +| electronics | 300 | 4,553 | +| building-services | 268 | 11,887 | +| obsolete | 779 | 27,492 | + +That is 11,947 processes and 420,063 exchanges, the same rows as before the layout +change. Every process in the release is a unit process: the `process_type` column holds +only `unit` today, although the contract also allows system processes and aggregated +inventory results. ### What a process row holds An identifier, a name, the reference product and its unit and amount, a geography code, -the process type, and optional technology text, validity date and comment. The reference -product is the technosphere match key: two processes can share a name and produce -different products, so linking is done on the product. +the process type, the sector, the datasource id and its version, and optional +technology text, validity date and comment. The reference product is the technosphere +match key: two processes can share a name and produce different products, so linking is +done on the product. ### What an exchange row holds @@ -79,10 +84,10 @@ the Brightway convention. Uncertainty is sparse: most rows carry no distribution ### Flow identifiers carry their unit Flow identifiers are deterministic name-based UUIDs rather than readable codes, so the -same substance in the same context always gets the same identifier. Across the eleven -sector folders there are about 14,600 distinct flow identifiers, and all but a handful -appear with exactly one unit, which means the unit is a property of the identifier -rather than of the row. Two consequences: +same substance in the same context always gets the same identifier. Across the release +there are about 14,600 distinct flow identifiers, and all but a handful appear with +exactly one unit, which means the unit is a property of the identifier rather than of +the row. Two consequences: - the same flow name can map to several identifiers, because compartment, sub-compartment and unit distinguish them; do not group by name; @@ -97,10 +102,12 @@ overlap at all. The bridge is the `bafu-2026-v1__ef-3.1` pair in [../sentier-mappings/](../sentier-mappings/), which relinks inventory flows onto EF 3.1 flow keys before any characterisation happens. -## Raw sector deliveries +## Deliveries -The repository also documents, in `data/README.md`, that a sector folder that has not -been populated yet ships only a `.gitkeep` and a metadata stub. Every folder in this -release carries real parquet. Parquet is committed directly: no git-LFS and no release -artifacts, by policy, so a file that would be too large is sharded or trimmed upstream -instead. +A delivery from [../sentier-importers/](../sentier-importers/) replaces its source +folder whole: the two parquet files and the metadata are copied over whatever the folder +held. Sources never share a file, so no merge step exists or is needed, and a second +datasource arrives as its own folder next to `bafu-2026-v1/`. A new release of the same +source is a new folder, `-`. Parquet is committed directly: no +git-LFS and no release artifacts, by policy, so a file that would be too large is +sharded or trimmed upstream instead. (source: sentier-inventory-pr-5) diff --git a/sentier/modules/sentier-inventory/gotchas.md b/sentier/modules/sentier-inventory/gotchas.md index b71a560..e061b28 100644 --- a/sentier/modules/sentier-inventory/gotchas.md +++ b/sentier/modules/sentier-inventory/gotchas.md @@ -1,45 +1,61 @@ --- title: sentier-inventory gotchas type: module-page -summary: "What surprises people: the obsolete folder, per-folder guarantees, absent optional columns and flow identifiers that match nothing else." +summary: "What surprises people: a layout change pending merge, the obsolete sector, per-folder guarantees, absent optional columns and flow identifiers that match nothing else." audience: [P1, P3] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5, sentier-importers-pr-29, sentier-brightway-pr-3] --- # sentier-inventory gotchas Each bullet says what happens, why, and what to do instead. Checked against the -repository at commit `6fced18`, 2026-09-17. (source: sentier-inventory-repo) +repository at commit `cf54f55`, 2026-10-12. (source: sentier-inventory-pr-5) -- **The rank-99 folder is not a sector.** `data/99-obsolete/` holds legacy link targets - kept so older references resolve. It carries 779 processes and about 27,000 exchanges, - and loading every folder blindly puts them in the same database as the current ones. - Decide explicitly whether you want it. +- **The layout changed on 2026-10-12, and the change is still on open pull requests.** + Until then `data/` held eleven sector folders, `01-agriculture` through + `10-building-services` plus `99-obsolete`, and the sector was the folder. Now `data/` + holds one folder per published release of one datasource, `data/-/`, + and the sector is a required column. The reason: sector parquet from several sources + did not stack. A delivery copied its sector file over the existing one, so the first + delivery of a second source into an existing sector folder would have erased the rows + already there. The change is pull request #5 here, #29 on the importers and #3 on the + loader, all open and pending merge as of 2026-10-12; a clone of the default branch + before the merge still shows the sector folders. (source: sentier-inventory-pr-5, + sentier-importers-pr-29, sentier-brightway-pr-3) -- **The guarantees are per folder, not per repository.** Primary key uniqueness and the - exchange to process foreign key are checked inside one sector folder. Nothing in the - contract says a process identifier is unique across sectors, and nothing prevents an - exchange in one folder from naming a flow that only exists in another. The rank prefix - is the documented tie break: lower wins when records overlap. +- **`obsolete` is a sector value, not a sector.** The 779 processes and about 27,000 + exchanges with `sector = obsolete` are legacy link targets kept so older references + resolve. They sit in the same folder as the current rows, so loading the folder + blindly puts them in the same database. Filter on the column if you do not want them, + and know that exchanges in other sectors may point at them: the loader in + [../sentier-brightway/](../sentier-brightway/) keeps them for exactly that reason. + (source: sentier-brightway-pr-3) -- **Nothing enforces the rank.** The rank is a number in the metadata and a prefix on - the folder name. No code in this repository applies it. A consumer that merges sectors - has to implement the precedence itself. +- **The guarantees are per folder, and a folder is one source.** Primary key uniqueness + and the exchange to process foreign key are checked inside one source folder. A + process identifier is not promised to be unique across sources, and a link from one + source to another is not a parquet join: it goes through + [../sentier-mappings/](../sentier-mappings/). + +- **There is no precedence between folders.** The rank prefix and its "lower wins" rule + are gone with the sector folders. A consumer names the source it wants, and the + version when two releases of one source are served side by side; nothing in the + repository decides for it. - **An optional column can be missing entirely.** The contract lists `valid_from` on the - process table, but no sector folder in this release actually carries that column. The - validator accepts a missing optional column, so read with a column check rather than - by position, and never assume a column exists because the schema mentions it. + process table, but the delivered table does not carry that column. The validator + accepts a missing optional column, so read with a column check rather than by position, + and never assume a column exists because the schema mentions it. - **An all-null column passes the type check.** The validator skips the type check for a column whose values are all null, because an all-null column carries no typed values to disagree with the contract. A physically mistyped but empty column therefore reaches you unflagged. -- **Uncertainty is sparse.** The five uncertainty columns are present in every sector but - are null for the large majority of rows. Treat an exchange as deterministic unless - `uncertainty_type` says otherwise, and do not read a null `scale` as zero. +- **Uncertainty is sparse.** The five uncertainty columns are present in the delivered + table but are null for the large majority of rows. Treat an exchange as deterministic + unless `uncertainty_type` says otherwise, and do not read a null `scale` as zero. - **Biosphere flow identifiers match nothing outside this repository.** They are deterministic name-based UUIDs from the delivery, and they do not overlap with the flow @@ -54,10 +70,11 @@ repository at commit `6fced18`, 2026-09-17. (source: sentier-inventory-repo) - **Link on the reference product, not on the process name.** The schema says so explicitly: the reference product is the technosphere match key. -- **The source of a row is not in the row.** Inventory is organised by sector rather than - by source, so no column says which upstream database a process came from. The sector - metadata names the source database for the whole folder, and the per-row import log - lives with the importers. +- **The source of a row is in the row, and in the folder name, and they must agree.** + `source` and `source_version` are required columns with exactly one value each per + folder, pinned by the validator to the metadata and to the folder name. What is not + in the row is which upstream file produced it; that import log lives with the + importers. - **There is no loader here, by design.** The repository is artifacts only. Writing a reader against the parquet is fine, but the supported path into an LCA tool is diff --git a/sentier/modules/sentier-inventory/layout.md b/sentier/modules/sentier-inventory/layout.md index cc1d077..75358e7 100644 --- a/sentier/modules/sentier-inventory/layout.md +++ b/sentier/modules/sentier-inventory/layout.md @@ -1,50 +1,42 @@ --- title: sentier-inventory layout type: module-page -summary: "The repository tree: eleven sector folders, four schema files, one validator." +summary: "The repository tree: one source folder, four schema files, one validator." audience: [P2, P3] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] --- # sentier-inventory layout -The whole repository, as of commit `6fced18` dated 2026-09-17. It is small: the data is -the repository. (source: sentier-inventory-repo) +The whole repository, as of commit `cf54f55` dated 2026-10-12, the head of pull request +#5 (branch `feat/source-folders`, not yet merged). It is small: the data is the +repository. (source: sentier-inventory-pr-5) ``` -data/ the inventory, one folder per sector - README.md the folder convention and why sectors, not sources - 01-agriculture/ rank 1, highest resolution precedence - 02-electricity/ - 03-chemicals/ - 04-transport/ - 05-energy/ - 06-waste/ - 07-construction/ - 08-materials/ - 09-electronics/ - 10-building-services/ - 99-obsolete/ legacy link targets kept so older references resolve +data/ the inventory, one folder per published release of one datasource + README.md the folder convention, the source id convention, why sources and not sectors + bafu-2026-v1/ BAFU:2026 v1, the only source folder today schema/ the data contract the importers validate against README.md how the plain-YAML column definitions work - common.yaml shared enumerations: flow direction, flow type, process type + common.yaml shared enumerations: flow direction, flow type, process type, sector process.yaml the columns of processes.parquet, primary key process_id exchange.yaml the columns of exchanges.parquet, joined on process_id - metadata.schema.json JSON Schema for a sector's metadata.json + metadata.schema.json JSON Schema for a source folder's metadata.json scripts/validate.py the CI validator, self-contained, no repository imports .github/workflows/ci.yml runs the validator on every pull request and branch push LICENSE MIT README.md what the repository is, how to read and validate it ``` -Every sector folder has exactly the same three files: +Every source folder has exactly the same three files: ``` -data/-/ - processes.parquet one row per process +data/-/ + processes.parquet one row per process, its sector in the sector column exchanges.parquet one row per exchange, technosphere and biosphere - metadata.json sector, title, rank, schema_version, row_counts, provenance + metadata.json source, source_version, title, publisher, license, citation, + source_url, provenance, retrieved, schema_version, sectors, row_counts ``` ## Key files @@ -52,17 +44,36 @@ data/-/ - `schema/process.yaml` and `schema/exchange.yaml` are the contract. They are read by the validator here and by the importers before they open a delivery pull request, so a column added on one side without the other fails. -- `schema/common.yaml` holds the three enumerations. A value outside them fails - validation, which is how a mis-typed direction or flow type is caught. -- `data/-/metadata.json` is the only place the numeric rank, the declared - row counts and the source citation live. +- `schema/common.yaml` holds the four enumerations, `sector` among them. A value outside + them fails validation, which is how a mis-typed direction, flow type or sector is + caught. Adding a sector is a schema pull request, as for the other enumerations. +- `data/-/metadata.json` is the only place the publisher, the + licence terms, the required citation, the declared row counts and the list of sectors + present live. The validator pins it to the parquet: one `source` and one + `source_version` per folder, equal to the metadata and to the folder name, and the + distinct `sector` values equal to `sectors`. - `scripts/validate.py` imports nothing from the repository and depends only on jsonschema, pyyaml and pyarrow, so it can be run from a bare checkout. ## Naming -The folder name is `-`, with `` a lower-kebab identifier and `` -the rank. The rank orders the folders and decides precedence when records overlap: lower -wins. The rank prefix is not part of the sector id stored inside the data. A background -database that exists only as a link target, such as ecoinvent, is not a sector and gets -no folder; it is resolved through [../sentier-mappings/](../sentier-mappings/). +The folder name is `-`. `` is the lower-kebab +datasource id `-` (`bafu-2026`), the same slug as the vocabulary's +Source IRI and the mapping repository's pair-folder prefix; `` is the +publisher's release (`v1`), never part of the id, so a re-release does not rename the +database a consumer's project depends on. A folder holds exactly one source and one +version. A delivery from the importers replaces the folder whole, sources never share a +file, and nothing is ever merged. A re-release is a new folder, and the pull request that +adds it removes the old one unless both are meant to be served. There is no rank prefix +and no precedence between folders: a consumer picks a source explicitly. A background +database that exists only as a link target, such as ecoinvent, is not a source here and +gets no folder; it is resolved through [../sentier-mappings/](../sentier-mappings/). +(source: sentier-inventory-pr-5) + +Until 2026-10-12 the same rows lived in eleven sector folders, `data/-/`, +ranked by a numeric prefix. That layout was dropped because sector parquet from several +sources did not stack: a delivery copied its sector file over the existing one, so the +first delivery of a second source into an existing sector folder would have erased the +rows already there. The sector moved into a column of the process table, and the older +"by sector, not by source" rule now applies to the vocabulary repository only. +(source: sentier-inventory-pr-5) diff --git a/sentier/modules/sentier-inventory/status.md b/sentier/modules/sentier-inventory/status.md index ddc0fc6..55d6aba 100644 --- a/sentier/modules/sentier-inventory/status.md +++ b/sentier/modules/sentier-inventory/status.md @@ -1,50 +1,65 @@ --- title: sentier-inventory status type: module-page -summary: "Where the inventory repository stands: one source database, eleven sectors, unit processes only." +summary: "Where the inventory repository stands: one source folder, schema 0.3.0 on an open pull request, unit processes only." audience: [P2] -updated: 2026-09-21 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5, sentier-importers-pr-29, sentier-brightway-pr-3] --- # sentier-inventory status -As of commit `6fced18`, dated 2026-09-17. (source: sentier-inventory-repo) +As of commit `cf54f55`, dated 2026-10-12, the head of pull request #5 (branch +`feat/source-folders`), which is open and not yet merged. (source: sentier-inventory-pr-5) ## Works today -- Eleven populated sector folders, 11,947 processes and about 420,000 exchanges, all - validating against the schema contract. +- One source folder, `data/bafu-2026-v1/`, with 11,947 processes and 420,063 exchanges, + validating against the schema contract at version `0.3.0`. - A complete, self-contained validator that continuous integration runs on every pull - request and on every push to a non-default branch. -- A stable delivery convention: importers open pull requests, nothing is hand-edited, - and the metadata declares row counts that are checked against the files. + request and on every push to a non-default branch, now also checking that a folder + holds one source and one version matching its name, and that the declared sectors + match the `sector` column. +- A stable delivery convention: importers open pull requests, nothing is hand-edited, a + delivery replaces its source folder whole, and the metadata declares row counts and + sectors that are checked against the files. - Downstream consumption works: the loader documented in - [../sentier-brightway/](../sentier-brightway/) reads every sector folder at a pinned - commit. + [../sentier-brightway/](../sentier-brightway/) opens the one folder matching a source + at a pinned commit, and verified the same 11,947 processes and the same bridge coverage + as before the layout change. (source: sentier-brightway-pr-3) ## In flight -- **One source, many sectors.** The release carries the Swiss federal inventory only. - The folder convention is built for several sources spanning the same sectors, and the - sector metadata already separates the provenance line from the sector identity. -- **Schema version `0.1.0`.** The contract is explicitly young: the enumerations allow - system processes and aggregated inventory results that no delivered row uses yet. +- **The source-folder layout, on three open pull requests.** Pull request #5 here moves + the data from eleven sector folders into `data/bafu-2026-v1/`, adds the `sector` + column and the per-source metadata, and raises the schema to `0.3.0`; #29 on + [../sentier-importers/](../sentier-importers/) collapses the 22 per-sector registry + blocks into two and emits the sector as a column; #3 on + [../sentier-brightway/](../sentier-brightway/) reads one source folder and re-pins the + inventory to `cf54f55`. The trigger was a question on 2026-10-12 about whether sector + parquet from several sources would stack: it would not, because a delivery overwrote + the sector file whole. All three are pending merge as of 2026-10-12, so the default + branches still show the old layout. (source: sentier-inventory-pr-5, + sentier-importers-pr-29, sentier-brightway-pr-3) +- **Schema version `0.3.0`.** The contract is still young: the enumerations allow + system processes and aggregated inventory results that no delivered row uses yet, and + only one source exercises the one-folder-per-source rule. ## Missing - No aggregated results and no system processes: every delivered process is a unit process. -- No `valid_from` dates in the delivered tables, although the contract defines the +- No `valid_from` dates in the delivered table, although the contract defines the + column. +- No second datasource. The layout is built for several sources side by side, each in + its own folder, but only BAFU:2026 v1 has been delivered. +- No per-row upstream provenance beyond `source` and `source_version`. Tracing a row + back to its upstream record means going to the importers' log rather than reading a column. -- No documented precedence implementation. The rank rule is stated in prose and applied - by consumers. -- No per-row provenance. Tracing a row back to its upstream record means going to the - importers' log rather than reading a column. - Uncertainty is present in the contract and largely absent from the data. ## Where help is welcome -The two tractable contributions are a documented, tested precedence helper for merging -sector folders, and populating the optional columns the contract already defines. +The two tractable contributions are delivering a second datasource into its own folder +through the importers, and populating the optional columns the contract already defines. Anything bigger belongs on [../../roadmap.md](../../roadmap.md). diff --git a/sentier/modules/sentier-inventory/use-cases/README.md b/sentier/modules/sentier-inventory/use-cases/README.md index d86df5e..08a9e4d 100644 --- a/sentier/modules/sentier-inventory/use-cases/README.md +++ b/sentier/modules/sentier-inventory/use-cases/README.md @@ -3,7 +3,7 @@ title: sentier-inventory use cases type: readme summary: End-to-end things a practitioner does with the Sentier inventory data. audience: [P1] -updated: 2026-09-22 +updated: 2026-10-12 sources: [] --- @@ -14,18 +14,17 @@ One file per use case, copied from ## Pages here -- [read-a-sector-folder-with-pandas.md](read-a-sector-folder-with-pandas.md): one sector's processes and exchanges in a dataframe. +- [read-a-source-folder-with-pandas.md](read-a-source-folder-with-pandas.md): one source's processes and exchanges in a dataframe, one sector filtered out of the column. - [validate-the-inventory-schema.md](validate-the-inventory-schema.md): run the repository's own validator, as CI does. -Further candidates, in the order they would help a reader: reading one sector into a -dataframe, validating a delivery before opening a pull request, merging two sector -folders under the rank rule, and finding the processes that produce a given reference -product. +Further candidates, in the order they would help a reader: validating a delivery before +opening a pull request, comparing two releases of one source side by side, and finding +the processes that produce a given reference product. ## Naming rule `lower-case-with-hyphens.md`, verb first, naming the outcome rather than the function -called: `read-a-sector-into-a-dataframe.md`, not `run-pyarrow.md`. +called: `read-a-source-into-a-dataframe.md`, not `run-pyarrow.md`. ## How to add here diff --git a/sentier/modules/sentier-inventory/use-cases/read-a-sector-folder-with-pandas.md b/sentier/modules/sentier-inventory/use-cases/read-a-sector-folder-with-pandas.md deleted file mode 100644 index fba9d45..0000000 --- a/sentier/modules/sentier-inventory/use-cases/read-a-sector-folder-with-pandas.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -title: Read a sector folder with pandas -type: use-case -summary: "Open one sector's processes and exchanges parquet in a dataframe, check them against the folder's declared row counts, and see the contracted columns." -audience: [P1] -updated: 2026-09-22 -sources: [sentier-inventory-repo] -verified: yes -verified_on: 2026-09-22 -verified_env: "uv 0.11.14, Python 3.11.15, Ubuntu 22.04, a clone of sentier-inventory at schema version 0.1.0, pandas and pyarrow via uv --with" ---- - -# Read a sector folder with pandas - -**Persona:** P1 practitioner who wants the inventory as data, not inside an LCA tool · -**GICS sector:** any; the folders are the database's own sectors, not GICS ones - -## Goal - -One [sector folder](../../../../vocabulary.md#sector-folder) in two dataframes, the -processes and their [exchanges](../../../../vocabulary.md#exchange), with the column -contract in front of you and the folder's own declared row counts as a check. - -## Prerequisites - -- A clone of sentier-inventory. There is no package and nothing to install: the - repository is data plus one validator. -- `uv`, and pandas and pyarrow, which `uv run --with` supplies for the length of the - command. - -## Steps - -1. See which sectors exist and in which order they rank: - - ```bash - ls data/ - ``` - -2. Size a sector cheaply, before loading it, by reading what it declares: - - ```bash - cat data/02-electricity/metadata.json - ``` - -3. Load the two tables: - - ```bash - uv run --with pandas --with pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/02-electricity/processes.parquet'); e = pd.read_parquet('data/02-electricity/exchanges.parquet'); print(p.shape, e.shape); print(list(p.columns)); print(list(e.columns))" - ``` - -4. Join them, which is what you are usually after: every exchange of one process, with - the process named: - - ```bash - uv run --with pandas --with pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/02-electricity/processes.parquet'); e = pd.read_parquet('data/02-electricity/exchanges.parquet'); print(e.merge(p[['process_id','name']], on='process_id').head().to_string())" - ``` - -5. If you do not want pandas, pyarrow alone is enough and reads only the columns you ask - for: - - ```bash - uv run --with pyarrow python -c "import pyarrow.parquet as pq; t = pq.read_table('data/02-electricity/exchanges.parquet', columns=['process_id','flow_name','direction','unit']); print(t.num_rows, t.column_names)" - ``` - -## Expected output - -Step 1 lists eleven folders with a numeric prefix: agriculture, electricity, chemicals, -transport, energy, waste, construction, materials, electronics, building services, and an -obsolete folder kept for completeness. The prefix is a rank, and it also sets resolution -precedence. - -Step 2 prints the folder's [metadata](../../../../vocabulary.md#metadata): the sector key, -a title and description, the rank, the [schema](../../../../vocabulary.md#schema) version -and the declared row counts. Those counts are validated against the files, so they are a -cheap and trustworthy size estimate. - -Step 3 gives the two shapes and the contracted columns. A process row carries its -identifier, name, [reference product](../../../../vocabulary.md#reference-product) and -reference unit and amount, its [geography](../../../../vocabulary.md#geography), its -process type, a technology description and a comment. An exchange row carries the process -identifier, the flow identifier and name, the flow type, the direction, the amount, the -unit, a location and five [uncertainty](../../../../vocabulary.md#uncertainty) columns. -Electricity, for scale, is roughly two thousand processes and about forty thousand -exchanges. - -Step 4 shows the join key is the process identifier on both sides. Step 5 is the same -data with a smaller memory footprint. - -## Pitfalls - -- **Read a sector, not the repository.** Loading every folder at once is rarely what you - want and is a lot of memory. Start from the metadata, pick the sector, then load. -- **The obsolete folder is in the list.** It is kept for completeness, not for use. Filter - it out unless you know why you want it. -- **A flow identifier is not resolvable here.** The exchange rows carry cross-source flow - identifiers; resolving them onto an impact assessment - [nomenclature](../../../../vocabulary.md#nomenclature) is the job of - [../../sentier-mappings/](../../sentier-mappings/), and the loader in - [../../sentier-brightway/](../../sentier-brightway/) does it for you. -- **A production row is typed, not inferred.** Do not identify the reference exchange by - the flow and the process sharing an identifier: the inventory contains technosphere - self-loops where a process consumes its own reference product, and the equality - misreads them. Use the exchange type. -- **The uncertainty columns are often empty.** Their presence is a schema promise, not a - guarantee that every row is characterised. - -## Related vocabulary - -[Sector folder](../../../../vocabulary.md#sector-folder), -[process](../../../../vocabulary.md#process), -[exchange](../../../../vocabulary.md#exchange), -[reference product](../../../../vocabulary.md#reference-product), -[flow code](../../../../vocabulary.md#flow-code), -[uncertainty](../../../../vocabulary.md#uncertainty), -[metadata](../../../../vocabulary.md#metadata), -[obsolete category](../../../../vocabulary.md#obsolete-category). - -## Verification - -Run on 2026-09-22 on Ubuntu 22.04 with uv 0.11.14 and Python 3.11.15, in a clone of -sentier-inventory at schema version 0.1.0. Steps 1, 2, 3 and 5 were executed as written -and took 3.8 s together including the dependency resolution. The electricity folder -loaded as 2,033 processes by 9 columns and 39,133 exchanges by 13 columns, with the -column names as described, and `data/` held the eleven folders listed above. Step 4 was -not run; it is the same two reads plus a dataframe merge. diff --git a/sentier/modules/sentier-inventory/use-cases/read-a-source-folder-with-pandas.md b/sentier/modules/sentier-inventory/use-cases/read-a-source-folder-with-pandas.md new file mode 100644 index 0000000..85d9ea9 --- /dev/null +++ b/sentier/modules/sentier-inventory/use-cases/read-a-source-folder-with-pandas.md @@ -0,0 +1,147 @@ +--- +title: Read a source folder with pandas +type: use-case +summary: "Open one source's processes and exchanges parquet in a dataframe, check them against the folder's declared row counts and sectors, filter one sector out of the column, and see the contracted columns." +audience: [P1] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] +verified: yes +verified_on: 2026-10-12 +verified_env: "uv 0.11.14, Python 3.9.25 as picked by uv, Ubuntu 22.04, a clone of sentier-inventory at commit cf54f55 (schema version 0.3.0, pull request #5), pandas and pyarrow via uv --with" +--- + +# Read a source folder with pandas + +**Persona:** P1 practitioner who wants the inventory as data, not inside an LCA tool · +**GICS sector:** any; the `sector` column holds the database's own sectors, not GICS ones + +## Goal + +One [source folder](../../../../vocabulary.md#source-folder) in two dataframes, the +processes and their [exchanges](../../../../vocabulary.md#exchange), with the column +contract in front of you, the folder's own declared row counts and sectors as a check, +and one sector picked out of the `sector` column. + +## Prerequisites + +- A clone of sentier-inventory. There is no package and nothing to install: the + repository is data plus one validator. +- `uv`, and pandas and pyarrow, which `uv run --with` supplies for the length of the + command. + +## Steps + +1. See which source releases exist, one folder each: + + ```bash + ls data/ + ``` + +2. Size a source cheaply, before loading it, by reading what it declares: + + ```bash + cat data/bafu-2026-v1/metadata.json + ``` + +3. Load the two tables: + + ```bash + uv run --with pandas --with pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/bafu-2026-v1/processes.parquet'); e = pd.read_parquet('data/bafu-2026-v1/exchanges.parquet'); print(p.shape, e.shape); print(list(p.columns)); print(list(e.columns)); print(p['sector'].value_counts().sort_index().to_string())" + ``` + +4. Pick one sector. The sector is a column, so it is a filter, and parquet predicate + pushdown reads only the matching row groups; the exchanges follow their processes: + + ```bash + uv run --with pandas --with pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/bafu-2026-v1/processes.parquet', filters=[('sector', '==', 'electricity')]); e = pd.read_parquet('data/bafu-2026-v1/exchanges.parquet', filters=[('process_id', 'in', p['process_id'].tolist())]); print(p.shape, e.shape)" + ``` + +5. Join them, which is what you are usually after: every exchange of one process, with + the process named and its sector beside it: + + ```bash + uv run --with pandas --with pyarrow python -c "import pandas as pd; p = pd.read_parquet('data/bafu-2026-v1/processes.parquet'); e = pd.read_parquet('data/bafu-2026-v1/exchanges.parquet'); print(e.merge(p[['process_id','name','sector']], on='process_id').head().to_string())" + ``` + +6. If you do not want pandas, pyarrow alone is enough and reads only the columns you ask + for: + + ```bash + uv run --with pyarrow python -c "import pyarrow.parquet as pq; t = pq.read_table('data/bafu-2026-v1/exchanges.parquet', columns=['process_id','flow_name','direction','unit']); print(t.num_rows, t.column_names)" + ``` + +## Expected output + +Step 1 lists one folder per published release of one datasource, named +`-`, plus the folder's own README. Today that is a single +folder, `bafu-2026-v1`. There is no rank and no precedence between folders; a second +source would appear as its own folder beside it. + +Step 2 prints the folder's [metadata](../../../../vocabulary.md#metadata): the source id +and version, a title, the publisher, the licence terms, the required citation, the +[schema](../../../../vocabulary.md#schema) version, the sorted list of sectors present and +the declared row counts. Counts and sectors are validated against the files, so they are +a cheap and trustworthy size estimate. + +Step 3 gives the two shapes, the contracted columns and the process count per sector. A +process row carries its identifier, name, [reference +product](../../../../vocabulary.md#reference-product) and reference unit and amount, its +[geography](../../../../vocabulary.md#geography), its process type, its sector, its source +and source version, a technology description and a comment. An exchange row carries the +process identifier, the flow identifier and name, the flow type, the direction, the +amount, the unit, a location and five [uncertainty](../../../../vocabulary.md#uncertainty) +columns. The whole release is 11,947 processes and 420,063 exchanges; the eleven sector +values run from 210 processes for agriculture to 3,188 for energy. + +Step 4 gives electricity, for scale: roughly two thousand processes and about forty +thousand exchanges, the same rows the former `02-electricity` folder held. + +Step 5 shows the join key is the process identifier on both sides. Step 6 is the same +data with a smaller memory footprint. + +## Pitfalls + +- **Filter on the column, do not look for a sector folder.** Before 2026-10-12 the + repository held one folder per sector; a path such as `data/02-electricity/` no longer + exists. The folder is the source, the sector is `processes.sector`. +- **The obsolete rows are in the folder.** `sector = obsolete` marks legacy link targets + kept for completeness, not for use. Filter them out unless you know why you want them, + and know that exchanges in other sectors may point at them. +- **Two releases of one source are two folders.** Pick the one whose metadata says the + `source_version` you mean; nothing picks for you. +- **A flow identifier is not resolvable here.** The exchange rows carry cross-source flow + identifiers; resolving them onto an impact assessment + [nomenclature](../../../../vocabulary.md#nomenclature) is the job of + [../../sentier-mappings/](../../sentier-mappings/), and the loader in + [../../sentier-brightway/](../../sentier-brightway/) does it for you. +- **A production row is typed, not inferred.** Do not identify the reference exchange by + the flow and the process sharing an identifier: the inventory contains technosphere + self-loops where a process consumes its own reference product, and the equality + misreads them. Use the exchange type. +- **The uncertainty columns are often empty.** Their presence is a schema promise, not a + guarantee that every row is characterised. + +## Related vocabulary + +[Source folder](../../../../vocabulary.md#source-folder), +[sector folder](../../../../vocabulary.md#sector-folder), +[process](../../../../vocabulary.md#process), +[exchange](../../../../vocabulary.md#exchange), +[reference product](../../../../vocabulary.md#reference-product), +[flow code](../../../../vocabulary.md#flow-code), +[uncertainty](../../../../vocabulary.md#uncertainty), +[metadata](../../../../vocabulary.md#metadata), +[obsolete category](../../../../vocabulary.md#obsolete-category). + +## Verification + +Run on 2026-10-12 on Ubuntu 22.04 with uv 0.11.14 and the Python 3.9.25 that uv picked, +in a clone of sentier-inventory at commit `cf54f55`, schema version 0.3.0. Steps 1, 2, +3, 4 and 6 were executed as written and took 1.0 s together with the dependencies +already cached. The folder loaded as 11,947 processes by 12 columns and 420,063 exchanges +by 13 columns, with the column names as described and the eleven sector values counted +as on [../datasets.md](../datasets.md); the electricity filter returned 2,033 processes +and 39,133 exchanges; `data/` held the one folder `bafu-2026-v1` and a README. Step 5 +was not run; it is the same two reads plus a dataframe merge. A first run on 2026-09-22 +against the former sector layout (schema 0.1.0) had loaded the `02-electricity` folder +to the same two counts. diff --git a/sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md b/sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md index 5a4a687..14b4f74 100644 --- a/sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md +++ b/sentier/modules/sentier-inventory/use-cases/validate-the-inventory-schema.md @@ -1,13 +1,13 @@ --- title: Validate the inventory schema type: use-case -summary: "Run the repository's own validator over every sector folder, the same command continuous integration runs on every pull request." +summary: "Run the repository's own validator over every source folder, the same command continuous integration runs on every pull request." audience: [P2] -updated: 2026-09-22 -sources: [sentier-inventory-repo] +updated: 2026-10-12 +sources: [sentier-inventory-repo, sentier-inventory-pr-5] verified: yes -verified_on: 2026-09-22 -verified_env: "uv 0.11.14, Python 3.11.15, Ubuntu 22.04, a clone of sentier-inventory at schema version 0.1.0, dependencies via uv --with" +verified_on: 2026-10-12 +verified_env: "uv 0.11.14, Python 3.9.25 as picked by uv, Ubuntu 22.04, a clone of sentier-inventory at commit cf54f55 (schema version 0.3.0, pull request #5), dependencies via uv --with" --- # Validate the inventory schema @@ -42,23 +42,25 @@ continuous integration uses, before a pull request rather than after it. ``` 3. When it fails on a delivery you are preparing, compare the folder's declared counts - with the files themselves: + and sectors with the files themselves: ```bash - cat data//metadata.json + cat data/-/metadata.json ``` ## Expected output -On success, one line saying how many [sector folders](../../../../vocabulary.md#sector-folder) +On success, one line saying how many [source folders](../../../../vocabulary.md#source-folder) validate against the [schema](../../../../vocabulary.md#schema) contracts, and exit status -zero. There are eleven folders today. +zero. There is one folder today, `bafu-2026-v1`. On failure, exit status non-zero and one line per violation, each naming the folder and what was wrong. The contracts it checks are the presence and shape of each folder's [metadata](../../../../vocabulary.md#metadata), the declared schema version, the column -set and types of the two parquet tables, and the agreement between the declared row -counts and the files. +set, types and enumerations of the two parquet tables, the agreement between the declared +row counts and the files, that `source` and `source_version` each hold one value equal +to the metadata and to the folder name, and that the distinct `sector` values equal the +declared `sectors` list. ## Pitfalls @@ -69,9 +71,13 @@ counts and the files. skipped. - **A passing validator is not a correct inventory.** It checks shape, not content. A process whose amounts are wrong, or whose flow identifiers point nowhere useful, passes. -- **Row counts are part of the contract.** Editing a parquet file without updating the - folder's declared counts fails, which is deliberate: the counts are how a reviewer sizes - a delivery without loading it. +- **Row counts and sectors are part of the contract.** Editing a parquet file without + updating the folder's declared counts or its `sectors` list fails, which is deliberate: + the metadata is how a reviewer sizes a delivery, and sees which sectors it holds, + without loading it. +- **The folder name is part of the contract too.** A folder named anything other than + `-` as its metadata and rows declare them fails, so a release + cannot be delivered under the wrong name. - **The same pattern, different dependencies, in the sibling repositories.** The methods and mappings repositories each carry their own `scripts/validate.py` with their own dependency list; do not copy this command between them. @@ -79,7 +85,7 @@ counts and the files. ## Related vocabulary [Schema](../../../../vocabulary.md#schema), -[sector folder](../../../../vocabulary.md#sector-folder), +[source folder](../../../../vocabulary.md#source-folder), [metadata](../../../../vocabulary.md#metadata), [data format](../../../../vocabulary.md#data-format), [process](../../../../vocabulary.md#process), @@ -87,7 +93,9 @@ counts and the files. ## Verification -Run on 2026-09-22 on Ubuntu 22.04 with uv 0.11.14 and Python 3.11.15, in a clone of -sentier-inventory at schema version 0.1.0. Steps 1 and 2 were executed as written and took -1.4 s; the validator reported that all eleven sector folders validate against the schema -contracts and exited zero. Step 3 was not needed, because nothing failed. +Run on 2026-10-12 on Ubuntu 22.04 with uv 0.11.14 and the Python 3.9.25 that uv picked, +in a clone of sentier-inventory at commit `cf54f55`, schema version 0.3.0. Steps 1 and 2 +were executed as written and took 0.3 s with the dependencies already cached; the +validator reported that one source folder validates against the schema contracts and +exited zero. Step 3 was not needed, because nothing failed. A first run on 2026-09-22 +against the former layout (schema 0.1.0) had reported eleven sector folders validating. diff --git a/sources.md b/sources.md index fb38137..810fc95 100644 --- a/sources.md +++ b/sources.md @@ -109,7 +109,10 @@ Rules: | activity-browser-conda-forge | activity-browser on conda-forge: stable releases and metadata | https://anaconda.org/conda-forge/activity-browser | 2026-09-21 | LGPL-3.0-or-later | | pypardiso-repo | PyPardiso repository and README (haasad/PyPardiso) | https://github.com/haasad/PyPardiso | 2026-09-21 | BSD 3-Clause (repository LICENSE.txt) | | sentier-vocab-repo | sentier-vocab repository: README, LinkML schemas, data folders, generator and docs/COVERAGE.md | https://github.com/sentier-dev/sentier_vocab | 2026-09-21 | MIT (repository LICENSE); imported terms keep their ontology's licence | -| sentier-inventory-repo | sentier-inventory repository: README, schema contracts, sector folders and the CI validator | https://github.com/sentier-dev/sentier-inventory | 2026-09-21 | MIT (repository LICENSE) | +| sentier-inventory-repo | sentier-inventory repository: README, schema contracts, source folders and the CI validator | https://github.com/sentier-dev/sentier-inventory | 2026-09-21 | MIT (repository LICENSE) | +| sentier-inventory-pr-5 | sentier-inventory pull request #5, branch feat/source-folders at commit cf54f55: one folder per source release, sector as a column, schema 0.3.0 | https://github.com/sentier-dev/sentier-inventory/pull/5 | 2026-10-12 | MIT (repository LICENSE) | +| sentier-importers-pr-29 | sentier-importers pull request #29, branch feat/source-folders at commit a27910f: BAFU inventory delivered as one source folder, sector as a column | https://github.com/sentier-dev/sentier-importers/pull/29 | 2026-10-12 | BSD 3-Clause | +| sentier-brightway-pr-3 | sentier-brightway pull request #3, branch feat/source-folders at commit 3239a4e: load one source folder, sector as a column, sentier-inventory re-pinned | https://github.com/sentier-dev/sentier-brightway/pull/3 | 2026-10-12 | MIT for the code; installed data keeps the licences of the Sentier data repositories | | sentier-methods-repo | sentier-methods repository: README, schema contracts, datasource folders and the CI validator | https://github.com/sentier-dev/sentier-methods | 2026-09-21 | MIT (repository LICENSE) | | sentier-mappings-repo | sentier-mappings repository: README, package profile and metadata schemas, pair folders and the CI validator | https://github.com/sentier-dev/sentier-mappings | 2026-09-21 | MIT (repository LICENSE) | | sentier-agribalyse-repo | sentier-agribalyse repository: README, BOOTSTRAP.md, pyproject.toml, command help and source tree | https://github.com/sentier-dev/sentier-agribalyse | 2026-09-21 | MIT for the code (pyproject.toml); ecoinvent-derived files are never distributed | diff --git a/vocabulary.md b/vocabulary.md index 50c347a..a2a7543 100644 --- a/vocabulary.md +++ b/vocabulary.md @@ -40,7 +40,7 @@ reviewer to confirm or replace it. ## Terms -[Activity](#activity) · [Activity class](#activity-class) · [Activity data](#activity-data) · [Activity link](#activity-link) · [Allocation](#allocation) · [Analysed decision](#analysed-decision) · [APOS](#apos) · [Assumption scenario](#assumption-scenario) · [Attributional modelling](#attributional-modelling) · [Backend](#backend) · [Background](#background) · [Backtest](#backtest) · [Benchmark](#benchmark) · [Best attainable consensus](#best-attainable-consensus) · [Biosphere](#biosphere) · [Bridge](#bridge) · [bw2setup](#bw2setup) · [By-product / waste](#by-product--waste) · [Category](#category) · [Characterisation factor table](#characterisation-factor-table) · [Characterisation function](#characterisation-function) · [Characterization factors](#characterization-factors) · [Circular Footprint Formula](#circular-footprint-formula) · [Co-function](#co-function) · [Co-product](#co-product) · [Common database release identifier](#common-database-release-identifier) · [Comparative assertion](#comparative-assertion) · [Comparative life cycle assessment](#comparative-life-cycle-assessment) · [Compartment](#compartment) · [Concept scheme](#concept-scheme) · [Conditional exchange](#conditional-exchange) · [Consequential modelling](#consequential-modelling) · [Constrained market](#constrained-market) · [Contribution analysis](#contribution-analysis) · [Core flow ontology](#core-flow-ontology) · [Coverage](#coverage) · [Coverage sidecar](#coverage-sidecar) · [Critical review](#critical-review) · [Crosswalk](#crosswalk) · [Cut-off](#cut-off) · [Data format](#data-format) · [Data quality](#data-quality) · [Data root](#data-root) · [Database](#database) · [Database dating](#database-dating) · [Database service layer](#database-service-layer) · [Datapackage](#datapackage) · [Dataset](#dataset) · [Dataset index](#dataset-index) · [Datasource](#datasource) · [Decision situation](#decision-situation) · [Declared unit](#declared-unit) · [Delivery](#delivery) · [Demand](#demand) · [Direct emissions](#direct-emissions) · [Disaggregate](#disaggregate) · [Disclosed to the public](#disclosed-to-the-public) · [Dry run](#dry-run) · [Dynamic biosphere matrix](#dynamic-biosphere-matrix) · [Dynamic score](#dynamic-score) · [Eco-point (UBP)](#eco-point-ubp) · [ecoSpold format](#ecospold-format) · [Edge](#edge) · [EF 3.1](#ef-31) · [EF-compliant dataset](#ef-compliant-dataset) · [eILCD](#eilcd) · [Elementary flow](#elementary-flow) · [Elementary flows](#elementary-flows) · [End-of-life product](#end-of-life-product) · [Environmental impact](#environmental-impact) · [Environmental Product Declaration (EPD)](#environmental-product-declaration-epd) · [Exchange](#exchange) · [Exchanges](#exchanges) · [Field](#field) · [File format](#file-format) · [File mode](#file-mode) · [Flow](#flow) · [Flow code](#flow-code) · [Flow mapper](#flow-mapper) · [Flows](#flows) · [Foreground](#foreground) · [Functional flow](#functional-flow) · [Functional unit](#functional-unit) · [Geography](#geography) · [Graph traversal](#graph-traversal) · [Guest access](#guest-access) · [ILCD-EL](#ilcd-el) · [ILCD format](#ilcd-format) · [Impact category](#impact-category) · [Importer](#importer) · [Information module](#information-module) · [Intermediate flows](#intermediate-flows) · [Inventory](#inventory) · [IRI namespace](#iri-namespace) · [LCA data](#lca-data) · [LCA database](#lca-database) · [LCA software](#lca-software) · [LCDN](#lcdn) · [LCI](#lci) · [LCI data](#lci-data) · [LCIA calculator](#lcia-calculator) · [LCIA method](#lcia-method) · [LCIA methods](#lcia-methods) · [LCIA results](#lcia-results) · [Life cycle inventory analysis results (LCI results)](#life-cycle-inventory-analysis-results-lci-results) · [Life cycle inventory (LCI) data set](#life-cycle-inventory-lci-data-set) · [Life cycle inventory (LCI) study](#life-cycle-inventory-lci-study) · [Linking](#linking) · [Manifest](#manifest) · [Mapping package](#mapping-package) · [Market activity](#market-activity) · [Matching rule](#matching-rule) · [Material for treatment (mft)](#material-for-treatment-mft) · [Materiality principle](#materiality-principle) · [Metadata](#metadata) · [Method](#method) · [Method id](#method-id) · [Migration](#migration) · [Modelling principles](#modelling-principles) · [Models](#models) · [Module](#module) · [Monofunctional process](#monofunctional-process) · [Multifunctional process](#multifunctional-process) · [New node each visit](#new-node-each-visit) · [Node](#node) · [Nomenclature](#nomenclature) · [Nomenclature-only mapping](#nomenclature-only-mapping) · [Non-functional flow](#non-functional-flow) · [Normalisation](#normalisation) · [Normalisation and weighting set](#normalisation-and-weighting-set) · [Obsolete category](#obsolete-category) · [Ordered pair](#ordered-pair) · [Overall environmental impact](#overall-environmental-impact) · [Pair](#pair) · [PARDISO](#pardiso) · [PCR](#pcr) · [PEF and OEF](#pef-and-oef) · [PEFCR](#pefcr) · [PEFCR / OEFSR](#pefcr--oefsr) · [Plugin](#plugin) · [Primary stem](#primary-stem) · [Process](#process) · [Processed arrays](#processed-arrays) · [Processes](#processes) · [Product](#product) · [Product flow](#product-flow) · [Production exchange guessing](#production-exchange-guessing) · [Production volume](#production-volume) · [Project](#project) · [Project directory](#project-directory) · [Project mode](#project-mode) · [Property](#property) · [Proprietary target](#proprietary-target) · [Randonneur package](#randonneur-package) · [Recycling, reuse, recovery](#recycling-reuse-recovery) · [Reference flow](#reference-flow) · [Reference function](#reference-function) · [Reference package](#reference-package) · [Reference product](#reference-product) · [Release component](#release-component) · [Release type](#release-type) · [Relevant](#relevant) · [Relink](#relink) · [Residual database](#residual-database) · [Resource group](#resource-group) · [Rest of the World (RoW)](#rest-of-the-world-row) · [Scenario difference file](#scenario-difference-file) · [Schema](#schema) · [Scoring package](#scoring-package) · [Screening LCA](#screening-lca) · [Secondary good](#secondary-good) · [Sector folder](#sector-folder) · [Sidecar](#sidecar) · [Single overall score](#single-overall-score) · [Skeleton](#skeleton) · [Software](#software) · [Solver](#solver) · [Strategy](#strategy) · [Sub-category](#sub-category) · [Sub-compartment](#sub-compartment) · [Sub-indicator](#sub-indicator) · [Substance](#substance) · [Substitution](#substitution) · [Substitution flow](#substitution-flow) · [Supply array](#supply-array) · [System](#system) · [System boundary](#system-boundary) · [System expansion](#system-expansion) · [System model](#system-model) · [System perspective](#system-perspective) · [Technosphere](#technosphere) · [Temporal distribution](#temporal-distribution) · [Temporal grouping](#temporal-grouping) · [Tier](#tier) · [Time-explicit LCA](#time-explicit-lca) · [Timeline](#timeline) · [Transformation verb](#transformation-verb) · [Transforming activity](#transforming-activity) · [Treatment activity](#treatment-activity) · [Uncertainty](#uncertainty) · [Unit normalisation](#unit-normalisation) · [Unit process, black box](#unit-process-black-box) · [Unit process, single operation](#unit-process-single-operation) · [Unit processes](#unit-processes) · [Unlinked edge](#unlinked-edge) · [Vintage](#vintage) · [Waste flow](#waste-flow) · [Weighting](#weighting) +[Activity](#activity) · [Activity class](#activity-class) · [Activity data](#activity-data) · [Activity link](#activity-link) · [Allocation](#allocation) · [Analysed decision](#analysed-decision) · [APOS](#apos) · [Assumption scenario](#assumption-scenario) · [Attributional modelling](#attributional-modelling) · [Backend](#backend) · [Background](#background) · [Backtest](#backtest) · [Benchmark](#benchmark) · [Best attainable consensus](#best-attainable-consensus) · [Biosphere](#biosphere) · [Bridge](#bridge) · [bw2setup](#bw2setup) · [By-product / waste](#by-product--waste) · [Category](#category) · [Characterisation factor table](#characterisation-factor-table) · [Characterisation function](#characterisation-function) · [Characterization factors](#characterization-factors) · [Circular Footprint Formula](#circular-footprint-formula) · [Co-function](#co-function) · [Co-product](#co-product) · [Common database release identifier](#common-database-release-identifier) · [Comparative assertion](#comparative-assertion) · [Comparative life cycle assessment](#comparative-life-cycle-assessment) · [Compartment](#compartment) · [Concept scheme](#concept-scheme) · [Conditional exchange](#conditional-exchange) · [Consequential modelling](#consequential-modelling) · [Constrained market](#constrained-market) · [Contribution analysis](#contribution-analysis) · [Core flow ontology](#core-flow-ontology) · [Coverage](#coverage) · [Coverage sidecar](#coverage-sidecar) · [Critical review](#critical-review) · [Crosswalk](#crosswalk) · [Cut-off](#cut-off) · [Data format](#data-format) · [Data quality](#data-quality) · [Data root](#data-root) · [Database](#database) · [Database dating](#database-dating) · [Database service layer](#database-service-layer) · [Datapackage](#datapackage) · [Dataset](#dataset) · [Dataset index](#dataset-index) · [Datasource](#datasource) · [Decision situation](#decision-situation) · [Declared unit](#declared-unit) · [Delivery](#delivery) · [Demand](#demand) · [Direct emissions](#direct-emissions) · [Disaggregate](#disaggregate) · [Disclosed to the public](#disclosed-to-the-public) · [Dry run](#dry-run) · [Dynamic biosphere matrix](#dynamic-biosphere-matrix) · [Dynamic score](#dynamic-score) · [Eco-point (UBP)](#eco-point-ubp) · [ecoSpold format](#ecospold-format) · [Edge](#edge) · [EF 3.1](#ef-31) · [EF-compliant dataset](#ef-compliant-dataset) · [eILCD](#eilcd) · [Elementary flow](#elementary-flow) · [Elementary flows](#elementary-flows) · [End-of-life product](#end-of-life-product) · [Environmental impact](#environmental-impact) · [Environmental Product Declaration (EPD)](#environmental-product-declaration-epd) · [Exchange](#exchange) · [Exchanges](#exchanges) · [Field](#field) · [File format](#file-format) · [File mode](#file-mode) · [Flow](#flow) · [Flow code](#flow-code) · [Flow mapper](#flow-mapper) · [Flows](#flows) · [Foreground](#foreground) · [Functional flow](#functional-flow) · [Functional unit](#functional-unit) · [Geography](#geography) · [Graph traversal](#graph-traversal) · [Guest access](#guest-access) · [ILCD-EL](#ilcd-el) · [ILCD format](#ilcd-format) · [Impact category](#impact-category) · [Importer](#importer) · [Information module](#information-module) · [Intermediate flows](#intermediate-flows) · [Inventory](#inventory) · [IRI namespace](#iri-namespace) · [LCA data](#lca-data) · [LCA database](#lca-database) · [LCA software](#lca-software) · [LCDN](#lcdn) · [LCI](#lci) · [LCI data](#lci-data) · [LCIA calculator](#lcia-calculator) · [LCIA method](#lcia-method) · [LCIA methods](#lcia-methods) · [LCIA results](#lcia-results) · [Life cycle inventory analysis results (LCI results)](#life-cycle-inventory-analysis-results-lci-results) · [Life cycle inventory (LCI) data set](#life-cycle-inventory-lci-data-set) · [Life cycle inventory (LCI) study](#life-cycle-inventory-lci-study) · [Linking](#linking) · [Manifest](#manifest) · [Mapping package](#mapping-package) · [Market activity](#market-activity) · [Matching rule](#matching-rule) · [Material for treatment (mft)](#material-for-treatment-mft) · [Materiality principle](#materiality-principle) · [Metadata](#metadata) · [Method](#method) · [Method id](#method-id) · [Migration](#migration) · [Modelling principles](#modelling-principles) · [Models](#models) · [Module](#module) · [Monofunctional process](#monofunctional-process) · [Multifunctional process](#multifunctional-process) · [New node each visit](#new-node-each-visit) · [Node](#node) · [Nomenclature](#nomenclature) · [Nomenclature-only mapping](#nomenclature-only-mapping) · [Non-functional flow](#non-functional-flow) · [Normalisation](#normalisation) · [Normalisation and weighting set](#normalisation-and-weighting-set) · [Obsolete category](#obsolete-category) · [Ordered pair](#ordered-pair) · [Overall environmental impact](#overall-environmental-impact) · [Pair](#pair) · [PARDISO](#pardiso) · [PCR](#pcr) · [PEF and OEF](#pef-and-oef) · [PEFCR](#pefcr) · [PEFCR / OEFSR](#pefcr--oefsr) · [Plugin](#plugin) · [Primary stem](#primary-stem) · [Process](#process) · [Processed arrays](#processed-arrays) · [Processes](#processes) · [Product](#product) · [Product flow](#product-flow) · [Production exchange guessing](#production-exchange-guessing) · [Production volume](#production-volume) · [Project](#project) · [Project directory](#project-directory) · [Project mode](#project-mode) · [Property](#property) · [Proprietary target](#proprietary-target) · [Randonneur package](#randonneur-package) · [Recycling, reuse, recovery](#recycling-reuse-recovery) · [Reference flow](#reference-flow) · [Reference function](#reference-function) · [Reference package](#reference-package) · [Reference product](#reference-product) · [Release component](#release-component) · [Release type](#release-type) · [Relevant](#relevant) · [Relink](#relink) · [Residual database](#residual-database) · [Resource group](#resource-group) · [Rest of the World (RoW)](#rest-of-the-world-row) · [Scenario difference file](#scenario-difference-file) · [Schema](#schema) · [Scoring package](#scoring-package) · [Screening LCA](#screening-lca) · [Secondary good](#secondary-good) · [Sector folder](#sector-folder) · [Sidecar](#sidecar) · [Single overall score](#single-overall-score) · [Skeleton](#skeleton) · [Software](#software) · [Solver](#solver) · [Source folder](#source-folder) · [Strategy](#strategy) · [Sub-category](#sub-category) · [Sub-compartment](#sub-compartment) · [Sub-indicator](#sub-indicator) · [Substance](#substance) · [Substitution](#substitution) · [Substitution flow](#substitution-flow) · [Supply array](#supply-array) · [System](#system) · [System boundary](#system-boundary) · [System expansion](#system-expansion) · [System model](#system-model) · [System perspective](#system-perspective) · [Technosphere](#technosphere) · [Temporal distribution](#temporal-distribution) · [Temporal grouping](#temporal-grouping) · [Tier](#tier) · [Time-explicit LCA](#time-explicit-lca) · [Timeline](#timeline) · [Transformation verb](#transformation-verb) · [Transforming activity](#transforming-activity) · [Treatment activity](#treatment-activity) · [Uncertainty](#uncertainty) · [Unit normalisation](#unit-normalisation) · [Unit process, black box](#unit-process-black-box) · [Unit process, single operation](#unit-process-single-operation) · [Unit processes](#unit-processes) · [Unlinked edge](#unlinked-edge) · [Vintage](#vintage) · [Waste flow](#waste-flow) · [Weighting](#weighting) ### Activity @@ -802,7 +802,8 @@ reviewer to confirm or replace it. ### Sector folder -- **Sentier (sentier-inventory)** — one numbered folder of the inventory repository, holding the processes table, the exchanges table and the metadata of one sector. The number ranks the folders and decides which wins when records overlap. — source: sentier-brightway-repo · provenance: first-use · review: needed +- **Sentier (sentier-inventory, until schema 0.3.0)** — one numbered folder of the inventory repository, holding the processes table, the exchanges table and the metadata of one sector; the number ranked the folders and decided which won when records overlap. Dropped on 2026-10-12 because sector files from several sources could not stack; since schema 0.3.0 the sector is the `sector` column of the processes table and the folder is a Source folder. — source: sentier-inventory-pr-5 · provenance: first-use · review: needed +- Disambiguation: the vocabulary repository still names its per-sector process term files by sector; that is a file, not a folder, and it is the one place the "by sector, not by source" rule still applies. ### Sidecar @@ -826,6 +827,10 @@ reviewer to confirm or replace it. - **Brightway (bw2calc >=2)** — the routine that solves the technosphere system for the supply array. The default is an LU factorisation of the sparse matrix, with dense, least squares, iterative and caching variants for systems it handles badly, and `pypardiso` or `scikit-umfpack` substituted where they are available. — source: bw2calc-docs · provenance: first-use · review: needed +### Source folder + +- **Sentier (sentier-inventory)** — one folder of the inventory repository, `data/-/`, holding the processes table, the exchanges table and the metadata of one published release of one datasource. Every process row carries the same `source` and `source_version` and its own `sector`; a delivery replaces the folder whole, sources never share a file, and there is no precedence between folders. — source: sentier-inventory-pr-5 · provenance: first-use · review: needed + ### Strategy - **Brightway (bw2io >=0.9)** — one function in an importer's ordered list that identifies, cleans or links the extracted data before it is written; 0.9.17 ships 108 named ones, `imp.strategies` is a plain editable list, and the order they run in changes the outcome. — source: bw2io-docs · provenance: first-use · review: needed