basic-lca-publisher

Proof of concept for an open-source publication platform for the BAFU:20XY life cycle inventory database: versioned processes, review with comments and issues, releases, LCA on every release, ecoSpold 1 and JSON export. Built with FastAPI, SQLAlchemy, SQLite, Jinja2 + htmx (one 50 kB script, no build step) and bw2calc.

Quick start

uv sync
uv run python dev/import_bafu.py --fresh \
    --release "2025 v1=/path/to/BAFU 2025 v1/ecospold" \
    --release "2026 v1=/path/to/BAFU 2026 v1/ecospold"
uv run uvicorn app.main:app --port 8000      # http://localhost:8000, API docs at /docs
uv run pytest                                # tests use a small synthetic database

Requires Python 3.14 and uv. The BAFU:20XY data is not included in this repository; download it from FOEN and keep it wherever you like.

Initial setup: dev/import_bafu.py

The import is a one-shot script, not part of the application. It builds the database from scratch:

1. Reference data. Downloads Brightway Flows 1.0 on first use and loads its 97,818 elementary flows, 138 impact categories (EF 3.1 and Stepwise 2006) and their characterisation factors.
2. Releases. Imports each BAFU release in the order given, oldest first. Every ecoSpold 1 file is repaired in memory (dev/ecospold_fix.py; the files on disk are never changed), parsed with pyecospold, and turned into product, process and exchange documents. Elementary flows are mapped to Brightway Flows; inputs are linked to the process that makes them. A process whose content did not change since the previous release keeps its version, so releases share data and the release diff shows only real changes. Each release is stored as published.
3. Issues. Inputs that match no process in the release, and elementary flows with no Brightway Flows match, are left out and recorded as blocking issues on the process, so they show up in the review queue.
4. Demo users. Creates the demo users for the user switcher.

Importing the three releases published so far (2025 v1, 2025 v2, 2026 v1) takes about four minutes and produces a SQLite database of about 1.3 GB.

Telling the script where the data is

Each release is a name and the folder that directly holds its ecoSpold XML files. The folder layout is up to you. Give one --release NAME=PATH per release, oldest first:

uv run python dev/import_bafu.py --fresh \
    --release "2025 v1=$HOME/data/bafu/2025-v1" \
    --release "2025 v2=$HOME/data/bafu/2025-v2" \
    --release "2026 v1=$HOME/data/bafu/2026-v1"

or set BAFU_RELEASES to the same pairs, separated by ;:

export BAFU_RELEASES="2025 v1=$HOME/data/bafu/2025-v1;2026 v1=$HOME/data/bafu/2026-v1"
uv run python dev/import_bafu.py --fresh

--release arguments take precedence over BAFU_RELEASES. The release name is what the application shows ("2026 v1"); names must be unique.

Options and environment variables

Option / variable	Default	Meaning
--release NAME=PATH (repeatable)	from BAFU_RELEASES	A release and the folder with its ecoSpold XML files.
BAFU_RELEASES	none	NAME=PATH pairs separated by ;, used when no --release is given.
--fresh	off	Delete the existing database first. Without it, the script refuses to import into a database that already has releases.
LCA_DATABASE_URL	sqlite:///data/lca.sqlite3	Database the script writes and the application reads.
BRIGHTWAY_FLOWS_DIR	data/brightway-flows	Where the Brightway Flows files are kept. Missing files are downloaded: harmonised-flows-simple.json.gz and lcia-factors.json.gz from Zenodo (62 MB), and consensus-flow-contexts.json (context labels) from the brightway-flows repository. Put the files there yourself to work offline.

Using the application

There is no authentication in the PoC. Pick a demo user at the top right (admin, editor, partner, reviewer); API writes take an X-Demo-User: <user id> header. Editors change processes and submit them, reviewers approve them or request changes, and admins build and publish releases. LCA_WARM_CACHES=0 stops the server from building the LCA matrices of the newest releases in the background at startup.

Comparing versions and releases

• /compare?from=2025 v2&to=2026 v1 gives an overview of two releases: processes added, removed, with changed inputs or emissions, or with only a changed description; which fields changed and how often; changes by product category; and how impact scores moved, as a histogram of relative changes plus the largest movers.
• /processes/<id>/compare?from=…&to=… compares one process. Each side is a release name, latest, or a version id. The process page links each pair of consecutive releases in which the process changed, and the newest release → latest when it was edited since.
• The same comparisons are in the API: /api/compare and /api/processes/<id>/compare.

Pedigree matrix

Pedigree scores are read from the start of exchange comments with pedigree_matrix and stored as an object on the exchange (reliability, completeness, …, sample_size, plus the basic uncertainty and a seventh value where BAFU writes them); the comment keeps only the text. BAFU's extended forms, such as (3,2,5,3,1,5,BU:3), BU:1,05, (4,4,3,2,4,5,11) and semicolon separators, are normalised before pedigree_matrix validates the scores. 277,457 exchanges get a pedigree; 315 comments with malformed matrices are left as they are. The uncertainty factor is calculated with the ecoinvent v2 factors from pedigree_matrix (its own calculate() uses Python 2's iteritems). ecoSpold export writes the matrix back into the comment.

Languages

The interface is available in English, German, French and Italian; the data (process names, comments, flow names) is shown as it is stored. People pick a language in the header; until they do, the browser's preferred language is used. Numbers follow Swiss conventions in each language (1’234.5; 1 234,5 in French), and amounts can be typed with either a decimal point or a decimal comma. The JSON API stays in English.

Translations are standard gettext catalogues, editable with any PO editor (Poedit, Weblate, …), in app/locale/<language>/LC_MESSAGES/messages.po. They are compiled when the application starts, so there are no .mo files to build. The German, French and Italian translations were machine-made and should be reviewed by native speakers.

After changing UI text:

uv run pybabel extract -F babel.cfg -k N_ -k "ngettext:1,2" -o app/locale/messages.pot .
uv run pybabel update -i app/locale/messages.pot -d app/locale
# translate the new entries, then:
uv run python dev/check_translations.py      # every entry translated, placeholders intact

To add a language, run uv run pybabel init -i app/locale/messages.pot -d app/locale -l <code> and add it to LANGUAGES and NUMBER_LOCALES in app/i18n.py.

Layout

app/domain.py           Pydantic dataclasses: Product, Process, Exchange, versions, review, diffs.
                        The unit of work, exposed more or less 1:1 by the API.
app/models.py           SQLAlchemy tables. Storage only; never leaves app/store.py and app/services.
app/store.py            Repository: domain <-> rows, content hashes, derived columns, optimistic locking.
app/services/           catalog (read side), workflow (drafts, review, issues, comments), releases,
                        editing (pure functions on domain objects), diff, results, export (ecoSpold 1).
app/pedigree.py         Pedigree matrix: read from and write to comments, uncertainty factor.
app/i18n.py, labels.py  UI translations (gettext), Swiss number formats, labels for enum values.
app/locale/             Translation catalogues (de, fr, it).
app/lca/                Matrices from SQLite -> bw_processing -> bw2calc; batch scores per release.
app/routes/             api.py (JSON, /api) and web.py (HTML).
dev/import_bafu.py      One-shot import (see above).
dev/ecospold_fix.py     Repairs BAFU ecoSpold 1 files so they validate; works in memory.
dev/check_translations.py  Checks the translation catalogues.

Data model in one paragraph

Products (a product made in a place, in a unit) and processes have a stable uuid7 identity and a chain of immutable versions. A version's content is a JSON document (data), validated by the domain dataclasses; columns outside it exist only for indexes and foreign keys and are derived from it on save. A process version owns its exchanges, one row each. An exchange references an elementary flow or one or more products; several products form a group that shares one amount. A release pins one version of every process and product, so publishing is cheap and published data never changes. The content hash ignores provenance and exchange keys, so the same data imported twice is one version.

What the import found

• All 2,679 BAFU elementary flows map to Brightway Flows, in 2025 and 2026, through Brightway Flows' own BAFU 2026 v1 correspondence (29 rows need a unit conversion).
• 2026 files link providers by ecoSpold number (100 %). 2025 files were renumbered sequentially, so providers are matched by name, location and unit; 45 inputs match nothing (the same ones in 2025 v1 and v2) and each opened a blocking issue.
• 2025 names carry a SimaPro-style location suffix (Clinker, at plant {CH}); it is removed so processes keep their identity across releases.
• 2025 v1 → 2025 v2: no content changes. 2025 v2 → 2026 v1: 200 processes added, 318 changed inputs or emissions, and in 10,465 the only change is that the German local name was replaced by the English name.
• The gist needed three more fixes for 2026 v1: undashed CAS numbers, a non-numeric percent, and references to sources missing from the file.

Known limits

• Exports write Brightway Flows names for elementary flows, not BAFU's original names.
• ecoSpold 1 has no groups; a group exports as one exchange per product.
• LCIA methods are the ones Brightway Flows publishes: EF 3.1 and Stepwise 2006.
• The first LCA view of a release builds its matrices (~6 s); later views take ~50 ms. A whole release scores in ~0.3 s for all 138 impact categories.

Licence

MIT, see LICENSE. Bundled third-party code: app/static/htmx.min.js (htmx, 0BSD) and dev/ecospold_fix.py (BSD 2-Clause, Cauldron Solutions; extended here with three fixes for BAFU 2026 v1). Data is not included: the BAFU:20XY database is published by FOEN under its own terms of use, and Brightway Flows is published under ODbL-1.0.