Cleaning-DB orchestrator module (revives odf_core, scoped to a staging Odoo we provision)

[bosd]

Revives bosd/odoo-data-flow#10 (odf_core), which was closed when that repo was archived. That closure was based on a faulty generalisation — that Odoo Online's ban on third-party Python modules rules out an Odoo module anywhere. It does not. It rules one out in the client's production database. This issue is scoped to a database we provision.

The pattern

A real client transformation shape:

legacy Odoo 12  ──►  Odoo 18 "cleaning DB"  ──►  Odoo 18 test  ──►  Odoo 18 prod
                     (intermediate)

The cleaning DB is not a file staging area. It is:

• the landing zone for transformed Odoo 12 data,
• where the data cleaning happens, against a real Odoo 18 schema,
• and the orchestrator for both the import from 12 and the push to test/prod.

Why it is worth building

It removes the consultant's workstation as a single point of failure. Today the orchestrator is somebody's laptop: the migration runs when that machine is on, on that machine's network, with that machine's credentials. Moving orchestration onto client infrastructure means scheduled runs continue regardless, and the client is already paying for the box.

For weekly delta migrations — where the pipeline runs for months before go-live — that is not a nicety. A laptop cannot be the thing a six-month migration depends on.

Secondary benefits over a file- or spreadsheet-based staging layer:

• The cleaning DB already is the target schema. No second mapping to maintain, no drift between staging shape and target shape.
• Validation runs against real Odoo constraints — _sql_constraints, required fields, onchange — instead of rules re-implemented elsewhere.
• The final push is Odoo → Odoo, which is already fluvo migrate.
• Odoo views, access rights, chatter and activities come free as the cleaning team's UI.

Scope

An Odoo module (18.0, and Community-compatible) providing:

• odf.connection — source/destination connection records, credentials stored securely.
• odf.flow.project — links a source connection to a destination connection, points at a flow definition (see #251, which this depends on).
• An ir.cron that runs due flows by invoking Fluvo, capturing exit code and output.
• A run log model: what ran, when, how long, how many records, what failed — with the fail file attached.
• The resolved company and language set surfaced on every run record (see #254, #255).

Explicitly out of scope

• Anything installed in a client's production Odoo. This module targets a cleaning/staging database only, and the README must say so.
• A flow-builder UI (the old odf_ui_builder). Far more work than it is worth right now.
• Conflict-resolution UI — a separate, later question, and it may be better served outside Odoo.

Open questions

1. How does the module invoke Fluvo? Shelling out (os.system) was the original sketch and is fragile. A queued job, or Fluvo as an importable library in the same venv, are both better. This decision affects packaging.
2. Does it depend on queue_job (OCA)? Adds a dependency but gives retries, concurrency and a job UI for free.
3. Which Odoo versions to support for the cleaning DB itself. It need not match the target — the cleaning DB is ours.

Relationship to the Grist staging portal

Both are staging layers outside production and they compose rather than compete: the cleaning DB is the orchestrator and the Odoo-constraint validation surface; a spreadsheet UI is better for business users doing bulk field-level cleaning. Neither blocks the other.