Adding a `-> Parent` field to `X` silently turns a stored `Parent.xs: -> X[]` into a derived collection, hiding its data; the plan shows 0 steps

[rrrodzilla]

Summary

Whether a -> X[] field is a stored list or a derived inverse collection (#34) is decided implicitly: it is derived exactly when X has one -> Parent field. Adding such a field to X for an unrelated purpose therefore changes the parent's field, and the migration does not report it:

• The stored ids in the parent's column are no longer read. The field now shows only children whose new FK points back, which is usually [].
• Writes to the field start failing with 422 is a derived inverse collection, so clients that maintained the list break.
• migrate reports the parent as metadata update, 0 migration steps. There is no destructive or requires_confirmation step, and --force is not needed. serve applies it silently at startup.
• The old column and its data are left in place, orphaned. Removing the new field again (apply --force, since that drops backup_for) brings the stored ids back, so the data is hidden, not migrated.

A second -> Parent field on X makes apply fail with ambiguous inverse relation for Team.members: schema Person has 2 fields pointing back at Team [backup_for, home], and there is no way to say which one is meant, or that neither is. schemaforge parse reports 0 errors for the same files.

Version checked

v0.45.0 release binary (x86_64 Linux, PostgreSQL build), source at tag v0.45.0 (10ec403), PostgreSQL 16.

Reproduction

v1:

@display("name")
schema Team {
    name:    text required
    members: -> Person[]
}

@display("name")
schema Person {
    name: text required
}

Create two Persons and a Team with members: [p1, p2]:

GET .../Team/entities/<team>
{"members":["person_01...a","person_01...b"],"name":"Core","members__display":["Ada","Grace"]}

v2 adds an unrelated back-reference to Person ("the team this person covers for"). Team is unchanged:

@display("name")
schema Person {
    name:       text required
    backup_for: -> Team
}

$ schemaforge migrate v2
Person (1 steps, safe)
  1. ADD RELATION 'backup_for' -> Team (One) [safe]

Team (metadata update, 0 migration steps)

After applying (or restarting serve on v2):

GET .../Team/entities/<team>
{"members":[],"name":"Core"}

PATCH .../Team/entities/<team>  {"fields":{"members":["person_01...a"]}}
422 {"error":"validation_failed","message":"validation failed: field 'members': is a derived inverse collection — write to the child schema's foreign-key field instead"}

The data is still in the table:

SELECT id, members FROM "Team";
 team_01... | {person_01...a,person_01...b}

Expected

A change in another schema should not silently change what a field means, or hide its data. Any of these would do:

• Explicit pairing. A stored -> X[] stays stored unless the parent says otherwise, for example members: -> Person[] @inverse("team"). This also removes the ambiguity error, because the author names the FK.
• Or, keep implicit pairing but surface it. When an existing stored -> X[] becomes derived, the plan emits a step classed destructive (or at least requires_confirmation) that names the field and its now-unused column, and ideally offers to drop it. serve then refuses it without --allow-destructive-migrations.
• Documentation of the pairing rule in the DSL reference. It is currently described only in code comments and one row of docs/site-guide.md.

Source

• crates/schema-forge-core/src/inverse_relations.rs:50-143 (pair_inverse_relations): 0 FKs back means stored TEXT[], 1 means derived, 2 or more is an Ambiguous error. It is applied to every parsed batch (crates/schema-forge-cli/src/commands/parse.rs:212, crates/schema-forge-acton/src/extension.rs:429, crates/schema-forge-acton/src/routes/schemas.rs:83).
• crates/schema-forge-core/src/migration.rs:764-770 (diff_modifiers_with_renames) skips derived fields, and :896-901 / :879-884 skip them in add/remove. A field that changes from stored to derived keeps the same FieldType, so diff_fields_with_renames (:737-747) emits nothing either.
• crates/schema-forge-acton/src/routes/entities.rs:764-772: writes to a derived field are rejected. :2221-2392 (populate_derived_collections) overwrites the field on read with the child-query result, so the stored column is never read.

Suggested fix

1. Make pairing opt-in with an explicit annotation on the parent field, naming the child FK. Keep the current inference for one release behind a deprecation warning from schemaforge parse when a pairing is inferred.
2. In the differ, compare derived_from between the stored and new definitions. Stored-to-derived should be a new step (e.g. DeriveCollection { field, drop_column }) classed destructive. Derived-to-stored should be a step that re-creates the column.
3. Document the rule and the migration behaviour in the DSL reference.