Skip to content

feat(config): Let every map state how it merges - #1131

Open
JeanMertz wants to merge 21 commits into
config-list-strategiesfrom
config-map-strategies
Open

JeanMertz wants to merge 21 commits into
config-list-strategiesfrom
config-map-strategies

Conversation

@JeanMertz

Copy link
Copy Markdown
Collaborator

A map-valued setting can say how it combines, the same way a list
can. Ten fields gain it: providers.mcp, conversation.tools,
providers.llm.aliases, plugins.command, template.values, and a
tool's parameters, questions, options and properties.

Removing an entry from any of them reaches the conversation, which an
entry-wise merge could not express: the conversation kept the tool or
server the user had deleted, and recomputed the same empty delta every
turn.

A conversation's own snapshot is emitted key by key rather than as a
replacement, so a server, tool or alias the workspace config gained
after the conversation was created still reaches it.

Also fixes how a generic type is named in the schema.
MergeableMap<ToolConfig> and MergeableMap<ToolParameterConfig> both
answered MergeableMap, so a consumer resolving a reference by name
walked a value against whichever it met first. That reached users
through stored conversations: either valid keys were deleted, or a stale
one survived for typed deserialization to reject, which discards the
whole stored config rather than the key. The arguments are appended to
the name.

merge_nested_indexmap and the four strategy-less delta helpers are
deleted; nothing needs to guess any more.

Prepending a rule to `access.fs` or `access.env`, inserting one before
the last, or reordering the list, produced a delta that reproduced the
set of rules and not their sequence. Rules of equal specificity break
toward the one declared last, so a `read = false` rule that should have
won lost, and a reorder was recorded as no change at all.

`rule_delta` decided between appending and replacing by asking whether
every previous rule still appeared somewhere in the new list. That is
true of any permutation. Appending reaches the new list only when that
list starts with the old one, which is what it now asks; every other
difference carries the whole list with `replace`.

The delta law is what a delta has to satisfy, so it is now a test:
folding `delta(prev, next)` onto `prev` must produce `next`, checked
across the nine ways one resolved snapshot can differ from another.
Three of them failed before this change. The suite is written against
the shape both sides of a delta arrive in, a resolved list stamped
`replace`, so it exercises the path the producer actually takes.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`assistant.instructions`, `assistant.system_prompt_sections` and
`conversation.attachments` each diffed by set difference, so a removed
element produced no delta and a reorder produced none either. All three
now go through `delta_mergeable_vec`, promoted out of `access.rs` where
the same rule was written for tool access grants: appending reaches the
new list only when that list starts with the old one, and every other
difference carries the whole list with `replace`.

A `MergeableVec` can say `replace` on the wire, which is why these need
no path report. The `unsets` mechanism exists for plain `Vec` fields,
which cannot.

The sweep that found them asks the question of every field the schema
names: clear it, resolve, take the delta, fold it back, and require the
resolved result to match. It is shaped like the producer, which diffs
two resolved configs, so a field carrying a `#[setting(default)]` comes
back holding that default rather than arriving cleared, and only a field
whose resolved type is optional reaches the delta as an absence.

`conversation.compaction.rules` is listed as a known gap rather than
fixed. Its built-in defaults carry `discard_when_merged`, so a resolved
empty list and the resolved defaults compare unequal while resolving
alike: routing it through the shared helper wrote a replace-with-empty
delta for no change at all, and turned 39 tests red doing it. Accounting
for the discard flag belongs with the collection conversion.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`--cfg assistant.name=null` cleared the field for one turn and was gone
by the next. The clear reached the runtime config, because `--cfg`
assigns directly rather than merging, but no delta could carry it:
schematic keeps the previous value when the next layer has none, so
`delta_opt` dropped the difference and the conversation resolved the old
value again.

Such a field now reports its path in the delta's `unsets`, and the fold
clears it before merging. `delta_opt_at` does that for a scalar, and
`delta_opt_partial_at` for a whole block that went away, which is how
`editor.cmd` is reached.

The wiring covers `user.name`, `assistant.name`, `conversation.
default_id`, `editor.cmd`, `style.inline_code.background`, the
`providers.llm.openrouter` keys, every `assistant.request` field, and
the model parameters, including their copies under the inquiry
assistant, the title generator and the reasoning summary model.

A sweep found them and now guards them: it asks of every field the
schema names whether clearing it survives a fold, resolving both sides
because that is what a later turn runs with. It populates its own
fixture, since a field has to hold something before clearing it proves
anything, and a field left unset by the fixture was how the first
version of this quietly covered nothing. Ten paths are listed as known
gaps with a reason each: `inherit` is never stored, `parameters.other`
is its own dispatch's catch-all so the path names a key inside the map,
`compaction.rules` needs its discard flag accounted for, and the tool
defaults block has no path-reporting delta yet.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`extends`, `inherit` and `loader` were each explained twice in
`PartialAppConfig`'s two delta methods, listed again in the
reachability test's exceptions, and a third time in the delta law's.
Four assertions of one fact, in prose, drifting independently.

`delta::LOAD_TIME_ONLY` names the set once and says why: each is read
while the file declaring it is loaded, and only its effect outlives
that, so carrying one into a conversation would re-apply a decision
already made. Both test suites now consult it rather than repeating it,
and the delta impls point at it instead of restating it. The field
assignments stay written out, since a struct literal has to name every
field and the compiler keeps the two methods honest about that.

The distinction the old prose blurred is now visible: all three are
load-time, but only `extends` and `loader` lack a key-value arm.
`inherit` is settable and simply never stored, which is why it appears
in one list and not the other.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`partial_via` substituted the partial's field type only when the field
was also `nested`, and `nested` requires the element type to be a
`Config`. A `Vec<String>` could therefore not carry a wrapper in its
partial, which is what a list needs in order to declare its own merge
strategy: five of JP's six plain-list config fields hold scalars.

The substitution now applies to a plain field too, and
`generate_from_partial_value` converts back to the field's own type on
the way out. The conversion wraps the inner access rather than the
container, since it is the value whose type differs and not its
`Option` or its `Box`, and the nullable case maps over the option
instead of converting it.

A field opting in gains a partial of `Option<Wrapper<T>>` while the
resolved config keeps `Vec<T>`, so callers see no change and the
wrapper is confined to the merge path.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`editor.envs` appended across layers with no way to say otherwise. It
now carries a strategy of its own, so a user can replace the inherited
list rather than extend it:

```toml
[editor]
envs = { value = ["MY_EDITOR"], strategy = "replace" }
```

The plain list still works and still appends, since that is the
declared default.

This is the first plain-list field to carry a wrapper in its partial,
which the `partial_via` change makes possible. Two pieces make it work
for a list of scalars rather than of configs: `try_some_mergeable_
strings` accepts either shape from `--cfg`, telling them apart the way
the type's own deserializer does, and `delta_opt_mergeable_vec` diffs
the field through the wrapper so a removal or a reorder is recorded as
a replacement instead of being dropped.

`editor.envs` is also the last list in the config whose removals were
recorded through `unsets`; that mechanism is now only needed for
scalars, and for the five remaining plain lists until they follow.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`providers.llm.anthropic.beta_headers` appended across layers with no
way to say otherwise. It now carries a strategy of its own:

```toml
[providers.llm.anthropic]
beta_headers = { value = ["interleaved-thinking-2025-05-14"], strategy = "replace" }
```

Deduplication is unchanged. `vec_with_strategy` deduplicates any
combining merge unless a config opts out, which is what
`append_vec_dedup` did for this field, so a plain list behaves exactly
as before.

`PartialAnthropicConfig` loses its `delta_with_unsets` override
entirely: with the last of its fields able to state `replace` on the
wire, nothing here needs a path reported, and the trait's default diff
is correct. That is the shape the rest of the conversion takes as it
lands, one field at a time.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`providers.mcp.<server>.arguments` and `.variables` appended across
layers with no way to say otherwise. Each now carries a strategy of its
own:

```toml
[providers.mcp.bookworm]
arguments = { value = ["serve"], strategy = "replace" }
```

That closes the last of the reported bug. Dropping an argument is now
recorded: the delta says `replace` and carries the whole list, where
before appending could not shorten one, the difference went unrecorded,
and every turn recomputed the same non-delta. `PartialMcpProviderConfig`
loses its `delta_with_unsets` override, since no field there needs a
path reported.

The new `ordered_vec_with_strategy` merge keeps duplicates unless a
config asks for deduplication, the opposite of the default. An argument
list is a command line: `["--flag", "x", "--flag", "y"]` means
something else once the second `--flag` is dropped. The opinion has to
live on the merge function rather than on the field's default, because
config layering merges partials before any defaults are filled in --
carrying it as a value let a plain list silently deduplicate, which a
test caught.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`assistant.model.parameters.stop_words` appended across layers with no
way to say otherwise. It now carries a strategy of its own, at every
site the model parameters are reached from -- the assistant, the inquiry
assistant, the title generator and the reasoning summary model:

```toml
[assistant.model.parameters]
stop_words = { value = ["\n\n"], strategy = "replace" }
```

The field also deduplicates now, matching every other converted list.
Two identical stop words have the same effect as one, so nothing about
generation changes; `providers.mcp.*.arguments` stays the sole exception,
because a repeated flag in a command line is not a duplicate.

`PartialParametersConfig` keeps its `delta_with_unsets` override for the
scalars around it, but `stop_words` no longer contributes to it: a
dropped word is recorded as a replacement at whichever site it was
reached from, with no path to report.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`config_load_paths` accumulated across layers with duplicates dropped
and no way to ask for anything else. It now carries a strategy of its
own, which matters because `--cfg <name>` walks the list in order and
takes the first directory holding a match:

```toml
config_load_paths = { value = [".jp/agents"], strategy = "prepend" }
```

A user-level config can put its own directory ahead of the workspace's
rather than behind it, and a workspace can replace an inherited list
instead of adding to it. The plain list still appends and deduplicates,
so nothing changes for a config that does not ask.

This is the last plain list in the configuration, so every list-valued
field now states how it merges. Four helpers that existed only to guess
at it are gone: `delta_opt_vec`, `delta_opt_vec_at`, `append_vec_dedup`
and `try_some_vec`. A list no longer needs a path reported in `unsets`
to record a removal -- it carries `replace` and the fold does the rest,
leaving `unsets` to the scalars, which genuinely cannot say it.

`try_some_mergeable_vec` takes an element parser, so a list of anything
that is not a `String` -- here `RelativePathBuf` -- can accept both the
plain and the `{ value, strategy }` shape from `--cfg`.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`conversation.labels` computed its own delta inline. The rule it
encoded is not specific to labels: an entry both states share carries
its own delta, an entry only the next state has is carried whole, and a
key the previous state had and the next one does not was dropped, which
entries cannot spell -- so the whole map carries `replace` rather than
letting a deep merge resurrect the key.

`delta_mergeable_map` states that once, alongside `delta_mergeable_vec`.
It also drops an entry whose own delta comes out empty, which the inline
version did not: a missing entry already means "unchanged", so an empty
one reads as a change that is not there and makes the enclosing partial
look non-empty. That is the same noise the map delta for
`providers.mcp` was fixed for.

Behaviour is otherwise unchanged; the helper is what the remaining
`IndexMap` fields need as they gain a strategy of their own.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Removing an MCP server, a tool, a model alias, a plugin, a template
value, a tool parameter, question or option from a config file now
reaches the conversation. Until now the removal was computed, found to
be unexpressible, and dropped: the conversation kept starting the
server the user had deleted, and recomputed the same non-delta on every
turn.

Map entries merge by key, which is what lets a server added to the
workspace config reach a conversation created before it existed. The
same property means no value a delta carries can take an entry away:
the key survives from the previous layer. The entry's own path is
reported in the delta's `unsets` instead, and the fold removes it
before merging, which `unset` already supported.

Every map in the configuration is covered, including the ones nested
inside a tool, and clearing a field of the `conversation.tools.'*'`
defaults block is recorded too. That block resolves through its own
type, which had no path-reporting delta, so a cleared `enable` or
`style.error.inline_results` there went unrecorded.

A map whose values are plain rather than nested partials gets the same
treatment through `delta_value_map_with_unsets`, replacing three copies
of the same inline entry-comparison loop.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`conversation.compaction.rules` is the one list field whose delta still
compares elements rather than saying `replace`, and the reason recorded
next to it was wrong. It named the built-in defaults' `discard_when_merged`
marker, which is not what stops the field.

What stops it is that the field's partial is a bare `MergeableVec`, so an
empty one cannot say whether the user asked for no rules or said nothing
about them. Every sparse partial that reaches the delta carries the empty
one, and replacing with it would record zero rules the user never asked
for, which makes a later `jp conversation compact` do nothing.

Found by routing the field through the shared helper and reading what
broke: `replace` with an empty list written into 37 conversation
snapshots, and a config event appended where the suppression path should
have left none. Reaching the field needs its partial to become an
`Option<MergeableVec<_>>`, the shape every converted list field has,
where `None` is absent and `Some([])` is a deliberate empty.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`providers.mcp` accepts a strategy the way every other collection field
does, so a workspace can drop the servers an outer layer configured
rather than merging with them:

```toml
[providers.mcp]
value = { bookworm = { type = "stdio", command = "just" } }
strategy = "replace"
```

The default is unchanged: entries merge by key, so a server added to a
later layer joins the ones an earlier layer set rather than replacing
them. That is what lets a server added to the workspace config reach a
conversation created before it existed, and there is now a test holding
that property.

Recording a removed server no longer needs a reported path. The map
states `replace` and carries the servers the user is left with, which
the fold applies without help, so `unsets` is left to the fields that
genuinely cannot speak for themselves.

The conversation's snapshot of the map merges per key rather than
stating `replace`, through the new `map_to_partial_per_key`. Stating
`replace` there would drop every server the config files declare and
the conversation does not, which is the behaviour the test above
catches. Merging per key is safe because each server's own lists
already state their strategies, so re-merging the snapshot over the
layer it came from reproduces it rather than doubling its arguments.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`plugins.command`, `providers.llm.aliases`, and a tool's `parameters`,
`questions` and nested `properties` accept a strategy the way every
other collection field does:

```toml
[providers.llm.aliases]
value = { opus = "anthropic/claude-opus-4" }
strategy = "replace"
```

The default is unchanged: entries merge by key, so an alias, plugin or
parameter configured in a later layer joins the ones an earlier layer
set. That is what lets a workspace config gain an entry and have it
reach a conversation created before it existed.

Recording a removed entry no longer needs a reported path. Each map
states `replace` and carries the entries the user is left with, which
the fold applies on its own. `unsets` is left to the fields that
genuinely cannot speak for themselves, and `PartialToolParameterConfig`
no longer needs a path-reporting delta at all.

Each conversation snapshot merges per key rather than stating
`replace`, so an entry the files declare and the conversation does not
survives the layering.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`template.values` and a tool's `options` accept a strategy the way
every other collection field does:

```toml
[template.values]
value = { branch = "main" }
strategy = "replace"
```

The default is unchanged: entries merge by key, so a value set in a
later layer joins the ones an earlier layer set.

Both maps hold free-form JSON rather than nested config, so their
entries have no partial to diff and are compared and carried whole
through `delta_mergeable_value_map`. A removed entry travels as a
`replace` with the entries the user is left with, so neither map needs
a reported path any more. `delta_value_map` and its unset-reporting
sibling are gone with them.

Every map in the configuration except `conversation.tools` itself now
states its own strategy, and each conversation snapshot merges per key
so an entry the files declare and the conversation does not survives
the layering.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`conversation.tools` accepts a strategy, which completes the set: every
collection field in the configuration now states how it merges.

```toml
[conversation.tools]
strategy = "replace"

[conversation.tools.value.my_tool]
source = "builtin"
```

The default is unchanged: tools merge by key, so a tool configured in a
later layer joins the ones an earlier layer set. Tool entries are
flattened to sit directly under `conversation.tools`, and the wrapper
needs both `value` and `strategy` to be recognised, so a tool may still
be named `value` or `strategy` on its own. Two tests hold both halves of
that.

`delta_map` and `delta_map_with_unsets` are gone. Every map states its
own strategy, so a removed entry travels in the value as a `replace` and
no map needs a reported path. What remains of `unsets` is what only it
can express: a scalar that went away.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`merge_nested_indexmap` merged two maps per key with no way for a config
to ask for anything else. Every map field now carries a `MergeableMap`,
whose `map_with_strategy` does the same per-key merge by default and
honours a declared `deep_merge`, `merge`, `keep` or `replace`, so the
older function has no callers left.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
A schema names each type it expands and refers back to that name
wherever the type appears again, so a name has to identify one type. A
generic named after its base alone does not: `MergeableMap<ToolConfig>`
and `MergeableMap<ToolParameterConfig>` both answered `MergeableMap`,
and a consumer resolving a reference by name walked a value against
whichever of them it met first.

That reached users through stored conversations. Stripping a stored
config walks it against the schema to drop keys a newer release wrote,
and resolving a tool's `parameters` to the map of tools instead walks
each parameter against `ToolConfig` — deleting valid keys, or leaving a
stale one behind for typed deserialization to reject, which discards the
whole stored config rather than the key.

The arguments are appended, so the two are `MergeableMap_ToolConfig` and
`MergeableMap_ToolParameterConfig`. An argument with no name of its own
contributes nothing, which leaves the base name for a type generic only
over primitives.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Stripping a stored config walks it against the schema and removes the
keys the schema has no field for, so a key an older release does not
know is dropped rather than left to fail typed deserialization, which
would discard the whole stored config.

A map that can state its own merge strategy is described as the plain
map beside the wrapper holding it under `value`. Both are objects on the
wire, so shape alone left the union ambiguous and the walk stopped:
every key inside a tool, server, alias or plugin went unvisited. The
variants are now told apart the way the wrapper's own deserializer does
it, by whether the value carries `value` and `strategy` together, so a
tool called `value` is still a tool.

A flattened map is resolved the same way, which is what lets the entries
of `conversation.tools` be walked at all.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
A tool the workspace config gained after a conversation was created
still reaches that conversation, as it did before the map could state
its own merge strategy. Filling took the conversation's map whole, so a
tool added later was invisible to every conversation that predated it.

A map that states a strategy is left alone instead: its owner said how
it combines, and filling gaps into it would answer differently. Only the
styles of the tools it holds are filled, which is what carries a single
`[conversation.tools.'*'.style]` key to each of them.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
@JeanMertz
JeanMertz force-pushed the config-map-strategies branch from e4804a1 to a5e22fb Compare September 8, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant