Multirack join service part 4 by andrewjstone · Pull Request #11143 · oxidecomputer/omicron

andrewjstone · 2026-08-24T20:28:48Z

Note to reviewers: This is not as bad as it looks there's at least 800 lines of JSON here.

This is the last bit of work for #10637 that got us to Friday's demo where
/64 addresses for sleds were published via DDM running over front ports.

There's a bunch of stuff in here, but at a high level it comes down to the following:

Adding a wicketd commission api endpoint for submitting a MultirackJoinRequest to the multirack join service.
The multirack join service being completed by putting the RackNetworkConfig into the bootstore so that it can be reconciled.
The reconciliation resulting in front ports being configured to allow_ddm_traffic via dpd-client
The reconciliation resulting in DDM state machines being started for those front ports via use of ddm-admin-client.

This code relies on new maghemite code on branch ry/external-peers. The necessary dendrite changes are already merged into main. The maghemite PR must be completed and merged before this goes in.

All of this was tested using a voxel PR that must be merged after this PR.

andrewjstone · 2026-09-21T19:23:12Z

This is finally ready for a review.

This relies on oxidecomputer/omicron#11143 which must be merged first. It removes the pre-existing shell commands and uses actual omicron reconcilliation of dendrite and maghemite to ensure interconnect ports are created and DDM routes exchanged over them.

jgallagher · 2026-10-01T16:36:18Z

 }

+fn switch_zone_ddm_router_id(baseboard: &Baseboard) -> String {
+    format!("sw{}", baseboard.identifier())


Is the sw prefix special or meaningful to ddm?

@rcgoodfellow added that one. I'm not sure what the goal was there.

I think we at least need some comments explaining this. I'm not sure if it's:

we're prefixing sw for humans to debug that this is something in a switch zone, but it doesn't matter for correctness

the prefix of sw is special to ddm in some way

we need any kind of prefix/suffix because if we used just baseboard.identifier().to_string() something would break because now the switch zone ID matches the gz ID

or something else

Also bump magehmite versions

jgallagher · 2026-10-05T15:54:03Z

    };

    let ports = UplinkPorts::new(ports)
        .context("rack network config must specify at least one uplink port")?;


Will this still fail if the config specifies only DDM ports and no uplink ports? (We should reject that combo somehow, right?)

Similarly, but not sure where, should we reject any configs targeting the multirack join service that don't specify any DDM ports? (Or in the future, don't specify any ports configured for connection to a proto rack?)

Ah you're correct. I fixed the implementation so it does so now in afdd508.

Unfortunately, the name is still confusingly UplinkPorts. But that really is not on the table to change right now.

I'll have to think a bit more about your second question tomorrow.

This change actually resulted in a couple of tests failing. In particular we can't just have a single port that has allow_ddm_traffic = true. That messes with the proptest change I made to populate that field rather than hardcode to false. I reverted that prior change so that we don't set allow_ddm_traffic = true in the Arbitrary. I don't think it's that big a deal for testing here.

Changed in 59d655b

jgallagher · 2026-10-05T16:34:10Z

 }

+fn switch_zone_ddm_router_id(baseboard: &Baseboard) -> String {
+    format!("sw{}", baseboard.identifier())


I think we at least need some comments explaining this. I'm not sure if it's:

we're prefixing sw for humans to debug that this is something in a switch zone, but it doesn't matter for correctness

the prefix of sw is special to ddm in some way

we need any kind of prefix/suffix because if we used just baseboard.identifier().to_string() something would break because now the switch zone ID matches the gz ID

or something else

jgallagher · 2026-10-05T16:37:36Z

+        // The two variants name their speed and FEC fields differently, so the
+        // required speed key is what selects the variant.


This seems pretty janky for a few reasons:

The comment may not even be right - the FEC fields are both fec, right?

Why would their speed fields be named differently when they're specifying the same thing?

If we fix 2, do we now break our ability to distinguish between the variants?

Should we either have some explicit kind = "ddm|uplink" field in UnvalidatedPortConfig, or make it itself an enum and push this up a level?

Agreed. This was gross, and I'm ashamed of sending it out for review that way. I changed this to properly serialize UserSpecifiedPortConfig with a tag, and then deserialize into UnvalidatedPortConfig as an intermediate so that we could handle untagged uplink versions still, which is what we currently use in our production RSS files.

This still needs a bit of testing on my side, but looks right to me.

This doesn't work right. The direction is correct, but I'm going to have to play around with the structure a bit.

Ok, this works now as of 3f57914.

I added intermediate untagged values to deserialize into and utilized adjacent tagging. This allows automated serialization and deserialization of UserSpecifiedPortConfig that also deals with legacy untagged uplinks.

andrewjstone · 2026-10-06T00:50:29Z

I think this is pretty close @jgallagher except for the question around the sw prefix.

Given all the changes though, I want to give it another run through voxel tomorrow.

andrewjstone added 4 commits August 20, 2026 18:16

Change UserPortConfig

5207b94

wip

69aa3c0

ddmd scrimlet reconciler

7e2940d

fix initialization order

dbc026b

andrewjstone assigned nickelization Aug 24, 2026

andrewjstone mentioned this pull request Aug 24, 2026

Multirack rss config oxidecomputer/voxel#42

Open

andrewjstone added 5 commits September 11, 2026 21:45

Remove duplicate type

819c76b

latest maghemite on the branch

e5d7425

make ddm-admin in switch zone available to global-zone

23dddf9

Put states in the right order

22d7f3f

remove unnecessary comment

91edece

andrewjstone commented Sep 11, 2026

View reviewed changes

Comment thread sled-agent/scrimlet-reconcilers/src/handle.rs Outdated

remove accidentally committed test configs

4239638

andrewjstone commented Sep 11, 2026

View reviewed changes

Comment thread sled-agent/src/services.rs Outdated

andrewjstone mentioned this pull request Sep 16, 2026

Update pinned Dendrite revision #11299

Merged

andrewjstone added 8 commits September 16, 2026 18:23

wip - use new ddm api

192684a

Merge branch 'main' into multirack-join-service-part-4

9be1ae4

fix build after merge from main and new maghemite branch changes

1dc1ee6

bump all maghemite related versions

4af7970

openapi, fmt, hack

68f769f

expectorate

e766abd

Merge branch 'main' into multirack-join-service-part-4

523af01

bump maghemite

cc8b829

andrewjstone mentioned this pull request Sep 19, 2026

derive switch zone ddm router id from baseboard id #11326

Merged

derive switch zone ddm router id from baseboard id (#11326)

a95995b

andrewjstone mentioned this pull request Sep 21, 2026

dynamic external ddm peers oxidecomputer/maghemite#911

Merged

andrewjstone commented Sep 21, 2026

View reviewed changes

Comment thread Cargo.toml Outdated

cleanup docs

43ed170

andrewjstone marked this pull request as ready for review September 21, 2026 19:22

andrewjstone requested a review from jgallagher September 21, 2026 19:22

andrewjstone unassigned nickelization Sep 29, 2026

jgallagher reviewed Oct 1, 2026

View reviewed changes

andrewjstone added 6 commits October 1, 2026 20:55

Some review fixes for John

31c0920

remove to_uplink_port_config

9474b73

Document our addrobj

bb5a921

Better serialization of toml

16447ed

arbitrary

9642f44

Do not allow conversion from DdmAuto

5a519df

andrewjstone mentioned this pull request Oct 2, 2026

Allow Changing the RackNetworkConfig for the Multirack Join Service until Nexus can connect #11407

Open

andrewjstone added 2 commits October 2, 2026 22:35

Merge branch 'main' into multirack-join-service-part-4

3a6582b

Also bump magehmite versions

clippy

7ce09e9

jgallagher reviewed Oct 5, 2026

View reviewed changes

andrewjstone added 5 commits October 5, 2026 22:19

review fixes

0c32a1d

Fix deserialization of UserSpecifiedPortConfig

3f57914

addrobj

09bb660

at least one uplink

afdd508

typo

70b539b

fix tests

59d655b

		// The two variants name their speed and FEC fields differently, so the
		// required speed key is what selects the variant.

Conversation

andrewjstone commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrewjstone commented Sep 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewjstone Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewjstone commented Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andrewjstone commented Aug 24, 2026 •

edited

Loading

andrewjstone Oct 6, 2026 •

edited

Loading