Skip to content

feat(aorta): multi_node configuration surface (schema, yaml, docs) - #328

Merged
speriaswamy-amd merged 2 commits into
surya/aorta-mn-02-single-node-fixesfrom
surya/aorta-mn-03-config
Sep 11, 2026
Merged

speriaswamy-amd merged 2 commits into
surya/aorta-mn-02-single-node-fixesfrom
surya/aorta-mn-03-config

Conversation

@speriaswamy-amd

Copy link
Copy Markdown
Collaborator

Stack 3/6 — splits #171. Base: #327.

Why

Declares the multi_node: block that the disaggregated torchrun launch consumes. Config plumbing only — nothing reads these values yet, so behavior is unchanged. Split out so the launch logic in #329 is reviewable on its own; review this one for names, defaults, and docs.

What changed

  • cvs/parsers/schemas.py — AortaMultiNodeConfigFile (extra="forbid", master_launch_mode restricted to auto/script/torchrun), wired in with a default factory so yamls predating the block still validate. validate_paths_exist() checks train_script only when the mode is explicitly torchrun.
  • cvs/runners/aorta.py — matching AortaMultiNodeConfig dataclass, plus AortaConfig.node_vpc_ips so master_addr can later resolve to the head node's RDMA-fabric address rather than its mgmt/SSH address (the same node_dict vs vpc_ip split rccl_perf.py already uses).
  • aorta_benchmark.yaml / docs/.../aorta.rst — the block with inline docs and a parameter table.
  • cvs/tests/benchmark/test_aorta.py — fixture maps the validated block onto the dataclass and populates node_vpc_ips from the cluster file.

Test

ruff clean. Unit tests 595 → 603. Adds cvs/parsers/unittests/ (per AGENTS.md, the package had none): schema defaults, unknown-key and bad-mode rejection, conditional train_script check, and backward compat for configs with no multi_node: block.

@speriaswamy-amd

Copy link
Copy Markdown
Collaborator Author

Closing/reopening to force GitHub to recompute mergeable state (stuck as 'dirty' despite a verified clean fast-forward merge locally).

Comment thread cvs/parsers/schemas.py
speriaswamy-amd and others added 2 commits September 10, 2026 19:09
Declares the `multi_node:` block that the disaggregated torchrun launch path
consumes. This commit is config plumbing only - nothing reads these values yet,
so behavior is unchanged. It is split out so the launch logic that follows is
reviewable on its own.

- schemas.py: AortaMultiNodeConfigFile (extra="forbid", master_launch_mode
  restricted to auto/script/torchrun), wired into AortaBenchmarkConfigFile with
  a default factory so yamls predating the block still validate. validate_paths_
  exist() checks train_script only when the mode is explicitly 'torchrun'.
- aorta.py: matching AortaMultiNodeConfig dataclass, plus AortaConfig.node_vpc_ips
  so master_addr can later resolve to the head node's RDMA-fabric address rather
  than its mgmt/SSH address (same node_dict vs vpc_ip split rccl_perf.py uses).
- aorta_benchmark.yaml / aorta.rst: the block with inline docs and a parameter
  table.
- tests/benchmark/test_aorta.py: fixture maps the validated block onto the
  dataclass and populates node_vpc_ips from the cluster file.

Adds cvs/parsers/unittests/ (the package had no unit tests) covering the new
schema defaults, rejection of unknown keys and bad launch modes, and the
conditional train_script path check.

Co-Authored-By: Claude <noreply@anthropic.com>
…er_addr

The class docstring said single-node clusters "ignore this block," which
is only true for the auto default -- an explicit master_launch_mode:
torchrun on one node still activates train_script/extra_env/master_addr.

master_addr's description said it defaults to "the head node hostname/IP
from the cluster file," without mentioning it prefers node_vpc_ips first.
Someone reading only that text could pin the SSH/management address and
break RDMA fabric rendezvous. Documented the real resolution order in the
schema, sample yaml, and rst docs.

@amd-droy amd-droy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

@speriaswamy-amd
speriaswamy-amd removed this pull request from stack #333 September 11, 2026 00:23
@speriaswamy-amd
speriaswamy-amd added this pull request to stack #414 September 11, 2026 00:23
@speriaswamy-amd
speriaswamy-amd merged commit 519fb88 into main Sep 11, 2026
4 checks passed
@cijohnson
cijohnson deleted the surya/aorta-mn-03-config branch September 15, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants