Skip to content

[Engine] reject duplicate GPUs in automatic replica groups - #3387

Open
j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:codex/strata-duplicate-replica-gpus
Open

j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:codex/strata-duplicate-replica-gpus

Conversation

@j-luwierski

Copy link
Copy Markdown
Contributor

Problem and result

On the current upstream main, gpu_groups() accepts overlapping GPUs when a numeric replica count derives groups from the configured GPU list. Reproduced before the fix: {"gpu":[0,0],"replicas":2} returns [[0],[0]]; {"gpu":[0,1,1,2],"replicas":2} returns [[0,1],[1,2]]. The server can then start independent replica engines on the same card. After this change, these configurations raise ReplicaConfigError before replica engines start.

Related discussion

PR #2953 was closed during the 0.2.0-beta tracker reset, not declined. The original change was revalidated against current main. Search of open and closed issues and PRs for gpu_groups, duplicate cards and replica GPU grouping found no other fix for this case.

Approach

Build numeric groups and pass them through the same non-empty and duplicate-card validation used for explicitly named groups. Add regression coverage for duplicate GPU IDs in list and comma-separated forms, normalized duplicate IDs, explicit groups, preserved card order, and disabled replica counts. Valid group ordering and the no-replica path remain unchanged. No engine backend or defaults are changed.

Validation

Passed:

  • Reproduced the issue on upstream main at e88ee39d1d6be6e3364183af3b39b62a7ed6f211, Linux Debian 13, Python 3.13.5: duplicate GPU configurations returned overlapping groups instead of raising an error.
  • uv run --with-requirements requirements.txt python -m unittest serve.test_replicas serve.test_parallel -q, on the change based on e88ee39d1d6be6e3364183af3b39b62a7ed6f211, Linux Debian 13, Python 3.13.5: 64 tests passed in 71.163 seconds. These tests use fake engines; no model or GPU was used.
  • git diff --check: passed.

Failed:

  • The initial python3 -m unittest serve.test_replicas serve.test_parallel -q attempt could not import jinja2 from the system Python. The same suite then passed when run with the pinned repository requirements in a temporary uv environment.

Not run:

  • Multi-GPU inference and CUDA, HIP, SYCL or Metal builds; this change touches only Python server code and was verified with fake engines.

Limits

This validates configuration handling and fake-engine server behavior. It does not establish GPU runtime behavior or performance on hardware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant