[flowerthrower]

🤖 AI text below 🤖

Description

Extend Core's shared SPANK component to run the QDMI checker as the job user with the effective job environment after configuration injection. Validation remains disabled unless an administrator supplies the checker path. Provider code stays outside Slurm daemons.

Each node runs one check per job step and shares its result across tasks. The checker uses Slurm's borrowed job environment directly. A failed check blocks tasks on that node; waits and child-process cleanup remain bounded.

Automatic validation requires the same site-established provider runtime, catalogue, and credentials for all tasks on each node, configured before the hook. For environments activated later in a batch script, the guide shows an explicit checker invocation after setup. This is a readiness snapshot, not a reservation, authorization, or atomic launch guarantee across nodes.

Fixes #2313. Stacked on #2600 and depends on #2599.

Validation

• Full Nox lint, changed-file C++ lint, and all 42 focused Python tests pass.
• The complete Slurm 25.11.2 Docker suite passes locally on Linux ARM64 with the final wheel. It covers four MPI ranks across two nodes with two checks, success/failure/timeout, identity and environment, separate batch/nested steps, descendant cleanup, license release, and usable nodes.
• Cancellation passes with a 300-second checker timeout and five-minute batch limit, beyond the 120-second cancellation observation window.
• The standalone SPANK module builds in the Docker fixture. The host-only build lacks Slurm development headers.
• Hosted checks are still running; the failed aggregate belongs to cancelled duplicate run 37966954465. These validation results are local.

Codex assisted implementation, validation, and this description. Human review and acceptance are pending.

Checklist

• The pull request only contains commits that are focused and relevant to this change.
• I have added appropriate tests that cover the new/changed functionality.
• I have updated the documentation to reflect these changes.
• The changes follow the project's style guidelines and introduce no new warnings.
• The changes are fully tested and pass the CI checks.
• I have reviewed my own code changes.

If PR contains AI-assisted content:

• Any agent that created, edited, or submitted GitHub content was explicitly authorized for that scope, as required by our AI Usage Guidelines.
• Every agent-authored or agent-edited public text body begins with the visible disclosure 🤖 *AI text below* 🤖 (titles are exempt).
• I have disclosed AI assistance in the PR description.
• I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[burgholzer]

🤖 AI text below 🤖

Early design feedback on 7d035f87: recommend making the supported context explicit—one provider configuration per node/job step, with the runtime, catalogue, and credentials established before validation and consistent across that step's tasks. This is a deliberate scope reduction from the current per-task configuration support.

That contract allows one shared bounded result per node/step and removes environment fingerprinting and fallback checks. Ordinary MPI rank metadata should not multiply provider queries. Preserve job identity, explicit job-environment passing, node-safe rejection, deadlines, cancellation, and child cleanup.

With the production changes described inline, remove test/slurm/validation_environment.cpp and its build/license plumbing, the task-specific configuration scenario, and the artificial 128 KiB rejection scenario. Keep two-node/four-task MPI success, failure, and timeout tests, expecting two checker calls. Also keep distinct batch/nested-step results, effective configuration, no payload on failure, license release, node health, and descendant cleanup. The source-inspection estimate is about 140–160 fewer production/fixture lines; no patch has been applied.

The 22 lightweight runner tests passed at this head. A compiled probe using the exact validation header confirmed that two otherwise identical task environments invoke the checker once, but changing only PMI_RANK causes two invocations. The full Slurm cluster was not rerun during this review.

[burgholzer]

🤖 AI text below 🤖

Re-reviewed 28935230. Both earlier requests are addressed: shared state now represents one result per node/step, and the guide explicitly requires the provider runtime/catalogue/credentials to exist before the hook. Keep the bounded wait, process groups, parent-death handling, and cleanup before reaping.

Focused compiled probes using the current header passed: eight callers with different PMI_RANK values produced one checker invocation; killing the winning task caused a waiting caller to fail within the bounded deadline. The lightweight runner suite passed 22/22, and current hosted Slurm checks pass; no full cluster was rerun locally.

Two follow-ups are inline: make the cancellation test independent of its own timeout, and merge the two validation-success allocations into the stronger MPI case. The former is a test-oracle gap, not evidence of broken production cancellation.

Optional documentation wording: use “The checker accepts” instead of “The checker still accepts”, and make “leave automatic validation disabled” explicitly administrator-facing. A job author cannot disable a site-wide validate= setting; this needs clarification, not a new option.

[burgholzer]

🤖 AI text below 🤖

Superseded by the consolidated implementation in #2599 (b965ed922). Automatic task-hook validation and its shared state are removed. Jobs use the explicit availability command after selecting their runtime and credentials; SPANK only supplies license-specific site defaults. The consolidated PR contains the reusable cluster and end-to-end tests. Closing this separate layer so the integration can be reviewed together.