Repository navigation
[2/3] Add request sampling controls to all pinned Drama engines - #27
Conversation
Restore the "answers as upstream does without flags" rule for the full-model patches and state the per-request temperature, top_p, top_k and seed fields from this PR (0013/0014, CUDA 0002, Metal 0004/0005) as the request-level exception: they need no launch flag, and requests without them are unchanged.
|
Thanks! The validation and the seed handling look right to me, and requests without the new fields keep exactly the old seeds and parameters. One non-blocking ask for TensorFold #6 (and so CUDA patch 0002): after this change |
|
My CUDA checks for #27 pass: seeds repeat byte for byte (including 2^53+1 and 2^64-1), out-of-range From my side this is ready. Please mark it ready for review whenever you're happy, and I'll merge it right after #26. The small README and patch-subject touch-ups for #28 I mentioned earlier I'll add once this one is in. |
|
Thanks for catching the unused helper, and for the CUDA checks and merge. The Redraws.salt cleanup is now in source companion #7 at e06a0cda and recipe #28 at 9ec2318. Because #27 has already merged, I appended CUDA patch 0004-remove-unused-redraw-salt.patch and its checksum rather than rewriting the published 0002. The helper has no remaining callers; the live request sampling paths are unchanged. Applying all four CUDA patches from the pinned b1f4b9b4 tree reproduces the updated source exactly (3,503 files compared). The legacy CPU server suite passes 379 tests with four skipped; the recipe suite ran 48 tests with 15 skipped and zero failures. |
Description and motivation
Part 2 of 3. Depends on #26; merge that first, then this PR, then #28. The incremental recipe change is 37a6832...cb20bda.
Expose per-request
temperature,top_p,top_kand exact unsigned 64-bitseedin all three pinned engines without changing their pins or default settings. Add full-model patches 0013/0014, Metal NVFP4 patches 0004/0005, and CUDA NVFP4 patch 0002, each with exact checksum coverage. Source companions: modern #1277 and legacy CUDA #6.Validation happens before engine submission. Tests inspect the actual sampler fields sent through whole/streamed requests, reference rebuilding, CFG and redraws. Explicit seeds retain all 64 bits; redraws add their take ordinal with wrapping arithmetic. Legacy long-form applies that policy separately to each segment. Documentation states the existing JSON-f64 underflow boundary and native sampler differences.
The public Sogni-AI/drama3-mlx API is the behavioral reference. Native Zig changes were implemented independently.
Testing
scripts/build-engine.shreapplies the complete selected series from each clean pin: full-model 66/66 build steps and Metal NVFP4 73/73 build steps passed on macOS arm64 with Zig 0.17.0. Native server and selected voice tools compile. Applied trees exactly equal reviewed source commits1c31b4d2and2af18fba.b1f4b9b4; tree equals reviewed22939c2a. Exact exported-tree CPU suite: 371/375 passed, four skipped.test-pull-hostaborts with ISCONN, andtf-flashnext-runfails on SharedEvent/Event. Both reproduce on the untouched base; this is not a passing whole-upstream build claim.During final CPU replays, two unchanged recall tests failed while the volume had about 3.3 GiB free, below their existing 4 GiB admission floor. Removing only task-created compiler caches restored 5.043 GiB; the identical commands passed with no source, environment or admission-guard change. Original failure receipts were retained.
Performance or quality claim
None. Equal settings do not promise equal audio across native backends.
Checklist