Skip to content

dory: KV pages follow-ups (#1284) - #1387

Merged
ashhart merged 2 commits into
ashhart:mainfrom
jschmied:dory-pages-followups
Oct 30, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
jschmied:dory-pages-followups

Conversation

@jschmied

Copy link
Copy Markdown
Contributor

The three open items from review of #1284, nothing else.

  1. max_slots is round_rows / window_rows (cuda_pages.slotsOf), not a literal 8. A test pins the derivation at other geometries.
  2. attach refuses a context that is not a whole number of 256-position pages (cuda_pages.perSlot, error.BadAttach). Served contexts come from capacity() (multiples of 1024) and are unaffected.
  3. Kernels.load(ctx, paged) loads window_pages.cu only when paging is on (TENSORFOLD_DORY_KV_PAGES). initPaged / openBytesPaged return error.PagesNotLoaded over kernels without it, instead of reaching an unset function.

Tests

  • Host: zig build test-dory-host covers the derivation and the refusal. Each fails with its check removed (derivation halved: "expected 8, found 4"; refusal removed: "expected error.BadAttach, found 133").
  • GPU (tf-cuda-test dory-pages, L=2): paging off loads no module, a paged model refuses such kernels, paging on loads it; fails when the load is made unconditional. The existing paged == contiguous checks still pass.
  • dory-serve L=2: paging on and paging off pass; the paging-off run has the same token hashes as main.
  • Full zig build test, zig fmt --check, and the lean check (0 new against main) are clean.

The default path only skips one module load at open, so no speed A/B was run.

@ashhart

ashhart commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Thanks @jschmied. These are exactly the three follow-ups we listed when #1284 landed, and nothing else. max_slots now comes from round_rows / window_rows (zig/src/families/dory/cuda_pages.zig:14), with a test that pins the derivation at other shapes. attach refuses a context that isn't a whole number of 256-position pages (perSlot, cuda_pages.zig:42, used at cuda_model.zig:145). And window_pages.cu loads only with TENSORFOLD_DORY_KV_PAGES on (cuda_kernels.zig:41, cuda_native.zig:107), while initPaged and openBytesPaged refuse kernels without it by name. The one-shot run path never pages, so it passes false, and served contexts come from capacity(), so nothing served changes shape. It merges cleanly onto current main, and CI is green on your head.

Next we'll run Dory 1.0 4B NVFP4 on a GB10: the default path against main by token hash, paging on against paging off, and dory-pages, dory-serve and dory-states with paging on, including your new module checks.

@ashhart
ashhart force-pushed the dory-pages-followups branch from cb36710 to f759da5 Compare October 30, 2026 14:42
@ashhart
ashhart merged commit abf8225 into ashhart:main Oct 30, 2026
9 checks passed
@ashhart

ashhart commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Thank you @jschmied. This is in TensorFold 1.2.0, released today: https://github.com/ashhart/TensorFold/releases/tag/v1.2.0

It landed with your commits under your own name, so you're listed as a TensorFold contributor, and it's credited to you in the release notes and the changelog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants