Repository navigation
dory: KV pages follow-ups (#1284) - #1387
Conversation
|
Thanks @jschmied. These are exactly the three follow-ups we listed when #1284 landed, and nothing else. Next we'll run Dory 1.0 4B NVFP4 on a GB10: the default path against main by token hash, paging on against paging off, and |
…refuses a context off the page size, window_pages only with paging (ashhart#1284)
cb36710 to
f759da5
Compare
|
Thank you @jschmied. This is in TensorFold 1.2.0, released today: https://github.com/ashhart/TensorFold/releases/tag/v1.2.0 It landed with your commits under your own name, so you're listed as a TensorFold contributor, and it's credited to you in the release notes and the changelog. |
The three open items from review of #1284, nothing else.
max_slotsisround_rows / window_rows(cuda_pages.slotsOf), not a literal 8. A test pins the derivation at other geometries.attachrefuses a context that is not a whole number of 256-position pages (cuda_pages.perSlot,error.BadAttach). Served contexts come fromcapacity()(multiples of 1024) and are unaffected.Kernels.load(ctx, paged)loadswindow_pages.cuonly when paging is on (TENSORFOLD_DORY_KV_PAGES).initPaged/openBytesPagedreturnerror.PagesNotLoadedover kernels without it, instead of reaching an unset function.Tests
zig build test-dory-hostcovers the derivation and the refusal. Each fails with its check removed (derivation halved: "expected 8, found 4"; refusal removed: "expected error.BadAttach, found 133").tf-cuda-test dory-pages, L=2): paging off loads no module, a paged model refuses such kernels, paging on loads it; fails when the load is made unconditional. The existing paged == contiguous checks still pass.dory-serveL=2: paging on and paging off pass; the paging-off run has the same token hashes as main.zig build test,zig fmt --check, and the lean check (0 new against main) are clean.The default path only skips one module load at open, so no speed A/B was run.