Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

## Unreleased

### Changed

- The full-model engine pins TensorFold v1.1.9 with an empty patch series: every behaviour of the
old patches 0001–0014 is upstream. Two differences remain: with the mid-silence rule on,
`X-Silence-Mid-Trips` reports only whether the returned take tripped (0 or 1), and a temperature
of `-0` or `-1e-400` reaches the sampler as `+0`. `scripts/build-engine.sh` accepts an empty
series only through an empty `SHA256SUMS`; a missing manifest still fails. The Linux native pilot
sets `TF_DRAMA_PREFILL_ROWS=128` for full and custom checkpoints to keep batched prefill; NVFP4
pins and the sglang reference backend are unchanged.

### Fixed

- CUDA NVFP4 long-form splitting preserves paired delivery tags, bracketed cues and special tokens,
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ series and check that the whole series still applies and builds from a clean che

```sh
cd build/tensorfold
git format-patch --start-number 12 -o ../../engine/patches HEAD~2 # two new commits; 0012 is the next free number
git format-patch --start-number 1 -o ../../engine/patches HEAD~2 # two new commits; the v1.1.9 series is empty, so 0001 is next
cd ../../engine/patches && sha256sum -- *.patch > SHA256SUMS
cd ../.. && ZIG=/path/to/zig-0.17.0/zig NVCC=/usr/local/cuda/bin/nvcc scripts/build-engine.sh
```
Expand Down
3 changes: 2 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ TensorFold
----------
The engine patches apply to TensorFold (https://github.com/ashhart/TensorFold), which is licensed under the
Apache License, Version 2.0. scripts/build-engine.sh clones TensorFold at the commit pinned in engine/UPSTREAM
and applies the patches with `git am`. On Linux, --weights nvfp4 instead clones the TensorFold fork
(v1.1.9) and applies the patches in engine/patches/ with `git am`; that series is currently empty, so the full
model builds unmodified TensorFold. On Linux, --weights nvfp4 instead clones the TensorFold fork
https://github.com/plotarmordev/TensorFold at the commit pinned in engine/nvfp4/UPSTREAM, also Apache-2.0,
and applies the ordered patches in engine/nvfp4/patches/ after checking their SHA256SUMS manifest. On Darwin,
--weights nvfp4 uses upstream TensorFold at engine/nvfp4/macos/UPSTREAM and applies the ordered patches in
Expand Down
63 changes: 32 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
**Fish Audio's Drama 3 text-to-speech on NVIDIA GB10 (native Zig/CUDA) or Apple silicon (native Zig/Metal).**

[TensorFold](https://github.com/ashhart/TensorFold) runs Drama 3 natively since v1.1.0. This repository pins
upstream TensorFold and adds a small series of opt-in patches for the full model. NVFP4 retains its
TensorFold v1.1.9 for the full model, which now carries every feature the old patch series added. NVFP4 retains its
measured CUDA engine pin with a checksummed patch series and uses a separate upstream pin plus Metal patches on Mac. It builds the engine
and starts a local speech server with an OpenAI-style `/v1/audio/speech` route, whole or streamed.

Expand Down Expand Up @@ -69,7 +69,7 @@ On Linux, `--weights nvfp4` builds [`engine/nvfp4/UPSTREAM`](engine/nvfp4/UPSTRE
guidance γ 1.5 and batched guidance, as measured; a `--cfg-gamma` of your own after `--` replaces both.
On Mac it builds [`engine/nvfp4/macos/UPSTREAM`](engine/nvfp4/macos/UPSTREAM) plus its patch series.
Metal's `--cfg-batched` pairs independent states in one command buffer, not a vectorized GEMM.
`--weights full` retains [`engine/UPSTREAM`](engine/UPSTREAM) and [`engine/patches`](engine/patches).
`--weights full` builds [`engine/UPSTREAM`](engine/UPSTREAM) with the empty [`engine/patches`](engine/patches) series.

When it prints `DRAMA_PILOT_READY`, synthesize from another terminal. The second command streams and plays as
it arrives:
Expand Down Expand Up @@ -159,34 +159,35 @@ See the [CUDA patch index](engine/nvfp4/patches/README.md) for scope and tests.

### Full-model engine

These patches build the full-model engine (`--weights full`). Upstream TensorFold already serves Drama 3. Each patch
below adds one feature that stays off until you pass its flag, so a server started without them answers as
upstream does. The exceptions are the first two and one request-level feature: batched prefill is on by default
(`TF_DRAMA_PREFILL_ROWS=1` restores upstream's single-row prefill), the replay fix is a bug fix, and per-request
`temperature`, `top_p`, `top_k` and `seed` (0013/0014) need no launch flag. A request without those fields is
answered as upstream answers it; upstream refuses them with a 400, and these patches serve them. The CUDA NVFP4
series (0002) and the Metal NVFP4 series (0004, 0005) carry the same request-level feature. The patches stack and
apply in order; each one's notes are in the engine tree's `CHANGELOG.md` or `docs/`.

| Patch | Adds | Opt in with | Upstream |
|---|---|---|---|
| 0001 (K0) | CUDA prompt prefill in bounded row chunks | on by default; `TF_DRAMA_PREFILL_ROWS` | [#1120](https://github.com/ashhart/TensorFold/pull/1120) |
| 0002 | Replay waits for poison fills before buffers are reused | always on | [#1119](https://github.com/ashhart/TensorFold/pull/1119) |
| 0003 (K1) | A configurable frame budget | `--max-frames N` | [#1134](https://github.com/ashhart/TensorFold/pull/1134) |
| 0004 (K7) | An allowlist for bracketed speech controls | `--allowed-controls LIST` | [#1152](https://github.com/ashhart/TensorFold/pull/1152) |
| 0005 (K9) | Attested voice catalogs and `GET /v1/voices` | `--voices DIR`, `--default-voice NAME` | [#1135](https://github.com/ashhart/TensorFold/pull/1135) |
| 0006 (K2) | A sustained-silence guard that ends a reply as `silence` | `--silence-floor-dbfs`, `--silence-seconds` | [#1157](https://github.com/ashhart/TensorFold/pull/1157) |
| 0007 (K3) | The guard's onset rule: silence before enough voiced audio | `--silence-onset-s`, `--silence-onset-voiced-s` | [#1158](https://github.com/ashhart/TensorFold/pull/1158) |
| 0008 (K4) | The guard's mid rule: an ending lost in a long silence | `--silence-mid-s`, `--silence-mid-min-voiced-s` | [#1159](https://github.com/ashhart/TensorFold/pull/1159) |
| 0009 (K5) | Whole replies the guard ended are drawn again | `--silence-redraws N`, `--silence-redraw-budget-s`, `--silence-redraw-controls` | [#1160](https://github.com/ashhart/TensorFold/pull/1160) |
| 0010 (K6) | Whole replies that stopped short of their text are drawn again | `--short-reply-ratio`, `--short-reply-chars-per-s` | [#1161](https://github.com/ashhart/TensorFold/pull/1161) |
| 0011 (K8) | A whole reply is drawn once more when a phrase opens far above the others | `--register-redraw-st`, `--register-redraw-high`, `--register-high-hz` | [#1162](https://github.com/ashhart/TensorFold/pull/1162) |
| 0012 | One bounded reference-prefix snapshot per Metal lane, with exact identity and safe invalidation | none (automatic) | [#1237](https://github.com/ashhart/TensorFold/pull/1237) |
| 0013 | Request-local speech sampling and exact integer seeds | HTTP `temperature`, `top_p`, `top_k`, `seed` | [#1277](https://github.com/ashhart/TensorFold/pull/1277) |
| 0014 | Parsed-JSON floating boundary documentation and regressions | none (tests/docs) | [#1277](https://github.com/ashhart/TensorFold/pull/1277) |

`drama3-voice`, a tool that writes attested voice files for `--voices`, is proposed upstream in
[#1153](https://github.com/ashhart/TensorFold/pull/1153); it is not part of this series.
The full-model engine (`--weights full`) is TensorFold v1.1.9 with no patches: every feature the old patch series
added is upstream. Each flagged feature stays off until you pass its flag, so a server started without them answers as plain
TensorFold does. The Linux recipe sets `TF_DRAMA_PREFILL_ROWS=128` for native full-model and custom-checkpoint
launches to keep batched prefill; TensorFold otherwise defaults to single-row prefill. NVFP4 and the sglang
reference backend do not receive this setting. Per-request `temperature`, `top_p`, `top_k` and `seed` need no
launch flag; the CUDA NVFP4 series (0002) and the Metal NVFP4 series (0004, 0005) carry the same request-level
feature. Each feature's notes are in the engine tree's `CHANGELOG.md` or `docs/`.

| In TensorFold 1.1.9 | Opt in with | Upstream |
|---|---|---|
| CUDA prompt prefill in bounded row chunks | `TF_DRAMA_PREFILL_ROWS` (the recipe sets `128`) | [#1120](https://github.com/ashhart/TensorFold/pull/1120) |
| Replay waits for poison fills before buffers are reused | always on | [#1119](https://github.com/ashhart/TensorFold/pull/1119) |
| A configurable frame budget | `--max-frames N` | [#1134](https://github.com/ashhart/TensorFold/pull/1134) |
| An allowlist for bracketed speech controls | `--allowed-controls LIST` | [#1152](https://github.com/ashhart/TensorFold/pull/1152) |
| Attested voice catalogs and `GET /v1/voices` | `--voices DIR`, `--default-voice NAME` | [#1135](https://github.com/ashhart/TensorFold/pull/1135) |
| A sustained-silence guard that ends a reply as `silence` | `--silence-floor-dbfs`, `--silence-seconds` | [#1157](https://github.com/ashhart/TensorFold/pull/1157) |
| The guard's onset rule: silence before enough voiced audio | `--silence-onset-s`, `--silence-onset-voiced-s` | [#1158](https://github.com/ashhart/TensorFold/pull/1158) |
| The guard's mid rule: an ending lost in a long silence | `--silence-mid-s`, `--silence-mid-min-voiced-s` | [#1159](https://github.com/ashhart/TensorFold/pull/1159) |
| Whole replies the guard ended are drawn again | `--silence-redraws N`, `--silence-redraw-budget-s`, `--silence-redraw-controls` | [#1160](https://github.com/ashhart/TensorFold/pull/1160) |
| Whole replies that stopped short of their text are drawn again | `--short-reply-ratio`, `--short-reply-chars-per-s` | [#1161](https://github.com/ashhart/TensorFold/pull/1161) |
| A whole reply is drawn once more when a phrase opens far above the others | `--register-redraw-st`, `--register-redraw-high`, `--register-high-hz` | [#1162](https://github.com/ashhart/TensorFold/pull/1162) |
| One bounded reference-prefix snapshot per Metal lane, with exact identity and safe invalidation | none (automatic) | [#1237](https://github.com/ashhart/TensorFold/pull/1237) |
| Request-local speech sampling and exact integer seeds, with parsed-JSON boundary docs | HTTP `temperature`, `top_p`, `top_k`, `seed` | [#1277](https://github.com/ashhart/TensorFold/pull/1277) |

Every old patch's behaviour is in v1.1.9 with two differences. With the mid-silence rule on, `X-Silence-Mid-Trips`
reports only whether the returned take tripped (0 or 1); takes discarded by silence/short/register redraws are no
longer counted. A temperature of `-0` or `-1e-400` reaches the sampler as `+0` rather than `-0`; both were already
greedy. `drama3-voice`, the upstream tool that writes attested voice files for `--voices`, is in TensorFold; the
full-model build script still builds only the native server and `drama3-generate`.

Pass serve flags to the launcher after `--`. An illustrative launch with the frame budget, the silence guard and
one redraw; the numbers are examples, not tuned values, so calibrate thresholds on your own voice:
Expand All @@ -195,7 +196,7 @@ one redraw; the numbers are examples, not tuned values, so calibrate thresholds
python3 recipe/pilot.py … -- --max-frames 2048 --silence-floor-dbfs -50 --silence-seconds 6 --silence-redraws 1
```

Patch 0004's cue list holds only words from Fish Audio's public tags; deployments may extend the cue list locally.
The speech-control allowlist holds only words from Fish Audio's public tags; deployments may extend it locally.

A voice catalog is a directory of `drama3-http-voice/1` files, each with a `source` attestation (`synthetic`,
`self` or `consented`); the format is in `docs/drama-http-voices.md` in the engine tree. Clone only a voice the
Expand Down
4 changes: 2 additions & 2 deletions engine/UPSTREAM
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# The upstream engine the patch series in engine/patches applies to.
# TensorFold v1.1.9, the upstream engine the patch series in engine/patches applies to.
TENSORFOLD_REPOSITORY=https://github.com/ashhart/TensorFold.git
TENSORFOLD_COMMIT=d1018b318fb51981c4a18f12e40269a4d3ee9713
TENSORFOLD_COMMIT=465e48b624202d027cb72144749753578846aa7c
Loading