test(fuzz): exercise live icomplete resizing - #23
Conversation
1df9a26 to
cd7760e
Compare
|
I ran a two-round adversarial review of the new fuzz oracles using six independent, persona-inspired review lenses. Each item below was reproduced with a temporary local mutation and then challenged by another reviewer. These are test-harness or CI-observability issues, not claims that the current production code is broken. Requested before merge1. Tie the live observations to the behavior being tested
Both halves fail negative controls:
There is a deterministic causal check that matches production's Please use an observation-driven handshake rather than a fixed sleep. Since production intentionally pins the session to 2. Preserve live failures and their seeds in CIThe runner supports Please set a result file, capture Nonblocking follow-upEvery case currently types the same Baseline live runs passed on the tested Emacs versions before mutations, including 40 local PTY cases. The isolated worktree was restored and verified clean after every probe. |
cd7760e to
0ad940b
Compare
0ad940b to
aa1e8bd
Compare
|
Phase-two oracle work is now on this branch at The requested findings are addressed:
The canaries and four generated cases pass locally through a pseudo-terminal. This qualifies the icomplete contract C14. Vertico, Emacs 31's built-in eager |
What this adds
This is stack PR 3 of 3. It drives a real icomplete-vertical minibuffer through
Emacs's command loop and checks the window after actual renders.
Batch Emacs cannot enter a live minibuffer, so CI runs terminal Emacs inside a
pseudo-terminal supplied by
script.There are no runtime changes in this PR.
This PR is based on #22, which is based on #21. Until those PRs merge,
GitHub's full diff includes the earlier commits. Review this PR's incremental
change here:
dangduc/fzfa@codex/fzfa-fuzz-native-producers...codex/fzfa-fuzz-live-icomplete
Simple example
One case starts with a target such as
alphaand many distractors:It then does this through Emacs's real input loop:
Cases choose among
alpha,quartz,violet, andmango, and vary thenumber of distractors.
What the oracle checks
The test records each fit call after the real function runs. One observation
contains:
The handshake has three checkpoints.
1. Initial empty query
The logical candidates must equal the complete input set, the display must use
more than one line, and the real fit must satisfy:
This distinguishes a working fit from a no-op. In the historical failure
shape, a one-line mini-window could remain short even when icomplete had many
rows to display.
2. Full narrow query
The target must still be present, at least one candidate must remain, and the
logical set must be smaller than the initial set. This prevents a display that
ignores the query from passing just because it has multiple lines.
3. Fresh empty query after deletion
The observation's sequence number must be newer than the narrow observation.
The full candidate set and multiline display must be restored, and the window
must be at least as tall as it was after the initial fit. Production uses a
grow-only session policy, so the contract is “restore the display without
collapsing,” not “shrink and grow again.”
Keys are queued only after the expected query has rendered. There is no fixed
sleep between keys. A watchdog sends
C-gand reports the current phase andexpected query if the handshake stops making progress.
Oracle qualification
Before generated cases run, three controlled canaries use the real minibuffer:
The intended oracle must reject each canary. This qualifies FZFA-C14 in the
contract catalog. It does not cover Vertico, Emacs 31's built-in eager
*Completions*frontend, or pixel-level rendering.The temporary observer advice is removed after every case. Each ordinary case
also requires fzfa's icomplete advice count and membership to return to their
starting values. A separate check installs the advice twice and removes it
twice to exercise nested-session reference counting.
CI failure output
CI writes the final live result to a file outside the pseudo-terminal, always
prints that file, and then returns
script's original exit status. A failureincludes its seed, target query, generated event list, and every recorded
observation, so the case can be replayed without reading terminal control
sequences.
CI and local use
Pull requests run eight live sessions on Emacs 29.1, 30.1, and snapshot. The
weekly job runs 40 sessions on each version.
Verification performed
Stack