Repository navigation
fix(audit): keep example commands, usage wraps and wrapped sentences out of the flag definitions - #181
Open
brettdavies wants to merge 3 commits into
Conversation
…out of the flag definitions
An indented line that starts with a dash was a definition unless it carried on the description above it. Three shapes that are not definitions got through:
- An example command carried over a line with a backslash. `aws s3 ls s3://amzn-s3-demo-bucket \` followed by `--recursive` declared `--recursive`.
- A usage synopsis that wraps onto a required option. argparse prints `usage: prog [-h] [--verbose] --input INPUT --mode {fast,slow}` and wraps to `--output OUTPUT --format FORMAT`, which declared `--output` and `--format` from a usage line.
- A sentence that wraps onto a flag name. java's ` -m or --module <module>/<mainclass> are passed as the arguments to` declared `-m`.
A phantom definition can credit a flag the help does not declare, and one `--name` among them switches off the single-dash rule for a Go `flag` help.
The classifier now reads each of those as text: a line under one that ends in a backslash, a line indented to the arguments of the usage line above it, and a line that reads on as a sentence directly under prose at the same indent.
It also stops losing a real layout. Under a short flag with no description, clap prints the long-only rows at the long column, and they were read as that flag's description: ` -x` followed by ` --format <FORMAT>` declared only `-x`. A short at column c now marks column c + 4 as a name column, where `-x, --long` puts its long name. A row there with a description of its own is a definition even when it sits at the description column of the row above.
Across the 71 fixture captures, six rows go: five in the aws capture (two examples and a wrapped sentence) and the java sentence. Nothing else changes.
This was referenced Oct 9, 2026
Open
brettdavies
added this pull request to stack #166
October 9, 2026 15:40
A second review pass found layouts where each of the three text rules dropped a real definition, and one place where the long-column rule turned a wrapped description into a flag. - **Backslash.** A line under one that ends in a backslash was text. A description can end in a backslash too (`Field delimiter, default \`), and the definition after it was lost. The rule applies only while no description is being read. - **Usage synopsis.** Every line indented to the usage line's arguments was text until a blank or shallower line. A tool that prints its option rows in the same block, at or right of that column, lost all of them. A row whose own description a gap sets off is a definition; a synopsis wrap never has one. - **Wrapped sentence.** A flag-led line with a one-space description was text under any same-indent line that was not a definition. Rows written `-h, --help: show this help` under an indented `Options:` label were all dropped, and each dropped row counted as prose for the next. A sentence can only wrap from a line that has not ended, so the line above has to run on: it must not end in `.`, `:`, `!` or `?`. - **Long column.** A row at a name column with a description of its own was a definition even at the description column of the row above. ` -1 means no limit.` under ` -n Number of results to return.` became a flag. A wrapped description line is never a definition, so that exception is removed. Under a definition with no description, a line at a name column is a definition only when it declares a name, so a bulleted description (`- fast: skip verification`) stays with its flag. The definitions read from the 71 fixtures and from the 910 registry captures are unchanged by this commit.
…ts names nmap closes a row's names with a colon and one space: `-iL <inputfilename>: Input from list of hosts/networks`. Under its `Ex: scanme.nmap.org, ...` line, which does not end a sentence, the wrapped-sentence rule read `-iL`, `-iR`, `--exclude` and `--excludefile` as the rest of that sentence, and each dropped row counted as prose for the next. A third review pass found it, and nmap's help lost four of its 87 definitions. A colon that ends a name or its placeholder now sets the description off, as a gap and the markers `#`, `=`, `--` and `-` already do. All three rules that ask whether a description is set off agree on it: the wrapped-sentence rule, the usage-wrap rule, and the rule for dash-led text at column 0. The last one means python's column-0 rows `--help-env: print help about Python environment variables and exit` are read as definitions. A header with no description (`-A, --all-namespaces=false:`) and a sentence that carries on after one space (`-m or --module <module>/<mainclass> are passed ...`) are not set off. A new case pins a sentence that wraps onto two flag-led lines, the only shape that needs a dropped row to count as prose for the next. The definitions read from the 71 fixtures and the 910 registry captures are unchanged. nmap's usage text (`docs/nmap.usage.txt` upstream, Nmap 7.991SVN) reads 87 definitions again. Of 121 help texts captured on the development machine, one changes: python3 gains `--help-env`, `--help-xoptions` and `--help-all`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
From the code review of the stack. An indented line that starts with a dash was a definition unless it carried on the description above it, so three shapes that are not definitions still got through:
aws s3 ls s3://amzn-s3-demo-bucket \followed by--recursivedeclared--recursive.usage: prog [-h] [--verbose] --input INPUT --mode {fast,slow}and wraps to--output OUTPUT --format FORMAT, which declared--outputand--formatfrom a usage line, while four audits' evidence says usage lines are not read.-m or --module <module>/<mainclass> are passed as the arguments todeclared-m.The parser on
devreads these lines the same way, so this is not something the stack broke. It is what the stack's README section and evidence text now promise, and one consequence is new: a single--nameamong such lines switches off the single-dash rule for a Goflaghelp.The classifier reads each shape as text, and each rule is narrow enough to keep the definitions that sit where such text could:
Field delimiter, default \), and the definition after it is still read.usage:line above it, unless a gap or a marker sets off a description of its own. A tool that prints its option rows in the same block as its usage line keeps them; a synopsis wrap never has such a description..,:,!or?). Rows written-h, --help: show this helpunder an indentedOptions:label are still read.A colon that ends a row's names sets its description off, as a gap and the markers
#,=,--and-already did. nmap is why: it prints-iL <inputfilename>: Input from list of hosts/networksand three more rows directly under anEx: scanme.nmap.org, ...line that does not end a sentence, and without the colon rule the wrapped-sentence rule drops all four. The same marker counts for dash-led text at column 0, so python3's--help-env: print help about Python environment variables and exitand the two rows beside it are read.It also stops losing a real layout, which is new in this stack (#168). Under a short flag with no description, clap prints the long-only rows at the long column, and they were read as that flag's description:
That help declared
-xand--helponly. A short at column c now marks column c + 4 as a name column, where-x, --longputs its long name. Under a definition that has no description yet, a line that declares a name at a name column is a definition and not that description. A wrapped description line is still never a definition:-1 means no limit.under-n Number of results to return.stays with-n, and so does a bulleted description (- fast: skip verification) under an undescribed flag.What this does not do:
-state, state-out, and -backup are legacy options ...is the known case.insert's--text --convert '...'follows a backslash and is now text, and an example line with no backslash above it is still read.Changelog
Fixed
usage:synopsis, from an example command continued with a backslash, and from a sentence that wraps onto a flag name. None of those lines declares a flag.-xabove--format <FORMAT>.--help-env: print help about Python environment variables and exit.Documentation
--helpnames the three text shapes, the long-column rule, and the colon among the description markers.Type of Change
fix: Bug fix (non-breaking change which fixes an issue)Related Issues/Stories
docs/plans/2026-10-06-0034-fix-help-flag-names-across-frameworks-plan.md(requirements R4 to R6; review follow-up)Testing
Test Summary:
classify.rsand one inheader.rs. Six excerpts of indented text declare nothing (two aws examples, the aws and java sentences, a sentence that wraps onto two flag-led lines, an argparse 3.14.8 usage wrap). Thirteen definitions that sit where such text could are still read, among them rows directly under a usage line, BINDhost's anddig's rows under a wrapped usage line, colon-described rows under an indented label, nmap's rows under itsEx:line, python3's column-0 colon rows, one-space rows whose descriptions wrap back to the name indent, and a definition after a description that ends in a backslash. An example's--out xno longer switches off the single-dash rule. The clap layout above declares all four of its flags, and a wrapped or bulleted description at the long column stays a description. A gap, a marker or a closing colon sets a description off; a header with no description and a sentence after one space are not set off.cargo test --test dogfoodpasses.Negatives observed failing. The branch has three commits, and the tests were run against the classifier and tokenizer before each. Against the base (#180):
Against the first commit, which wrote the three rules without the narrowing:
Against the second commit, which had no colon marker:
The two-line sentence case fails only against the base. It pins that a dropped row counts as prose for the row under it, so it was also run against the head with that carry-over removed:
Expected moves. Written before the corpus run.
No row moves. Across the full registry capture set (910 help texts) the change removes 14 definitions and adds none, and no audit asks for any of the 14:
sqlite-utils insert-files --help-c name:name \and five more-clines, then--pk namesqlite-utils transform --help--drop column1 \,--rename column2 column_renamedsqlite-utils convert --help--import=textwrapsqlite-utils insert --help--text a "text" variable.) and an example (--text --convert '...')cargo-binstall --help--version option.xr --help--color always is set) and human-only banners ...Each was read against its capture. cargo-binstall and xr declare
--versionand--coloron real definition lines elsewhere in the same help, so their rows hold. Replaying the capture set through the base and head builds moves no row. The second and third commits read the same definitions from all 910 captures as the first: no registry tool prints the layouts they fix. nmap is not in the registry; its upstream usage text (docs/nmap.usage.txt, Nmap 7.991SVN) reads 87 definitions at the base and at the head.Corpus before/after. Run after the table above was written, and run twice. Base
8134d09b7510, headafbfeffb80a4. Imagesha256:05db16cf226cd14a93a2682aea41b72d3e6b4d240cadba006f2881d3ffa996b3, networkbridge. 98 tools selected, 95 scored under both builds, 1 rerun three times (mods).The first run reported one moved row, and it is the one row on the A/A noise list:
Moved rows (1)
p3-should-about-long-aboutp3-about-long-about-hand--helpproduce byte-identical output. SHOULD-tier — clap renders the short summary on-hand the full description on--helpwhenlong_aboutis set; collapsing them gives agents no concise list-level grep target.head: null
Derived fields moved (4)
badge.score_pctbandsummary.passsummary.warnThe harness reports a noise-listed row as moved only when the two builds share no result. In this run all four base runs returned warn and all four head runs returned pass. That split comes from the row, not from this change. The row compares mods's
-hand--helpoutput byte for byte, and no file this PR touches is read by that audit. Each build returns both results in other runs: the base build returned warn, pass, pass, warn as the head of #180's run, and this head returned pass, pass, pass, warn as the base of #182's run.The second run, with the same two builds in the same image:
Moved rows (0)
None.
Derived fields moved (0)
None.
In the second run the mods row returned warn, pass, pass, pass under the base and warn, pass, warn, pass under the head, so it is reported as not moved. No row and no derived field moved, as expected. In neither run is a row unstable, a harness bug flagged, or a tool scored under one build only.
cursor,nvidia-smiandxai-grok-buildare absent from the image and ran under neither build.Files Modified
Modified:
src/runner/help_probe/flags/classify.rs: the three text shapes with their narrowing, the long column after a short, and their tests.src/runner/help_probe/flags/header.rs: a colon that ends the names sets the description off.README.mdsrc/runner/help_probe/snapshots/aws__s3_ls_help.snap,java__-help.snapCreated:
Renamed:
Deleted:
Breaking Changes
Deployment Notes
Checklist