Repository navigation
deblank: find the pages that hold nothing, and drop them - #42
Merged
Merged
Conversation
BlankPages draws each page and counts the pixels that are not paper;
Doc.DropBlank removes them and says which went, numbered as the reader saw
them. `pdfops deblank` exposes both, with -list as the dry run.
Drawing is the only test that agrees with a reader. A page whose content stream
is full of operators painting white, or drawing outside the media box, looks
blank and is blank; a page carrying a picture of nothing is blank too, and no
amount of reading the content stream says so.
⛔ THE DEFAULT RESOLUTION WAS MEASURED, AND THE FIRST ONE WAS WRONG. This
started at 50 DPI with a comment claiming that was "enough to see a full stop".
Measured on A4, at 50 DPI an eight-point full stop covers EXACTLY ONE PIXEL —
the same mark a speck of dust makes — so no threshold at that resolution can
tell punctuation from dirt. At 150 it covers four and a comma covers eight, so
there is a factor of four to set MaxInk from. TestMeasureTheInkLadder prints
the whole ladder at 50, 100, 150 and 300 so that a renderer change moving
either end of the gap fails there rather than silently reclassifying pages.
⛔ The zero value of BlankOptions is the STRICTEST setting, not a middle one: a
page with a single inked pixel is kept. This is the input to deleting pages, so
the conservative direction is the only safe default — a page wrongly called
blank destroys content, while a blank page wrongly kept merely leaves a blank
page. The scan threshold (1e-6, about two pixels) is documented with its
measurements instead, because whether its margin holds depends on the scanner.
It will not call a page of solid colour blank: the test is distance from WHITE,
not uniformity, and a measure of variance would delete a deliberate black
separator sheet. It will not empty the document. It does not count annotations
as ink unless asked.
⛔ TWO MUTATIONS SURVIVED THE FIRST SUITE, and both were real test weaknesses:
- defaultTolerance = 0 changed nothing, because every fixture rendered to
EXACTLY white, so no test could see the field and the doc comment's claim
was unproven. Measured: a mark at 247/255 inks 2.2e-03 at tolerance zero
and exactly nothing at twelve. That page is now asserted paper by default
and ink at a tolerance of one level;
- dropping the alpha channel from inkFraction changed nothing, because the
only transparent pixel any test used was transparent BLACK, whose colour
channels are ink anyway. White-and-transparent is the case the rule is
about, and it is now its own test.
Three fixtures were wrong before the code was:
- a "solid page" made by stamping U+2588 FULL BLOCK drew NOTHING, because
Helvetica's encoding has no such character — the page really was blank and
the test was right to fail. The uniformity claim is now a synthetic raster,
which does not depend on what a font happens to contain, plus an
end-to-end page covered by a black picture;
- fixture(t, 2) in the command's tests carries a content stream that is not
drawing operators, so its pages render as blank paper. A fixture whose
"content" is invisible is no fixture for a tool that measures ink;
- Picture(data, 1) means one image pixel per INCH, so a 64-pixel square
became a 64-inch page and the renderer refused it.
And DropBlank lost its own emptiness check: Bytes already refuses a document
with no pages, and a second copy of that rule was a branch no test could reach.
A rule stated twice is a rule that will disagree with itself. ⛔ I nearly
removed the write's error check as unreachable too — Bytes also refuses a form
field with nowhere to go, which is a non-empty document.
This closes the last of the three entries go-pdfkit/conformance's parity table
had written down as not covered with a reason: "remove-blank-pages needs a
blank DETECTOR over rendered pixels, and a detector is a measurement, not a
wiring job."
100% statement coverage on both packages with no uncovered statements, go vet
and gofmt clean. Six mutations caught.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BlankPagesdraws each page and counts the pixels that are not paper;Doc.DropBlankremoves them and says which went, numbered as the reader saw them.pdfops deblankexposes both, with-listas the dry run.Drawing is the only test that agrees with a reader: a page whose content stream is full of operators painting white, or drawing outside the media box, looks blank and is blank, and a page carrying a picture of nothing is blank too.
This closes the last of the three entries
go-pdfkit/conformance's parity table had written down as not covered with a reason — and that reason was the specification:⛔ The default resolution was measured, and my first one was wrong
This started at 50 DPI, with a comment claiming that was "enough to see a full stop". Measured on A4:
At 50 DPI a full stop is exactly the mark a speck of dust makes, so no threshold at that resolution can tell punctuation from dirt.
TestMeasureTheInkLadderprints the whole ladder at 50, 100, 150 and 300, so a renderer change that moved either end fails there instead of silently reclassifying somebody's pages.⛔ The zero value is the strictest setting
A page with a single inked pixel is kept. This is the input to deleting pages: a page wrongly called blank destroys content, while a blank page wrongly kept merely leaves a blank page. The scan threshold (
-max-ink 1e-6, about two pixels — forgives a speck, keeps a full stop) is documented with its measurements rather than made the default, because whether that margin holds depends on the scanner, which this package cannot see.Three more refusals: it will not call a page of solid colour blank (the test is distance from white, not uniformity — a measure of variance would delete a deliberate black separator sheet); it will not empty the document; and it does not count annotations as ink unless asked.
⛔ Two mutations survived the first suite, and both were real weaknesses
defaultTolerance = 0Three fixtures were wrong before the code was
fixture(t, 2)in the command's tests carries a content stream that is not drawing operators, so its pages render as blank paper. A fixture whose "content" is invisible is no fixture for a tool that measures ink;Picture(data, 1)means one image pixel per inch, so a 64-pixel square became a 64-inch page and the renderer refused it.One rule removed, one nearly removed by mistake
DropBlanklost its own emptiness check:Bytesalready refuses a document with no pages, and a second copy was a branch no test could reach. ⛔ I nearly removed the write's error check as unreachable too —Bytesalso refuses a form field with nowhere to go, which is a non-empty document. That is the third time this session I have almost called reachable code dead.Checks
100 % statement coverage on both packages with zero uncovered statements,
go vetandgofmtclean. Six mutations caught.🤖 Generated with Claude Code