Skip to content

tables: find the grids the text lines up into, and say what it cannot see - #48

Merged
tannevaled merged 1 commit into
mainfrom
the-grids-the-text-lines-up-into
Oct 11, 2026
Merged

tannevaled merged 1 commit into
mainfrom
the-grids-the-text-lines-up-into

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

pdfops tables writes the grids a page's text lines up into: CSV (RFC 4180,
a blank line between one table and the next), JSON (with the page and the
box, so a caller pulling tables out of a long document knows where to look), or
GFM pipe tables. -to writes one file per table, named the way images
names its own.

pdfops tables accounts.pdf
pdfops tables -as md -pages 3 accounts.pdf
pdfops tables -as json -to tables/ accounts.pdf

The detection is extract v0.6.0's.
This is the command, and three decisions of its own.

⛔ A page that could not be read is said on the error stream

It otherwise looks exactly like a page with no tables on it — and standard
output is where the rows are going, so a warning there would end up inside what
somebody is about to parse.

⛔ The first row becomes the Markdown header

A pipe table has nowhere to put a table with no header row, and a grid found by
looking at alignment does not know whether its first row is one. So it picks
the reading that renders, and says that it picked.

The cells go through go-richdoc/markdown
rather than a strings.Join here, so a cell holding a | is escaped by the
code that already knew it had to be. That is also the path pdf-to-markdown
will take.

⛔ A table can be infinitely wide

A document may say a word sits 10^308 points along, and a table with one word at
minus that and another at plus it is infinitely wide. JSON has no spelling for
an infinity, so -as json fails on such a file rather than writing
"width": +Inf into something that has already begun parsing it. -as csv
never mentions the box and still works.

A PDF real has no exponent, so the fixture writes all 309 digits out: 1e308
in a content stream is not a number at all, and a fixture written that way
quietly tests nothing — which is how the first version of that test passed
while proving the opposite.

How it is checked

100% of statements with no uncovered blocks, and 19 mutations run against it,
all caught. Four of them needed tests that did not exist:

a writer that refuses everything pdfops tables … | head closes the pipe, and a renderer that returns nought from a write that did not happen reports success for an empty file
the exact Markdown shape asserting that | Name | Age | appears cannot see the header row being repeated as the first body row
the two-space indent the test parsed the JSON, so nothing was checking the formatting BentoPDF's own documentation specifies
the infinity above

Parity

Covers BentoPDF's extract-tables and pdf-to-csv. pdf-to-markdown and
prepare-pdf-for-ai are the next two and will reuse this path.

🤖 Generated with Claude Code

… see

pdfops tables writes them as CSV (RFC 4180, a blank line between one table and
the next), as JSON (with the page and the box, so a caller pulling tables out
of a long document knows where to look), or as GFM pipe tables. -to writes one
file per table, named the way images names its own.

The detection is extract v0.6.0's. This is the command, and three decisions of
its own:

  a page that could not be read is said on the ERROR stream. It otherwise
  looks exactly like a page with no tables on it, and standard output is
  where the rows are going.

  the first row becomes the Markdown header. A pipe table has nowhere to put
  a table with no header row, and a grid found by looking at alignment does
  not know whether its first row is one -- so it picks the reading that
  renders and says so.

  the cells go through go-richdoc/markdown rather than a strings.Join here,
  so a cell holding a | is escaped by the code that already knew it had to
  be. That is also the path pdf-to-markdown will take.

A document may say a word sits 10^308 points along, and a table with one word
at minus that and another at plus it is infinitely wide: -as json fails on
such a file rather than writing "width": +Inf into something that has already
begun parsing it. A PDF real has no exponent, so the fixture writes the 309
digits out -- 1e308 in a content stream is not a number at all, and a fixture
written that way tests nothing.

100% of statements with no uncovered blocks, 19 mutations caught. Four needed
tests that did not exist: a writer that refuses everything (pdfops tables |
head closes the pipe), the exact Markdown shape, the two-space indent, and the
infinity above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 4478534 into main Oct 11, 2026
1 check passed
@tannevaled
tannevaled deleted the the-grids-the-text-lines-up-into branch October 11, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant