markdown: read the pages back as a document, and decode what the file says

[tannevaled]

pdfops markdown writes the pages as Markdown — headings, paragraphs, lists
and tables — through
go-pdfkit/document v0.1.0. With
-json it writes one record per page instead: the page's Markdown plus the
document's own metadata, which is the shape a retrieval pipeline already
expects (LlamaIndex's document schema, the one PyMuPDF4LLM's
LlamaMarkdownReader produces).

pdfops markdown report.pdf > report.md
pdfops markdown -pages 3-7 report.pdf
pdfops markdown -json report.pdf > report-for-indexing.json

One verb for both of BentoPDF's pdf-to-markdown and prepare-pdf-for-ai,
because prepare-pdf-for-ai is per-page Markdown with metadata.

⛔ The range is applied after the document is read, never before

What counts as a heading is decided by the sizes the whole document uses,
so asking for page nine alone would otherwise give it different headings from
asking for all of them — and a caller doing a document a page at a time would
get a different answer for every page. A test asserts that page one reads the
same both ways.

⛔ And it decodes the metadata

A title written in UTF-16 behind a byte order mark came out of ToString as
\xfe\xffL\x00a\x00…. info printed it that way too, for as long as that
verb has existed. Both go through
reader.TextString now, and
info on a pandoc-produced file says

creator    LaTeX via pandoc

where it used to print the raw UTF-16.

How it is checked

100% of statements with no uncovered blocks, 17 mutations caught. Four needed
tests that did not exist:

the indentation	the test parsed the JSON, so nothing checked the formatting
[] rather than null	and the only way to reach it is a document with no pages, since a valid range always names a page that exists
the base name, not the whole path	file_path would have leaked the operator's directory layout
the lowered keys	Title and title are the same field to a caller

Real output

# Note d’argumentaire — Reconnaissance de la fonction de cadre man-ager

Agent : DELAVENNAT David — IR hors classe, BAP E, …

## 1. Objet de la demande

- Fonction de cadre manager : OUI (actuellement NON) ;
- Poste mutualisé entre plusieurs unités : OUI (actuellement NON).

| Activité RFCM « Responsable de plateforme » | Réalisation |
| --- | --- |
| Concevoir et opérer un système de stockage et de | Stockage 10 Po (ZFS) + full flash VAST Data ; |

🤖 Generated with Claude Code