Skip to content

[pdf] Upgrade to pypdfium2 v5 - #22

Merged
salkinium merged 2 commits into
mainfrom
feature/pypdfium2-v5
Oct 4, 2026
Merged

salkinium merged 2 commits into
mainfrom
feature/pypdfium2-v5

Conversation

@salkinium

Copy link
Copy Markdown
Member

pypdfium2 v4 is getting a bit long in the tooth, so this moves to v5.14.0. The API breaks were the easy part, the newer pdfium underneath also quietly changed its text extraction, which is of course exactly what all the layout heuristics are tuned to, lol.

  • Adapt to the v5 API: bookmarks instead of outline items, container argument for page objects, get_pos() -> get_bounds(), text render mode via the text object, pages registered with their document.
  • Compute the loose character box the way the old pdfium did (one font size tall, single precision rounding and all), since the new one is ~10% taller and breaks paragraph and table cell detection.
  • Add back the space between a bullet point and its text, since pdfium generates fewer spaces now and the list detection needs that one.
  • Fix Document.destinations, which passed the wrong ctypes types and never worked, whoops.
  • Freeze the missing cxxheaderparser in tools/requirements.txt.
  • Regression test pages are byte-identical.
  • Converted the 220 documents whose archived HTML changed in the last year with v4 and v5: 2576 of 2833 chapter files are identical.

Missing: the remaining 813 differing hunks (of 5.7M lines) are almost all whitespace from spaces that pdfium no longer generates (formulas, . . dot leaders), plus ~40 sub/superscript placements inside math formulas. A general gap-based space insertion fixed those but added spurious spaces elsewhere (TIM1 /2), so meh, only the bullets get one.

Character.weight now reports the real 400/700 instead of 440/690 and the file identifiers are 16 bytes instead of 15, neither is used by the converter.

Adapt to the API changes: the outline items are now bookmarks, page
objects take a container argument, get_pos() became get_bounds(), the
text render mode moved to the text object and pages must be registered
with their document.

Newer pdfium computes the loose character box from the real font ascent
and descent, which breaks the layout heuristics. Compute the previous
definition instead, including its single precision rounding. It also
generates fewer spaces, so add back the one between a bullet point and
its text for the list detection.

Fix the named destination lookup, which passed the wrong ctypes types.
@salkinium
salkinium merged commit 041fdb7 into main Oct 4, 2026
10 checks passed
@salkinium
salkinium deleted the feature/pypdfium2-v5 branch October 4, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant