Skip to content

[pdf] Own character geometry and stacked line detection - #23

Draft
salkinium wants to merge 2 commits into
mainfrom
feature/pdf-own-geometry
Draft

salkinium wants to merge 2 commits into
mainfrom
feature/pdf-own-geometry

Conversation

@salkinium

Copy link
Copy Markdown
Member

Follow-up to #22: instead of emulating the loose character box of the old pdfium (float32 rounding and all, meh), this computes our own box from what the layout heuristics actually want: the font size and the advance width, placed by the full text matrix.

  • Character._em_box() is one font size tall and one advance width wide, which also works for rotated text, so Page._fix_bboxes() and its cache are gone.
  • Lines of the same font size are merged into the longer line instead of comparing their heights for equality, which depended on rounding errors.
  • Lines of the same font size whose characters are printed on top of each other stay separate lines (numerator and denominator of a formula, overlapping index entries). This resolves the FIXME about formulas colliding vertically.
  • Regression test pages are byte-identical.
  • Converted the 220 documents whose archived HTML changed in the last year: 2441 of 2833 chapter files are identical to [pdf] Upgrade to pypdfium2 v5 #22, 1219 hunks differ in 142 documents.

What changes in the output:

#22 this PR
Headings starting with <sub> 468 208
Interleaved sub/sup garbage runs 32 33

The remaining 208 are real subscripts like V<sub>BAT</sub>. Fractions now come out as numerator / dashes / denominator on separate lines instead of a <sub>/<sup> salad.

Missing:

  • Full regeneration of the HTML archive, hence the draft. Only those 220 documents were converted so far.
  • Fractions are readable but not structured, and a denominator sometimes ends up at the end of the paragraph instead of under its bar.
  • A wrapped mm² in a table cell is still wrong, just differently.
  • General gap-based space insertion: fixes Armdouble but breaks Cortex®-M7 into Cortex -M7 since the ® is filtered out, so only the bullet points get their space back for now.

The layout heuristics need the font size and the advance width of a
character, so compute a box from these and place it with the text
matrix instead of relying on the loose box of pdfium. This also works
for rotated text, which removes the need to fix up their boxes.

Lines of the same font size are now merged into the longer line, since
comparing the line heights for equality depended on rounding errors.
Lines of the same font size, whose characters are printed on top of
each other, are not sub- or superscript but separate lines, like the
numerator and denominator of a formula or overlapping index entries.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant