Skip to content

fix(#1856): parse XML from a bounded buffer so a large CLUT round-trips - #1995

Merged
xsscx merged 1 commit into
masterfrom
fix/1856-xml-large-text-node
Aug 6, 2026
Merged

fix(#1856): parse XML from a bounded buffer so a large CLUT round-trips#1995
xsscx merged 1 commit into
masterfrom
fix/1856-xml-large-text-node

Conversation

@colourbill-ctrl

Copy link
Copy Markdown
Contributor

Part of #1856. This is the XML half only — the JSON cap landed in #1981 and its
follow-ups moved to #1994. Item 1 of the issue ("confirm whether to enable
XML_PARSE_HUGE, CMake default or toggle?") is left open deliberately: the finding
below is that there may be nothing to enable, but that call is @ChrisCoxArt's to make.

Detail and full measurements in
#1856 (comment).

Problem

iccToXml can write a document iccFromXml then refuses. A profile whose CLUT
serialises to a single text node above libxml2's XML_MAX_TEXT_LENGTH (10 MB) is
rejected on read, so the two tools disagree about what a valid profile is.

Reproducible in 2.5 s from tracked profiles:

iccApplyToLink link33.icc 0 33 0 "repro" 0 1 1 1 \
    Testing/CMYK-3DLUTs/CMYK-3DLUTs.icc 1 Testing/sRGB_v4_ICC_preference.icc 1
iccToXml   link33.icc  link33.xml     # 21.7 MB, longest text node 21,705,957 bytes
iccFromXml link33.xml  back.icc       # refused

The tracked corpus is already close to the limit — Testing/Display/LaserProjector.xml
carries a 7,473,697-byte node, 74.7% of the cap.

Why not XML_PARSE_HUGE

The limit is enforced on the path that appends successive character-data chunks to a
text node as they arrive from the streaming reader. One contiguous buffer never takes
that path. Same bytes, same options, libxml2 2.9.14:

single text node xmlReadFile xmlReadMemory
9 MB parsed parsed
11 MB refused parsed
40 MB refused parsed
120 MB refused parsed

So the round trip is recoverable without the flag, which would additionally have
disabled the nesting-depth and name-length caps for every input. (The existing rationale
comment also credited it with holding off billion-laughs; that is actually done by not
setting XML_PARSE_NOENT, and the comment is corrected here.)

Change

icXmlReadFileBounded reads the document, bounds it at the existing
icXmlMaxTextFileBytes, and parses the buffer. Both xmlReadFile sites use it — the
profile entry point and the calculator <Import> path — with parser options unchanged.

CIccProfileXml::LoadXml applied no document bound at all before this:
icXmlMaxTextFileBytes guarded only embedded file references, and libxml2's caps bound a
node rather than a file, so a large document made of small nodes was never limited. The
change trades a per-node cap for a whole-document cap, which is the stricter promise, and
a rejection now reports the bound instead of failing as a bare parse error.

The buffer is allocated nothrow and a refusal reported, matching the other bounded
text-buffer sites in the file and preserving the reader's contract of returning NULL
rather than throwing. libxml2 copies the buffer into the document, so it is released
before returning — confirmed under ASAN by clobbering and freeing the buffer before
walking the tree.

Evidence

before after
21.7 MB profile XML refused round-trips, byte-identical ignoring dateTime + MD5
peak RSS, that file 50 MB 69 MB
peak RSS, 8 MiB of ~1M tiny nodes 59.0x input 34.9x input
212 tracked .xml, parse outcome 196 ok / 16 fail identical
deep nesting / long name / billion laughs refused refused

The RSS cost on the text-dense case is real and expected: the raw buffer and the tree are
held at once. That is the same asymmetry raised for loadJsonFrom in #1994, and it
applies here too. On the node-dense shape the buffered path is cheaper.

Test

iccdev.xml-parser-regressions (5 cases) covers the round trip and, deliberately, the
three hardening cases — so a later attempt to solve a size problem by reaching for
XML_PARSE_HUGE turns this red rather than quietly widening what the parser accepts.

  • Red on the unfixed tree: 2 fail / 3 pass. Green after: 5/5.
  • Clean under -fsanitize=address,undefined with detect_leaks=1, verified against a
    deliberate-leak control so the clean result is meaningful.
  • Full local suite green apart from spectral-tiff-preview, which fails on this machine
    for a missing imagecodecs module unrelated to this change.

Two judgement calls, stated rather than buried

  1. No new CMake knob. Reused the existing icXmlMaxTextFileBytes rather than adding an
    ICC_XML_MAX_FILE_MB mirroring ICC_JSON_MAX_FILE_MB. That introduces no new number
    and no new configuration surface. If you would rather XML had the same configurability
    JSON has, it is a small follow-up — a preference question, not a technical one.
  2. 256 MiB is a refusal that did not exist before. At the ~3x profile-to-XML expansion
    measured here that is roughly an 85 MB profile; nothing tracked approaches it, and the
    28 MB link in the original report would emit ~87 MB, well inside. Worth an explicit nod
    rather than a silent tightening.

Not addressed here, noted while working: LoadXml's RelaxNG validation paths return
false without xmlFreeDoc(doc) — pre-existing, out of scope for this issue, and better
raised separately than folded into this diff.

iccToXml could write a document iccFromXml then refused. A profile whose
CLUT serialises to a single text node larger than libxml2's
XML_MAX_TEXT_LENGTH (10 MB) was rejected on read, so the tools disagreed
about what a valid profile is.

That limit is enforced on the path that appends successive character-data
chunks to a text node as they arrive from the streaming reader. Handing
libxml2 one contiguous buffer never takes that path: measured on 2.9.14
with identical bytes and options, xmlReadFile refuses an 11 MB node while
xmlReadMemory accepts 11, 40 and 120 MB. So the round trip is restored
without XML_PARSE_HUGE, which would have disabled the nesting-depth and
name-length caps for every input.

icXmlReadFileBounded reads the document, bounds it at the existing
icXmlMaxTextFileBytes, and parses the buffer. Both xmlReadFile sites use
it - the profile entry point and the calculator <Import> path - with the
parser options unchanged: XML_PARSE_NONET, no XML_PARSE_NOENT, no
XML_PARSE_DTDLOAD. The buffer is allocated nothrow and a refusal is
reported, matching the other bounded text-buffer sites in this file and
keeping the reader's contract of returning NULL rather than throwing at a
size where allocation can plausibly fail. libxml2 copies the buffer into
the document, so it is released before returning; that was confirmed under
ASAN by clobbering and freeing the buffer before walking the tree.

This trades a per-node cap for a whole-document cap, which is the stricter
promise. CIccProfileXml::LoadXml applied no document bound at all before:
icXmlMaxTextFileBytes guarded only embedded file references, and libxml2's
caps bound a node rather than a file, so a large document made of small
nodes was never limited. A rejection now also reports the bound instead of
failing as a bare parse error.

Measured on a grid-33 CMYK->RGB device link built from tracked profiles,
whose XML carries a 21,705,957-byte text node:

  - round trip restored, byte-identical ignoring creation dateTime and MD5
  - peak RSS 50 -> 69 MB on that file, the raw buffer and tree now both
    held; on 8 MiB of ~1M small nodes it falls from 59.0x to 34.9x input
  - all 212 tracked .xml parse identically, 196 ok / 16 fail before and after
  - 3000-level nesting, a 60000-byte element name and a billion-laughs
    document are all still refused

iccdev.xml-parser-regressions covers the round trip and those three
hardening cases, so that a later attempt to solve a size problem by
reaching for XML_PARSE_HUGE turns red rather than quietly widening what
the parser accepts. It passes clean under -fsanitize=address,undefined
with detect_leaks=1.
@github-actions github-actions Bot added Testing CTest, regression, or test coverage Source C or C++ source code changes Scripts Shell, PowerShell, or repository automation scripts Configuration Repository, CMake, YAML, JSON, or tool configuration Build Build system, CMake, compiler, or packaging Unix Linux, macOS, Bash, or POSIX shell scope pending CI checks still running passed All CI checks passed and removed pending CI checks still running labels Aug 6, 2026

@xsscx xsscx left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2026-08-06 10:39:31 UTC

@xsscx
xsscx merged commit a7abbee into master Aug 6, 2026
36 checks passed
@xsscx

xsscx commented Aug 6, 2026

Copy link
Copy Markdown
Member

Not addressed here, noted while working: LoadXml's RelaxNG validation paths return
false without xmlFreeDoc(doc) — pre-existing, out of scope for this issue, and better
raised separately than folded into this diff.

@colourbill-ctrl Thank You, Please open Issue & PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build Build system, CMake, compiler, or packaging Configuration Repository, CMake, YAML, JSON, or tool configuration passed All CI checks passed Scripts Shell, PowerShell, or repository automation scripts Source C or C++ source code changes Testing CTest, regression, or test coverage Unix Linux, macOS, Bash, or POSIX shell scope

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants