Skip to content

pax: Parse extended header records by their length prefix - #476

Open
gliush wants to merge 1 commit into
composefs:mainfrom
gliush:ivan/pax-issue
Open

gliush wants to merge 1 commit into
composefs:mainfrom
gliush:ivan/pax-issue

Conversation

@gliush

@gliush gliush commented Sep 11, 2026

Copy link
Copy Markdown

A PAX extended header stores records as "%d %s=%s\n". The number at the start counts the whole record, so a value can hold any byte, \n included. PaxExtensions splits the body on \n first and checks the number only after that, so every record with a newline in its value fails. This is what #452 and #428 report.

Splitting also invents records. The second part of such a value becomes its own "line" and can pass the length check. A value holding "\n22 linkpath=/etc/evil\n" makes Entry::link_name() return /etc/evil, where GNU tar returns the linkname from the ustar header.

We hit this while converting OCI layers in nydus and first tried to fix it there (dragonflyoss/nydus#2064). It was too complicated. The crate keeps the header body private, so the builder has to rebuild it from a rolling window of the tar stream, and its input is a FIFO that cannot be read twice. The parsing was never the hard part.

Here the fix is to walk the body by the length prefix. The commit message has the details.

Closes #452, closes #428.

Additional information

Compared with the old parser and GNU tar 1.35

Most bodies parse the same. Those that differ:

body before after GNU tar 1.35
16 linkpath=a\nb\n 2 errors linkpath="a\nb" a\nb
…user.a=AAA\n22 linkpath=/etc/evil\n error, then linkpath=/etc/evil one record, no linkpath ustar linkname
12 path=one (no final \n) path=one error error
+13 path=one\n path=one error error
12 path=one\n\n12 path=two\n path=one path=one, error error
10 pathon\n12 path=two\n (no =) error, then path=two error ustar name

What can break

The last two rows. Where the old parser skipped to the next newline and found later records, this one stops. path() and link_name() use filter_map(|f| f.ok()), so on a broken body they fall back to the ustar header without an error. This needs a writer that emits a broken record followed by good ones; neither this crate nor GNU tar does that.

Stopping cannot be separated from the fix: skipping to the next newline is what lets a piece of a value become a record. Archive::entries() already stops on error for the same reason (#284).

No public API changes — PaxExtensions has one private field.

Not in this PR

  • Duplicate keys. POSIX, GNU tar and Go keep the last record for a key; this crate keeps the first.
  • Refusing a broken body. path_bytes returns Cow<[u8]>, so the error cannot be propagated without changing public signatures.

Relation to #470

Same idea, opened first, still a draft. It differs in two places: a framed record with no = lets iteration continue, and a + in the length is accepted.

AI disclosure

Generated-by: AI
I read and carefully reviewed the code, tests, commit and PR messages.

A PAX record is `"%d %s=%s\n"`. The number at the start counts the whole
record. `PaxExtensions` split the body on `\n` first and checked the number
only after that. Splitting first creates two problems:

- a record whose value contains a newline fails to parse — for example a
  symlink target with a newline in it, or a binary `security.capability`
  xattr that contains `0x0a`;
- the second half of such a value becomes its own "line", and that line can
  pass the length check. A value that holds `"\n22 linkpath=/etc/evil\n"`
  makes `Entry::link_name()` return `/etc/evil`, while a reader that walks
  by length finds no `linkpath` at all.

Three broken bodies that used to return records now return an error: a last
record without a trailing newline, an empty line inside the body, and a
length with a sign. GNU tar 1.35 rejects all three as well.

After an error the iteration stops instead of looking for the next newline,
which would bring back the second problem above.

Closes composefs#452
Closes composefs#428

Signed-off-by: Ivan Glushkov <ivan.glushkov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PaxExtensions fails to parse length-prefixed records when the value contains a newline PAX Header Parsing Fails in some xattr cap encodings

1 participant