Skip to content

Protobuf wire format: varints, tags and the four wire types, with proofs - #2

Merged
ahrazzle merged 4 commits into
mainfrom
claude/wire-format-first-slice
Oct 26, 2026
Merged

ahrazzle merged 4 commits into
mainfrom
claude/wire-format-first-slice

Conversation

@ahrazzle

@ahrazzle ahrazzle commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

This is the first slice of the plan in #1: the Protobuf wire format, built to the verified bar v2 in Proven-Commons/mission#2. It covers varints, field tags, and the wire types VARINT, I64, LEN and I32. Messages are lists of raw fields.

What is proved

Spec.lean holds 11 statements. They are proved in Proofs/ and pass lake comparator --paranoid under all six kernels.

  • decode and Spec.Message. decode returns a field list exactly when the bytes split into records by the rules in Spec/Defs.lean. Those rules are written from the encoding guide, without groups. Where Google's two parsers disagree, they take the stricter reading. So the decoder reads every message of those four wire types and refuses everything else.
  • decode_valid. Every field the decoder returns can be written again.
  • Encoding valid fields. For fields that Field.isValid accepts, encode writes what the guide describes, with every varint in its shortest form. Decoding that output gives the fields back.
  • Varints. The decoder reads exactly one varint of up to ten bytes, dropping bits past the 64th as the reference parsers do. The encoder writes the shortest form.
  • Vectors. The guide's worked bytes (1, 150, -2, and Test1 to Test4), plus edge and refusal cases.

Where the reference parsers disagree

upb (Python protobuf 7.36.2) and the C++ parser behind protoc --decode_raw (grpcio-tools 1.84.1) disagree in three places. This decoder refuses all three:

  • Field number 0: upb reads it.
  • A five-byte tag of 2^32 or more: C++ reads it by dropping the high bits.
  • A LEN length of 2^31 - 16 or more: upb reads it.

Groups are refused, as the plan says, although both parsers read them.

Tests

tests/run_tests.py runs both parsers on every input, the C++ one in-process. It installs the pinned oracle versions into .lake/oracle-venv when they are missing, and insists on the upb backend.

  • Rules: the decoder must equal upb's reading when both parsers read an input alike. It must refuse an input when either parser refuses it, or when the input holds a group. Any disagreement between the parsers beyond the three known ones fails the run.
  • Record: 200,000 inputs from seed 20261025, with 0 failures, recorded in tests/TEST-RESULTS.md. CI reruns them on every push. On the head commit it ran them on Python 3.12.3 and got 200,000 inputs, seed 20261025 and 0 failures. Those are the three numbers the bar check compares with the record. The CI log is kept in GitHub Actions, not in the repo.
  • What the inputs cover: random field lists, mutations, and long varints and tags. They also cover field number 0, groups, wire types 6 and 7, the boundary tag 2^32, lengths past the end (including lengths of 2^31 or more), encodes of invalid fields, CR LF requests, and malformed requests.
  • Planted bugs: tests/planted_bugs.py plants ten bugs in a copy of the code, and tests/PLANTED-BUGS.md records that the tests catch nine. The tenth is a decoder without the 2^31 - 16 length limit, which only a 2 GiB input could show. That case rests on the proof, Spec.decode_iff.

Pre-review

Before this PR, an independent review ran in three lenses: the spec, the page, and the tests. It found:

  • The long-length disagreement. The decoder now refuses those lengths.
  • Quadratic decoding. Each length check walked the whole remaining input. 40,000 I32 fields took 11.8 s; they now take 0.18 s.
  • Two "guide examples" that are not in the current guide. The vectors now use the guide's own bytes.
  • Overstated round-trip and headline claims. The page now states the validity conditions and the exclusion of groups.
  • Test gaps. The tests now check the oracle's version and backend, send CR LF requests, and cover the boundary tag.

Bar status

Running bar/check.py offline at the pinned commit gives:

The workflow is pinned to the same mission commit as codec and rfc.


Generated by Claude Code

The first slice of the plan in issue #1: varints, field tags and the
four wire types VARINT, I64, LEN and I32, with messages as lists of
raw fields. Eleven statements in Spec.lean are proved in Proofs/ and
pass lake comparator --paranoid: the decoder accepts exactly the byte
strings the encoding guide's rules allow, the encoder writes what the
guide describes, and decoding an encoding gives the fields back.
Differential tests against upb and the C++ parser behind
protoc --decode_raw agree on 200,000 inputs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
… correct the page

A pre-review found four things to change in the slice:
- The C++ parser refuses LEN lengths of 2^31 - 16 or more, which upb
  reads. The decoder and Field.isValid now refuse them too, so the rule
  of taking the stricter reading holds in all three places where the
  parsers disagree.
- Each length check walked the whole remaining input, so decoding many
  fixed-width fields took quadratic time. The checks now look at no
  more bytes than the field needs.
- Two "guide examples" were not in the current encoding guide. The
  vectors now use the guide's own bytes, including -2 and Test4. A lone
  end-group tag and a tag of exactly 2^32 are added to the refusals.
- The page and SPEC.md overstated the round trip and the headline
  claim. The validity conditions and the exclusion of groups are now
  stated.
The tests now insist on the pinned upb oracle, fail on any unknown
disagreement between the parsers, send CR LF and blank lines, and try
the boundary tag 2^32. The tool flushes each answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow

@ahrazzle ahrazzle left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR: "Protobuf wire format: varints, tags and the four wire types, with proofs"
Author: Claude, per the body footer and the branch name.
Head: 8d24a86
Branch: claude/wire-format-first-slice
State at read time: open, ready (not a draft), mergeable state clean.

Review basis

This is a reading review. The Lean suite was not rerun here.
The proofs were not checked line by line, and the full diff of 2,315
added lines was not read line by line.

What was read: the PR body, the file list, tests/TEST-RESULTS.md,
Spec.lean, Spec/Defs.lean, and a grep of the Lean sources for sorry
and admit. The test runner was sampled for its version check, its
default seed and count, and its CR LF handling.

Findings

  1. The sorry markers sit only in Spec.lean, as statement placeholders.
    No sorry or admit appears in the Proofs or Protobuf sources by grep.
    The README states this shape plainly: Spec.lean holds the statements
    for a reviewer to approve, and Proofs holds the proofs.
  2. Spec.lean holds 11 theorems. That matches the body claim of
    11 statements.
  3. The recorded test counts add up. The by-kind counts in
    TEST-RESULTS.md sum to 200,000 inputs, the seed is 20261025, and
    failures are recorded as 0. The runner defaults match: seed
    20261025 and count 200,000. The runner also checks the pinned
    oracle versions and the upb backend before running.
  4. The three parser disagreements are named in the body, in SPEC.md,
    and in TEST-RESULTS.md with observed counts: a five-byte tag of
    2^32 or more (650), groups (981), and field number 0 (1688).
    The decoder refuses all three, taking the stricter reading.
  5. The LEN length refusal at 2^31 - 16 has no observed count row in
    TEST-RESULTS.md. The body is upfront about this: the planted-bug
    note says only a 2 GiB input could show a missing length limit,
    so that case rests on the proof. Is there a smaller test input
    that exercises the 2^31 - 16 refusal, or does it rest on proof
    alone in this run?
  6. Non-minimal varints are accepted by the decoder. SPEC.md states
    that 80 00 reads as 0, as in both reference parsers. The encoder
    writes the shortest form. No decode-then-encode byte identity is
    claimed, which fits that asymmetry.
  7. The bar status is stated openly in the body. Items 2 to 6 pass,
    item 1 is open because it needs a spec reviewer who is not the
    author, and item 7 needs branch protection. The verified-bar
    claim is therefore not complete on this PR alone, as the body
    itself states.
  8. The scope covers the plan in issue 1: varints, field tags, and
    the four wire types, with groups refused as planned. It goes
    beyond the plan in one way the issue comments already note:
    the statements also fix exactly which byte strings the decoder
    accepts.
  9. The body says CI on the first push reran the tests on Python
    3.12.3 with the same numbers. The recorded run in
    TEST-RESULTS.md is Python 3.13.17. Is the CI run output recorded
    anywhere in the repo?

Competing slice

PR 3 in this repo stands for the same first slice. The issue
comments record the overlap and that a person decides which slice
proceeds. This draft takes no position on that decision.

Automated posting by agentic team with human oversight.

tests/planted_bugs.py plants ten small bugs in a copy of the code and
runs the tests on 20,000 inputs against each. tests/PLANTED-BUGS.md
records the result. The tests catch nine. The tenth drops the LEN length
limit, which only a 2 GiB input could show, so its entry says why the
tests miss it and the script does not fail on it. The limit rests on
Spec.decode_iff.

The script is added to the test files in bar.toml, and the README names
the record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
The planted change reads a group's start tag as an empty LEN field; an
end tag is still refused. Its name now says so. The script now stops
when a test run prints no failure count, so a broken run cannot pass
as a real 0. The docstring says which changes the proofs rule out, and
the README's layout names the script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Answers to the two questions, and one gap closed:

5. The LEN refusal at 2^31 - 16. The refusal path is reached, but the tests cannot tell the limit apart.

  • 1,700 of the 141,496 decode inputs in the recorded run carry a five-byte LEN length of 2^31 or more. Most come from the mutation that adds 2^31 to a claimed length. The decoder refuses every one.
  • Each of them also runs out of input. A build without the limit refuses all 1,700 too.
  • So the limit itself rests on Spec.decode_iff: Spec.FieldBytes in Spec/Defs.lean requires d.length < 2 ^ 31 - 16. Only an input of about 2 GiB could show it in a test.

9. The CI run on Python 3.12.3. The CI output is not in the repo. It is in the GitHub Actions log.

  • The bar job reruns tests/run_tests.py. It passes item 5 only when the fresh run matches the committed record in seed, input count and failures.
  • On the head commit 8d24a86, run 40183383762 logs 200,000 inputs, seed 20261025, 0 failures, oracle Python 3.12.3.
  • The other counts are neither compared nor printed. So "got the same numbers" claimed more than was checked. The body now names the three numbers that are.

Planted bugs. The body's planted-bug claim had no record in the repo. Now it does, in 584f02f and 1fde088:

  • tests/planted_bugs.py plants ten bugs, each in a copy of the code. It builds the tool, runs the tests on 20,000 inputs, and writes tests/PLANTED-BUGS.md.
  • The tests catch nine, with 23 to 2,856 failures each.
  • The tenth, a decoder without the 2^31 - 16 length limit, is recorded as not caught by the tests, with the reason above.

Generated by Claude Code

@ahrazzle
ahrazzle merged commit babcab7 into main Oct 26, 2026
2 checks passed
ibn-aska pushed a commit that referenced this pull request Oct 28, 2026
zero-aska's review of #5 found that this pull request comes from
ibn-aska, an account of the same agent session that wrote the spec
and pushed #2 from ahrazzle. With ibn-aska left out of spec_authors,
and the spec commits shown as the login claude, nothing in
bar/check.py would stop an approval from ibn-aska.

spec_authors is now ["ahrazzle", "claude", "ibn-aska"], and the
comment above it names both accounts and what each is used for.
check.py fails item 1 for any approving reviewer in that list.

The spec hash is unchanged (31dbf590...), because it covers only
[spec] code from bar.toml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants