Skip to content

Add rmi package: JRMP wire-protocol parser - #18

Merged
phith0n merged 3 commits into
masterfrom
rmi-parser-support
Apr 18, 2026
Merged

phith0n merged 3 commits into
masterfrom
rmi-parser-support

Conversation

@phith0n

@phith0n phith0n commented Apr 17, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a new rmi package that parses the JRMP (Java RMI) wire protocol end-to-end — built on top of the existing serz Java-serialization parser, with two parsing modes (buffered byte inputs and live TCP streams), wireshark-dissector-style output, and integration tests against real rmiregistry captures from both JDK 17 and JDK 8.

Scope

Protocol coverage

  • JRMI handshake (client→server 7-byte header + endpoint echo)
  • ProtocolAck (server→client)
  • MsgCall (0x50): full Java-serialization-embedded call framing; all five java.rmi.registry.Registry stub methods (bind / list / lookup / rebind / unbind) semantically decoded
  • MsgReturnData (0x51): normal / exceptional / void payload variants
  • MsgPing / MsgPingAck / MsgDgcAck (0x52 / 0x53 / 0x54)
  • Explicit Registry-only gate: ObjID == REGISTRY_ID AND methodHash == RegistryInterfaceHash must both match before a Call is treated as Registry

Entry points

Both delegate to one parseTransmission(stream, streaming bool):

API Backing input Arg loop strategy Non-Registry Call Return
rmi.FromBytes(data []byte) Pre-read buffer (.bin file, io.ReadAll-ed slice) Sentinel (stop on non-TC_* byte or EOF) Supported via raw tree Supported
rmi.FromStream(r io.Reader) Live net.Conn / pipe Exact count (registryArgCount(op)) Errors out Errors out

The split is deliberate: on a live TCP reader the sentinel would block forever after the last arg because the next-frame flag hasn't arrived yet; exact-count reading sidesteps this by consuming precisely the bytes the Call owns.

Output format (wireshark-dissector style)

Every byte gets a semantic label, printed exactly once. The leading TC_BLOCKDATA is decomposed in place into ObjID + UID + @Operation + @MethodHash (each with its own hex slice). Subsequent TCContent args carry inline labels like TC_STRING - 0x74 (Registry.lookup arg 0: "name"). A compact @Decoded summary at the top references complex args (bind's Remote stub etc.) by handler, so the stub tree is rendered once inside @Serialization, not duplicated.

Implementation highlights

  • serz.NewObjectStreamFromStream (serz/buffer.go): 3-line constructor that wraps an existing *commons.Stream. This is what lets the rmi parser share a single byte cursor between outer JRMP framing and inner serz parsing — without it, peeking into either layer would strand bytes in the wrong buffer.
  • Registry dispatch correction: initial hand-crafted tests assumed modern JRMP's operation=-1 + method hash format. Real captures proved sun.rmi.registry.RegistryImpl_Stub uses legacy operation=op_index + interface_hash (the five methods share one RegistryInterfaceHash = 0x44154DC9D4E63BDF). Caught by integration tests (closed-loop bug in hand-crafted fixtures).
  • socat 1.8 SYSTEM: colon-parsing quirk: the capture script escapes TCP\:localhost\:1099 because socat 1.8 splits the SYSTEM: argument on :. Documented in _tools/rmi-capture/README.md.
  • Filed issue commons.Stream.Seek 向前 seek 会静默破坏读取状态 #17 for the commons.Stream.Seek forward-seek footgun discovered during development.

Files

New package rmi/ (core parser):

  • parser.go — FromBytes / FromStream / parseTransmission / Transmission.ToString
  • call.go — CallMessage + readCall (mode-aware)
  • return.go — ReturnMessage + readReturn
  • message.go — message dispatcher
  • handshake.go — Handshake / Acknowledge / readModifiedUTF
  • dgc.go — Ping / PingAck / DgcAck
  • objid.go — ObjID / UID value types
  • registry.go — method decoders + registryArgCount
  • printer.go — dissector-style rendering helpers
  • model.go — protocol constants

Tests:

  • rmi_test.go — 21 hand-crafted unit tests
  • integration_test.go — 14 integration tests × 2 JDKs (27 subtests, via forEachJDK)
  • streaming_test.go — 9 streaming tests including io.Pipe delivery + 5-second blocking-guard timeout

Capture tooling (under _tools/rmi-capture/, auto-ignored by Go toolchain):

  • capture.sh — orchestrates rmiregistry + socat tee-proxy + per-op JVM
  • Driver.java — one-op-per-JVM RMI client
  • README.md — regeneration instructions, socat 1.8 gotcha documented

Fixtures under testcases/rmi/{jdk17,jdk8}/ — 20 real captures (5 ops × 2 directions × 2 JDKs).

Other changes:

  • main.go — new rmi CLI subcommand mirroring dump's -f / -B flags
  • serz/buffer.go — NewObjectStreamFromStream constructor
  • CLAUDE.md — architecture guide (new file)
  • .gitignore — ignore *.class artifacts

Test plan

  • go test ./... — all 5 packages green (rmi has 80 subtests)
  • go test -race ./... — green
  • golangci-lint run --no-config ./rmi/... — 0 issues
  • CLI smoke: go run main.go rmi -f testcases/rmi/jdk17/lookup-c2s.bin produces the dissector-style dump
  • io.Pipe test proves streaming parser doesn't block on live-reader semantics
  • Cross-JDK parity: all integration assertions pass against both Zulu 17 and Zulu 8 captures
  • No regression in serz ysoserial byte-exact round-trip tests

🤖 Generated with Claude Code

A read-only parser for the JRMP Stream-protocol (0x4B) covering the full
message surface: handshake, acknowledge, Call, ReturnData, Ping, PingAck,
DgcAck. All five java.rmi.registry.Registry stub methods (bind, list,
lookup, rebind, unbind) are semantically decoded; non-Registry Remote
calls fall back to the raw serialization tree.

Two entry points, one unified parseTransmission:
- FromBytes(data): buffered input; sentinel-based arg loop (tolerant of
  non-Registry calls and ReturnData because the input eventually EOFs).
- FromStream(r io.Reader): live connections (net.Conn etc.); derives each
  message's byte boundary from protocol framing alone, so it never blocks
  after finishing a frame. Registry calls only — non-Registry and Return
  return explicit errors directing the caller to FromBytes.

Supporting changes:
- serz.NewObjectStreamFromStream lets the embedded serialization parse
  share a commons.Stream byte cursor with its caller, so bytes peeked
  into one layer don't strand at a buffer handoff.
- rmi CLI subcommand mirrors dump (-f file / -B base64).
- Output is wireshark-dissector style: @serialization walks the stream
  once with inline semantic labels on each TCContent; the leading
  TC_BLOCKDATA is decomposed in place into ObjID + op + hash.
- _tools/rmi-capture: Go-free socat tee-proxy + Java Driver for
  regenerating testcases/rmi fixtures (jdk17 / jdk8 subdirs).
- 80+ unit and integration tests, including JDK 17 / JDK 8 real-capture
  parity and a streaming io.Pipe test with 5s timeout guard.
- CLAUDE.md documents architecture, scope boundaries, and the "don't
  refactor back to serz.FromBytes" warning for the embedded stream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@phith0n

phith0n commented Apr 18, 2026

Copy link
Copy Markdown
Owner Author

Code review

Found 1 issue:

  1. CLAUDE.md references a function rmi/call.go:readEmbeddedSerialization that does not exist in the rmi/ package. The sentinel-loop logic described (walking serz.ReadTCContent and stopping on a byte outside [JAVA_TC_BASE, JAVA_TC_MAX]) is actually implemented inline inside readCallArgs in rmi/call.go and the mirror section of readReturn in rmi/return.go. The important "Do not refactor this to use serz.FromBytes" warning is therefore pinned to a ghost anchor — a future maintainer searching for readEmbeddedSerialization to understand the constraint will not find it.

zkar/CLAUDE.md

Lines 69 to 73 in 248852e

- **`DgcAckMessage`** (0x54) — raw 14-byte UID written outside any `ObjectOutputStream` framing (the only JRMP frame that does *not* go through `serz`).
**The critical design point worth internalizing**: a single Java serialization stream has no explicit end marker — `serz.FromReader` terminates only on `io.EOF`. Inside JRMP, the next byte after a Call/Return body is the next message's flag (`0x50..0x54`), which is neither `io.EOF` nor a valid `TC_*` tag. `rmi/call.go:readEmbeddedSerialization` walks `serz.ReadTCContent` in a loop and stops when `PeekN(1)` returns a byte outside `[serz.JAVA_TC_BASE, serz.JAVA_TC_MAX]` = `[0x70, 0x7F]`. TC_* and JRMP-flag ranges are disjoint, so the check is unambiguous. **Do not refactor this to use `serz.FromBytes`** — doing so would fail on any stream with more than one frame.
**Registry dispatch is op-index + interface hash, not per-method hash.** The JDK ships a precompiled `sun.rmi.registry.RegistryImpl_Stub` whose wire format is `operation = 0..4` (indexing into `{bind, list, lookup, rebind, unbind}`) paired with a single `int64 RegistryInterfaceHash` shared by all five methods. Modern JRMP's `operation = -1 + per-method hash` pattern applies only to dynamic-proxy stubs and is NOT used by Registry. `rmi/model.go` exposes the five op-index constants (`LookupOpIndex`, etc.) and the `RegistryInterfaceHash` constant (calibrated against a live Zulu OpenJDK 17 capture; see `testcases/rmi/*.bin` and `_tools/rmi-capture/`). If a real capture's `CallMessage.Decoded` is nil for an obvious Registry call, print `MethodHash` as hex and compare — recalibration is a one-line edit.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

phith0n added 2 commits April 18, 2026 21:09
- rmi/call.go, rmi/return.go: accept leading TC_BLOCKDATA longer than the
  fixed header, since non-Registry remotes with primitive params and
  methods with primitive return types legally append those raw bytes to
  the header block. Previously rejected as malformed.
- rmi/rmi_test.go: cover both cases.
- rmi/registry.go: reformat doc comment to Go 1.19+ gofmt style.
- CLAUDE.md: point to the real buffered-mode sentinel loop in
  readCallArgs / readReturn instead of a nonexistent function.
@phith0n
phith0n merged commit c018867 into master Apr 18, 2026
17 checks passed
@phith0n
phith0n deleted the rmi-parser-support branch April 18, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant