From 91f1966dda33607b6ee54270aceab2fe9c225582 Mon Sep 17 00:00:00 2001 From: phith0n Date: Sun, 19 Apr 2026 03:12:38 +0800 Subject: [PATCH 1/2] rmi: split Decoder opening for server-side flows Opening() reads handshake+ClientEndpoint in one call, which deadlocks on a server-side live reader: a conforming Java client blocks after its 7-byte handshake waiting for the server's ProtocolAck before it sends the endpoint echo. Servers need to inject a write between the two reads, which the unified Opening() call forbade. Split the opening phase into three mutually exclusive primitives: ReadHandshake (server read), ReadClientEndpoint (server read after Ack write), and ReadAcknowledge (client read). A stage enum enforces call ordering. Opening() remains the one-shot entry point for fully-buffered captures and client-direction reads; Next() still auto-advances from any stage to stageReady so bare captures just work. Also add ToBytes encoders on Handshake/Acknowledge/Endpoint (zero-value Magic/Protocol/Flag default to JRMI_MAGIC/ProtocolStream/AckFlag so server code can write &Acknowledge{Host, Port}.ToBytes() without ceremony), and a ToString on Endpoint for symmetry with the other two. Tests: TestDecoderServerFlowWithInterleavedAck uses two io.Pipes to reproduce the real client timing and assert no deadlock. Boundary tests cover the stage-machine rejections, Next() auto-advance, and encoder round-trip. ExampleDecoder_serverFlow demonstrates the new sequence. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 24 +++++- rmi/decoder.go | 188 +++++++++++++++++++++++++++++++++++++++----- rmi/decoder_test.go | 166 ++++++++++++++++++++++++++++++++++++++ rmi/example_test.go | 73 +++++++++++++++-- rmi/handshake.go | 44 +++++++++++ rmi/parser.go | 35 ++++++--- rmi/rmi_test.go | 12 +++ 7 files changed, 505 insertions(+), 37 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1c94442..e40dd35 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -82,7 +82,7 @@ Adding a new message type (e.g. if SingleOp/Multiplex support is ever added): ex The endpoint-echo heuristic in `parser.go:maybeReadClientEndpoint` peeks the byte right after the handshake: if it falls in `[0x50, 0x54]` (JRMP message flag range) we skip the echo, otherwise read `writeUTF + int32`. This lets hand-crafted test fixtures omit the echo. The ambiguity only collides on pathological (≥ 20480-char) hostnames. -**Live-TCP ergonomics.** On a long-lived `net.Conn` a typical Registry session has the shape handshake → Call → (peer waits for Return) → Return → close. `FromBytes` would deadlock on the second peek after the first frame; `Decoder` is the only viable choice: +**Live-TCP ergonomics.** On a long-lived `net.Conn` a typical Registry session has the shape handshake → Call → (peer waits for Return) → Return → close. `FromBytes` would deadlock on the second peek after the first frame; `Decoder` is the only viable choice. For *client-side* reading (capture already in hand, or reading responses from a server), the one-shot `Opening()` is fine: ```go d := rmi.NewDecoder(conn) @@ -96,7 +96,27 @@ for { } ``` -`Decoder.Next()` returns one frame per call. Registry Calls return as soon as their own bytes arrive (no peek past last arg). ReturnData uses the sentinel and may block between frames on a live reader until the next flag byte arrives or the reader's deadline fires — the caller is responsible for the deadline. `Opening()` is optional; if omitted, the first `Next()` transparently consumes and discards any handshake prefix. Calling `Opening()` twice, or after `Next()`, returns an error. +*Server-side `Opening()` deadlocks.* A conforming Java client (`sun.rmi.transport.tcp.TCPChannel`) writes the 7-byte handshake and then blocks reading the server's `ProtocolAck` before writing its `ClientEndpoint` echo. `Opening()` consumes handshake + endpoint in one call, so its second peek waits for bytes the client refuses to send until the server writes the Ack. For servers (and any caller that must inject writes into the handshake phase), use the fine-grained primitives: + +```go +d := rmi.NewDecoder(conn) +hs, err := d.ReadHandshake() // consumes only the 7-byte handshake +// ... write the Ack to conn before reading further ... +remote := conn.RemoteAddr().(*net.TCPAddr) +_, _ = conn.Write((&rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)}).ToBytes()) +ep, err := d.ReadClientEndpoint() // now safe: client unblocks after reading Ack +for { + _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) + msg, err := d.Next() + ... +} +``` + +`ReadAcknowledge()` is the symmetric client-side primitive. The three opening primitives and `Opening()` are **mutually exclusive** — the Decoder tracks a stage (`stageInitial` → `stageAfterHandshake` → `stageReady`) and refuses out-of-order calls. `Next()` auto-advances to `stageReady` from either earlier stage (auto-consuming a `ClientEndpoint` after `ReadHandshake` if the caller skipped `ReadClientEndpoint`), so omitting the opening entirely still works for bare captures. + +`Handshake`, `Acknowledge`, and `Endpoint` all expose `ToBytes()` for writing: zero-valued `Magic`/`Protocol`/`Flag` fields default to `JRMI_MAGIC` / `ProtocolStream` / `AckFlag`, so server code can usually construct with just `Host` + `Port` (see `TestOpeningEncodersRoundTrip` for the exact layouts). + +`Decoder.Next()` returns one frame per call. Registry Calls return as soon as their own bytes arrive (no peek past last arg). ReturnData uses the sentinel and may block between frames on a live reader until the next flag byte arrives or the reader's deadline fires — the caller is responsible for the deadline. **`ToString()` rendering is wireshark-dissector style — every byte is printed exactly once, with semantic labels inline.** `Call/ReturnMessage.ToString()` emits a compact `@Decoded` summary (method + scalar args; complex args referenced by handler) at the top, then `@Serialization` with a custom walk (`rmi/printer.go`): diff --git a/rmi/decoder.go b/rmi/decoder.go index 681ac38..e7072bc 100644 --- a/rmi/decoder.go +++ b/rmi/decoder.go @@ -19,7 +19,32 @@ import ( // one message before the next arrives and can apply SetReadDeadline on the // underlying net.Conn between calls to bound how long they wait. // -// Blocking semantics: +// # Opening-phase primitives vs. Opening() +// +// The JRMP opening has three pieces that are interleaved with a server-side +// write on the wire: +// +// client → server: JRMI magic + version + 0x4B (7 bytes) +// server → client: 0x4E + writeUTF(host) + int32(port) (Acknowledge) +// client → server: writeUTF(host) + int32(port) (ClientEndpoint) +// +// A conforming Java client blocks after sending the handshake until the +// server's Acknowledge arrives, and only then writes its ClientEndpoint. +// That means Opening() — which reads Handshake+ClientEndpoint in one call — +// deadlocks on a server-side reader: the second PeekN waits for bytes the +// client will not send until it has received the Ack. +// +// For server-side use (or any caller that needs to inject a write between +// the handshake reads), use the fine-grained primitives: +// +// ReadHandshake() — consumes only the 7-byte handshake +// ReadClientEndpoint() — consumes the post-Ack endpoint echo +// ReadAcknowledge() — consumes the server→client Acknowledge frame +// +// Opening() remains the convenience entry point for fully-buffered captures +// and for client→server consumers that already have the bytes in hand. +// +// # Blocking semantics // // - CallMessage: the Registry dispatch gate fails fast for non-Registry // headers, and a valid Registry Call reads an exact arg count — Next() @@ -40,10 +65,29 @@ import ( // Callers that process Returns over a live connection should set // SetReadDeadline on the underlying net.Conn. // -// Typical usage: +// # Typical usage +// +// Client → server capture (bytes already in hand): +// +// d := rmi.NewDecoder(bytes.NewReader(data)) +// opening, err := d.Opening() +// if err != nil { return err } +// for { +// msg, err := d.Next() +// if errors.Is(err, io.EOF) { break } +// if err != nil { return err } +// handle(msg) +// } +// +// Server side on a live net.Conn: // // d := rmi.NewDecoder(conn) -// opening, err := d.Opening() // optional; omit to skip +// hs, err := d.ReadHandshake() +// if err != nil { return err } +// remote := conn.RemoteAddr().(*net.TCPAddr) +// ack := &rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)} +// if _, err := conn.Write(ack.ToBytes()); err != nil { return err } +// ep, err := d.ReadClientEndpoint() // if err != nil { return err } // for { // _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) @@ -53,10 +97,21 @@ import ( // handle(msg) // } type Decoder struct { - stream *commons.Stream - openingDone bool + stream *commons.Stream + stage decoderStage } +// decoderStage tracks how far through the opening phase the Decoder has +// progressed. Each Read* primitive and Opening() check this before +// advancing it; Next() refuses to run until the stage reaches stageReady. +type decoderStage int + +const ( + stageInitial decoderStage = iota // nothing consumed yet + stageAfterHandshake // 7-byte handshake consumed; ClientEndpoint still pending + stageReady // opening phase done; Next() is the only valid call +) + // NewDecoder returns a Decoder that reads from r. The Decoder holds the // reader until its last frame is consumed or the caller drops it; the // underlying bytes are only consumed on Opening() / Next() calls. @@ -64,24 +119,33 @@ func NewDecoder(r io.Reader) *Decoder { return &Decoder{stream: commons.NewStreamFromReader(r)} } -// Opening reads the optional handshake-phase prefix: Handshake + client -// endpoint echo (client→server direction) or Acknowledge (server→client -// direction), or neither for a bare capture that starts mid-stream. +// Opening reads the optional handshake-phase prefix in one call: Handshake + +// client endpoint echo (client→server direction) or Acknowledge +// (server→client direction), or neither for a bare capture that starts +// mid-stream. +// +// Opening is a convenience entry point for fully-buffered captures and +// consumers that already have every byte. It is NOT safe on a server-side +// live reader: a conforming client sends its handshake and then blocks +// waiting for the server's Acknowledge, so the peek inside +// maybeReadClientEndpoint deadlocks. Servers must use ReadHandshake + +// ReadClientEndpoint with their own Acknowledge write in between. // // Calling Opening is optional — if the caller omits it, the first Next() // call will read and discard any opening transparently. Explicit Opening // is the way to access the Handshake / Acknowledge / ClientEndpoint fields. // -// It is an error to call Opening twice, or to call it after Next(). +// It is an error to call Opening twice, after Next(), or after any of the +// ReadHandshake / ReadClientEndpoint / ReadAcknowledge primitives. func (d *Decoder) Opening() (*Opening, error) { - if d.openingDone { - return nil, errors.New("rmi.Decoder: Opening already called") + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: Opening is only valid as the first call") } - d.openingDone = true var t Transmission if err := readOpening(d.stream, &t); err != nil { return nil, err } + d.stage = stageReady return &Opening{ Handshake: t.Handshake, Acknowledge: t.Acknowledge, @@ -89,18 +153,80 @@ func (d *Decoder) Opening() (*Opening, error) { }, nil } -// Next returns the next JRMP message. On the first call, if Opening hasn't -// been invoked, any handshake prefix is read and discarded. Returns io.EOF -// when the reader closes cleanly at a frame boundary. Non-Registry Calls -// error out at parse time; ReturnData may block on the sentinel peek -// between frames — see the Decoder doc for details. +// ReadHandshake consumes the 7-byte client→server opening +// (JRMI magic + uint16 version + 1-byte sub-protocol flag) and nothing else. +// This is the first call a server should make after accepting a connection: +// it returns as soon as the 7 bytes arrive, leaving the server free to +// write its Acknowledge before reading the ClientEndpoint echo. Call +// ReadClientEndpoint (or Next, which consumes any remaining opening +// implicitly) once the Ack has been written. +// +// It is an error to call ReadHandshake after Opening, any other Read* +// primitive, or Next. +func (d *Decoder) ReadHandshake() (*Handshake, error) { + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: ReadHandshake must be the first call") + } + h, err := readHandshake(d.stream) + if err != nil { + return nil, err + } + d.stage = stageAfterHandshake + return h, nil +} + +// ReadClientEndpoint consumes the client's post-Acknowledge endpoint echo +// (writeUTF host + int32 port). Must be preceded by ReadHandshake; the +// server is expected to have written its Acknowledge between the two +// calls. Returns (nil, nil) if the client omits the echo (matching the +// lenient behavior of fully-buffered captures). +// +// It is an error to call ReadClientEndpoint outside the +// ReadHandshake → ReadClientEndpoint sequence. +func (d *Decoder) ReadClientEndpoint() (*Endpoint, error) { + if d.stage != stageAfterHandshake { + return nil, errors.New("rmi.Decoder: ReadClientEndpoint must follow ReadHandshake") + } + ep, err := maybeReadClientEndpoint(d.stream) + if err != nil { + return nil, fmt.Errorf("client endpoint echo: %w", err) + } + d.stage = stageReady + return ep, nil +} + +// ReadAcknowledge consumes the server→client Acknowledge frame +// (0x4E + writeUTF host + int32 port). This is the primitive a client-side +// reader uses to inspect the server's view of its endpoint before any +// message traffic. Must be the first call on the Decoder. +// +// It is an error to call ReadAcknowledge after Opening, any other Read* +// primitive, or Next. +func (d *Decoder) ReadAcknowledge() (*Acknowledge, error) { + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: ReadAcknowledge must be the first call") + } + a, err := readAcknowledge(d.stream) + if err != nil { + return nil, err + } + d.stage = stageReady + return a, nil +} + +// Next returns the next JRMP message. On the first call, if no opening +// primitive has advanced the Decoder past stageReady, any remaining +// handshake-phase bytes are read and discarded — including a lone +// ClientEndpoint when ReadHandshake was called but ReadClientEndpoint was +// skipped. Returns io.EOF when the reader closes cleanly at a frame +// boundary. Non-Registry Calls error out at parse time; ReturnData may +// block on the sentinel peek between frames — see the Decoder doc for +// details. func (d *Decoder) Next() (Message, error) { - if !d.openingDone { - var discard Transmission - if err := readOpening(d.stream, &discard); err != nil { + if d.stage != stageReady { + if err := d.finishOpening(); err != nil { return nil, err } - d.openingDone = true } if _, err := d.stream.PeekN(1); err != nil { // io.EOF flows through unwrapped so callers can check with @@ -114,6 +240,26 @@ func (d *Decoder) Next() (Message, error) { return readMessage(d.stream) } +// finishOpening fast-forwards from whichever stage the caller left the +// Decoder in to stageReady, consuming only the bytes the remaining opening +// phase owns. It exists so Next() can be called on a Decoder whose caller +// didn't (or couldn't) explicitly finish the opening. +func (d *Decoder) finishOpening() error { + switch d.stage { + case stageInitial: + var discard Transmission + if err := readOpening(d.stream, &discard); err != nil { + return err + } + case stageAfterHandshake: + if _, err := maybeReadClientEndpoint(d.stream); err != nil { + return fmt.Errorf("client endpoint echo: %w", err) + } + } + d.stage = stageReady + return nil +} + // Opening captures the three optional handshake-phase fields of a JRMP // Transmission. Any or all may be nil: client→server traffic typically sets // Handshake + ClientEndpoint; server→client sets Acknowledge; a capture diff --git a/rmi/decoder_test.go b/rmi/decoder_test.go index 299bd8b..8032ab5 100644 --- a/rmi/decoder_test.go +++ b/rmi/decoder_test.go @@ -157,6 +157,172 @@ func TestDecoderRejectsNonRegistryCall(t *testing.T) { require.Contains(t, err.Error(), "not a Registry call") } +// TestDecoderServerFlowWithInterleavedAck reproduces the real rmiregistry +// server timing: a standard Java client sends its 7-byte handshake and +// then BLOCKS until the server writes a ProtocolAck — only after reading +// the Ack does it send its ClientEndpoint echo. Opening() would deadlock +// on this shape because it reads handshake and endpoint in a single call. +// The server-side flow (ReadHandshake → write Ack → ReadClientEndpoint → +// Next) must complete without blocking past what the client will send at +// each step. +func TestDecoderServerFlowWithInterleavedAck(t *testing.T) { + // c2s: what the client writes. s2c: what the server writes (captured + // here so we can assert the Ack bytes the server produced). + c2sRead, c2sWrite := io.Pipe() + s2cRead, s2cWrite := io.Pipe() + + clientDone := make(chan error, 1) + // Client goroutine: mimic sun.rmi.transport.tcp.TCPChannel — write + // handshake, read ack in full, then write endpoint + a Ping so the + // server has a message to parse and we can verify the whole flow. + go func() { + defer func() { _ = c2sWrite.Close() }() + if _, err := c2sWrite.Write(buildHandshake()); err != nil { + clientDone <- err + return + } + // Drain the server's Ack before writing anything else — this is + // the critical ordering constraint. Ack = 1 flag + 2 length + + // len("127.0.0.1")=9 + 4 port = 16 bytes for the fixture below. + ack := make([]byte, 16) + if _, err := io.ReadFull(s2cRead, ack); err != nil { + clientDone <- err + return + } + if _, err := c2sWrite.Write(buildClientEndpointEcho("client.local", 55555)); err != nil { + clientDone <- err + return + } + if _, err := c2sWrite.Write(buildPing()); err != nil { + clientDone <- err + return + } + clientDone <- nil + }() + + d := NewDecoder(c2sRead) + + // Stage 1: read just the handshake. Must return as soon as 7 bytes + // arrive — a peek past that would block. + hs, err := d.ReadHandshake() + require.NoError(t, err) + require.Equal(t, uint16(2), hs.Version) + require.Equal(t, ProtocolStream, hs.Protocol) + + // Stage 2: server writes its Ack. Without this step the client will + // sit forever on its io.ReadFull and the next ReadClientEndpoint + // would deadlock. + ack := &Acknowledge{Host: "127.0.0.1", Port: 1234} + go func() { + _, _ = s2cWrite.Write(ack.ToBytes()) + _ = s2cWrite.Close() + }() + + // Stage 3: consume the post-Ack endpoint echo. + ep, err := d.ReadClientEndpoint() + require.NoError(t, err) + require.NotNil(t, ep) + require.Equal(t, "client.local", ep.Host) + require.Equal(t, int32(55555), ep.Port) + + // Stage 4: messages flow normally from here. + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) + + select { + case err := <-clientDone: + require.NoError(t, err) + case <-time.After(2 * time.Second): + t.Fatal("client goroutine stalled — server flow did not advance as expected") + } +} + +// TestDecoderOpeningAfterReadHandshakeErrors: the three opening-phase +// entry points are mutually exclusive. Once ReadHandshake has committed +// the Decoder to the fine-grained flow, Opening() is no longer valid. +func TestDecoderOpeningAfterReadHandshakeErrors(t *testing.T) { + d := NewDecoder(bytes.NewReader(buildHandshake())) + _, err := d.ReadHandshake() + require.NoError(t, err) + _, err = d.Opening() + require.Error(t, err) +} + +// TestDecoderReadHandshakeAfterOpeningErrors: symmetric — once Opening has +// finished the opening phase, none of the Read* primitives may run. +func TestDecoderReadHandshakeAfterOpeningErrors(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildHandshake()) + buf.Write(buildClientEndpointEcho("h", 1)) + d := NewDecoder(&buf) + _, err := d.Opening() + require.NoError(t, err) + _, err = d.ReadHandshake() + require.Error(t, err) +} + +// TestDecoderReadClientEndpointRequiresHandshake: the endpoint primitive +// has no way to locate itself in the stream without the preceding +// handshake read — guard against callers skipping the prerequisite. +func TestDecoderReadClientEndpointRequiresHandshake(t *testing.T) { + d := NewDecoder(bytes.NewReader(buildClientEndpointEcho("h", 1))) + _, err := d.ReadClientEndpoint() + require.Error(t, err) +} + +// TestDecoderNextAutoConsumesClientEndpoint: if a server calls +// ReadHandshake but then skips ReadClientEndpoint (e.g. doesn't care +// about the echo), Next() should transparently consume the echo before +// parsing the first message. +func TestDecoderNextAutoConsumesClientEndpoint(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildHandshake()) + buf.Write(buildClientEndpointEcho("h", 1)) + buf.Write(buildPing()) + + d := NewDecoder(&buf) + _, err := d.ReadHandshake() + require.NoError(t, err) + + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) +} + +// TestDecoderReadAcknowledge: client-side flow — read the server's Ack +// as the first call, then message traffic. +func TestDecoderReadAcknowledge(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildAck("1.2.3.4", 4242)) + buf.Write(buildPing()) + + d := NewDecoder(&buf) + ack, err := d.ReadAcknowledge() + require.NoError(t, err) + require.Equal(t, "1.2.3.4", ack.Host) + require.Equal(t, int32(4242), ack.Port) + + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) +} + +// TestOpeningEncodersRoundTrip: the new ToBytes encoders must produce the +// exact layout the fixture builders produce (and therefore the exact +// layout the parsers consume). +func TestOpeningEncodersRoundTrip(t *testing.T) { + // Handshake: defaults on zero-valued Magic/Protocol. + require.Equal(t, buildHandshake(), (&Handshake{Version: 2}).ToBytes()) + // Acknowledge: default-Flag path and explicit-Flag path both emit + // the buildAck layout. + require.Equal(t, buildAck("127.0.0.1", 1234), (&Acknowledge{Host: "127.0.0.1", Port: 1234}).ToBytes()) + require.Equal(t, buildAck("host", 9), (&Acknowledge{Flag: AckFlag, Host: "host", Port: 9}).ToBytes()) + // Endpoint: matches buildClientEndpointEcho. + require.Equal(t, buildClientEndpointEcho("client.local", 55555), + (&Endpoint{Host: "client.local", Port: 55555}).ToBytes()) +} + // TestDecoderChunkedDelivery feeds a real rmiregistry capture through an // io.Pipe in 7-byte chunks with tiny sleeps between, mimicking TCP // segmentation where bytes arrive across multiple Read calls. A Decoder diff --git a/rmi/example_test.go b/rmi/example_test.go index 253bbe2..26a97c9 100644 --- a/rmi/example_test.go +++ b/rmi/example_test.go @@ -97,11 +97,17 @@ func ExampleNewDecoder() { } // ExampleDecoder_liveConnection sketches the pattern for reading JRMP -// traffic off a live net.Conn — the only supported way to parse from a -// long-lived TCP connection. The two things to notice are (a) the caller -// applies SetReadDeadline between frames to bound how long to wait for -// the next one, and (b) non-Registry Call headers and Return sentinel -// timeouts surface as normal errors the caller can inspect. +// traffic off a live net.Conn — the client→server reading direction, or +// any case where every byte is already available before parsing starts. +// The two things to notice are (a) the caller applies SetReadDeadline +// between frames to bound how long to wait for the next one, and (b) +// non-Registry Call headers and Return sentinel timeouts surface as +// normal errors the caller can inspect. +// +// For the server-side reading direction — where the server must write a +// ProtocolAck between reading the client's handshake and the client's +// endpoint echo — see ExampleDecoder_serverFlow. Opening() is NOT safe +// on a server-side live reader. // // This example does not connect to a real server; it's structured as a // compile-check so the idiom stays accurate as the API evolves. @@ -141,3 +147,60 @@ func ExampleDecoder_liveConnection() { // got frame 0x52 // peer closed } + +// ExampleDecoder_serverFlow demonstrates the server-side read ordering: +// ReadHandshake → write Acknowledge → ReadClientEndpoint → Next. A +// conforming Java client (sun.rmi.transport.tcp.TCPChannel) blocks after +// its 7-byte handshake until it has read the server's ProtocolAck, only +// then writing its endpoint echo. Opening() would deadlock on this +// ordering because it peeks past the handshake before the client has +// unblocked. +// +// Here the client's bytes are pre-buffered for determinism; on a real +// net.Conn the Ack write (conn.Write(ack.ToBytes())) is what actually +// lets the client's ReadClientEndpoint bytes reach us. +func ExampleDecoder_serverFlow() { + // Bytes the client sends across the two handshake halves plus one + // Ping. Constructed with the new ToBytes encoders so the example + // doubles as their usage demo. + var clientBytes bytes.Buffer + clientBytes.Write((&rmi.Handshake{Version: 2}).ToBytes()) + clientBytes.Write((&rmi.Endpoint{Host: "client.local", Port: 55555}).ToBytes()) + clientBytes.WriteByte(0x52) // MsgPing + + d := rmi.NewDecoder(&clientBytes) + + hs, err := d.ReadHandshake() + if err != nil { + fmt.Println("handshake:", err) + return + } + fmt.Println("handshake version:", hs.Version) + + // Server-side Ack. In real code: + // remote := conn.RemoteAddr().(*net.TCPAddr) + // _, _ = conn.Write((&rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)}).ToBytes()) + // Zero-valued Flag defaults to AckFlag, so Host/Port is all we need. + ack := (&rmi.Acknowledge{Host: "127.0.0.1", Port: 1234}).ToBytes() + fmt.Println("ack byte count:", len(ack)) + + ep, err := d.ReadClientEndpoint() + if err != nil { + fmt.Println("endpoint:", err) + return + } + fmt.Printf("client endpoint: %s:%d\n", ep.Host, ep.Port) + + msg, err := d.Next() + if err != nil { + fmt.Println("next:", err) + return + } + fmt.Printf("got frame 0x%02X\n", msg.Op()) + + // Output: + // handshake version: 2 + // ack byte count: 16 + // client endpoint: client.local:55555 + // got frame 0x52 +} diff --git a/rmi/handshake.go b/rmi/handshake.go index 1ca23d5..8a75638 100644 --- a/rmi/handshake.go +++ b/rmi/handshake.go @@ -125,3 +125,47 @@ func (a *Acknowledge) ToString() string { b.Printf("@Port - %d - %s", a.Port, commons.Hexify(a.Port)) return b.String() } + +// ToBytes encodes the handshake for writing to a JRMP peer: JRMI magic + +// uint16 version + 1-byte sub-protocol flag. Zero-valued Magic and Protocol +// fields default to JRMI_MAGIC and ProtocolStream so callers can construct +// a Handshake with only the Version field set. +func (h *Handshake) ToBytes() []byte { + magic := h.Magic + if len(magic) == 0 { + magic = JRMI_MAGIC + } + proto := h.Protocol + if proto == 0 { + proto = ProtocolStream + } + var buf bytes.Buffer + buf.Write(magic) + _ = binary.Write(&buf, binary.BigEndian, h.Version) + buf.WriteByte(proto) + return buf.Bytes() +} + +// ToBytes encodes the server→client acknowledgement as +// 0x4E + writeUTF(host) + int32(port). A zero-valued Flag defaults to +// AckFlag so a server can construct with only Host/Port. +func (a *Acknowledge) ToBytes() []byte { + flag := a.Flag + if flag == 0 { + flag = AckFlag + } + var buf bytes.Buffer + buf.WriteByte(flag) + writeModifiedUTF(&buf, a.Host) + _ = binary.Write(&buf, binary.BigEndian, a.Port) + return buf.Bytes() +} + +// writeModifiedUTF encodes s using the same uint16-length-prefixed modified +// UTF-8 layout readModifiedUTF consumes. Hostnames are overwhelmingly +// ASCII, so we emit the raw bytes unchanged — matching the lenient read +// path. +func writeModifiedUTF(buf *bytes.Buffer, s string) { + _ = binary.Write(buf, binary.BigEndian, uint16(len(s))) + buf.WriteString(s) +} diff --git a/rmi/parser.go b/rmi/parser.go index a9e7d8e..a6cde13 100644 --- a/rmi/parser.go +++ b/rmi/parser.go @@ -29,6 +29,31 @@ type Endpoint struct { Port int32 } +// ToBytes encodes the endpoint as writeUTF(host) + int32(port) — the exact +// layout a client writes immediately after receiving the server's +// Acknowledge, and the layout ReadClientEndpoint consumes. +func (e *Endpoint) ToBytes() []byte { + var buf bytes.Buffer + writeModifiedUTF(&buf, e.Host) + _ = binary.Write(&buf, binary.BigEndian, e.Port) + return buf.Bytes() +} + +// ToString formats the endpoint as a wireshark-dissector-style block +// headed by "@Endpoint", matching the style of Handshake/Acknowledge. +func (e *Endpoint) ToString() string { + b := commons.NewPrinter() + b.Printf("@Endpoint") + b.IncreaseIndent() + b.Printf("@Host") + b.IncreaseIndent() + b.Printf("@Length - %d - %s", len(e.Host), commons.Hexify(uint16(len(e.Host)))) + b.Printf("@Value - %s - %s", e.Host, commons.Hexify(e.Host)) + b.DecreaseIndent() + b.Printf("@Port - %d - %s", e.Port, commons.Hexify(e.Port)) + return b.String() +} + // Transmission is a parsed JRMP byte stream from either direction of a // Stream-protocol connection. The three opening fields capture the handshake // phase — any or all can be nil depending on which side of the conversation @@ -167,15 +192,7 @@ func (t *Transmission) ToString() string { b.Print(t.Acknowledge.ToString()) } if t.ClientEndpoint != nil { - b.Printf("@ClientEndpoint") - b.IncreaseIndent() - b.Printf("@Host") - b.IncreaseIndent() - b.Printf("@Length - %d - %s", len(t.ClientEndpoint.Host), commons.Hexify(uint16(len(t.ClientEndpoint.Host)))) - b.Printf("@Value - %s - %s", t.ClientEndpoint.Host, commons.Hexify(t.ClientEndpoint.Host)) - b.DecreaseIndent() - b.Printf("@Port - %d - %s", t.ClientEndpoint.Port, commons.Hexify(t.ClientEndpoint.Port)) - b.DecreaseIndent() + b.Print(t.ClientEndpoint.ToString()) } if len(t.Messages) > 0 { b.Printf("@Messages") diff --git a/rmi/rmi_test.go b/rmi/rmi_test.go index 5f3bfdd..93c4f66 100644 --- a/rmi/rmi_test.go +++ b/rmi/rmi_test.go @@ -153,6 +153,18 @@ func TestAcknowledge(t *testing.T) { require.Equal(t, int32(12345), tr.Acknowledge.Port) } +// TestEndpointToString covers the @Endpoint dump — every piece of state +// (host length, host value, port) must appear with both decimal and hex, +// matching the wireshark-dissector style of Handshake/Acknowledge. +func TestEndpointToString(t *testing.T) { + s := (&Endpoint{Host: "client.local", Port: 55555}).ToString() + require.Contains(t, s, "@Endpoint") + require.Contains(t, s, "@Host") + require.Contains(t, s, "@Length - 12 - 0x00 0c") + require.Contains(t, s, "@Value - client.local - 0x63 6c 69 65 6e 74 2e 6c 6f 63 61 6c") + require.Contains(t, s, "@Port - 55555 - 0x00 00 d9 03") +} + // ---------- ping / pingack / dgcack ---------- func TestPing(t *testing.T) { From 7c54fc2a73aa1cfd4b532ddd7fc3c2aae80ca061 Mon Sep 17 00:00:00 2001 From: phith0n Date: Sun, 19 Apr 2026 03:39:19 +0800 Subject: [PATCH 2/2] docs: trim rmi section in CLAUDE.md Shrink ~75 lines to ~30. Cut what the code already says (per-frame field breakdowns, the maybeReadClientEndpoint hostname-length edge case, the shared-cursor note, a redundant restatement of Decoder.Next behavior, the full client-side Opening() example that's already in the Decoder godoc). Keep what future work needs to avoid landmines: Registry-only scope + dispatch gate + JDK recalibration hint, FromBytes vs Decoder deadlock, both server-side Opening() and readReturn sentinel traps on live readers, the no-end-marker invariant that forbids replacing the sentinel with serz.FromBytes, and the wireshark-dump anti-pattern. Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 77 ++++++++++++------------------------------------------- 1 file changed, 16 insertions(+), 61 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e40dd35..f5e192c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,78 +53,33 @@ Key contracts: ### `rmi` — JRMP (Java RMI) wire-protocol parser -Read-only parser for a single direction of a JRMP Stream-protocol (`0x4B`) byte stream **addressed at `java.rmi.registry.Registry`**. Consumes either a client→server or server→client capture and produces a `Transmission` tree: optional `Handshake` (client side) or `Acknowledge` (server side), optional `ClientEndpoint` echo, then a `Messages []Message` list. +Read-only parser for a single direction of a JRMP Stream-protocol (`0x4B`) byte stream **addressed at `java.rmi.registry.Registry`**. Produces a `Transmission`: optional `Handshake` + `ClientEndpoint` (client→server) or `Acknowledge` (server→client), then `Messages []Message` (`CallMessage` / `ReturnMessage` / `PingMessage` / `PingAckMessage` / `DgcAckMessage`). Every frame satisfies `Message` (`Op() byte` + `ToString() string`). -**Scope: Registry only.** This module deliberately supports only one Remote interface — the well-known `sun.rmi.registry.RegistryImpl_Stub`. Any `MsgCall` whose header doesn't pass the dispatch gate (ObjID == `REGISTRY_ID` **and** methodHash == `RegistryInterfaceHash` **and** op ∈ [0..4]) is rejected at parse time in `readCall`. Non-Registry Remote interfaces each have their own method-hash table that we don't carry, and the project's use case (intercepting `rmiregistry` traffic for exploitation or analysis) is fully served by Registry coverage. If you need a general JRMP parser, this is not it. +**Scope: Registry only.** Any `MsgCall` whose header fails the dispatch gate (ObjID == `REGISTRY_ID` **and** methodHash == `RegistryInterfaceHash` **and** op ∈ [0..4]) is rejected at parse time in `readCall`. Registry uses the legacy `int32 operation = 0..4 + int64 RegistryInterfaceHash` form (op-index + shared interface hash); modern JRMP's `operation = -1 + per-method hash` is dynamic-proxy-stub only and NOT used here. `RegistryInterfaceHash` in `model.go` is calibrated against Zulu OpenJDK 17 — if it starts failing after a JDK upgrade, print `MethodHash` as hex and recalibrate (one-line edit). To add a new message type, extend `MsgXxx` in `model.go`, implement `Message`, add a case to `rmi/message.go:readMessage` — single extension point. -**Two entry points.** Pick by input shape, not by input source: -- `rmi.FromBytes(data []byte)` — for bytes already in memory (`.bin` captures, `io.ReadAll` of an HTTP body, etc.). Loops until `io.EOF` and returns the whole `Transmission`. **Never use with a live `net.Conn`**: the loop blocks on the next `PeekN(1)` after the last frame and deadlocks when the peer stays open waiting for a reply — which is the typical synchronous-RPC pattern. -- `rmi.Decoder` (`rmi/decoder.go`) — frame-by-frame, for live readers. `NewDecoder(r)` wraps any `io.Reader`; `Opening()` consumes the optional handshake prefix; `Next()` returns one `Message` per call or `io.EOF`. This is the only sensible choice for a long-lived TCP connection: callers apply `SetReadDeadline` on the underlying `net.Conn` between `Next()` calls to bound how long they wait for the next frame. +**Two entry points. Pick by input shape, not source.** +- `rmi.FromBytes(data []byte)` — fully-buffered captures only. **Deadlocks on a live `net.Conn`** (loops until EOF, the peek after the last frame blocks while the peer waits for a reply). +- `rmi.Decoder` — frame-by-frame over any `io.Reader`. Only sensible choice for live TCP. See the `Decoder` godoc for full usage. -Both rely on `serz.NewObjectStreamFromStream(*commons.Stream)` so the embedded serialization parse shares a byte cursor with the outer framing reader — no double-buffering, no peeked-byte loss at handoff. +**Opening phase has two live-reader traps.** -**Arg/payload-reading strategy.** -- `readCallArgs` — **exact count, no peek**. Once `readCall` passes the Registry dispatch gate, `registryArgCount(op)` gives the stub method's known arity, and `readCallArgs` reads precisely that many TCContents. The parser returns as soon as the frame's own bytes arrive — critical on a live TCP reader where a Registry client sends one Call and then waits for the server's response before sending anything else. A peek-ahead scheme would deadlock. -- `readReturn` — **sentinel**. Payload count is 0 (void method: bind/rebind/unbind) or 1 (list/lookup value / exception Throwable), and that choice depends on the originating Call's return type. Direction-agnostic parsing can't correlate Returns to outstanding Calls, so we fall back to a sentinel: read TCContents until `PeekN(1)` yields a byte outside `[JAVA_TC_BASE, JAVA_TC_MAX]` (= `[0x70, 0x7F]`) or `io.EOF`; assert count ≤ 1. TC_* and JRMP-flag (`[0x50, 0x54]`) ranges are disjoint, so the check is unambiguous. **On a live reader the terminating peek blocks** until the next frame's flag byte arrives, the peer closes (`io.EOF`), or the reader's deadline fires. Callers that process Returns over a live connection must set `SetReadDeadline` on the underlying `net.Conn`. - -Every JRMP frame implements `Message` (`Op() byte` + `ToString() string`). Five frame types: - -- **`CallMessage`** (0x50) — wraps an embedded serialization stream. The first `TC_BLOCKDATA` holds 34 bytes of primitive writes (`ObjID(22) + int32 op + int64 methodHash`). Remaining `TCContent` entries are the method arguments. On a successful parse, `Operation` is always one of the five `{Bind, List, Lookup, Rebind, Unbind}OpIndex` constants and `Decoded` is populated; a non-fatal decoder error (e.g. malformed string arg) can still leave `Decoded` nil while `Raw` / `ObjectArgs` hold the raw tree. `ToString()` renders the stream in wireshark-dissector style — see the rendering note below. -- **`ReturnMessage`** (0x51) — same embedded-stream shape, 15 bytes of primitives (`returnType + UID`) then ≤1 payload `TCContent` (value / exception / none-for-void). -- **`PingMessage`** (0x52), **`PingAckMessage`** (0x53) — single-byte frames, no payload. -- **`DgcAckMessage`** (0x54) — raw 14-byte UID written outside any `ObjectOutputStream` framing (the only JRMP frame that does *not* go through `serz`). - -**The critical design point worth internalizing**: a single Java serialization stream has no explicit end marker — `serz.FromReader` terminates only on `io.EOF`. Inside JRMP, the next byte after a Return body is the next message's flag (`0x50..0x54`), which is neither `io.EOF` nor a valid `TC_*` tag. The sentinel in `readReturn` (`rmi/return.go`) walks `serz.ReadTCContent` and stops when `PeekN(1)` returns a byte outside `[serz.JAVA_TC_BASE, serz.JAVA_TC_MAX]`. **Do not refactor this to use `serz.FromBytes`** — doing so would fail on any stream with more than one frame. (Calls don't need the sentinel because `registryArgCount` gives the exact count.) - -**Registry dispatch is op-index + interface hash, not per-method hash.** The JDK ships a precompiled `sun.rmi.registry.RegistryImpl_Stub` whose wire format is `operation = 0..4` (indexing into `{bind, list, lookup, rebind, unbind}`) paired with a single `int64 RegistryInterfaceHash` shared by all five methods. Modern JRMP's `operation = -1 + per-method hash` pattern applies only to dynamic-proxy stubs and is NOT used by Registry. `rmi/model.go` exposes the five op-index constants (`LookupOpIndex`, etc.) and the `RegistryInterfaceHash` constant (calibrated against a live Zulu OpenJDK 17 capture; see `testcases/rmi/*.bin` and `_tools/rmi-capture/`). If a real Registry capture starts failing the hash check after a JDK upgrade, print `MethodHash` as hex and recalibrate — it's a one-line edit. - -Adding a new message type (e.g. if SingleOp/Multiplex support is ever added): extend the `MsgXxx` constants in `model.go`, define `*XxxMessage` implementing `Message`, add a case to the `switch` in `rmi/message.go:readMessage`. The dispatcher is the single extension point. - -The endpoint-echo heuristic in `parser.go:maybeReadClientEndpoint` peeks the byte right after the handshake: if it falls in `[0x50, 0x54]` (JRMP message flag range) we skip the echo, otherwise read `writeUTF + int32`. This lets hand-crafted test fixtures omit the echo. The ambiguity only collides on pathological (≥ 20480-char) hostnames. - -**Live-TCP ergonomics.** On a long-lived `net.Conn` a typical Registry session has the shape handshake → Call → (peer waits for Return) → Return → close. `FromBytes` would deadlock on the second peek after the first frame; `Decoder` is the only viable choice. For *client-side* reading (capture already in hand, or reading responses from a server), the one-shot `Opening()` is fine: +First, `Opening()` reads `Handshake + ClientEndpoint` in one call — **deadlocks on server-side reads** because a conforming Java client (`sun.rmi.transport.tcp.TCPChannel`) blocks after its 7-byte handshake waiting for the server's `ProtocolAck` before writing the endpoint echo. Servers must use the split primitives: ```go -d := rmi.NewDecoder(conn) -opening, _ := d.Opening() // optional — read handshake/ack/endpoint -for { - _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) - msg, err := d.Next() - if errors.Is(err, io.EOF) { break } - if err != nil { return err } - handle(msg) -} +hs, _ := d.ReadHandshake() // 7 bytes only +_, _ = conn.Write((&rmi.Acknowledge{Host: h, Port: p}).ToBytes()) // unblocks client +ep, _ := d.ReadClientEndpoint() // echo now arrives ``` -*Server-side `Opening()` deadlocks.* A conforming Java client (`sun.rmi.transport.tcp.TCPChannel`) writes the 7-byte handshake and then blocks reading the server's `ProtocolAck` before writing its `ClientEndpoint` echo. `Opening()` consumes handshake + endpoint in one call, so its second peek waits for bytes the client refuses to send until the server writes the Ack. For servers (and any caller that must inject writes into the handshake phase), use the fine-grained primitives: - -```go -d := rmi.NewDecoder(conn) -hs, err := d.ReadHandshake() // consumes only the 7-byte handshake -// ... write the Ack to conn before reading further ... -remote := conn.RemoteAddr().(*net.TCPAddr) -_, _ = conn.Write((&rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)}).ToBytes()) -ep, err := d.ReadClientEndpoint() // now safe: client unblocks after reading Ack -for { - _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) - msg, err := d.Next() - ... -} -``` - -`ReadAcknowledge()` is the symmetric client-side primitive. The three opening primitives and `Opening()` are **mutually exclusive** — the Decoder tracks a stage (`stageInitial` → `stageAfterHandshake` → `stageReady`) and refuses out-of-order calls. `Next()` auto-advances to `stageReady` from either earlier stage (auto-consuming a `ClientEndpoint` after `ReadHandshake` if the caller skipped `ReadClientEndpoint`), so omitting the opening entirely still works for bare captures. - -`Handshake`, `Acknowledge`, and `Endpoint` all expose `ToBytes()` for writing: zero-valued `Magic`/`Protocol`/`Flag` fields default to `JRMI_MAGIC` / `ProtocolStream` / `AckFlag`, so server code can usually construct with just `Host` + `Port` (see `TestOpeningEncodersRoundTrip` for the exact layouts). - -`Decoder.Next()` returns one frame per call. Registry Calls return as soon as their own bytes arrive (no peek past last arg). ReturnData uses the sentinel and may block between frames on a live reader until the next flag byte arrives or the reader's deadline fires — the caller is responsible for the deadline. +`ReadAcknowledge()` is the symmetric client-side primitive. A stage enum (`stageInitial → stageAfterHandshake → stageReady`) enforces ordering; `Opening()` and the three `Read*` primitives are mutually exclusive. `Next()` auto-advances from any earlier stage to `stageReady`. `Handshake` / `Acknowledge` / `Endpoint` all expose `ToBytes()` with zero-value defaults (`JRMI_MAGIC` / `ProtocolStream` / `AckFlag`) so server code can skip those fields. -**`ToString()` rendering is wireshark-dissector style — every byte is printed exactly once, with semantic labels inline.** `Call/ReturnMessage.ToString()` emits a compact `@Decoded` summary (method + scalar args; complex args referenced by handler) at the top, then `@Serialization` with a custom walk (`rmi/printer.go`): +**Arg/payload reading strategies are not interchangeable.** +- `readCallArgs`: **exact count**, no peek. `registryArgCount(op)` gives the stub method's arity; the parser returns as soon as the frame's own bytes arrive — critical because a Registry client sends one Call and then waits for the Return before writing anything else. +- `readReturn`: **sentinel**. Payload count (0 for void bind/rebind/unbind, 1 for list/lookup/Throwable) depends on the originating Call, which direction-agnostic parsing can't correlate. So we peek after the 15-byte primitive header and stop when the next byte falls outside `[JAVA_TC_BASE, JAVA_TC_MAX]` (= `[0x70, 0x7F]`). Works because TC_* and JRMP-flag (`[0x50, 0x54]`) ranges are disjoint. **On live readers this peek blocks until the next frame's flag byte, EOF, or a read deadline fires** — callers must `SetReadDeadline`. -- The leading `TC_BLOCKDATA` that carries the Call's 34-byte `ObjID + op + hash` (or the Return's 15-byte `returnType + UID`) is replaced in-place with its decomposed fields, each showing both decimal *and* the matching byte slice. -- Every subsequent `TCContent` arg has its header line annotated: `TC_STRING - 0x74 (Registry.lookup arg 0: "name")`, `TC_OBJECT - 0x73 (Registry.bind arg 1: "obj")`. -- Remote-stub TCContent subtrees (bind/rebind `obj`) appear exactly once, inside `@Serialization`; `@Decoded.Args` references them by handler — **do not reintroduce full-subtree dumping in `DecodedCall.ToString`**, that's the duplication this layout fixes. +**Critical invariant.** A Java serialization stream has no end marker — `serz.FromReader` terminates only on EOF. Inside JRMP the byte after a Return body is the next message's flag (`0x50..0x54`), which is neither EOF nor a valid `TC_*` tag. **Do not refactor `readReturn` to use `serz.FromBytes`** — it would fail on any stream with more than one frame. (Calls don't need the sentinel because the arg count is known.) -Fields outside `@Serialization` (`Handshake.Magic/Version/Protocol`, `Acknowledge`/`ClientEndpoint.Host/Port`) keep their raw hex because no other section carries those bytes. +**`ToString()` rendering is wireshark-dissector style — every byte is printed exactly once, semantic labels inline.** `Call/ReturnMessage.ToString()` emits a compact `@Decoded` summary at the top, then `@Serialization` via `rmi/printer.go`: the leading `TC_BLOCKDATA` is replaced by its decomposed primitive fields (decimal + hex for each), every subsequent `TCContent` arg gets an inline annotation (`TC_STRING - 0x74 (Registry.lookup arg 0: "name")`), and remote-stub subtrees appear **exactly once** inside `@Serialization` (referenced by handler in `@Decoded.Args`). **Do not reintroduce full-subtree dumping in `DecodedCall.ToString`** — that's the duplication this layout fixes. ### `class` — JVM `.class` parser