diff --git a/CLAUDE.md b/CLAUDE.md index 1c94442..f5e192c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,58 +53,33 @@ Key contracts: ### `rmi` — JRMP (Java RMI) wire-protocol parser -Read-only parser for a single direction of a JRMP Stream-protocol (`0x4B`) byte stream **addressed at `java.rmi.registry.Registry`**. Consumes either a client→server or server→client capture and produces a `Transmission` tree: optional `Handshake` (client side) or `Acknowledge` (server side), optional `ClientEndpoint` echo, then a `Messages []Message` list. +Read-only parser for a single direction of a JRMP Stream-protocol (`0x4B`) byte stream **addressed at `java.rmi.registry.Registry`**. Produces a `Transmission`: optional `Handshake` + `ClientEndpoint` (client→server) or `Acknowledge` (server→client), then `Messages []Message` (`CallMessage` / `ReturnMessage` / `PingMessage` / `PingAckMessage` / `DgcAckMessage`). Every frame satisfies `Message` (`Op() byte` + `ToString() string`). -**Scope: Registry only.** This module deliberately supports only one Remote interface — the well-known `sun.rmi.registry.RegistryImpl_Stub`. Any `MsgCall` whose header doesn't pass the dispatch gate (ObjID == `REGISTRY_ID` **and** methodHash == `RegistryInterfaceHash` **and** op ∈ [0..4]) is rejected at parse time in `readCall`. Non-Registry Remote interfaces each have their own method-hash table that we don't carry, and the project's use case (intercepting `rmiregistry` traffic for exploitation or analysis) is fully served by Registry coverage. If you need a general JRMP parser, this is not it. +**Scope: Registry only.** Any `MsgCall` whose header fails the dispatch gate (ObjID == `REGISTRY_ID` **and** methodHash == `RegistryInterfaceHash` **and** op ∈ [0..4]) is rejected at parse time in `readCall`. Registry uses the legacy `int32 operation = 0..4 + int64 RegistryInterfaceHash` form (op-index + shared interface hash); modern JRMP's `operation = -1 + per-method hash` is dynamic-proxy-stub only and NOT used here. `RegistryInterfaceHash` in `model.go` is calibrated against Zulu OpenJDK 17 — if it starts failing after a JDK upgrade, print `MethodHash` as hex and recalibrate (one-line edit). To add a new message type, extend `MsgXxx` in `model.go`, implement `Message`, add a case to `rmi/message.go:readMessage` — single extension point. -**Two entry points.** Pick by input shape, not by input source: -- `rmi.FromBytes(data []byte)` — for bytes already in memory (`.bin` captures, `io.ReadAll` of an HTTP body, etc.). Loops until `io.EOF` and returns the whole `Transmission`. **Never use with a live `net.Conn`**: the loop blocks on the next `PeekN(1)` after the last frame and deadlocks when the peer stays open waiting for a reply — which is the typical synchronous-RPC pattern. -- `rmi.Decoder` (`rmi/decoder.go`) — frame-by-frame, for live readers. `NewDecoder(r)` wraps any `io.Reader`; `Opening()` consumes the optional handshake prefix; `Next()` returns one `Message` per call or `io.EOF`. This is the only sensible choice for a long-lived TCP connection: callers apply `SetReadDeadline` on the underlying `net.Conn` between `Next()` calls to bound how long they wait for the next frame. +**Two entry points. Pick by input shape, not source.** +- `rmi.FromBytes(data []byte)` — fully-buffered captures only. **Deadlocks on a live `net.Conn`** (loops until EOF, the peek after the last frame blocks while the peer waits for a reply). +- `rmi.Decoder` — frame-by-frame over any `io.Reader`. Only sensible choice for live TCP. See the `Decoder` godoc for full usage. -Both rely on `serz.NewObjectStreamFromStream(*commons.Stream)` so the embedded serialization parse shares a byte cursor with the outer framing reader — no double-buffering, no peeked-byte loss at handoff. +**Opening phase has two live-reader traps.** -**Arg/payload-reading strategy.** -- `readCallArgs` — **exact count, no peek**. Once `readCall` passes the Registry dispatch gate, `registryArgCount(op)` gives the stub method's known arity, and `readCallArgs` reads precisely that many TCContents. The parser returns as soon as the frame's own bytes arrive — critical on a live TCP reader where a Registry client sends one Call and then waits for the server's response before sending anything else. A peek-ahead scheme would deadlock. -- `readReturn` — **sentinel**. Payload count is 0 (void method: bind/rebind/unbind) or 1 (list/lookup value / exception Throwable), and that choice depends on the originating Call's return type. Direction-agnostic parsing can't correlate Returns to outstanding Calls, so we fall back to a sentinel: read TCContents until `PeekN(1)` yields a byte outside `[JAVA_TC_BASE, JAVA_TC_MAX]` (= `[0x70, 0x7F]`) or `io.EOF`; assert count ≤ 1. TC_* and JRMP-flag (`[0x50, 0x54]`) ranges are disjoint, so the check is unambiguous. **On a live reader the terminating peek blocks** until the next frame's flag byte arrives, the peer closes (`io.EOF`), or the reader's deadline fires. Callers that process Returns over a live connection must set `SetReadDeadline` on the underlying `net.Conn`. - -Every JRMP frame implements `Message` (`Op() byte` + `ToString() string`). Five frame types: - -- **`CallMessage`** (0x50) — wraps an embedded serialization stream. The first `TC_BLOCKDATA` holds 34 bytes of primitive writes (`ObjID(22) + int32 op + int64 methodHash`). Remaining `TCContent` entries are the method arguments. On a successful parse, `Operation` is always one of the five `{Bind, List, Lookup, Rebind, Unbind}OpIndex` constants and `Decoded` is populated; a non-fatal decoder error (e.g. malformed string arg) can still leave `Decoded` nil while `Raw` / `ObjectArgs` hold the raw tree. `ToString()` renders the stream in wireshark-dissector style — see the rendering note below. -- **`ReturnMessage`** (0x51) — same embedded-stream shape, 15 bytes of primitives (`returnType + UID`) then ≤1 payload `TCContent` (value / exception / none-for-void). -- **`PingMessage`** (0x52), **`PingAckMessage`** (0x53) — single-byte frames, no payload. -- **`DgcAckMessage`** (0x54) — raw 14-byte UID written outside any `ObjectOutputStream` framing (the only JRMP frame that does *not* go through `serz`). - -**The critical design point worth internalizing**: a single Java serialization stream has no explicit end marker — `serz.FromReader` terminates only on `io.EOF`. Inside JRMP, the next byte after a Return body is the next message's flag (`0x50..0x54`), which is neither `io.EOF` nor a valid `TC_*` tag. The sentinel in `readReturn` (`rmi/return.go`) walks `serz.ReadTCContent` and stops when `PeekN(1)` returns a byte outside `[serz.JAVA_TC_BASE, serz.JAVA_TC_MAX]`. **Do not refactor this to use `serz.FromBytes`** — doing so would fail on any stream with more than one frame. (Calls don't need the sentinel because `registryArgCount` gives the exact count.) - -**Registry dispatch is op-index + interface hash, not per-method hash.** The JDK ships a precompiled `sun.rmi.registry.RegistryImpl_Stub` whose wire format is `operation = 0..4` (indexing into `{bind, list, lookup, rebind, unbind}`) paired with a single `int64 RegistryInterfaceHash` shared by all five methods. Modern JRMP's `operation = -1 + per-method hash` pattern applies only to dynamic-proxy stubs and is NOT used by Registry. `rmi/model.go` exposes the five op-index constants (`LookupOpIndex`, etc.) and the `RegistryInterfaceHash` constant (calibrated against a live Zulu OpenJDK 17 capture; see `testcases/rmi/*.bin` and `_tools/rmi-capture/`). If a real Registry capture starts failing the hash check after a JDK upgrade, print `MethodHash` as hex and recalibrate — it's a one-line edit. - -Adding a new message type (e.g. if SingleOp/Multiplex support is ever added): extend the `MsgXxx` constants in `model.go`, define `*XxxMessage` implementing `Message`, add a case to the `switch` in `rmi/message.go:readMessage`. The dispatcher is the single extension point. - -The endpoint-echo heuristic in `parser.go:maybeReadClientEndpoint` peeks the byte right after the handshake: if it falls in `[0x50, 0x54]` (JRMP message flag range) we skip the echo, otherwise read `writeUTF + int32`. This lets hand-crafted test fixtures omit the echo. The ambiguity only collides on pathological (≥ 20480-char) hostnames. - -**Live-TCP ergonomics.** On a long-lived `net.Conn` a typical Registry session has the shape handshake → Call → (peer waits for Return) → Return → close. `FromBytes` would deadlock on the second peek after the first frame; `Decoder` is the only viable choice: +First, `Opening()` reads `Handshake + ClientEndpoint` in one call — **deadlocks on server-side reads** because a conforming Java client (`sun.rmi.transport.tcp.TCPChannel`) blocks after its 7-byte handshake waiting for the server's `ProtocolAck` before writing the endpoint echo. Servers must use the split primitives: ```go -d := rmi.NewDecoder(conn) -opening, _ := d.Opening() // optional — read handshake/ack/endpoint -for { - _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) - msg, err := d.Next() - if errors.Is(err, io.EOF) { break } - if err != nil { return err } - handle(msg) -} +hs, _ := d.ReadHandshake() // 7 bytes only +_, _ = conn.Write((&rmi.Acknowledge{Host: h, Port: p}).ToBytes()) // unblocks client +ep, _ := d.ReadClientEndpoint() // echo now arrives ``` -`Decoder.Next()` returns one frame per call. Registry Calls return as soon as their own bytes arrive (no peek past last arg). ReturnData uses the sentinel and may block between frames on a live reader until the next flag byte arrives or the reader's deadline fires — the caller is responsible for the deadline. `Opening()` is optional; if omitted, the first `Next()` transparently consumes and discards any handshake prefix. Calling `Opening()` twice, or after `Next()`, returns an error. +`ReadAcknowledge()` is the symmetric client-side primitive. A stage enum (`stageInitial → stageAfterHandshake → stageReady`) enforces ordering; `Opening()` and the three `Read*` primitives are mutually exclusive. `Next()` auto-advances from any earlier stage to `stageReady`. `Handshake` / `Acknowledge` / `Endpoint` all expose `ToBytes()` with zero-value defaults (`JRMI_MAGIC` / `ProtocolStream` / `AckFlag`) so server code can skip those fields. -**`ToString()` rendering is wireshark-dissector style — every byte is printed exactly once, with semantic labels inline.** `Call/ReturnMessage.ToString()` emits a compact `@Decoded` summary (method + scalar args; complex args referenced by handler) at the top, then `@Serialization` with a custom walk (`rmi/printer.go`): +**Arg/payload reading strategies are not interchangeable.** +- `readCallArgs`: **exact count**, no peek. `registryArgCount(op)` gives the stub method's arity; the parser returns as soon as the frame's own bytes arrive — critical because a Registry client sends one Call and then waits for the Return before writing anything else. +- `readReturn`: **sentinel**. Payload count (0 for void bind/rebind/unbind, 1 for list/lookup/Throwable) depends on the originating Call, which direction-agnostic parsing can't correlate. So we peek after the 15-byte primitive header and stop when the next byte falls outside `[JAVA_TC_BASE, JAVA_TC_MAX]` (= `[0x70, 0x7F]`). Works because TC_* and JRMP-flag (`[0x50, 0x54]`) ranges are disjoint. **On live readers this peek blocks until the next frame's flag byte, EOF, or a read deadline fires** — callers must `SetReadDeadline`. -- The leading `TC_BLOCKDATA` that carries the Call's 34-byte `ObjID + op + hash` (or the Return's 15-byte `returnType + UID`) is replaced in-place with its decomposed fields, each showing both decimal *and* the matching byte slice. -- Every subsequent `TCContent` arg has its header line annotated: `TC_STRING - 0x74 (Registry.lookup arg 0: "name")`, `TC_OBJECT - 0x73 (Registry.bind arg 1: "obj")`. -- Remote-stub TCContent subtrees (bind/rebind `obj`) appear exactly once, inside `@Serialization`; `@Decoded.Args` references them by handler — **do not reintroduce full-subtree dumping in `DecodedCall.ToString`**, that's the duplication this layout fixes. +**Critical invariant.** A Java serialization stream has no end marker — `serz.FromReader` terminates only on EOF. Inside JRMP the byte after a Return body is the next message's flag (`0x50..0x54`), which is neither EOF nor a valid `TC_*` tag. **Do not refactor `readReturn` to use `serz.FromBytes`** — it would fail on any stream with more than one frame. (Calls don't need the sentinel because the arg count is known.) -Fields outside `@Serialization` (`Handshake.Magic/Version/Protocol`, `Acknowledge`/`ClientEndpoint.Host/Port`) keep their raw hex because no other section carries those bytes. +**`ToString()` rendering is wireshark-dissector style — every byte is printed exactly once, semantic labels inline.** `Call/ReturnMessage.ToString()` emits a compact `@Decoded` summary at the top, then `@Serialization` via `rmi/printer.go`: the leading `TC_BLOCKDATA` is replaced by its decomposed primitive fields (decimal + hex for each), every subsequent `TCContent` arg gets an inline annotation (`TC_STRING - 0x74 (Registry.lookup arg 0: "name")`), and remote-stub subtrees appear **exactly once** inside `@Serialization` (referenced by handler in `@Decoded.Args`). **Do not reintroduce full-subtree dumping in `DecodedCall.ToString`** — that's the duplication this layout fixes. ### `class` — JVM `.class` parser diff --git a/rmi/decoder.go b/rmi/decoder.go index 681ac38..e7072bc 100644 --- a/rmi/decoder.go +++ b/rmi/decoder.go @@ -19,7 +19,32 @@ import ( // one message before the next arrives and can apply SetReadDeadline on the // underlying net.Conn between calls to bound how long they wait. // -// Blocking semantics: +// # Opening-phase primitives vs. Opening() +// +// The JRMP opening has three pieces that are interleaved with a server-side +// write on the wire: +// +// client → server: JRMI magic + version + 0x4B (7 bytes) +// server → client: 0x4E + writeUTF(host) + int32(port) (Acknowledge) +// client → server: writeUTF(host) + int32(port) (ClientEndpoint) +// +// A conforming Java client blocks after sending the handshake until the +// server's Acknowledge arrives, and only then writes its ClientEndpoint. +// That means Opening() — which reads Handshake+ClientEndpoint in one call — +// deadlocks on a server-side reader: the second PeekN waits for bytes the +// client will not send until it has received the Ack. +// +// For server-side use (or any caller that needs to inject a write between +// the handshake reads), use the fine-grained primitives: +// +// ReadHandshake() — consumes only the 7-byte handshake +// ReadClientEndpoint() — consumes the post-Ack endpoint echo +// ReadAcknowledge() — consumes the server→client Acknowledge frame +// +// Opening() remains the convenience entry point for fully-buffered captures +// and for client→server consumers that already have the bytes in hand. +// +// # Blocking semantics // // - CallMessage: the Registry dispatch gate fails fast for non-Registry // headers, and a valid Registry Call reads an exact arg count — Next() @@ -40,10 +65,29 @@ import ( // Callers that process Returns over a live connection should set // SetReadDeadline on the underlying net.Conn. // -// Typical usage: +// # Typical usage +// +// Client → server capture (bytes already in hand): +// +// d := rmi.NewDecoder(bytes.NewReader(data)) +// opening, err := d.Opening() +// if err != nil { return err } +// for { +// msg, err := d.Next() +// if errors.Is(err, io.EOF) { break } +// if err != nil { return err } +// handle(msg) +// } +// +// Server side on a live net.Conn: // // d := rmi.NewDecoder(conn) -// opening, err := d.Opening() // optional; omit to skip +// hs, err := d.ReadHandshake() +// if err != nil { return err } +// remote := conn.RemoteAddr().(*net.TCPAddr) +// ack := &rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)} +// if _, err := conn.Write(ack.ToBytes()); err != nil { return err } +// ep, err := d.ReadClientEndpoint() // if err != nil { return err } // for { // _ = conn.SetReadDeadline(time.Now().Add(30 * time.Second)) @@ -53,10 +97,21 @@ import ( // handle(msg) // } type Decoder struct { - stream *commons.Stream - openingDone bool + stream *commons.Stream + stage decoderStage } +// decoderStage tracks how far through the opening phase the Decoder has +// progressed. Each Read* primitive and Opening() check this before +// advancing it; Next() refuses to run until the stage reaches stageReady. +type decoderStage int + +const ( + stageInitial decoderStage = iota // nothing consumed yet + stageAfterHandshake // 7-byte handshake consumed; ClientEndpoint still pending + stageReady // opening phase done; Next() is the only valid call +) + // NewDecoder returns a Decoder that reads from r. The Decoder holds the // reader until its last frame is consumed or the caller drops it; the // underlying bytes are only consumed on Opening() / Next() calls. @@ -64,24 +119,33 @@ func NewDecoder(r io.Reader) *Decoder { return &Decoder{stream: commons.NewStreamFromReader(r)} } -// Opening reads the optional handshake-phase prefix: Handshake + client -// endpoint echo (client→server direction) or Acknowledge (server→client -// direction), or neither for a bare capture that starts mid-stream. +// Opening reads the optional handshake-phase prefix in one call: Handshake + +// client endpoint echo (client→server direction) or Acknowledge +// (server→client direction), or neither for a bare capture that starts +// mid-stream. +// +// Opening is a convenience entry point for fully-buffered captures and +// consumers that already have every byte. It is NOT safe on a server-side +// live reader: a conforming client sends its handshake and then blocks +// waiting for the server's Acknowledge, so the peek inside +// maybeReadClientEndpoint deadlocks. Servers must use ReadHandshake + +// ReadClientEndpoint with their own Acknowledge write in between. // // Calling Opening is optional — if the caller omits it, the first Next() // call will read and discard any opening transparently. Explicit Opening // is the way to access the Handshake / Acknowledge / ClientEndpoint fields. // -// It is an error to call Opening twice, or to call it after Next(). +// It is an error to call Opening twice, after Next(), or after any of the +// ReadHandshake / ReadClientEndpoint / ReadAcknowledge primitives. func (d *Decoder) Opening() (*Opening, error) { - if d.openingDone { - return nil, errors.New("rmi.Decoder: Opening already called") + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: Opening is only valid as the first call") } - d.openingDone = true var t Transmission if err := readOpening(d.stream, &t); err != nil { return nil, err } + d.stage = stageReady return &Opening{ Handshake: t.Handshake, Acknowledge: t.Acknowledge, @@ -89,18 +153,80 @@ func (d *Decoder) Opening() (*Opening, error) { }, nil } -// Next returns the next JRMP message. On the first call, if Opening hasn't -// been invoked, any handshake prefix is read and discarded. Returns io.EOF -// when the reader closes cleanly at a frame boundary. Non-Registry Calls -// error out at parse time; ReturnData may block on the sentinel peek -// between frames — see the Decoder doc for details. +// ReadHandshake consumes the 7-byte client→server opening +// (JRMI magic + uint16 version + 1-byte sub-protocol flag) and nothing else. +// This is the first call a server should make after accepting a connection: +// it returns as soon as the 7 bytes arrive, leaving the server free to +// write its Acknowledge before reading the ClientEndpoint echo. Call +// ReadClientEndpoint (or Next, which consumes any remaining opening +// implicitly) once the Ack has been written. +// +// It is an error to call ReadHandshake after Opening, any other Read* +// primitive, or Next. +func (d *Decoder) ReadHandshake() (*Handshake, error) { + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: ReadHandshake must be the first call") + } + h, err := readHandshake(d.stream) + if err != nil { + return nil, err + } + d.stage = stageAfterHandshake + return h, nil +} + +// ReadClientEndpoint consumes the client's post-Acknowledge endpoint echo +// (writeUTF host + int32 port). Must be preceded by ReadHandshake; the +// server is expected to have written its Acknowledge between the two +// calls. Returns (nil, nil) if the client omits the echo (matching the +// lenient behavior of fully-buffered captures). +// +// It is an error to call ReadClientEndpoint outside the +// ReadHandshake → ReadClientEndpoint sequence. +func (d *Decoder) ReadClientEndpoint() (*Endpoint, error) { + if d.stage != stageAfterHandshake { + return nil, errors.New("rmi.Decoder: ReadClientEndpoint must follow ReadHandshake") + } + ep, err := maybeReadClientEndpoint(d.stream) + if err != nil { + return nil, fmt.Errorf("client endpoint echo: %w", err) + } + d.stage = stageReady + return ep, nil +} + +// ReadAcknowledge consumes the server→client Acknowledge frame +// (0x4E + writeUTF host + int32 port). This is the primitive a client-side +// reader uses to inspect the server's view of its endpoint before any +// message traffic. Must be the first call on the Decoder. +// +// It is an error to call ReadAcknowledge after Opening, any other Read* +// primitive, or Next. +func (d *Decoder) ReadAcknowledge() (*Acknowledge, error) { + if d.stage != stageInitial { + return nil, errors.New("rmi.Decoder: ReadAcknowledge must be the first call") + } + a, err := readAcknowledge(d.stream) + if err != nil { + return nil, err + } + d.stage = stageReady + return a, nil +} + +// Next returns the next JRMP message. On the first call, if no opening +// primitive has advanced the Decoder past stageReady, any remaining +// handshake-phase bytes are read and discarded — including a lone +// ClientEndpoint when ReadHandshake was called but ReadClientEndpoint was +// skipped. Returns io.EOF when the reader closes cleanly at a frame +// boundary. Non-Registry Calls error out at parse time; ReturnData may +// block on the sentinel peek between frames — see the Decoder doc for +// details. func (d *Decoder) Next() (Message, error) { - if !d.openingDone { - var discard Transmission - if err := readOpening(d.stream, &discard); err != nil { + if d.stage != stageReady { + if err := d.finishOpening(); err != nil { return nil, err } - d.openingDone = true } if _, err := d.stream.PeekN(1); err != nil { // io.EOF flows through unwrapped so callers can check with @@ -114,6 +240,26 @@ func (d *Decoder) Next() (Message, error) { return readMessage(d.stream) } +// finishOpening fast-forwards from whichever stage the caller left the +// Decoder in to stageReady, consuming only the bytes the remaining opening +// phase owns. It exists so Next() can be called on a Decoder whose caller +// didn't (or couldn't) explicitly finish the opening. +func (d *Decoder) finishOpening() error { + switch d.stage { + case stageInitial: + var discard Transmission + if err := readOpening(d.stream, &discard); err != nil { + return err + } + case stageAfterHandshake: + if _, err := maybeReadClientEndpoint(d.stream); err != nil { + return fmt.Errorf("client endpoint echo: %w", err) + } + } + d.stage = stageReady + return nil +} + // Opening captures the three optional handshake-phase fields of a JRMP // Transmission. Any or all may be nil: client→server traffic typically sets // Handshake + ClientEndpoint; server→client sets Acknowledge; a capture diff --git a/rmi/decoder_test.go b/rmi/decoder_test.go index 299bd8b..8032ab5 100644 --- a/rmi/decoder_test.go +++ b/rmi/decoder_test.go @@ -157,6 +157,172 @@ func TestDecoderRejectsNonRegistryCall(t *testing.T) { require.Contains(t, err.Error(), "not a Registry call") } +// TestDecoderServerFlowWithInterleavedAck reproduces the real rmiregistry +// server timing: a standard Java client sends its 7-byte handshake and +// then BLOCKS until the server writes a ProtocolAck — only after reading +// the Ack does it send its ClientEndpoint echo. Opening() would deadlock +// on this shape because it reads handshake and endpoint in a single call. +// The server-side flow (ReadHandshake → write Ack → ReadClientEndpoint → +// Next) must complete without blocking past what the client will send at +// each step. +func TestDecoderServerFlowWithInterleavedAck(t *testing.T) { + // c2s: what the client writes. s2c: what the server writes (captured + // here so we can assert the Ack bytes the server produced). + c2sRead, c2sWrite := io.Pipe() + s2cRead, s2cWrite := io.Pipe() + + clientDone := make(chan error, 1) + // Client goroutine: mimic sun.rmi.transport.tcp.TCPChannel — write + // handshake, read ack in full, then write endpoint + a Ping so the + // server has a message to parse and we can verify the whole flow. + go func() { + defer func() { _ = c2sWrite.Close() }() + if _, err := c2sWrite.Write(buildHandshake()); err != nil { + clientDone <- err + return + } + // Drain the server's Ack before writing anything else — this is + // the critical ordering constraint. Ack = 1 flag + 2 length + + // len("127.0.0.1")=9 + 4 port = 16 bytes for the fixture below. + ack := make([]byte, 16) + if _, err := io.ReadFull(s2cRead, ack); err != nil { + clientDone <- err + return + } + if _, err := c2sWrite.Write(buildClientEndpointEcho("client.local", 55555)); err != nil { + clientDone <- err + return + } + if _, err := c2sWrite.Write(buildPing()); err != nil { + clientDone <- err + return + } + clientDone <- nil + }() + + d := NewDecoder(c2sRead) + + // Stage 1: read just the handshake. Must return as soon as 7 bytes + // arrive — a peek past that would block. + hs, err := d.ReadHandshake() + require.NoError(t, err) + require.Equal(t, uint16(2), hs.Version) + require.Equal(t, ProtocolStream, hs.Protocol) + + // Stage 2: server writes its Ack. Without this step the client will + // sit forever on its io.ReadFull and the next ReadClientEndpoint + // would deadlock. + ack := &Acknowledge{Host: "127.0.0.1", Port: 1234} + go func() { + _, _ = s2cWrite.Write(ack.ToBytes()) + _ = s2cWrite.Close() + }() + + // Stage 3: consume the post-Ack endpoint echo. + ep, err := d.ReadClientEndpoint() + require.NoError(t, err) + require.NotNil(t, ep) + require.Equal(t, "client.local", ep.Host) + require.Equal(t, int32(55555), ep.Port) + + // Stage 4: messages flow normally from here. + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) + + select { + case err := <-clientDone: + require.NoError(t, err) + case <-time.After(2 * time.Second): + t.Fatal("client goroutine stalled — server flow did not advance as expected") + } +} + +// TestDecoderOpeningAfterReadHandshakeErrors: the three opening-phase +// entry points are mutually exclusive. Once ReadHandshake has committed +// the Decoder to the fine-grained flow, Opening() is no longer valid. +func TestDecoderOpeningAfterReadHandshakeErrors(t *testing.T) { + d := NewDecoder(bytes.NewReader(buildHandshake())) + _, err := d.ReadHandshake() + require.NoError(t, err) + _, err = d.Opening() + require.Error(t, err) +} + +// TestDecoderReadHandshakeAfterOpeningErrors: symmetric — once Opening has +// finished the opening phase, none of the Read* primitives may run. +func TestDecoderReadHandshakeAfterOpeningErrors(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildHandshake()) + buf.Write(buildClientEndpointEcho("h", 1)) + d := NewDecoder(&buf) + _, err := d.Opening() + require.NoError(t, err) + _, err = d.ReadHandshake() + require.Error(t, err) +} + +// TestDecoderReadClientEndpointRequiresHandshake: the endpoint primitive +// has no way to locate itself in the stream without the preceding +// handshake read — guard against callers skipping the prerequisite. +func TestDecoderReadClientEndpointRequiresHandshake(t *testing.T) { + d := NewDecoder(bytes.NewReader(buildClientEndpointEcho("h", 1))) + _, err := d.ReadClientEndpoint() + require.Error(t, err) +} + +// TestDecoderNextAutoConsumesClientEndpoint: if a server calls +// ReadHandshake but then skips ReadClientEndpoint (e.g. doesn't care +// about the echo), Next() should transparently consume the echo before +// parsing the first message. +func TestDecoderNextAutoConsumesClientEndpoint(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildHandshake()) + buf.Write(buildClientEndpointEcho("h", 1)) + buf.Write(buildPing()) + + d := NewDecoder(&buf) + _, err := d.ReadHandshake() + require.NoError(t, err) + + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) +} + +// TestDecoderReadAcknowledge: client-side flow — read the server's Ack +// as the first call, then message traffic. +func TestDecoderReadAcknowledge(t *testing.T) { + var buf bytes.Buffer + buf.Write(buildAck("1.2.3.4", 4242)) + buf.Write(buildPing()) + + d := NewDecoder(&buf) + ack, err := d.ReadAcknowledge() + require.NoError(t, err) + require.Equal(t, "1.2.3.4", ack.Host) + require.Equal(t, int32(4242), ack.Port) + + msg, err := d.Next() + require.NoError(t, err) + require.Equal(t, MsgPing, msg.Op()) +} + +// TestOpeningEncodersRoundTrip: the new ToBytes encoders must produce the +// exact layout the fixture builders produce (and therefore the exact +// layout the parsers consume). +func TestOpeningEncodersRoundTrip(t *testing.T) { + // Handshake: defaults on zero-valued Magic/Protocol. + require.Equal(t, buildHandshake(), (&Handshake{Version: 2}).ToBytes()) + // Acknowledge: default-Flag path and explicit-Flag path both emit + // the buildAck layout. + require.Equal(t, buildAck("127.0.0.1", 1234), (&Acknowledge{Host: "127.0.0.1", Port: 1234}).ToBytes()) + require.Equal(t, buildAck("host", 9), (&Acknowledge{Flag: AckFlag, Host: "host", Port: 9}).ToBytes()) + // Endpoint: matches buildClientEndpointEcho. + require.Equal(t, buildClientEndpointEcho("client.local", 55555), + (&Endpoint{Host: "client.local", Port: 55555}).ToBytes()) +} + // TestDecoderChunkedDelivery feeds a real rmiregistry capture through an // io.Pipe in 7-byte chunks with tiny sleeps between, mimicking TCP // segmentation where bytes arrive across multiple Read calls. A Decoder diff --git a/rmi/example_test.go b/rmi/example_test.go index 253bbe2..26a97c9 100644 --- a/rmi/example_test.go +++ b/rmi/example_test.go @@ -97,11 +97,17 @@ func ExampleNewDecoder() { } // ExampleDecoder_liveConnection sketches the pattern for reading JRMP -// traffic off a live net.Conn — the only supported way to parse from a -// long-lived TCP connection. The two things to notice are (a) the caller -// applies SetReadDeadline between frames to bound how long to wait for -// the next one, and (b) non-Registry Call headers and Return sentinel -// timeouts surface as normal errors the caller can inspect. +// traffic off a live net.Conn — the client→server reading direction, or +// any case where every byte is already available before parsing starts. +// The two things to notice are (a) the caller applies SetReadDeadline +// between frames to bound how long to wait for the next one, and (b) +// non-Registry Call headers and Return sentinel timeouts surface as +// normal errors the caller can inspect. +// +// For the server-side reading direction — where the server must write a +// ProtocolAck between reading the client's handshake and the client's +// endpoint echo — see ExampleDecoder_serverFlow. Opening() is NOT safe +// on a server-side live reader. // // This example does not connect to a real server; it's structured as a // compile-check so the idiom stays accurate as the API evolves. @@ -141,3 +147,60 @@ func ExampleDecoder_liveConnection() { // got frame 0x52 // peer closed } + +// ExampleDecoder_serverFlow demonstrates the server-side read ordering: +// ReadHandshake → write Acknowledge → ReadClientEndpoint → Next. A +// conforming Java client (sun.rmi.transport.tcp.TCPChannel) blocks after +// its 7-byte handshake until it has read the server's ProtocolAck, only +// then writing its endpoint echo. Opening() would deadlock on this +// ordering because it peeks past the handshake before the client has +// unblocked. +// +// Here the client's bytes are pre-buffered for determinism; on a real +// net.Conn the Ack write (conn.Write(ack.ToBytes())) is what actually +// lets the client's ReadClientEndpoint bytes reach us. +func ExampleDecoder_serverFlow() { + // Bytes the client sends across the two handshake halves plus one + // Ping. Constructed with the new ToBytes encoders so the example + // doubles as their usage demo. + var clientBytes bytes.Buffer + clientBytes.Write((&rmi.Handshake{Version: 2}).ToBytes()) + clientBytes.Write((&rmi.Endpoint{Host: "client.local", Port: 55555}).ToBytes()) + clientBytes.WriteByte(0x52) // MsgPing + + d := rmi.NewDecoder(&clientBytes) + + hs, err := d.ReadHandshake() + if err != nil { + fmt.Println("handshake:", err) + return + } + fmt.Println("handshake version:", hs.Version) + + // Server-side Ack. In real code: + // remote := conn.RemoteAddr().(*net.TCPAddr) + // _, _ = conn.Write((&rmi.Acknowledge{Host: remote.IP.String(), Port: int32(remote.Port)}).ToBytes()) + // Zero-valued Flag defaults to AckFlag, so Host/Port is all we need. + ack := (&rmi.Acknowledge{Host: "127.0.0.1", Port: 1234}).ToBytes() + fmt.Println("ack byte count:", len(ack)) + + ep, err := d.ReadClientEndpoint() + if err != nil { + fmt.Println("endpoint:", err) + return + } + fmt.Printf("client endpoint: %s:%d\n", ep.Host, ep.Port) + + msg, err := d.Next() + if err != nil { + fmt.Println("next:", err) + return + } + fmt.Printf("got frame 0x%02X\n", msg.Op()) + + // Output: + // handshake version: 2 + // ack byte count: 16 + // client endpoint: client.local:55555 + // got frame 0x52 +} diff --git a/rmi/handshake.go b/rmi/handshake.go index 1ca23d5..8a75638 100644 --- a/rmi/handshake.go +++ b/rmi/handshake.go @@ -125,3 +125,47 @@ func (a *Acknowledge) ToString() string { b.Printf("@Port - %d - %s", a.Port, commons.Hexify(a.Port)) return b.String() } + +// ToBytes encodes the handshake for writing to a JRMP peer: JRMI magic + +// uint16 version + 1-byte sub-protocol flag. Zero-valued Magic and Protocol +// fields default to JRMI_MAGIC and ProtocolStream so callers can construct +// a Handshake with only the Version field set. +func (h *Handshake) ToBytes() []byte { + magic := h.Magic + if len(magic) == 0 { + magic = JRMI_MAGIC + } + proto := h.Protocol + if proto == 0 { + proto = ProtocolStream + } + var buf bytes.Buffer + buf.Write(magic) + _ = binary.Write(&buf, binary.BigEndian, h.Version) + buf.WriteByte(proto) + return buf.Bytes() +} + +// ToBytes encodes the server→client acknowledgement as +// 0x4E + writeUTF(host) + int32(port). A zero-valued Flag defaults to +// AckFlag so a server can construct with only Host/Port. +func (a *Acknowledge) ToBytes() []byte { + flag := a.Flag + if flag == 0 { + flag = AckFlag + } + var buf bytes.Buffer + buf.WriteByte(flag) + writeModifiedUTF(&buf, a.Host) + _ = binary.Write(&buf, binary.BigEndian, a.Port) + return buf.Bytes() +} + +// writeModifiedUTF encodes s using the same uint16-length-prefixed modified +// UTF-8 layout readModifiedUTF consumes. Hostnames are overwhelmingly +// ASCII, so we emit the raw bytes unchanged — matching the lenient read +// path. +func writeModifiedUTF(buf *bytes.Buffer, s string) { + _ = binary.Write(buf, binary.BigEndian, uint16(len(s))) + buf.WriteString(s) +} diff --git a/rmi/parser.go b/rmi/parser.go index a9e7d8e..a6cde13 100644 --- a/rmi/parser.go +++ b/rmi/parser.go @@ -29,6 +29,31 @@ type Endpoint struct { Port int32 } +// ToBytes encodes the endpoint as writeUTF(host) + int32(port) — the exact +// layout a client writes immediately after receiving the server's +// Acknowledge, and the layout ReadClientEndpoint consumes. +func (e *Endpoint) ToBytes() []byte { + var buf bytes.Buffer + writeModifiedUTF(&buf, e.Host) + _ = binary.Write(&buf, binary.BigEndian, e.Port) + return buf.Bytes() +} + +// ToString formats the endpoint as a wireshark-dissector-style block +// headed by "@Endpoint", matching the style of Handshake/Acknowledge. +func (e *Endpoint) ToString() string { + b := commons.NewPrinter() + b.Printf("@Endpoint") + b.IncreaseIndent() + b.Printf("@Host") + b.IncreaseIndent() + b.Printf("@Length - %d - %s", len(e.Host), commons.Hexify(uint16(len(e.Host)))) + b.Printf("@Value - %s - %s", e.Host, commons.Hexify(e.Host)) + b.DecreaseIndent() + b.Printf("@Port - %d - %s", e.Port, commons.Hexify(e.Port)) + return b.String() +} + // Transmission is a parsed JRMP byte stream from either direction of a // Stream-protocol connection. The three opening fields capture the handshake // phase — any or all can be nil depending on which side of the conversation @@ -167,15 +192,7 @@ func (t *Transmission) ToString() string { b.Print(t.Acknowledge.ToString()) } if t.ClientEndpoint != nil { - b.Printf("@ClientEndpoint") - b.IncreaseIndent() - b.Printf("@Host") - b.IncreaseIndent() - b.Printf("@Length - %d - %s", len(t.ClientEndpoint.Host), commons.Hexify(uint16(len(t.ClientEndpoint.Host)))) - b.Printf("@Value - %s - %s", t.ClientEndpoint.Host, commons.Hexify(t.ClientEndpoint.Host)) - b.DecreaseIndent() - b.Printf("@Port - %d - %s", t.ClientEndpoint.Port, commons.Hexify(t.ClientEndpoint.Port)) - b.DecreaseIndent() + b.Print(t.ClientEndpoint.ToString()) } if len(t.Messages) > 0 { b.Printf("@Messages") diff --git a/rmi/rmi_test.go b/rmi/rmi_test.go index 5f3bfdd..93c4f66 100644 --- a/rmi/rmi_test.go +++ b/rmi/rmi_test.go @@ -153,6 +153,18 @@ func TestAcknowledge(t *testing.T) { require.Equal(t, int32(12345), tr.Acknowledge.Port) } +// TestEndpointToString covers the @Endpoint dump — every piece of state +// (host length, host value, port) must appear with both decimal and hex, +// matching the wireshark-dissector style of Handshake/Acknowledge. +func TestEndpointToString(t *testing.T) { + s := (&Endpoint{Host: "client.local", Port: 55555}).ToString() + require.Contains(t, s, "@Endpoint") + require.Contains(t, s, "@Host") + require.Contains(t, s, "@Length - 12 - 0x00 0c") + require.Contains(t, s, "@Value - client.local - 0x63 6c 69 65 6e 74 2e 6c 6f 63 61 6c") + require.Contains(t, s, "@Port - 55555 - 0x00 00 d9 03") +} + // ---------- ping / pingack / dgcack ---------- func TestPing(t *testing.T) {