Skip to content

rmi: split Decoder opening for server-side flows - #21

Merged
phith0n merged 2 commits into
masterfrom
decoder-server-flow
Apr 18, 2026
Merged

phith0n merged 2 commits into
masterfrom
decoder-server-flow

Conversation

@phith0n

@phith0n phith0n commented Apr 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Split Decoder's opening phase into three mutually exclusive primitives (ReadHandshake / ReadClientEndpoint / ReadAcknowledge) so an RMI Registry server can write its ProtocolAck between the two client reads. The existing Opening() stays as the one-shot entry point for fully-buffered captures and client-direction reads.
  • Add ToBytes() encoders on Handshake / Acknowledge / Endpoint so servers can write framing (&Acknowledge{Host, Port}.ToBytes()) without reimplementing the modified-UTF layout. Also add Endpoint.ToString() for symmetry.
  • Add ExampleDecoder_serverFlow demonstrating the new read ordering and update CLAUDE.md's Live-TCP section.

Why

Opening() reads Handshake + ClientEndpoint in one call. A conforming Java client (sun.rmi.transport.tcp.TCPChannel) sends its 7-byte handshake and then blocks waiting for the server's ProtocolAck before writing the endpoint echo. On a server-side live net.Conn, the peek inside Opening() waits for bytes the client will never send, so the caller deadlocks. There was no way to implement a working Registry server on top of Decoder without this split.

Design

  • decoderStage enum (stageInitial → stageAfterHandshake → stageReady) guards call ordering: the three Read* primitives and Opening() are mutually exclusive, and each returns an error if called out of order.
  • Next() auto-advances from either earlier stage to stageReady, so bare captures (no handshake) and callers who skip ReadClientEndpoint after ReadHandshake still work transparently.
  • ToBytes() encoders default zero-valued Magic/Protocol/Flag to JRMI_MAGIC / ProtocolStream / AckFlag so construction from scratch stays ergonomic; round-trip fidelity via ToBytes(FromBytes(x)) is unaffected because parsing rejects bad values before they reach the struct.

Test plan

  • TestDecoderServerFlowWithInterleavedAck — reproduces the real client timing using two io.Pipe pairs (client writes handshake, blocks on io.ReadFull of the Ack, then writes endpoint + Ping); asserts no deadlock and correct field values at each stage.
  • Boundary tests for the stage machine: TestDecoderOpeningAfterReadHandshakeErrors, TestDecoderReadHandshakeAfterOpeningErrors, TestDecoderReadClientEndpointRequiresHandshake, TestDecoderNextAutoConsumesClientEndpoint, TestDecoderReadAcknowledge.
  • TestOpeningEncodersRoundTrip — byte-exact match of the three ToBytes() outputs against the existing fixture builders.
  • TestEndpointToString — wireshark-dissector format with decimal + hex for length/value/port.
  • ExampleDecoder_serverFlow runnable example; all pre-existing examples and integration tests green (go test ./...).

🤖 Generated with Claude Code

phith0n and others added 2 commits April 19, 2026 03:12
Opening() reads handshake+ClientEndpoint in one call, which deadlocks on
a server-side live reader: a conforming Java client blocks after its
7-byte handshake waiting for the server's ProtocolAck before it sends
the endpoint echo. Servers need to inject a write between the two reads,
which the unified Opening() call forbade.

Split the opening phase into three mutually exclusive primitives:
ReadHandshake (server read), ReadClientEndpoint (server read after Ack
write), and ReadAcknowledge (client read). A stage enum enforces call
ordering. Opening() remains the one-shot entry point for fully-buffered
captures and client-direction reads; Next() still auto-advances from
any stage to stageReady so bare captures just work.

Also add ToBytes encoders on Handshake/Acknowledge/Endpoint (zero-value
Magic/Protocol/Flag default to JRMI_MAGIC/ProtocolStream/AckFlag so
server code can write &Acknowledge{Host, Port}.ToBytes() without
ceremony), and a ToString on Endpoint for symmetry with the other two.

Tests: TestDecoderServerFlowWithInterleavedAck uses two io.Pipes to
reproduce the real client timing and assert no deadlock. Boundary tests
cover the stage-machine rejections, Next() auto-advance, and encoder
round-trip. ExampleDecoder_serverFlow demonstrates the new sequence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shrink ~75 lines to ~30. Cut what the code already says (per-frame field
breakdowns, the maybeReadClientEndpoint hostname-length edge case, the
shared-cursor note, a redundant restatement of Decoder.Next behavior,
the full client-side Opening() example that's already in the Decoder
godoc). Keep what future work needs to avoid landmines: Registry-only
scope + dispatch gate + JDK recalibration hint, FromBytes vs Decoder
deadlock, both server-side Opening() and readReturn sentinel traps on
live readers, the no-end-marker invariant that forbids replacing the
sentinel with serz.FromBytes, and the wireshark-dump anti-pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@phith0n
phith0n merged commit cd1560d into master Apr 18, 2026
17 checks passed
@phith0n
phith0n deleted the decoder-server-flow branch April 18, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant