Skip to content

Shared WebSocket RPC sessions in workerd seem to fail under large payload volume #158

Closed

Description

@edmundhung

While working on a new feature that uses remote bindings to populate OpenNext cache entries into R2, we noticed that a shared WebSocket RPC session could fall over in workerd under large payload volume. I'm not confident whether the right fix belongs in capnweb or in workerd, but it seems to point more at the shared WebSocket session path than anything higher-level.

What seems to be happening is:

  • a long-lived shared WebSocket RPC session works fine in Node
  • the same pattern does not hold up in workerd once the payloads get large enough
  • if we avoid reusing one long-lived session and instead use fresh sessions in smaller batches, the problem goes away

I put together a repro in #157. In the failing test, workerd eventually OOMs and the client side ends with fetch failed / ECONNREFUSED:

workerd/jsg/setup.c++:38: fatal: V8 fatal error; location = Ineffective mark-compacts near heap limit; message = : allocation failed: JavaScript heap out of memory

Activity

  1. kentonv commented on Mar 20, 2026

    @kentonv
    Member

    Have you tried asking an AI to analyze the issue? What did it say?

  2. edmundhung commented on Mar 20, 2026

    @edmundhung
    MemberAuthor

    It suggests the issue to be most likely in the shared WebSocket session path, specifically around backpressure / in-flight data handling. As it doesn’t see any obvious transport-level flow control in the WebSocketTransport:

    capnweb/src/websocket.ts

    Lines 41 to 57 in cfa1b95

    class WebSocketTransport implements RpcTransport {
    constructor (webSocket: WebSocket) {
    this.#webSocket = webSocket;
    if (webSocket.readyState === WebSocket.CONNECTING) {
    this.#sendQueue = [];
    webSocket.addEventListener("open", event => {
    try {
    for (let message of this.#sendQueue!) {
    webSocket.send(message);
    }
    } catch (err) {
    this.#receivedError(err);
    }
    this.#sendQueue = undefined;
    });
    }

    I did notice there's already flow-control / backpressure logic around streams, so I wonder if the WebSocket transport might need something similar?

  3. kentonv commented on Apr 17, 2026

    @kentonv
    Member

    Opus 4.7 analyzed this and concluded it's actually due to the Promise.race() memory leak fixed by #154. Apparently, the leak actually leaks not just promise nodes but the entire received messages. OMG.

    O.

    M.

    G.

    https://x.com/KentonVarda/status/2044935325554823193

  4. kentonv commented on Apr 17, 2026

    @kentonv
    Member

    The reason it works on Node and not workerd is only because Node doesn't use pointer compression and as such is able to reach larger heap sizes. If you make the test run longer it should eventually crash Node too.

  5. dimitropoulos commented on May 22, 2026

    @dimitropoulos
    Collaborator

    my reading of the above is that this can now be closed now that #154 (and #168) are closed. if that's not right please let me know and we can reopen and investigate :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions