Skip to content

perf: avoid copying WebSocket deflate/inflate output - #1235

Open
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:perf/bytestring-output-stream
Open

perf: avoid copying WebSocket deflate/inflate output#1235
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:perf/bytestring-output-stream

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Motivation

PerMessageDeflate buffers the output of the Deflater/Inflater in a ByteArrayOutputStream and then calls ByteString.fromArrayUnsafe(output.toByteArray). ByteArrayOutputStream.toByteArray always copies the buffer, so every compressed or decompressed WebSocket frame pays a full copy of its payload.

Modification

Add an internal ByteStringOutputStream (@InternalApi, private[http]) that extends ByteArrayOutputStream and exposes toByteStringUnsafe:

  • when most of the buffer is used, the buffer is wrapped via ByteString.fromArrayUnsafe(buf, 0, count) with no copy;
  • otherwise the bytes are copied to a right-sized array so a large buffer is not retained by a small payload.

The stream must not be written to or reused after toByteStringUnsafe, which is documented on the method. Both call sites (inflate and deflate in PerMessageDeflate) allocate the stream per call and discard it immediately afterwards.

HeaderCompression also uses a ByteArrayOutputStream, but it reuses one instance across frames via reset(), so it is left unchanged here.

Result

No copy of the payload per WebSocket frame when permessage-deflate is enabled, and no oversized buffer retained when a frame only fills a small part of it.

Note that toByteStringUnsafe can return a non-compact ByteString1, so a consumer that calls toArrayUnsafe() on the result will copy - that is the same single copy as before, just moved, and it is avoided entirely for consumers that do not need an array.

Tests

  • sbt "http-core/testOnly org.apache.pekko.http.impl.util.ByteStringOutputStreamSpec org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec" - 51 tests succeeded, 0 failed
  • New ByteStringOutputStreamSpec covers the empty, exactly-filled, grown-buffer and small-write-in-a-large-buffer cases, including that the copied result is unaffected by later writes to the stream
  • scalafmt --list --mode diff-ref=upstream/main - no files reported
  • sbt headerCreateAll - used to add the headers for the new files
  • MiMa not run - the change is confined to @InternalApi private[http] code and adds no public API

References

None - the ByteStringOutputStream implementation is adapted from Apache Pekko gRPC (Apache License 2.0), apache/pekko-grpc#862

Motivation:
PerMessageDeflate buffers the output of the Deflater/Inflater in a
ByteArrayOutputStream and then calls ByteString.fromArrayUnsafe on
toByteArray. ByteArrayOutputStream.toByteArray always copies the buffer,
so every compressed or decompressed WebSocket frame pays a full copy of
its payload.

Modification:
Add an internal ByteStringOutputStream that extends
ByteArrayOutputStream and exposes toByteStringUnsafe, wrapping the
internal buffer in a ByteString without copying when most of the buffer
is used, and copying to a right-sized array otherwise so a large buffer
is not retained by a small payload. Use it for the inflate and deflate
paths in PerMessageDeflate, which allocate the stream per call and
discard it immediately afterwards.

Result:
No copy of the payload per WebSocket frame when permessage-deflate is
enabled, and no oversized buffer retained when a frame only fills a
small part of it.

Tests:
- sbt "http-core/testOnly org.apache.pekko.http.impl.util.ByteStringOutputStreamSpec org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec" - 51 tests succeeded
- scalafmt --list --mode diff-ref=upstream/main - no files reported
- sbt headerCreateAll - headers added for the new files

References:
None - the ByteStringOutputStream implementation is adapted from Apache
Pekko gRPC (Apache License 2.0), apache/pekko-grpc#862
@pjfanning

Copy link
Copy Markdown
Member Author

@mkurz does this look like a viable micro-optimisation on the per message deflate?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant