perf: avoid copying WebSocket deflate/inflate output - #1235
Open
pjfanning wants to merge 1 commit into
Open
Conversation
Motivation: PerMessageDeflate buffers the output of the Deflater/Inflater in a ByteArrayOutputStream and then calls ByteString.fromArrayUnsafe on toByteArray. ByteArrayOutputStream.toByteArray always copies the buffer, so every compressed or decompressed WebSocket frame pays a full copy of its payload. Modification: Add an internal ByteStringOutputStream that extends ByteArrayOutputStream and exposes toByteStringUnsafe, wrapping the internal buffer in a ByteString without copying when most of the buffer is used, and copying to a right-sized array otherwise so a large buffer is not retained by a small payload. Use it for the inflate and deflate paths in PerMessageDeflate, which allocate the stream per call and discard it immediately afterwards. Result: No copy of the payload per WebSocket frame when permessage-deflate is enabled, and no oversized buffer retained when a frame only fills a small part of it. Tests: - sbt "http-core/testOnly org.apache.pekko.http.impl.util.ByteStringOutputStreamSpec org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec" - 51 tests succeeded - scalafmt --list --mode diff-ref=upstream/main - no files reported - sbt headerCreateAll - headers added for the new files References: None - the ByteStringOutputStream implementation is adapted from Apache Pekko gRPC (Apache License 2.0), apache/pekko-grpc#862
Member
Author
|
@mkurz does this look like a viable micro-optimisation on the per message deflate? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
PerMessageDeflatebuffers the output of theDeflater/Inflaterin aByteArrayOutputStreamand then callsByteString.fromArrayUnsafe(output.toByteArray).ByteArrayOutputStream.toByteArrayalways copies the buffer, so every compressed or decompressed WebSocket frame pays a full copy of its payload.Modification
Add an internal
ByteStringOutputStream(@InternalApi,private[http]) that extendsByteArrayOutputStreamand exposestoByteStringUnsafe:ByteString.fromArrayUnsafe(buf, 0, count)with no copy;The stream must not be written to or reused after
toByteStringUnsafe, which is documented on the method. Both call sites (inflateanddeflateinPerMessageDeflate) allocate the stream per call and discard it immediately afterwards.HeaderCompressionalso uses aByteArrayOutputStream, but it reuses one instance across frames viareset(), so it is left unchanged here.Result
No copy of the payload per WebSocket frame when
permessage-deflateis enabled, and no oversized buffer retained when a frame only fills a small part of it.Note that
toByteStringUnsafecan return a non-compactByteString1, so a consumer that callstoArrayUnsafe()on the result will copy - that is the same single copy as before, just moved, and it is avoided entirely for consumers that do not need an array.Tests
sbt "http-core/testOnly org.apache.pekko.http.impl.util.ByteStringOutputStreamSpec org.apache.pekko.http.impl.engine.ws.WebSocketServerSpec"- 51 tests succeeded, 0 failedByteStringOutputStreamSpeccovers the empty, exactly-filled, grown-buffer and small-write-in-a-large-buffer cases, including that the copied result is unaffected by later writes to the streamscalafmt --list --mode diff-ref=upstream/main- no files reportedsbt headerCreateAll- used to add the headers for the new files@InternalApi private[http]code and adds no public APIReferences
None - the
ByteStringOutputStreamimplementation is adapted from Apache Pekko gRPC (Apache License 2.0), apache/pekko-grpc#862