Skip to content

perf: find SSE line terminators with ByteString.indexOf - #1226

Open
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:perf/sse-line-parser-indexof
Open

perf: find SSE line terminators with ByteString.indexOf#1226
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:perf/sse-line-parser-indexof

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Motivation

LineParser scanned its buffer one byte at a time:

bs(at) match {
  case CR_BYTE if at < bs.length - 1 && bs(at + 1) == LF_BYTE => ...

The buffer is built as buffer ++ grab(in), so it is a multi-fragment ByteString whenever a line spans more than one chunk. ByteString$ByteStrings.apply has no offset index or position cache - it walks the fragment vector from the first fragment on every access - so scanning an SSE line that arrives in N chunks costs O(bytes × N).

ByteString.indexOf(byte, from) is the opposite: it walks fragments once and, inside a fragment, uses SWARUtil to test 8 bytes per step.

Modification

Jump to the next CR or LF with indexOf instead of testing every byte. The three identical line-emitting blocks are factored into a local lineAt helper.

Line termination semantics are unchanged - CR, LF, CRLF, a lone CR followed by a non-LF byte, and a CRLF split across two chunks (the lastCharWasCr flag) all behave exactly as before. The oversized-line handling (maxLineSize and every OversizedSseStrategy) is untouched, just moved into the helper.

Result

LineParserBenchmark, JDK 21, -f 1 -wi 3 -i 3:

lineSize chunkSize before after
1 KB 512 0.265 ms/op 0.189 ms/op within noise
1 KB 8192 0.295 ms/op 0.183 ms/op within noise
128 KB 512 71.210 ms/op 1.013 ms/op ~70x
128 KB 8192 3.269 ms/op 0.342 ms/op ~10x
1 MB 512 9821.281 ms/op 54.361 ms/op ~180x

Small lines are unaffected; the gain scales with how many chunks a line is split across, which is exactly the shape of a real SSE stream over a network.

Note this is independent of the pending pekko-core change that gives ByteStrings.apply a fragment hint. That hint helps monotonic forward scans, but indexOf still avoids one virtual call per byte and tests 8 bytes at a time, and this change also benefits the 1.x line, which is pinned to pekko 1.1.5.

Tests

  • sbt "http-tests / Test / testOnly org.apache.pekko.http.scaladsl.unmarshalling.sse.*" - 48 passed (46 existing, plus 2 new in LineParserSpec: a CRLF split across two chunks, and lines parsed out of an explicitly multi-fragment ByteString, asserted non-compact so the test really covers the rope path).
  • sbt "http-bench-jmh/Jmh/run …" - numbers above, measured before and after on the same machine.
  • scalafmt --mode diff-ref=upstream/main - clean.

References

None - found while auditing ByteString usage across the code base

Motivation:
LineParser scanned its buffer one byte at a time with `bs(at)`. The buffer is
built as `buffer ++ grab(in)`, so it is a multi-fragment ByteString whenever a
line spans several chunks, and `ByteStrings.apply` walks the fragment list from
the first fragment on every access. Parsing an SSE line that arrives in many
chunks therefore costs O(bytes * fragments).

Modification:
Locate the next CR or LF with ByteString.indexOf, which is fragment aware and
scans several bytes at a time, instead of testing every byte. The three
identical line-emitting blocks are factored into a local helper. Line
termination semantics (CR, LF, CRLF, and a CRLF split across chunks) are
unchanged.

Result:
Large SSE lines delivered in small chunks parse dramatically faster; small
lines are unaffected.

  lineSize  chunkSize      before        after
      1 KB        512    0.265 ms     0.189 ms
      1 KB       8192    0.295 ms     0.183 ms
    128 KB        512   71.210 ms     1.013 ms
    128 KB       8192    3.269 ms     0.342 ms
      1 MB        512  9821.281 ms   54.361 ms

Tests:
- sbt "http-tests / Test / testOnly org.apache.pekko.http.scaladsl.unmarshalling.sse.*" - 48 passed (46 existing plus 2 new)
- sbt "http-bench-jmh/Jmh/run -f 1 -wi 3 -i 3 -p lineSize=... -p chunkSize=... .*LineParserBenchmark.*" - numbers above, JDK 21
- scalafmt --mode diff-ref=upstream/main - clean

References:
None - found while auditing ByteString usage across the code base
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant