Skip to content

LineOutputHandler is quadratic in line length: a 16 MB line with no newline takes ~221 s (OutputHandler: 39 ms) and stalls the child #92

Description

@matt-edmondson

What's wrong

RunCommand/LineOutputHandler.cs:65-96, ProcessDataByLine:

buffer += data;          // copies the whole pending line on every chunk
int i = 0; ...
while (i < buffer.Length) // rescans from index 0 every chunk
...
buffer = buffer[lineStart..];

While a line is still incomplete, each ~4 KB read copies the entire pending buffer and rescans it from the start. The total work is O(L²/chunk) in the line length L.

Reproduction

ExecuteAsync("sh", ["-c", "head -c N /dev/zero | tr '\\0' a; echo"], handler):

Line size OutputHandler LineOutputHandler
2 MB 70 ms 3.7 s
4 MB 10 ms 14.0 s
8 MB 27 ms 56.0 s
16 MB 39 ms 221 s

A direct unit test that feeds 1024 chunks of 4096 'a' characters and then "\n" into HandleStandardOutputData took 15.35 s.

Why it matters

Single-line output is common: minified JSON, jq -c, base64 blobs, and progress output that uses no newlines. The handler runs on the read loop, so while it grinds the pipe isn't drained. The child then blocks on a full pipe and the whole command looks hung. The old Contains/Split implementation was also quadratic, so this predates the recent CRLF fix (da03a26).

Suggested fix

Keep one StringBuilder per stream for the partial line, plus a "pending CR" flag carried between calls. Scan only the newly arrived data, append up to each line break, then emit builder.ToString() and clear. This preserves the CRLF-split-across-reads behaviour and makes the work linear.

Acceptance: feeding 4 MB in 4 KB chunks with one trailing newline completes in well under a second, and the existing CRLF and line-splitting tests still pass. The fix can land together with #88, which touches the same buffer.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingreadyFully specified; implement as written

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions