Skip to content

SplitStringOnSpaces measures and splits by bytes, cutting multi-byte runes in half #14

Description

@rawsun007

SplitStringOnSpaces counts bytes rather than characters, so any multi-byte rune is both mis-measured and liable to be cut in half. The halves are invalid UTF-8, which is fatal downstream: it is what disconnects GoMud's web client on the quests command (GoMudEngine/GoMud#631).

Minimal reproduction against v1.3.2:

report("bar-25", strings.Repeat("░", 25), 25)   // 25 chars, width 25 -> should be 1 chunk
bar-25: input runes=25 bytes=75 maxLen=25 -> 3 chunk(s)
   chunk 0: validUTF8=false runes=9 bytes=25 "░░░░░░░░\xe2"
   chunk 1: validUTF8=false runes=11 bytes=25 "\x96\x91░░░░░░░\xe2\x96"
   chunk 2: validUTF8=false runes=9 bytes=25 "\x91░░░░░░░░"

The same string as ASCII returns one chunk, as expected:

ascii-25: input runes=25 bytes=25 maxLen=25 -> 1 chunk(s)
   chunk 0: validUTF8=true runes=25 bytes=25 "xxxxxxxxxxxxxxxxxxxxxxxxx"

Three places treat one byte as one visible column:

  • visibleLen (splitstring.go:501) - for i := 0; i < len(input); i++ { ... count++ }
  • splitPoints (splitstring.go:168) - recordVisible(ch byte) does consumed++ per byte
  • SplitStringOnSpaces (splitstring.go:297) - writeVisible(b byte) writes one byte, increments the count, and may call split() between two bytes of one rune

Tag detection itself is fine, since the tag syntax is ASCII. It is the visible-character accounting that is byte-based.

Two consequences, and the first is easy to miss because it looks like a cosmetic problem:

  1. Any non-ASCII text wraps early - a costs 3 columns, an é costs 2 - so a line of accented text wraps at roughly half the requested width.
  2. A hard split (no space available) lands mid-rune and emits invalid UTF-8.

For 2, here is the wire capture from GoMud. I logged in over /ws, gave a character two quest steps, and ran quests; the frame carrying the progress bar arrives broken at byte 428:

INVALID frame 2 len=1055 at byte 428: b';8m\x1b[49m\xe2\x96\x91...\xe2\x96\x91\xe2\x1b[0m       '

which renders as

    ██████████░░░░░░░░░░░░░░<?>
    <?><?> 40%

A browser closes the socket with 1007 "Invalid frame payload data" at that point. GoMud's ProgressBar uses and , and internal/templates/panel.go passes the row through SplitStringOnSpaces, so every quest with a progress bar hits it.

I have not opened a PR, because this repo has no contributing or AI policy file and I did not want to assume one. If you would like it, the direction I would take is to iterate runes (for _, r := range input) and count runes rather than bytes in all three functions, writing whole runes so a split can never land inside one - leaving the tag matcher byte-oriented as it is. Whether the column count should be runes or display width (wide CJK, combining marks) is a design call I would rather you make than guess at, so tell me which and I will send it with tests.

Written with AI assistance (Claude Code, Claude Opus 5). Every output above was produced locally against v1.3.2 and GoMud at main; nothing is inferred.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions