Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
zurfyx
force-pushed
the
zurfyx/y-replay-keep-chars
branch
from
October 30, 2026 02:44
7fa612e to
740f9d6
Compare
zurfyx
force-pushed
the
zurfyx/y-replay-keep-chars
branch
from
October 30, 2026 03:15
740f9d6 to
218a4bf
Compare
zurfyx
force-pushed
the
zurfyx/y-replay-keep-chars
branch
from
October 30, 2026 03:39
218a4bf to
4616877
Compare
zurfyx
force-pushed
the
zurfyx/y-replay-keep-chars
branch
from
October 30, 2026 04:06
4616877 to
b6ea70a
Compare
…t nodes made again with other text
## Description
When an update removes a line's text nodes and makes new ones in their place with some of the text changed (an agent's rewrite of a block from its JSON with one word changed, a code highlighter's new tokens after a paste), `@lexical/y` wrote every character of a removed node whose text was not all there again. A removed node's text was aligned with what replaced it whole or not at all (`TextReplay#stretch`, invariant 4). A collaborator's letter typed at the same time into an unchanged word then stayed in front of the rewritten text, out of its word, and a character a collaborator deleted there at the same time came back:
- `t63 t71z t13z t69z t15z`, made again as one node with `t81z` for `t69z`, while a collaborator typed `Q` after `t63`, gave `Qt63 t71z t13z t81z t15z`;
- the same line while a collaborator deleted the `7` of `t71z` gave the `7` back.
Now the text of nodes made in a stretch also keeps a removed node's text where it continues the common start or end in part. One more step runs after the existing alignment wherever a stretch has text of nodes made and text of removed nodes. It only fills gaps between what the existing passes keep, it counts whole words (a new `diffWords` in `Inline.ts`: a run of letters, digits and `_`, or any single other character, compared whole), and every run it adds to has to start and end at word boundaries of both whole lines: the new line as it is after the update (text made and deleted again left out) and the old line. A word or character is whatever goes with it: its marks, a joiner or stacking sign and what it joins, an emoji's parts (the platform's grapheme segmenter where there is one, and these rules either way).
- a word that gained or lost letters (`t1z` made again as `t91z`) is a new word, so a collaborator's concurrent replacement of the old word takes no letter from the new one;
- `one` made again right before a set `two` (`onetwo`), `one` of an old `onex` whose `x` was a removed node of its own, and `red` of an old `foored` made again inside a new `ored` are new words too;
- text pasted before a word that starts with the paste's first letter (`t68z ` before `t50z` in a highlighted code line) no longer writes that letter again;
- removed text that the made text holds twice is kept only where the whole words around it are there once: a piece of a word a highlighter split off (` t` of `t49z`) keeps its characters next to pasted tokens that hold ` t` too, but words an app made twice keep none.
A text set (`setTextContent`) that adds letters inside a word still writes them as typed: `t1z` set to `t91z` while a collaborator replaces `t1z` with `t94z` gives `9t94z`. That is unchanged here and kept as a known failure in the new tests.
## Changeset
`.changeset/lexical-y-keep-words-of-remade-text.md`: `[fix]` for `@lexical/y`, with a `Verify:` paragraph.
## Test plan
New cases in `packages/lexical-y/src/__tests__/unit/regressions/TextIdentity.test.ts` (S9-7): one node and three nodes made again with one word changed, while a peer types into or deletes from an unchanged word; a removed node that shares no text before one that holds all of it; a piece of a word made again after text that holds it too; a paste before a word that starts with the paste's first letter; a word made again as another word that holds its letters while a peer replaces the old word, also while the editor composes in another line; a made word joined to the set word after it; removed words made again twice; a word whose first or last letter was a removed node of its own; a removed node that was one word with the node before it; a made word whose separator was deleted again; an Arabic word joined by a ZWJ, an emoji sequence, a kana with a combining mark, Khmer and Myanmar conjuncts and Thai and Lao consonants with AM whose first character changed, and a Khmer conjunct whose stacked consonant changed; a Khmer conjunct across an unchanged node, and one whose middle is an unchanged node; removed words made again twice after a different word; without a grapheme segmenter, a stacking sign joins across the marks after it. The S7-39 cases now expect only the changed word to be written.
### Before
```
$ pnpm vitest run --project unit packages/lexical-y/src/__tests__/unit/regressions/TextIdentity.test.ts -t "S9-7|S7-39"
Tests 25 failed | 40 passed | 2 expected fail | 131 skipped (198)
AssertionError: expected { del: 't63z ', ins: 't80z ' } to deeply equal { del: 't63z', ins: 't80z' }
AssertionError: expected [ 'Qt63 t71z t13z t81z t15z', …(2) ] to deeply equal [ 't63Q t71z t13z t81z t15z', …(2) ]
AssertionError: expected [ 't63 t71z t13z t81z t15z', …(2) ] to deeply equal [ 't63 t1z t13z t81z t15z', …(2) ]
AssertionError: expected [ 't12z Qt13z t81z t15z', …(2) ] to deeply equal [ 't12z t1Q3z t81z t15z', …(2) ]
AssertionError: expected [ 'one two X', 'one two X', …(1) ] to deeply equal [ 'oe two X', 'oe two X', 'oe two X' ]
AssertionError: expected [ 't48zZQ t78z t49z t50z', …(2) ] to deeply equal [ 't48zQ t78z tZ49z t50z', …(2) ]
```
### After
```
$ pnpm vitest run --project unit packages/lexical-y/src/__tests__/unit/regressions/TextIdentity.test.ts -t "S9-7|S7-39"
Tests 65 passed | 2 expected fail | 131 skipped (198)
$ pnpm vitest run --project unit packages/lexical-y
Tests 7542 passed | 48 expected fail | 4 skipped (7594)
```
zurfyx
force-pushed
the
zurfyx/y-replay-keep-chars
branch
from
October 30, 2026 04:32
b6ea70a to
1781c5e
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Design proposal, draft: needs Gerard's decision. It lifts a deliberate rule of the
@lexical/ywriter (a removed text node's text is aligned with what replaced it whole or not at all). Please do not merge before that decision.Description
When an update removes a line's text nodes and makes new ones in their place with some of the text changed,
@lexical/ywrote every character of each removed node whose text was not all there again. Agents do this when they rewrite a block from its JSON with one word changed, and code highlighters do it when they retokenize after a paste. A collaborator's letter typed at the same time into an unchanged word then showed up in front of the rewritten text, out of its word. A character a collaborator deleted there at the same time came back, and a comment or format they added there was dropped.TextReplay#stretchaligns, between characters that stayed in place, the text that left (removed nodes, the old text asetTextContentreplaced) with the text that arrived (set text, text of nodes made). Invariant 4 drops a removed node's text from that alignment unless all of it shows up at the common start or end. This PR keeps that rule for text sets and adds one step for the text of nodes made:The new step runs only where a stretch has text of nodes made and text of removed nodes. It only fills gaps between what the existing passes keep and never changes what they keep. It counts whole words (a new
diffWords: a run of letters, digits and_, or any single other character, compared whole), and every run it adds to has to start and end at word boundaries of both whole lines: the new line as it is after the update (text made and deleted again left out) and the old line. A word or character is whatever goes with it: its marks, a joiner or stacking sign and what it joins, an emoji's parts (the platform's grapheme segmenter where there is one, and these rules either way):t1zmade again ast91z) is a new word, so a collaborator's concurrent replacement of the old word takes no letter from the new one;onemade again right before a settwo(onetwo),oneof an oldonexwhosexwas a removed node of its own, andredof an oldfooredmade again inside a neworedare new words too;t68zbeforet50zin a highlighted code line) no longer writes that letter again;tpiece of a highlightedt49zkeeps its characters next to pasted tokens that all holdt, but words an app made twice keep none;t69z→t81z; B typesQaftert63Qt63 t71z t13z t81z t15zt63Q t71z t13z t81z t15z7oft71z7comes backt12z+t13z t69z+t15zmade again as one node; B typesQint13zt12z Qt13z t81z t15zt12z t1Q3z t81z t15zxyz+one twomade again asone two X; B deletes thenone two Xoe two Xt48z+t+49z t50zmade again ast48zQ t78z t49z t50z; B typesZafter the oldtt48zZQ t78z t49z t50zt48zQ t78z tZ49z t50zt68z t69z t70zpasted beforet50z, tokens made again; B deletes thetoft50ztcomes backt1zmade again ast91z(also while the editor composes in another line); B replacest1zwitht94zone+two!made again asone+ settwo(onetwo); B deletes the oldoneonetwo tailonetwo tailredmade again asblue red red(a template that repeats the word); B deletes the oldredblue red red tailblue red red tailblue one+xmade again asgold one; B deletes the oldonexgold one tailgold one tailfoo+red,redmade again asblue red ored; B deletes the oldredfooblue red ored tailfooblue red ored tailone bluemade again asone!gold, then the!deleted; B deletes the oldoneonegold tailonegold tailblue بار(a ZWJ after the first letter) made again asgold دار; B deletes the old wordgold دار tailgold دار tailt1zset tot91z(setTextContent); B replacest1zwitht94z9t94z9t94z(unchanged, known failure)Why invariant 4 exists, and what the new step keeps from it
A version that aligns removed text in part everywhere, as set text is, broke 15 existing tests. Each shows a job of the rule:
MadeNodeEdits, "a node removed before a node set whole": a removedxyzin front of a node set fromone twotoone twoXemptied the common start, so the set node's text was written again and a peer's concurrent delete came back (oe twoXbecameone twoX).RewriteMerge, "agent rewrite of a line with a bold word" (6 client orders): the set text's.aligned with the end of the removedverify timing.and moved a peer's concurrent typing to the start of the rewritten line.TextIdentity, "a move, a move of all of its destination, then that node set again": a character came out twice.In this PR the new step never gives removed text to set text, so 1 to 3 run exactly as on main. Only the S7-39 cases change: the rewrite now writes
t63z→t80zinstead oft63z→t80z, with the same convergence and fresh-reader checks.Invariants
Evidence
@lexical/yunit suite: 7,542 passed, 48 expected fail (46 on main plus the 2 new known failures);pnpm run ci-checkpasses.Stored bytes, per test: Doc state hashes over the
@lexical/yunit suite with deterministic client ids, 7,049 tests that are stable across two runs of main (67,568 docs). 10 tests differ, and every one makes text nodes again in place of removed ones with other text: the 6 S7-39 cases, 3TextNewlinescases (a node made again withone\ntwoforone two, which now keepsoneandtwo), and a scratch S9-7 repro. In every differing test the new step is what changed the alignment.Golden fuzz corpus, 8,100 seeds over 7 lanes (moves, gc, block ids, long, comments as keys): state bytes, editor JSON, warnings, selection and node map are identical to main.
Fuzz.test.tsseeds 1 to 3,000 (gc 1,000) and 50,000 to 51,999: the same failures as main (seed 2277 and 8 blocks between 50,000 and 51,999, all failing on main).Collaboration soak, 1,500 randomized sessions (3 profiles, 18 rounds, 3 peers and agents; the same seeds on main):
No new kind of flag. Sessions diverge after the first update this PR writes differently, so per-session counts of the noisy code-annotation flags move in both directions. The one new undo flag is an undo of a move that drops a comment added later to the moved copy, in a session that diverged.
Randomized rewrite property check, 1,000 cases: a line of unique words in 1 to 4 text nodes is made again as 1 to 3 nodes with one or two word edits (replace, insert, delete, a letter added inside a word), while a peer types into, deletes from or replaces an old word. Oracles: the peer's letter stays in a word the rewrite kept, the peer's delete there stays, every word the rewrite kept or added is there whole.
The 7 remaining one-edit cases all start from what the existing passes keep: a removed node whose text is a piece of a word, matched whole at the common start or end (
. t3oft37ztaken for the start oft38z); the new step does not build on such a piece. With two edits, the words between them are written again: the diff keeps a common start and end only.With [lexical-code-core][lexical-code-prism][lexical-code-shiki][lexical-y] Bug Fix: Highlighting keeps the NodeState of every character in a code block #9558 applied (code highlighting keeps per-character NodeState), the session that lost a token after a paste no longer loses it, and one agent warning is gone; three self-check differences on a set-valued NodeState in code lines remain with and without this PR.
Review: an independent model reviewed it in five rounds, each re-running all earlier cases. Round 1 found two losses main does not have (composition in another line turned off word safety; a made word joined to the set word after it); round 2 two more (a removed neighbour deleted whole made a piece of a word look whole; a one-character context at an unchanged neighbour lent old letters to a different new word) and a gap in the copy rule; round 3 text made and deleted again in the same update faking a word boundary, and a joiner, mark or emoji part letting a changed character keep part of the old one; round 4 the same for Khmer and Myanmar conjuncts and Thai and Lao AM; round 5 a conjunct spanning an unchanged node, a stacking sign followed by a mark where there is no grapheme segmenter, and a slowdown of plain typing on long lines. All are fixed here, each loss with a regression test. Its 1,891 probe cases, re-run on this version: every reported loss passes; 45 strict-oracle rows fail only on this PR, all judged non-defects by the reviewer (a peer's letter typed where the rewrite deleted text now sits next to the unchanged word before it rather than at the line start; a word piece main keeps completed into its whole old word; old history kept around a character typed exactly into a made word), and 12 more call internals that changed; 134 fail on both and 152 only on main.
Risks
tpiece oft49z). An app that types a copy of a word fragment right before the original could see a peer's concurrent edit land on the copy.stretchand word helpers inInline.ts(about 650 added lines, a fifth of them comments). Nothing runs where no node is made in place of removed ones (the line texts and grapheme starts are built only for a stretch that needs them). Linear in the stretch: the run check scans twice at most, the copy check runs oneindexOfpair per kept run of the text refused as there twice, and the grapheme starts of the last four texts are cached (regressions/Scalingpasses; the round-3 and round-4 reviews measured the validation at about three boundary checks per word, flat from 400 to 6,400 words).Intl.Segmenter(all current browsers and Node), no word boundary falls inside one of its graphemes; the fixed rules (marks, joiners, stacking signs, Thai and Lao AM, emoji modifiers, tags, regional indicators) apply everywhere. On a platform without it, or with an older Unicode version, the fixed rules still hold.What it would close
Changeset
.changeset/lexical-y-keep-words-of-remade-text.md:[fix]for@lexical/y, with aVerify:paragraph.Test plan
New S9-7 cases in
packages/lexical-y/src/__tests__/unit/regressions/TextIdentity.test.ts; the S7-39 cases now expect only the changed word to be written.Before
After
Browser and e2e suites were not run locally; CI runs them.