Skip to content

test: split the editor suite so only the frame-driven half is Linux-only - #193

Merged
matt-edmondson merged 4 commits into
mainfrom
claude/happy-rubin-w67sx1
Sep 14, 2026
Merged

matt-edmondson merged 4 commits into
mainfrom
claude/happy-rubin-w67sx1

Conversation

@matt-edmondson

@matt-edmondson matt-edmondson commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

The second half of the macOS redness on main. #191 fixes the Schema.Cpp.Test half; this fixes the Schema.Editor.Test half. Both are needed for a green macOS cell — #191's run confirms Schema.Cpp.Test now passes there, leaving only the editor suite.

What was wrong

The shared dotnet.yml runs UI suites on Linux only, and says so as one glob on the non-Linux runners:

ktsubuild test all --workspace "$GITHUB_WORKSPACE" --verbose --exclude "**/*.UITests/*"

This repository's UI suite was called Schema.Editor.Test and sat at the repository root, so the glob matched nothing — it was the one ktsu UI suite running on Windows and macOS. On macOS it also failed, two ImGui probe assertions that pass on Linux and Windows, and that is what kept main red.

What it takes to match, and how I got it wrong first

Both the suffix and a leading directory are required. ktsubuild's **/ does not match zero path segments, so a project at the repository root is never excluded however it is named.

I named this exact risk in the first version of this PR and then checked it wrongly — against DotNet.Glob and Microsoft.Extensions.FileSystemGlobbing, neither of which is what ktsubuild uses. Both match a root-level path, so the check "passed" and confirmed nothing. The rename-only head then ran the suite on macOS anyway:

--exclude matched no projects in Schema.sln. Patterns: **/*.UITests/*
Running 4 test project(s), skipping 0.

What settles it is the tool's own output on the same shared workflow, from ImGuiApp's macOS run:

Excluding tests/ImGuiMarkdownDemo.UITests/ImGuiMarkdownDemo.UITests.csproj from the test run.
Excluding tests/ImGui.Widgets.UITests/ImGui.Widgets.UITests.csproj from the test run.
All 11 test project(s) passed! (7 excluded by --exclude.)

Every excluded path carries a leading directory. Hence tests/Schema.Editor.UITests/.

The split, and why it is not just a move

Excluding the whole project would have applied the rule to thirteen tests that are not UI tests. Twenty of the twenty-two files drive real frames through EditorHarness or WidgetHarness; two do not, and say so themselves:

file says
RecentFilesTests "Pure logic - no ImGui context needed"
SchemaFileTests "Nothing here needs a frame - this is the one part of the editor that was always plain logic"

Those two are also the platform-sensitive ones. SchemaFileTests does real filesystem I/O through AbsoluteFilePath and a scratch directory; RecentFilesTests branches on OperatingSystem.IsWindows() to build the paths it asserts on — a test written to check Windows behaviour, which a blanket exclusion would have stopped running on Windows.

project tests runs on
Schema.Editor.Test (repo root) 13 every platform
tests/Schema.Editor.UITests 189 Linux only

Keeping Schema.Editor.Test at the root is now load-bearing rather than incidental: the root is precisely where the exclusion cannot reach.

Why this rather than editing the workflow

.github/workflows/dotnet.yml is byte-identical across all 48 ktsu .NET repositories. Editing the glob here would reintroduce exactly the drift that consolidating it removed. Moving the project applies the rule that already exists instead of writing a second one.

What this does not do

It turns the cell green by changing where the 189 frame-driven tests run, not by fixing the two macOS assertions. I could not reproduce those locally — Linux passes, and forcing the submenu-timing condition (30 settle frames instead of 3) did not break it either, so I have no verified diagnosis to offer.

There is a real latent fragility next to them, worth its own look: EditorHarness.Click waits on Probe.Matches(item).Count > 0, which asks whether the probe has ever recorded a name, while the harness's own IsOnScreen documents that as the wrong question —

"Not the same question as whether the probe has ever seen it: the probe remembers every name it has recorded, so a popup that has been dismissed still matches by name."

The macOS failure is exactly that shape (was not drawn in the most recent frame, so its recorded position is stale). Plausible cause; unverified, so not pushed here.

Verification

  • Full solution builds clean in Release, no warnings
  • Schema.Editor.Test — 13/13; tests/Schema.Editor.UITests — 189/189 (202 before the split, 202 after)
  • Schema.Cpp.Test — 103/103, Schema.Test — 476/476
  • The four InternalsVisibleTo declarations name both assemblies; solution entries and five documents updated
  • The exclusion itself is verified by CI, not locally — that is the whole lesson above

🤖 Generated with Claude Code

https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua

The shared `dotnet.yml` runs UI test suites on Linux only, and says so as
one glob on the non-Linux runners:

    ktsubuild test all --workspace "$GITHUB_WORKSPACE" --verbose --exclude "**/*.UITests/*"

This repository's UI suite was called `Schema.Editor.Test`, which that
glob does not match, so it was the one ktsu UI suite running on Windows
and macOS. On macOS it also failed - two ImGui probe assertions that pass
on Linux and Windows - which is what made `main` red there.

Renaming the project is what applies the existing rule rather than
editing the workflow: `.github/workflows/dotnet.yml` is byte-identical
across all 48 ktsu .NET repositories, and a repo-local edit would
reintroduce exactly the drift that consolidation removed.

The suite still runs in full on Linux on every pull request, which is
where the rule puts every other ktsu UI suite. It is unchanged otherwise:
202 tests, all passing, renamed namespace only.

Verified the glob rather than assuming it. Against both DotNet.Glob and
Microsoft.Extensions.FileSystemGlobbing, relative and runner-absolute:

    MATCH  Schema.Editor.UITests/Schema.Editor.UITests.csproj
    MATCH  /Users/runner/work/Schema/Schema/Schema.Editor.UITests/...
    no     Schema.Editor.Test/Schema.Editor.Test.csproj

The old name matching nothing is why it ran everywhere.

Also updated: the four `InternalsVisibleTo` declarations naming the
assembly, the solution entry, and the five documents that named the
project.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
The rename in the previous commit applied the UI-suite rule to the whole
editor suite, and thirteen of its tests are not UI tests.

Twenty of the twenty-two files drive real frames through EditorHarness or
WidgetHarness. Two do not, and say so themselves:

  RecentFilesTests - "Pure logic - no ImGui context needed"
  SchemaFileTests  - "Nothing here needs a frame - this is the one part
                      of the editor that was always plain logic"

Those two are also the platform-sensitive ones, which is what makes the
over-application worth undoing rather than accepting. SchemaFileTests does
real filesystem I/O through AbsoluteFilePath and a scratch directory, and
RecentFilesTests branches on OperatingSystem.IsWindows() to build the paths
it asserts on - a test written to check Windows behaviour that the rename
would have stopped running on Windows.

So they move to Schema.Editor.Test, which no glob excludes and which
therefore runs everywhere, and Schema.Editor.UITests keeps the 189 that
need a frame.

  Schema.Editor.Test     13 tests, every platform
  Schema.Editor.UITests 189 tests, Linux only

Both pass. The four InternalsVisibleTo declarations now name both
assemblies, and the documents describe both projects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua
@matt-edmondson matt-edmondson changed the title ci: name the editor UI suite Schema.Editor.UITests so the Linux-only rule applies test: split the editor suite so only the frame-driven half is Linux-only Sep 14, 2026
The rename alone did not work. macOS run 34827691047 reported:

    --exclude matched no projects in Schema.sln. Patterns: **/*.UITests/*
    Running 4 test project(s), skipping 0.

and ran the 189 frame-driven tests anyway.

ktsubuild's `**/` does not match zero path segments, so a project at the
repository root is never excluded however it is named. Every path the
shared workflow successfully excludes elsewhere has a leading directory -
from ImGuiApp's macOS run on the same workflow:

    Excluding tests/ImGuiMarkdownDemo.UITests/ImGuiMarkdownDemo.UITests.csproj
    Excluding tests/ImGui.Widgets.UITests/ImGui.Widgets.UITests.csproj
    All 11 test project(s) passed! (7 excluded by --exclude.)

So the suffix was necessary and not sufficient: the path needs both.

This was the one risk I named and then checked wrongly. I tested the
pattern against DotNet.Glob and Microsoft.Extensions.FileSystemGlobbing,
both of which match a root-level path, and neither is what ktsubuild uses.
The evidence above is from the tool itself.

Schema.Editor.Test stays at the root, where nothing excludes it, which is
what keeps its 13 frameless tests running on every platform.

Verified: solution builds clean, 189/189 from the new location.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua

Copy link
Copy Markdown
Contributor Author

Correcting myself: the rename alone did not work, and the glob check I claimed as verification was worthless.

The macOS run on the previous head reported:

--exclude matched no projects in Schema.sln. Patterns: **/*.UITests/*
Running 4 test project(s), skipping 0.

and ran all 189 frame-driven tests anyway.

ktsubuild's **/ does not match zero path segments. A project at the repository root is never excluded however it is named — the .UITests suffix was necessary and not sufficient.

I did name this exact risk in the PR body ("**/ matching zero leading segments is where glob implementations differ") and then checked it against DotNet.Glob and Microsoft.Extensions.FileSystemGlobbing. Neither is what ktsubuild uses, so the check confirmed nothing about the thing it was supposed to confirm. Testing two arbitrary matchers is not testing the matcher.

The evidence that actually settles it comes from the tool, on the same shared workflow — ImGuiApp's macOS run:

Excluding tests/ImGuiMarkdownDemo.UITests/ImGuiMarkdownDemo.UITests.csproj from the test run.
Excluding tests/ImGui.Widgets.UITests/ImGui.Widgets.UITests.csproj from the test run.
All 11 test project(s) passed! (7 excluded by --exclude.)

Every successfully excluded path carries a leading directory. So the suite moves to tests/Schema.Editor.UITests/, matching the layout that demonstrably works.

Schema.Editor.Test stays at the repository root, where nothing excludes it — which is now doing real work rather than being incidental, since it is what keeps those 13 frameless tests running on every platform.

Pushed as 4c05fba. Solution builds clean; 189/189 from the new location. The PR body is updated.

The remaining macOS failures on this head should be Schema.Cpp.Test only, which is #191's <compare> fix and not this PR's to carry.


Generated by Claude Code

The exclusion works now - macOS run 34828747684 reported:

    Excluding tests/Schema.Editor.UITests/Schema.Editor.UITests.csproj from the test run.

Schema.Editor.Test passed there, so the thirteen frameless tests still run
on macOS, and Schema.Test passed on all three frameworks. The only failures
left on that cell are Schema.Cpp.Test on net9.0 and net10.0, which is #191's
fix and not this branch's work.

Rather than wait for #191 to merge, its change is ported here so this PR can
go green on its own. It is copied verbatim - same placement, same comment -
so the two are byte-identical and merge without a conflict: the post-image
blob is 6d74192 on both sides. Once #191 lands this commit is a no-op.

Schema.Cpp.Test: 101/101.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxqZVzN8CJcDxTb5gtWua

Copy link
Copy Markdown
Contributor Author

The exclusion works. macOS on 4c05fba:

Excluding tests/Schema.Editor.UITests/Schema.Editor.UITests.csproj from the test run.

and the split holds up on that cell:

assembly macOS
ktsu.Schema.Editor.Test (net10.0) passed — the 13 frameless tests still run
ktsu.Schema.Test (net8.0 / net9.0 / net10.0) passed
ktsu.Schema.Editor.UITests not run — excluded, as intended
ktsu.Schema.Cpp.Test (net9.0 / net10.0) failed, 1 error each

1643 tests, 2 failures, both Schema.Cpp.Test — #191's <compare> fix, not this branch's work.

Rather than leave this PR red waiting on another one, I have ported #191's change here (9c95f52) so this can go green on its own. It is copied verbatim — same placement, same comment — so the two sides are byte-identical and merge without a conflict. The post-image blob is 6d74192 on both:

diff --git a/Schema.Cpp/CppFileBuilder.cs b/Schema.Cpp/CppFileBuilder.cs
index f333f0d..6d74192 100644

Once #191 lands, this commit is a no-op. #191 remains the right place for that fix to be reviewed; this is only so the two PRs stop blocking each other.

Schema.Cpp.Test 101/101 locally.


Generated by Claude Code

@sonarqubecloud

Copy link
Copy Markdown

@matt-edmondson
matt-edmondson merged commit 2c68fee into main Sep 14, 2026
13 checks passed
@matt-edmondson
matt-edmondson deleted the claude/happy-rubin-w67sx1 branch September 14, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants