Add the model comparison board to the benchmark hero chart - #1079
Merged
Conversation
The hero chart gains a second view. "Model comparison" plots structural validity against measured cost for 30 models on OpenUI; "Format comparison" keeps the existing 6-model, 3-format frontier. Both views share one 832px frame so switching tabs does not move the page, and both hang their axis labels left of a flush-left plot. Model board specifics: - Axes rescale to the current selection. The cost axis picks whole or half decades so ticks stay evenly spaced on the log scale, and the validity axis floors to the lowest visible score instead of always starting at 0. - Every point carries its model name. Labels are placed beside their dot and routed around other labels, the axis ticks and the dots themselves, with a leader line whenever a label has to travel to find space. - The Pareto frontier connects the non-dominated priced models: those with no other model that is both cheaper and more valid. - Models scoring below 70% structural validity start deselected, so the board opens on the range a reader is actually choosing between. All 30 stay in the table, the downloads and the filter, and the URL preserves selection. Marks are drawn in provider brand colours, including the Gemini gradient. Also publishes the benchmark as data: /benchmarks/data.json with a published JSON Schema, CSV, and agent.md, plus focused /benchmarks/language and /benchmarks/framework pages and three scripts that check those surfaces stay answerable and in sync with the page. The disclosure tables now borrow the marketing tables' frame one size down, so exact numbers read as part of the page rather than raw output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The drawn line is the frontier of the models currently selected; the table column is computed over all 30 so the exports stay stable. Naming the column "Frontier (all 30)" keeps those two facts from reading as a contradiction when a reader narrows the selection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both hero charts assumed a desktop page. The fixes: Axis placement is now chosen by measurement rather than assumed. The hanging-label layout spends the page gutter, so the component measures how much gutter there actually is and only hangs labels when there is room for them; otherwise the axis moves inside the canvas. This was not only a phone problem: between roughly 700px and 1300px the page padding shrinks well before the chart does, and labels hung 72px into a 40px gutter were landing off the side of the window. - The plot is shorter on a narrow screen. It was taller there than on desktop, which stretched twenty models into a vertical smear. - First and last cost ticks align inward instead of centring on the edge gridline, so they no longer hang off the canvas. This also fixes the $0.001 label that overflowed on desktop. - Only the frontier is labelled on a narrow screen. Twenty names cannot fit beside twenty dots at 390px; the rest keep their names in the tap card, the accessible label and the table. - Where a label genuinely has nowhere to go, it is dropped rather than printed over a neighbour. Its fallback box also measured the wrong side for right-anchored text. - The format view keeps its model marks on a narrow screen, and keys both the formats and the models, since neither is labelled in the plot there. - The chart header drops its ", by model" suffix on a narrow screen, where it collided with the "better" marker. The active tab already says which comparison this is. - The measured-width floor was wider than the container on a 320px screen, so the svg overflowed it. Verified with no label collisions, no clipping and no horizontal page scroll at every width from 320 to 1920, on both tabs. Tables: each one now carries its own scroller instead of the whole disclosure body scrolling as a sheet, so a wide table slides sideways while its heading and note stay put, and two tables scroll independently. On a phone the scroller bleeds to the card edges and pads itself back in, so the last column can reach the screen edge. The disclosure also gets a real accordion chevron, pointing down when collapsed and up when open, and a spacing scale that steps down from disclosure to section to heading to note. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t copy Reverts the brand-coloured logos on the format chart and its legend back to muted ink, and removes the brand palette and the Mark `brand` path with them rather than leaving an unused colour table behind. Provider dots are picked to read against a white page, so on a dark surface they sank into the background. Every dot carrying a provider hue is now lifted by one rule in dark mode, rather than a second hand-tuned palette that would drift from the first. Model marks on the format plot are placed by the same collision pass the point labels use. Fixed offsets were fine while the marks were spread out, but at the cheap end of the axis several models land within a few pixels of each other and their logos stacked into a pile, most visibly on a phone. A mark with nowhere near its own point is dropped rather than dragged somewhere it would read as belonging to a different one. Also: - The disclosure chevron moves to the trailing edge of its label. - The full-bleed table now escapes exactly the card's own inline padding, published as a custom property, instead of a hardcoded 18px. Every chart card on /benchmarks is laid out edge to edge with zero padding, so the fixed bleed was leaving an 18px strip on the right that the table could never reach. - speedHarness and dispute still pointed at thesysdev/openui. They now derive from BENCHMARK_REPOSITORY like every other link, so they cannot drift from it again. The agent-facing copy still described the old hand-kept filter: "five compact or local models are hidden" and a fixed "20-100%" scale. Neither has been true since the filter became a 70% threshold and the axes started rescaling to the selection, so agents were being told something false about the page. The prose, the field description, the agent Markdown and the two answer contracts now describe the rule instead of a count. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tables were cut off on both sides on a phone. My previous attempt fixed the wrong inset: it escaped the card's own padding, which is zero on this page, while the thing actually holding the table in was the section's 16px page margin further out. And the two marketing tables — including "Structural validity by model" — never opted into a bleed at all. Rather than compensate for a measured inset that can change, the scroller now breaks out to the viewport: 50% resolves against its own containing block, so calc(50% - 50vw) is exactly the distance to the window edge however much gutter, padding and margin sit in between, and the same amount is padded back so the first column still lines up with the heading above it. Nothing to hardcode and nothing to keep in sync. Two things were quietly defeating it: - .dataScroller carried max-width: 100%, and that rule sits after the mobile block in source order, so at equal specificity it beat the full-bleed width. Removed — a block element already fills its container. - The disclosure tables switched at 640px and the marketing tables at 767px, so between those widths half the tables bled and half did not. They now share one breakpoint. Verified on all ten tables from 320 to 1920, with and without classic scrollbars: every table reaches both screen edges at or below 767px, none do above it, no horizontal page overflow at any width, and the first column stays aligned with its heading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drops PUBLISHED_SCORER_TAG and PUBLISHED_REPO_ROOT. rawData, rawOutputs, harness, briefs and catalog now resolve under generative-ui-bench/tree/main, so every provenance link reads from one place and there is no second regime to keep straight. The methodology note loses its "preserved at <tag>" line with them. Verified across the benchmark surface: no tag links remain and all 22 links resolve under tree/main. Note for whoever picks up the score update: the lang-core 0.2.11 vs 0.2.15 caveat is still asserted in eight places, and it stops being true the moment the tables carry main's numbers. It needs to come out in the same commit as the new scores, or the page will warn about a divergence that no longer exists while linking to data that matches it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every row is now scored by one build of the shipped OpenUI parser, lang-core 0.2.16. The site and the benchmark repository are on the same regime, so the 0.2.11-vs-0.2.15 caveat is gone from all eight places it was asserted; it described a divergence that no longer exists. Board: 30 models to 31. ox-alpha joins as a stealth free preview at 91.8. Twenty-two scores move. Most are half a point either way, but three are not: Gemini 3.6 Flash 95.1 to 78.8, Gemma 4 31B 53.8 to 46.7 and DeepSeek V4 Pro 89.7 to 84.2. Cross-format slice: Google's seat moves from Gemini 3.6 Flash to Gemini 3.7 Flash. Gemini 3.6 Flash keeps its place on the model board. Kimi K3 and Muse Spark 1.2 each lose one complete run on their OpenUI leg. The other three seats are unchanged, and every A2UI and json-render leg is untouched, which is what pinned the mapping while checking this. The model count was written out in twelve places and had already gone stale once. It now derives from MODEL_BOARD_SIZE, and the three audit scripts assert the surfaces agree with each other rather than with a number typed into the script. Verified every published number against the source board: all 31 scores and costs, all six cross-format rows, and the three board means reconcile. The chart still has zero label collisions at 31 models, at 390, 768 and 1440, both in the default view and with everything selected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Back to 30 models. The stealth preview came in with the rescore; removing it takes the provider hue and the changelog line with it, and the model count derives from the board so nothing else needed touching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arison-board # Conflicts: # docs/lib/benchmark-data.ts
ox-alpha is off the board; back to 30 models. Its provider hue and changelog line go with it, and the model count derives from the board so nothing else needed touching. Merges main, which moved the benchmark to its own repo in #1080 and bumped lang-core to 0.2.16 in #1081. #1080 repointed the same links this branch had already repointed, which is what conflicted. Resolved on main's naming (REPO_ROOT / REPO_TREE), keeping the two extra keys the methodology page needs. Also restores 43 files this branch had no business touching: an earlier prettier run was pointed at whole directories rather than the files being changed, and reformatted a good part of app/ and components/ along the way. They are back to main's version, and the diff is 38 benchmark files again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Aditya-thesys
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Branched off current
main, so it carries the restructured nav, the Observability page and the integrations directory. 37 files, no conflicts.What this adds
The hero chart on
/benchmarksgains a second view:Both share one 832px frame, so switching tabs doesn't move the page.
Model board
Benchmark as data
/benchmarks/data.jsonwith a published JSON Schema, plus CSV andagent.md; focused/benchmarks/languageand/benchmarks/frameworkpages; and three scripts that check those surfaces stay answerable and in sync with the page.Tables
The disclosure tables now borrow the marketing tables' frame one size down (same
--openui-border-defaultand--openui-highlight-subtletokens, 12px radius and type instead of 16/15), with measured values in muted ink.Relationship to #1077
#1077 should be closed in favour of this. It was raised from
blog/benchmarks-page, whose merge base predatesmainreceivingbenchmarks/openui-bench/raw/**, so it reports 3,956 changed files and conflicts. Its page content is already onmainvia #1027. This branch carries the same intent as 37 files off currentmain.Verification
prettier --check,tsc --noEmit,eslintandnext build(325 static pages) all pass. Page renders with no console errors; label placement and axis spacing checked headlessly in both the 20-model default and the all-30 state.🤖 Generated with Claude Code