Skip to content

Add CodSpeed continuous performance benchmarks - #389

Open
codspeed-hq[bot] wants to merge 1 commit into
mainfrom
codspeed-wizard-1787902124960
Open

Add CodSpeed continuous performance benchmarks#389
codspeed-hq[bot] wants to merge 1 commit into
mainfrom
codspeed-wizard-1787902124960

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Aug 28, 2026

Copy link
Copy Markdown

This adds continuous performance measurement to the repository with CodSpeed, so parser and renderer regressions show up on every pull request.

What was added

Benchmark suite (benchmarks/*.bench.ts, Vitest bench)

The suite focuses on comark's own hot paths rather than comparisons with other parsers (the existing mitata scripts in benchmarks/ are left untouched and remain the place for cross-library comparisons). 28 benchmarks in total:

  • parse.bench.ts — parsing small, medium and large documents, plus the effect of autoClose: false, registerDefaultPlugins: false, and creating a fresh parser per call.
  • render.bench.tsrenderHtmlFromDocument, renderAnsiFromDocument and renderMarkdown on pre-parsed documents, plus one combined parse + render pass.
  • streaming.bench.tsautoCloseMarkdown on partial and complete documents, streaming-mode parsing, and re-parsing a full sequence of growing chunks (the LLM streaming path).
  • plugins.bench.ts — per-plugin cost for toc, emoji, punctuation, footnotes, security and summary, individually and combined, against a baseline.

Fixtures live in benchmarks/fixtures.ts and are deterministic so results stay comparable across runs.

Tooling

  • @codspeed/vitest-plugin added as a root dev dependency (through the catalog) and wired into vitest.config.ts. The plugin is a no-op outside of a CodSpeed run, so local vitest bench and the existing test/ suite are unaffected.
  • benchmark.include scoped to benchmarks/**/*.bench.ts, matching how test.include is already scoped.
  • New pnpm bench script.
  • .github/workflows/codspeed.yml runs the benchmarks on main, on pull requests and on workflow_dispatch, using CodSpeedHQ/action@v5 in simulation mode with OIDC authentication. The Node/pnpm setup mirrors ci.yml.
  • CodSpeed badge added to the README.

Dependency notes

  • @codspeed/vitest-plugin has not widened its vite peer range to v8 yet. Without a workaround pnpm resolves the root workspace back to vite 7, which would silently downgrade vitest and nuxt at the root. A packageExtensions entry in pnpm-workspace.yaml widens that peer range, and vite is now an explicit root dev dependency pinned to the catalog version (^8.1.4) so the resolution stays on vite 8.
  • The lockfile diff is larger than the change suggests: @codspeed/core pulls in axios, which changes a number of hashed peer-resolution keys. No package versions are downgraded.

Verification

Run locally with the CodSpeed CLI in simulation mode (codspeed run --mode simulation -- pnpm bench): all 28 benchmarks were collected and reported successfully. pnpm lint and pnpm typecheck pass.

Next steps

  • Merge this PR so the baseline is recorded on main; the first pull requests afterwards will get performance reports.
  • Optionally trigger the workflow manually (workflow_dispatch) to backfill historical data.
  • Consider adding the Shiki highlighting plugin to the suite once you are comfortable with the extra CI time it costs.

@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
comark Ready Ready Preview Aug 28, 2026 7:48am
comark-json-render Ready Ready Preview Aug 28, 2026 7:48am
comark-nextjs Ready Ready Preview Aug 28, 2026 7:48am
comark-nuxt Ready Ready Preview Aug 28, 2026 7:48am
comark-svelte Ready Ready Preview Aug 28, 2026 7:48am
comark-sveltekit Ready Ready Preview Aug 28, 2026 7:48am
comark-twoslash Ready Ready Preview Aug 28, 2026 7:48am
comark-vue Ready Ready Preview Aug 28, 2026 7:48am

@pkg-pr-new

pkg-pr-new Bot commented Aug 28, 2026

Copy link
Copy Markdown

Open in StackBlitz

comark

npm i https://pkg.pr.new/comark@389

@comark/angular

npm i https://pkg.pr.new/@comark/angular@389

@comark/ansi

npm i https://pkg.pr.new/@comark/ansi@389

@comark/html

npm i https://pkg.pr.new/@comark/html@389

@comark/nuxt

npm i https://pkg.pr.new/@comark/nuxt@389

@comark/react

npm i https://pkg.pr.new/@comark/react@389

@comark/svelte

npm i https://pkg.pr.new/@comark/svelte@389

@comark/vue

npm i https://pkg.pr.new/@comark/vue@389

commit: 0a3bdec

@codspeed-hq

codspeed-hq Bot commented Aug 28, 2026

Copy link
Copy Markdown
Author

Congrats! CodSpeed is installed 🎉

🆕 28 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Open in CodSpeed

@codspeed-hq
codspeed-hq Bot marked this pull request as ready for review August 28, 2026 07:48
@codspeed-hq
codspeed-hq Bot requested a review from farnabaz as a code owner August 28, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant