Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
19360fd
docs: add spec for postgres support
MrLukeSmith Jul 10, 2026
06f27be
docs: address spec review feedback on postgres support design
MrLukeSmith Jul 10, 2026
dd239a1
docs: add implementation plan for postgres support
MrLukeSmith Jul 10, 2026
8f79adb
feat: add PostgresClient mirroring MysqlClient
MrLukeSmith Jul 10, 2026
413f22f
feat: add Engines registry for per-engine facts
MrLukeSmith Jul 10, 2026
6e9c04e
refactor: rename Profile's mysql_cpus/mysql_memory to db_cpus/db_memory
MrLukeSmith Jul 10, 2026
2c8df67
refactor: inject Digests SQL via constructor instead of hardcoding MySQL
MrLukeSmith Jul 10, 2026
09cdc2c
feat: add database field to Result
MrLukeSmith Jul 10, 2026
23c4bc3
feat: wire Runner to the selected database engine
MrLukeSmith Jul 10, 2026
3c04325
feat: add --database flag and rename --mysql-cpus/--mysql-memory to -…
MrLukeSmith Jul 10, 2026
304aabc
feat: show database engine in compare reports; rename mysql_cpu to db…
MrLukeSmith Jul 10, 2026
3cdc6bb
feat: add postgres service to docker-compose.yml
MrLukeSmith Jul 10, 2026
1d4862d
feat: select database adapter via BENCH_DATABASE
MrLukeSmith Jul 10, 2026
2b739b2
feat: add pg gem for Postgres adapter support
MrLukeSmith Jul 10, 2026
e4d515d
feat: make db_setup adapter-aware for Postgres
MrLukeSmith Jul 10, 2026
0ea5550
docs: document postgres support in README and mise tasks
MrLukeSmith Jul 10, 2026
d87a056
fix: cast total_exec_time to numeric for Postgres ROUND
MrLukeSmith Jul 10, 2026
56c2e68
fix: parse psql output as CSV instead of naive newline/tab split
MrLukeSmith Jul 10, 2026
0bbb1a0
test: guard the Postgres ROUND numeric-cast regression
MrLukeSmith Jul 10, 2026
92bdf30
fix: avoid macos postgres fork crashes
MrLukeSmith Jul 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ gem "railties", "~> 8.0.0"
gem "activerecord", "~> 8.0.0"
gem "activejob", "~> 8.0.0"
gem "trilogy"
gem "pg"

case (sq_source = ENV.fetch("SOLID_QUEUE_SOURCE", "upstream"))
when "upstream"
Expand Down
51 changes: 34 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@ reports.
## Prerequisites

- [mise](https://mise.jdx.dev) for the pinned Ruby version
- Docker for MySQL 8.0
- Docker for MySQL 8.0 and Postgres 16
- `libpq` dev headers on the host, to compile the `pg` gem's native extension
(`libpq-dev` on Debian/Ubuntu, `postgresql` via Homebrew on macOS)

Workers run on the host. Only MySQL is containerized.
Workers run on the host. Only the database (MySQL or Postgres) is containerized.

## Setup

Expand All @@ -20,25 +22,28 @@ mise run setup

If mise reports that `mise.toml` is not trusted, run `mise trust` once in the checkout, then retry.

`setup` bundles the default upstream source and pulls the MySQL image.
`setup` bundles the default upstream source and pulls both the MySQL and Postgres images.

## Run a Benchmark

```sh
# latest official gem
# latest official gem, against MySQL (default)
bin/bench run baseline --source upstream

# same run against Postgres, for a like-for-like comparison
bin/bench run baseline --source upstream --database postgres

# pinned RubyGems release
bin/bench run baseline --source upstream@1.2.4

# local fork; must be clean unless --allow-dirty is passed
bin/bench run sprawl --source path:~/Projects/solid_queue
```

Each run starts a fresh MySQL volume, loads schema from the selected gem source, starts the
solid_queue supervisor, runs the scenario, waits for drain, and writes
`results/<run-id>/result.json` plus logs. Results include the resolved gem version and, for
`path:` sources, the git SHA.
Each run starts a fresh database volume (MySQL or Postgres, per `--database`; default `mysql`),
loads schema from the selected gem source, starts Solid Queue workers, runs the scenario, waits
for drain, and writes `results/<run-id>/result.json` plus logs. Results include the resolved gem
version, the database engine used, the process launcher, and, for `path:` sources, the git SHA.

## Scenarios

Expand All @@ -59,17 +64,28 @@ bin/bench run sprawl --source path:~/Projects/solid_queue --set seeds=5 --set fa

## Profiles and Topology

Profiles live in `profiles/*.yml` and bundle MySQL resources with worker topology so comparisons
stay comparable. The default profile uses MySQL pinned to 1 CPU / 1 GB, 10 worker processes x 2
threads, and 1 dispatcher.
Profiles live in `profiles/*.yml` and bundle database resource limits with worker topology so
comparisons stay comparable, independent of which engine a run targets. The default profile
pins the database container to 1 CPU / 1 GB, with 10 worker processes x 2 threads and 1
dispatcher.

```sh
bin/bench run baseline --source upstream --profile default
bin/bench run baseline --source upstream --profile smoke
bin/bench run baseline --source upstream --workers 50 --mysql-cpus 2 --mysql-memory 2g
bin/bench run baseline --source upstream --workers 50 --db-cpus 2 --db-memory 2g
bin/bench run baseline --source upstream --database postgres --profile default
```

`bin/bench compare` refuses profile mismatches unless `--force` is passed.
`bin/bench compare` refuses profile mismatches unless `--force` is passed. Comparing a MySQL run
against a Postgres run under the same profile is not treated as a mismatch — the report simply
shows which engine each side used.

By default, the harness uses Solid Queue's fork supervisor. On macOS with Postgres, the harness
uses `direct` launching instead: each worker and dispatcher is started as a fresh Ruby process to
avoid the `pg`/libpq crashes seen when forked children connect after Rails has booted. The chosen
launcher is recorded in `result.json` under `profile.process_launcher`, so mixed launcher
comparisons are guarded like other topology changes. Override with
`BENCH_SOLID_QUEUE_LAUNCHER=supervisor` or `BENCH_SOLID_QUEUE_LAUNCHER=direct` when needed.

## Compare Runs

Expand All @@ -78,8 +94,9 @@ bin/bench list
bin/bench compare results/<A>/result.json results/<B>/result.json
```

Reports are written to `reports/<A>__vs__<B>/report.md` and `report.html`. They include metric
deltas, MySQL CPU and ready-depth overlay charts, and top SQL statements by total database time.
Reports are written to `reports/<A>__vs__<B>/report.md` and `report.html`. They include which
database engine each run used, metric deltas, DB CPU and ready-depth overlay charts, and top SQL
statements by total database time.

Use `--force` only when you intentionally want to compare different profiles:

Expand All @@ -99,9 +116,9 @@ Repeats keep every raw result and print median throughput across completed runs.

- Throughput, including a per-second series
- Latency percentiles for enqueue-to-start and enqueue-to-finish
- MySQL container CPU samples from `docker stats`
- Database container CPU samples from `docker stats`
- Queue depth samples for ready, scheduled, claimed, blocked, failed, and completed jobs
- Top statement digests from MySQL `performance_schema`
- Top statement digests from MySQL `performance_schema` or Postgres `pg_stat_statements`

## Adding a Scenario

Expand Down
22 changes: 20 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,30 @@ services:
MYSQL_ROOT_PASSWORD: bench
MYSQL_DATABASE: bench
command: --performance-schema=ON --max-connections=500
cpus: "${BENCH_MYSQL_CPUS:-1.0}"
mem_limit: "${BENCH_MYSQL_MEMORY:-1g}"
cpus: "${BENCH_DB_CPUS:-1.0}"
mem_limit: "${BENCH_DB_MEMORY:-1g}"
ports:
- "127.0.0.1:${BENCH_MYSQL_PORT:-13306}:3306"
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-pbench"]
interval: 2s
timeout: 2s
retries: 60

postgres:
image: postgres:16
container_name: sq-bench-postgres
environment:
POSTGRES_USER: bench
POSTGRES_PASSWORD: bench
POSTGRES_DB: bench
command: postgres -c shared_preload_libraries=pg_stat_statements -c max_connections=500
cpus: "${BENCH_DB_CPUS:-1.0}"
mem_limit: "${BENCH_DB_MEMORY:-1g}"
ports:
- "127.0.0.1:${BENCH_POSTGRES_PORT:-15432}:5432"
healthcheck:
test: ["CMD", "pg_isready", "-U", "bench"]
interval: 2s
timeout: 2s
retries: 60
Loading