Skip to content

fix: support pre-release tags in release pipeline and keep PR benchmarks under 10 minutes - #697

Merged
dgafka merged 3 commits into
mainfrom
dgafka/release-pipeline-and-benchmark-fixes
Aug 22, 2026
Merged

fix: support pre-release tags in release pipeline and keep PR benchmarks under 10 minutes#697
dgafka merged 3 commits into
mainfrom
dgafka/release-pipeline-and-benchmark-fixes

Conversation

@dgafka

@dgafka dgafka commented Aug 22, 2026

Copy link
Copy Markdown
Member

Why is this change proposed?

Ecotone 2.0 will be tested through beta tags (e.g. 2.0.0-beta.1) released from a dedicated branch, and the release pipeline assumes every release is a stable tag cut from main:

  • The branch alias written on release (<tag>-dev) sorts below a pre-release constraint like ~2.0.0-beta.1, so split testing and per-package composer install would stop resolving the local monorepo packages and fail (verified with composer update --dry-run).
  • The monorepo split action pushes every release to the read-only repositories' main branch, so a beta from a 2.0 branch and the next stable 1.x release would overwrite each other's content there.

Separately, the PR benchmark workflow grew from ~5 to ~50-70 minutes after the high throughput publishing and outbox relay benchmarks were added, making every PR wait on it.

Description of Changes

Release pipeline (pre-release support):

  • bin/update-required-packages.php writes the branch alias as MAJOR.MINOR.x-dev (e.g. 2.0.x-dev), which satisfies both stable (~1.326.0) and pre-release (~2.0.0-beta.1) mutual constraints — verified for both flows end-to-end
  • release.yml forwards the tag's source branch to the split action, so stable releases keep landing on the read-only repos' main while pre-releases land on a matching branch
  • split action bumped v2.4.0 → v2.4.5 — older versions exit before reaching the create-missing-branch path

PR benchmarks (~50-70 min → under 10):

  • removed --retry-threshold=5: it re-runs any iteration deviating ≥5% from the variant mean in an unbounded loop, and broker-bound subjects have 15-30% natural variance
  • Iterations(2) for the two new benchmark classes; publishing scenarios keep 1000 messages per iteration, outbox relay works on 1000 messages (forwarding batch size is 100, so the message-by-message vs batched vs single-batch contrast is preserved: measured 4.8s vs 0.2s)
  • baseline results for a given main commit are cached and reused across PR runs, so the typical run benchmarks only the PR side
  • DbConnectBenchmark respects DATABASE_DSN (previously hardcoded localhost and failed in containerized environments)
  • contribution terms check accepts the lowercase x GitHub writes when the checkbox is ticked through the UI

Measured full suite: ~3 minutes per pass, was ~25-35. Note: the first run after this merges still benchmarks the old heavy suite as baseline once (~30 min), then the cache takes over.

Pull Request Contribution Terms

  • I have read and agree to the contribution terms outlined in CONTRIBUTING.

dgafka added 3 commits August 22, 2026 12:45
…anches

- update-required-packages.php writes branch alias as MAJOR.MINOR.x-dev so path
  repositories keep resolving local monorepo packages when mutual dependencies
  point to a pre-release version (e.g. ~2.0.0-beta.1)
- release.yml forwards the tag's source branch to the monorepo split action so
  stable releases from main and pre-releases from other branches do not
  overwrite each other's branch in the read-only repositories
- bump split action to v2.4.5 which is able to create missing branches in the
  read-only repositories (v2.4.0 exits before reaching branch creation)
The workflow grew from ~5 to ~50-70 minutes after adding the high throughput
publishing and outbox relay benchmarks. Three compounding causes:

- retry-threshold=5 rejects any iteration deviating 5% from the variant mean
  and re-runs it in an unbounded loop; broker-bound subjects have 15-30%
  natural variance, so nearly every variant retried many times
- outbox relay filled and drained 10 000 messages per iteration, with the
  message-by-message subject alone costing ~100s per iteration
- the whole suite runs twice per PR (main baseline + PR)

Changes:
- iterations reduced to 2 for both new benchmark classes; publishing scenarios
  keep 1000 messages per iteration, outbox relay works on 1000 messages
  (forwarding batch size is 100, so the batched vs single-batch vs
  message-by-message contrast is preserved)
- retry threshold removed from both phpbench invocations
- baseline results for a given main commit are cached and reused across PR
  runs, so the typical run benchmarks only the PR side

Measured suite time: ~3 minutes per pass (was ~25-35 minutes)
GitHub writes a lowercase x when the checkbox is ticked through the UI,
so UI-checked agreements failed the uppercase-only check
@dgafka
dgafka merged commit 7c13ef9 into main Aug 22, 2026
10 of 11 checks passed
@dgafka
dgafka deleted the dgafka/release-pipeline-and-benchmark-fixes branch August 22, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant