fix(deploy): make readiness waits configurable (default 90s) - #24
Merged
Merged
Conversation
The staging boot, rollback-restore and production health waits each polled /healthz for only 20s (40 x 0.5s). A cold uvicorn boot on a loaded FORGE exceeds that: it aborted the d8ad9fd security release at staging, and the same budget can report a healthy rollback as failed or roll back a good production release. - CERTFORGE_READY_TIMEOUT_S (default 90) drives all three waits; a healthy service still returns on the first successful probe. - On a staging readiness failure, print the /healthz probe result and keep service.log under $STATE_ROOT/deploy-logs/ before the EXIT trap removes the scratch directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0152mN1wMwj9vE2YmV2xZF4F
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
deploy/deploy_forge.shwaited only 20s (40 × 0.5s) for/healthzin three places:ROLLBACK FAILEDOn 2026-09-24 the d8ad9fd security release aborted at staging twice for exactly this reason. The same candidate passed staging and prod live-smoke once the window was widened.
Change
CERTFORGE_READY_TIMEOUT_S(default 90) drives all three waits. Success still returns on the first healthy probe, so healthy deploys are no slower./healthzprobe result and keepsservice.logunder$STATE_ROOT/deploy-logs/staging-<release>.log(the EXIT trap previously deleted it, so these failures left no evidence).test_deploy_gate.pypins one of them).Verification
bash -n deploy/deploy_forge.sh: OKpytest tests/test_deploy_gate.py: 10 passed🤖 Generated with Claude Code
https://claude.ai/code/session_0152mN1wMwj9vE2YmV2xZF4F