CLOUDP-441133: fix service account teardown failing with 401 - #36
Merged
Merged
Conversation
The CI job "Setup a Service Account and create a project" failed 100% of the time at its final teardown step, blocking every unrelated PR. generate-service-account.sh creates the service account against $MONGODB_ATLAS_OPS_MANAGER_URL (cloud-dev), but terminate-service-account.sh hardcoded https://cloud.mongodb.com/. The account was created in cloud-dev and the delete attempted against prod, where the configured API keys have no rights, so every run left an orphaned ORG_OWNER service account behind. Also fixed in the same area: - Both scripts now use curl --fail-with-body, so a non-2xx response is a non-zero exit while the response body is still available for diagnostics. terminate-service-account.sh previously parsed the status out of the body with `jq -r '.error'`; on a successful 204 the body is empty, so the check became `[ "" -ge 300 ]`, which errored on stderr and then fell through to the success branch by accident. A failure returning a non-JSON body (e.g. an HTML 503) was reported as success. - Masked the service account secret with ::add-mask:: so it is no longer echoed unredacted into this public repo's run logs. Unlike the API keys, this value is minted at runtime and passed between steps as an output, so GitHub does not mask it automatically. - Added the missing MONGODB_ATLAS_OPS_MANAGER_URL guard to the terminate script, and corrected the generate script's guard, which reported MONGODB_ATLAS_ORG_ID as the missing variable.
fmenezes
force-pushed
the
CLOUDP-441133
branch
from
September 1, 2026 16:09
83cee83 to
e3bd386
Compare
fmenezes
enabled auto-merge (squash)
September 1, 2026 16:12
cveticm
reviewed
Sep 1, 2026
| error_code=$(echo "$output" | jq -r '.error') | ||
|
|
||
| if [ "$error_code" -ge 300 ]; then | ||
| -X DELETE "${MONGODB_ATLAS_OPS_MANAGER_URL}api/atlas/v2/orgs/${MONGODB_ATLAS_ORG_ID}/serviceAccounts/${CLIENT_ID}" |
Collaborator
There was a problem hiding this comment.
huh, I'm suprised that this only started failing lately cuz this bug was implemented (by me oops) nearly a year ago.
Any hypothesis on why this cropped up now?
I thought it might have been that we hit a max number of SAs for the dev org but it doesn't look to be the case
Collaborator
Author
There was a problem hiding this comment.
I think this has happened before we just never paid attention
Collaborator
There was a problem hiding this comment.
I'm not so sure. We should've been blocked on merging PRs since the bug was introduced.
Either way, just curious :)
cveticm
approved these changes
Sep 1, 2026
cveticm
left a comment
cveticm
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, thanks for the fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
Setup a Service Account and create a projectjob fails 100% of the time at its finalDelete the Service Accountstep, blocking every unrelated PR in this repo (currently #34 and #35). It is not flaky — confirmed across 4 rerun attempts on #34 and again on #35.build/generate-service-account.shcreates the service account against${MONGODB_ATLAS_OPS_MANAGER_URL}(https://cloud-dev.mongodb.com/, fromvars.CLOUD_DEV_URL), butbuild/terminate-service-account.shhardcodedhttps://cloud.mongodb.com/. So the account is created in cloud-dev and the delete is attempted against production, where the configured API keys have no rights: