Repository navigation
Conversation
Contributor
Module Scorecard Check
|
| Theme | Before | After |
|---|---|---|
| Presentation & Onboarding | 12 / 17 | 12 / 17 |
| Integration | 15 / 25 | 15 / 25 |
| Credential Hygiene | 12 / 20 | 12 / 20 |
| Restricted-Environment | 13.5 / 20 | 11 / 20 |
| Engineering Quality | 10 / 10 | 10 / 10 |
| Overall | 68 / 100 | 65 / 100 |
Full scorecard for this PR
| Presentation & Onboarding | Agent Integration | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|---|
| 12 / 17 | 15 / 25 | 12 / 20 | 11 / 20 | 10 / 10 | 65 / 100 |
Drilldown
Presentation & Onboarding — 12 / 17
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | 12 | Every major auth/provider mode has a dedicated README example with sensible defaults: standalone API key, OAuth token, AI Gateway, api_key_helper, AWS Bedrock, Google Vertex AI, Microsoft Foundry, custom API gateway, managed settings, and telemetry. Each example is self-contained and shows the module invocation. |
| Visual preview | 5 | 0 | No image, GIF, or video is embedded in the README. The icon frontmatter field (../../../../.icons/claude.svg) is a registry listing icon, not an in-action preview. |
Agent Integration — 15 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| AI governance | 10 | 5 | AI Gateway is thoroughly documented (endpoint wiring, authentication_config, mutual-exclusivity rules, apiKeyHelper behavior). Agent Firewall / Boundary support is explicitly stated as dropped in the v5 WARNING block ("drops support for Coder Tasks and Boundary. We plan to add Boundary support back in a follow-up"). Per rubric, dropped support counts as absent. Exactly one of two → half. |
| Dashboard entry point | 5 | 5 | The "Standalone mode with a launcher app" example defines a coder_app resource with slug, display_name, icon, open_in = "slim-window", and a command that cds into the workdir and launches claude. |
| Session continuity | 5 | 0 | No README section documents resuming an existing session, a --resume/session-ID flag, or use of a persistent session manager (tmux, screen, boo). The coder_app example launches a fresh claude process each time. |
| Managed configuration | 5 | 5 | managed_settings writes a highest-precedence policy file to /etc/claude-code/managed-settings.d/10-coder.json (documented with a full example including permissions, env, model). mcp and mcp_config_remote_path configure MCP servers at user scope. api_key_helper registers a managed credential helper. All documented with examples. |
Credential Hygiene — 12 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | 8 | anthropic_api_key, claude_code_oauth_token, foundry_api_key, and foundry_auth_token all carry sensitive = true in main.tf. However, multiple README examples inline a placeholder key (anthropic_api_key = "xxxx-xxxxx-xxxx"), which per the calibration anchor caps this criterion at half. |
| Non-hardcoded auth path | 4 | 4 | The README documents several non-raw-key paths: api_key_helper (Vault / AWS Secrets Manager / cloud IAM scripts), AI Gateway (workspace session token), AWS Bedrock via IRSA / instance profile, Google Vertex via Application Default Credentials / Workload Identity, and Microsoft Foundry via managed identity / Entra ID. Each has a dedicated example. |
Restricted-Environment Readiness — 11 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 5 | 0 | The install script hardcodes https://claude.ai/install.sh. No module input variable overrides this download URL. claude_code_version pins the version argument but not the source URL; claude_binary_path specifies where a pre-installed binary lives, not where to download from. No variable can be named that replaces the download URL. |
| Bring-your-own binary | 10 | 10 | install_claude_code = false disables the installer entirely. Combined with claude_binary_path (validated to only be settable when install is off), the module skips download and validates the pre-installed binary. Documented in the "Advanced Configuration" example and the variable description. |
| Egress transparency | 3 | 0 | No dedicated README section enumerates external endpoints contacted at install or runtime. Endpoints (claude.ai/install.sh, Anthropic API, AI Gateway, MCP remote URLs, OTLP collector) are scattered across individual examples but never consolidated with air-gapped/restricted-environment guidance. |
| Runs without sudo | 2 | 1 | The install script uses sudo only for writing to /etc/claude-code/managed-settings.d/ (managed settings, API key helper registration) and for removing stale files there. Each sudo call is guarded by command_exists sudo with a non-sudo fallback (mkdir -p, echo >, rm -f). The core install (curl + installer + ~/.local/bin) and all auth/MCP configuration work without root. Sudo is used for optional features with a working non-root fallback → half. |
Engineering Quality — 10 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | 6 | Every variable has a description. Defaults are sensible (install_claude_code = true, claude_code_version = "latest", workdir = null). Extensive validation blocks enforce mutual exclusivity across all provider backends, authentication_config enum, claude_binary_path/install_claude_code coupling, and Foundry endpoint/credential constraints. |
| Test coverage | 4 | 4 | main.tftest.hcl (30+ plan-level assertions) covers business logic: env-var creation, validation failures, managed-settings encoding, script-output ordering, provider mutual-exclusivity. main.test.ts (20+ container-level tests) covers end-to-end behavior: install success/failure paths, binary resolution, MCP registration, AI Gateway apiKeyHelper token resolution, managed-settings write/cleanup, permission-failure reporting, symlink safety. Clear separation of concerns. |
Overall — 65 / 100
Raw 60 / 92 → round(60 / 92 × 100) = 65
Tip
You can run this locally by telling your agent: "review this module against .github/scorecard/SCORECARD.md".
Scored against SCORECARD.md with solstice-1. Language-model scores are advisory.
matifali
marked this pull request as ready for review
October 23, 2026 10:40
Edd88-pixel
reviewed
Oct 23, 2026
| else | ||
| jq -n --arg helper "$${helper}" '{apiKeyHelper: $helper}' > "$${settings_tmp}" | ||
| fi | ||
| mv "$${settings_tmp}" "$${settings}" |
Collaborator
There was a problem hiding this comment.
Could we preserve the symlink when updating settings.json and skip rewriting it when no change is needed? The mv replaces a symlink with a regular file, even when a custom helper is already configured. For users managing this file through dotfiles, subsequent updates to the original file no longer reach Claude Code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Configure
apiKeyHelperin Claude Code user settings when AI Gateway is enabled. The helper first triescoder login tokenand falls back toANTHROPIC_AUTH_TOKENwhen the CLI fails or returns an empty token, so settings contain no credential values and clients can recognize the authentication source.Preserve existing custom helpers and other settings, remove only the module default when AI Gateway is turned off, and write updates atomically. Prepare module version
5.5.3; dependent changes are https://github.com/coder/dogfood/pull/739 and coder/coder#31169.Validation: 46 Terraform tests, plus container tests covering fallback, custom settings, malformed settings, repeat installation, cleanup, and the other authentication modes.
Generated with Codex using GPT-6.