Skip to content

Fix/aw bootstrap hierarchy cmek - #220

Open
lynn-goog wants to merge 3 commits into
mainfrom
fix/aw-bootstrap-hierarchy-cmek
Open

Fix/aw bootstrap hierarchy cmek#220
lynn-goog wants to merge 3 commits into
mainfrom
fix/aw-bootstrap-hierarchy-cmek

Conversation

@lynn-goog

Copy link
Copy Markdown
Collaborator

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.

Fixes # (GitHub issue id)

Type of Change

  • [ x] Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Deployment & Compliance Impact

  • Applicable Regimes:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Moderate
    • FedRAMP High
    • FedRAMP Moderate
    • DoD IL4
    • [x ] DoD IL5
    • [ x] General / All
  • NIST 800-53r5 Controls: (If this PR helps satisfy or modifies control implementations, list them here)

Checklist

Code Quality & Reusability

  • [ x] My code adheres to the Maximize Reusability principle. I have not redefined common elements and have reused existing base configurations and modules where possible.
  • [x ] I have checked that no existing module or configuration in modules/ or fast/ can be leveraged for this change.
  • [ x] My code follows the established naming conventions outlined in documentation/naming-convention.md.

Documentation

  • I have updated the README.md of the modified module or blueprint.
  • I have added/updated documentation for inputs (variables) and outputs.

Security

  • [x ] My change adheres to GCP security best practices and the principle of least privilege.
  • [ x] I have ensured compliance with the targeted regime (FedRAMP Moderate, FedRAMP High, IL5, etc.).

Testing

  • [x ] I have tested my changes locally.
  • I have included details of my testing in this PR.

Testing Performed

Please describe the tests that you ran to verify your changes.

All syntax, type constraints, and relationships have been fully verified with terraform validate and live GCP API calls.

lynn-goog and others added 3 commits September 4, 2026 14:41
… constraint naming, and service agents

- modules/folder: Filter for CONSUMER_FOLDER in Assured Workloads resources to prevent resolving the encryption keys project as the parent folder ID.
- modules/organization-se: Strip hyphens from org policy custom constraint names and policy names using replace(var.prefix, "-", "") to adhere to GCP alphanumeric constraint naming requirement.
- modules/project: Add cloudkms to disabled default service agent bindings and remove obsolete bq_sa data source that triggered 403 Access Denied on new projects.
… logging buckets, and KMS IAM bindings

- organization.tf, budgets.tf, outputs.tf: Centralize Assured Workloads consumer folder resolution into local.assured_workload_folder, filtering by CONSUMER_FOLDER to prevent using encryption keys project ID as parent folder.
- log-export.tf: Use coalesce(var.logging_kms_key, module.logging-kms.key_ids["log-sink"]) to correctly pass CMEK key IDs to logging buckets and sinks, add module.logging-kms to depends_on, and explicitly enable logging.googleapis.com on the audit logs project.
- kms.tf: Make KMS IAM bindings conditional on contains(local.log_types, ...) to avoid creating bindings for unconfigured sink types, and grant roles/cloudkms.cryptoKeyEncrypterDecrypter to the Cloud Logging service agent on the log-sink crypto key.
…n, and constraint naming

- modules/folder/main.tf: Wrap format inside try() to prevent unhandled fatal error when one() returns null for non-existent CONSUMER_FOLDER.
- fast/stages-aw/0-bootstrap/kms.tf: Remove duplicate logging IAM binding from KeyRing level, preserving least-privilege key-level binding under keys["log-sink"].
- fast/stages-aw/0-bootstrap/log-export.tf: Add depends_on = [module.logging-kms] to log-export-dataset, log-export-gcs, and log-export-pubsub to eliminate race conditions before KMS IAM bindings propagate.
- modules/organization-se: Use regex replace /[^a-zA-Z0-9]/ and startswith("custom.") to ensure custom constraints conform strictly to GCP naming rules.
@aghassemlouei

Copy link
Copy Markdown
Collaborator

After pushing some minor changes I went ahead and validated this branch (fix/aw-bootstrap-hierarchy-cmek at commit 52b1b3058491d0d610ab0f4752438e9cffb1c372) against a live test Organization using IL5 Assured Workloads configuration (us-east4).

Summary of Fixes and Verification Results

  1. Folder Hierarchy Fallback (modules/folder/main.tf):
    • Wrapped format("folders/%s", ...) inside try() so that if one([...]) returns null, Terraform falls back cleanly rather than throwing an unhandled fatal error on format("%s", null).
  2. KMS Least-Privilege IAM (fast/stages-aw/0-bootstrap/kms.tf):
    • Removed duplicate roles/cloudkms.cryptoKeyEncrypterDecrypter binding at the KeyRing level for the Cloud Logging service agent, preserving only the crypto-key level binding (module.logging-kms.google_kms_crypto_key_iam_member.members["logging"]).
  3. Log Sink KMS Dependency Ordering (fast/stages-aw/0-bootstrap/log-export.tf):
    • Added explicit depends_on = [module.logging-kms] to log-export-dataset, log-export-gcs, and log-export-pubsub modules to prevent race conditions during initial deployment before KMS IAM bindings propagate.
  4. Custom Organization Policy Constraint Naming (modules/organization-se/):
    • Updated string replacement to replace(var.prefix, "/[^a-zA-Z0-9]/", "") and startswith(..., "custom.") to ensure custom constraint IDs strictly conform to GCP alphanumeric naming rules (^custom\.[a-zA-Z0-9]+$).

Plan Output (0-bootstrap)

  # module.logging-kms.google_kms_crypto_key_iam_member.members["logging"] will be created
  + resource "google_kms_crypto_key_iam_member" "members" {
      + crypto_key_id = (known after apply)
      + etag          = (known after apply)
      + id            = (known after apply)
      + member        = (known after apply)
      + role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
    }

  # module.organization.google_org_policy_custom_constraint.constraint["custom.kmsRotation"] will be created
  + resource "google_org_policy_custom_constraint" "constraint" {
      + action_type    = "ALLOW"
      + condition      = "((resource.rotationPeriod != duration('0s')) && (resource.rotationPeriod <= duration('7776000s')))"
      + description    = "The rotation period for Cloud KMS keys must be set to 90 days or less."
      + display_name   = "Enforce maximum rotation period for KMS keys"
      + id             = (known after apply)
      + name           = "custom.kmsRotationse220"
      + parent         = "organizations/<REDACTED_ORG_ID>"
      + resource_types = [
          + "cloudkms.googleapis.com/CryptoKey",
        ]
    }

  # module.organization.google_org_policy_custom_constraint.constraint["custom.restrictFirewallRanges"] will be created
  + resource "google_org_policy_custom_constraint" "constraint" {
      + action_type    = "DENY"
      + condition      = "resource.direction.matches('INGRESS') && resource.sourceRanges.all(range, range == '0.0.0.0/0') && resource.allowed.exists(entry, entry != null)"
      + description    = "Firewall rules must be more specific than 0.0.0.0/0."
      + display_name   = "Restrict Firewall rules using the source-range 0.0.0.0/0"
      + id             = (known after apply)
      + name           = "custom.restrictFirewallRangesse220"
      + parent         = "organizations/<REDACTED_ORG_ID>"
      + resource_types = [
          + "compute.googleapis.com/Firewall",
        ]
    }

  # module.organization-logging.google_logging_organization_settings.default[0] will be created
  + resource "google_logging_organization_settings" "default" {
      + disable_default_sink       = (known after apply)
      + id                         = (known after apply)
      + kms_key_name               = (known after apply)
      + kms_service_account_id     = (known after apply)
      + logging_service_account_id = (known after apply)
      + name                       = (known after apply)
      + organization               = "<REDACTED_ORG_ID>"
      + storage_location           = "us-east4"
    }

Plan: 253 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + alert_email             = "<REDACTED_EMAIL>"
  + assured_workload        = (known after apply)
  + automation              = {
      + federated_identity_pool      = null
      + federated_identity_providers = {}
      + outputs_bucket               = "se220-prod-iac-core-outputs-0"
      + project_id                   = "se220-prod-iac-core-0"
      + project_number               = (known after apply)
      + service_accounts             = {
          + bootstrap   = "se220-prod-bootstrap-0@se220-prod-iac-core-0.iam.gserviceaccount.com"
          + bootstrap-r = "se220-prod-bootstrap-0r@se220-prod-iac-core-0.iam.gserviceaccount.com"
          + resman      = "se220-prod-resman-0@se220-prod-iac-core-0.iam.gserviceaccount.com"
          + resman-r    = "se220-prod-resman-0r@se220-prod-iac-core-0.iam.gserviceaccount.com"
        }
    }
  + custom_roles            = {
      + gcve_network_admin            = "organizations/<REDACTED_ORG_ID>/roles/gcveNetworkAdmin"
      + organization_admin_viewer     = "organizations/<REDACTED_ORG_ID>/roles/organizationAdminViewer"
      + organization_iam_admin        = "organizations/<REDACTED_ORG_ID>/roles/organizationIamAdmin"
      + service_project_network_admin = "organizations/<REDACTED_ORG_ID>/roles/serviceProjectNetworkAdmin"
      + storage_viewer                = "organizations/<REDACTED_ORG_ID>/roles/storage_viewer"
      + tag_viewer                    = "organizations/<REDACTED_ORG_ID>/roles/tagViewer"
      + tenant_network_admin          = "organizations/<REDACTED_ORG_ID>/roles/tenantNetworkAdmin"
    }
  + kms_protection_level    = "SOFTWARE"
  + outputs_bucket          = "se220-prod-iac-core-outputs-0"
  + project_ids             = {
      + automation     = "se220-prod-iac-core-0"
      + billing-export = "se220-prod-billing-exp-0"
      + log-export     = "se220-prod-audit-logs-0"
    }

@aghassemlouei aghassemlouei added bug Something isn't working Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week labels Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Level of Effort - Medium Moderate task requiring thought and testing; typically takes a couple of days to a week Priority - Medium Standard features and non-blocking bugs; important for the current milestone but not urgent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants