From c002a9867f6729c7fd499eca8e8656648dac4f2a Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Fri, 28 Aug 2026 12:32:05 -0700 Subject: [PATCH 01/18] Add spec: reconcile hydrated state against reality on rad startup Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/spec.md | 97 +++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 specs/006-state-restoration/spec.md diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md new file mode 100644 index 0000000000..8772067ac6 --- /dev/null +++ b/specs/006-state-restoration/spec.md @@ -0,0 +1,97 @@ +# Feature Specification: Reconcile hydrated state against reality on `rad startup` + +**Feature Branch**: `state-restoration` +**Created**: 2026-08-28 +**Status**: Draft - awaiting approval +**Input**: Design notes from the sync with Will Tsai and Nicole James on the "Delete workflow 409 loops forever when a resource is stranded in a non-terminal state" bug (recording 2026-08-28). + +## Purpose + +Repo Radius (the ephemeral k3d control plane the GitHub workflows spin up on every run) restores durable state from an OCI archive at the start of each run via [`rad startup`](../../pkg/cli/cmd/startup/startup.go). The restore is currently a one-way load: PostgreSQL dumps and Terraform state Secrets are put back into the fresh control plane exactly as they were persisted at the end of the previous run. + +That is not sufficient when the previous run was interrupted while a resource was mid-operation. The archive can preserve a resource in a non-terminal state — for example `provisioningState: "Updating"` — that never actually completed. On the next run, the control plane accepts that state as authoritative, so every subsequent operation against the resource is blocked with `409 Conflict / target resource is in progress`. The delete workflow loops on that 409 forever and the application becomes undeletable through Radius. + +This feature adds a reconciliation pass at the end of `rad startup`: for every resource the archive restored in a non-terminal `provisioningState`, query the resource's actual current state (through the resource provider that owns it) and update the state store to match reality — including marking the resource as gone when the underlying resource does not exist. + +The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. It does not change the delete command, does not add a `--force` flag, and does not touch the control-plane runtime for regular (non-Repo) Radius, which already reconciles asynchronously through its own resource providers. + +## Non-goals + +- **A `rad app delete --force` flag was considered and explicitly rejected.** With two concurrent deletes (for example, a user re-runs `rad app delete` after their terminal died), a force option that bypasses state can convert an in-progress happy-path delete into a broken one by overwriting the state store while the first delete is still driving to a terminal state. Fixing hydration removes the need for the flag. +- **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it, do not warn. +- **Regular Radius (persistent control plane) is out of scope.** Its resource providers reconcile continuously; this bug is specific to the ephemeral archive-hydrated topology. +- **The concurrent-`rad app delete` behavior is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). + +## Decisions + +### Reconciliation runs at the end of `rad startup`, before it returns success + +`rad startup` today performs three stages, in order: + +1. [`ScaleDown`](../../pkg/cli/cmd/startup/stateclient.go) the resource-provider deployments so no live pgx connections hold the databases open. +2. [`RestoreDatabases`](../../pkg/cli/cmd/startup/stateclient.go) — load the PostgreSQL dumps. +3. [`RestoreTerraform`](../../pkg/cli/cmd/startup/stateclient.go) — recreate the Terraform recipe state Secrets. +4. [`ScaleUp`](../../pkg/cli/cmd/startup/stateclient.go) — bring the resource providers back online. + +Reconciliation adds a fifth stage that runs after ScaleUp and after the resource providers have finished their readiness probes: enumerate every resource in the restored state store whose `provisioningState` is non-terminal (`Accepted`, `Provisioning`, `Updating`, `Deleting`, or any other state that is neither `Succeeded` nor `Failed`), query its owning resource provider for the resource's current state, and rewrite the state-store entry accordingly. The whole reconciliation completes before `rad startup` exits, so downstream steps in the workflow (deploy, delete, or plain `rad` commands) observe an accurate state store from their first request. + +Running after ScaleUp is required because the reconciliation queries flow through the same UCP/RP path that regular clients use — the resource providers must be ready to answer. Reconciliation must not talk directly to the database; that would re-introduce the class of bug Nicole flagged, where a database write bypasses the resource provider's state machine. + +### What "reality" is for each resource + +For each hydrated non-terminal resource, `rad startup` issues a GET against the resource-provider surface (UCP → RP) and interprets the response: + +| Hydrated `provisioningState` | RP response | Action | +| ---------------------------- | -------------------- | -------------------------------------------------------------------------------------------------- | +| any non-terminal | `200`, terminal | Rewrite the state-store entry with the RP-reported terminal state (`Succeeded` / `Failed` / etc.). | +| any non-terminal | `200`, still non-terminal | Leave as-is. The hydrated state is accurate. | +| any non-terminal | `404` | Delete the state-store entry. The resource does not exist. | +| any non-terminal | error (network, 5xx) | Leave as-is; log a warning. Reconciliation is best-effort and never blocks startup. | + +Terminal-state entries (`Succeeded`, `Failed`) are not reconciled. The archive captured a settled state; if it drifts, the next operation the user issues will refresh it through the normal path. + +### Reconciliation is best-effort and does not fail `rad startup` + +`rad startup` today already treats a failed archive open as fatal, because without the archive the control plane has nothing to serve. Reconciliation is different: a reconciliation failure on any single resource, or on the reconciliation pass as a whole, must not fail startup. The workflow's subsequent commands then run against the un-reconciled state, which is no worse than today's behavior. Every reconciliation outcome (skipped, unchanged, updated, deleted, failed to query) is written to the `rad startup` log so it is visible in the workflow log. + +This preserves the guarantee that a run can always at least *try* to make progress. It also means the fix is safe to ship without a fallback flag: at worst the reconciliation pass is a no-op. + +### Scope of resources reconciled + +The reconciler enumerates every resource in every restored resource-provider database. It does not filter by resource type, by application, or by environment. Filtering would require the reconciler to know which resource types can transition non-terminal states on their own (they all can) and would create carve-outs to keep in sync as new resource types land. + +Applications and environments are themselves tracked resources and are included. If an application is hydrated in `Updating`, its record is reconciled the same way as any container or database inside it. + +### Reconciliation semantics for children + +An application's child resources (containers, databases, gateways, etc.) each have their own `provisioningState` and are reconciled independently. The reconciler does not need to walk the application graph; it enumerates directly from the resource store. If an application is hydrated in `Succeeded` but a container inside it was hydrated in `Updating`, only the container is reconciled. + +## System Context + +### Where the bug manifests today + +- The GitHub delete workflow ([.github/extension/delete-azure.yml](../../.github/extension/delete-azure.yml), [.github/extension/delete-aws.yml](../../.github/extension/delete-aws.yml)) runs [`restore-state`](../../.github/extension/actions/restore-state/action.yml) which shells out to `rad startup`, then [`delete-resource`](../../.github/extension/actions/delete-resource/action.yml) which shells out to `rad app delete --yes --preview`. +- The failure mode: `rad app delete` retries a `409 Conflict / target resource is in progress` indefinitely because the hydrated state store reports a resource in a non-terminal state that never actually existed (or that has since settled underneath). The specific transcript case was an application whose deployment had failed, leaving nothing in the cloud, while the state store insisted the resource was `Updating`. + +### Where the change lives + +- The `rad startup` command in [pkg/cli/cmd/startup/startup.go](../../pkg/cli/cmd/startup/startup.go) and its state client in [pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go). A new stage (call it `ReconcileHydrated`) is added there. +- The reconciler enumerates state through the standard UCP list endpoints and queries individual resources via the standard RP GET endpoints. It does not import the resource providers' internal packages. +- No workflow-level change. [restore-state/action.yml](../../.github/extension/actions/restore-state/action.yml) already runs `rad startup`; the reconciliation is transparent. +- No CLI change to `rad app delete`. Once reconciliation runs, the delete sees an accurate state store and proceeds normally. + +### Why regular Radius is unaffected + +A persistently running Radius control plane has resource providers that reconcile their world continuously — the async operation controller polls, the health controller polls, and stuck non-terminal states resolve within the RP's own polling interval. The archive-hydrate topology short-circuits that: the state is loaded from disk and immediately trusted. Reconciliation on hydrate closes that gap only for the ephemeral topology. + +## Acceptance + +- Deleting an application whose state archive contains at least one child resource in a non-terminal `provisioningState` succeeds when the underlying cloud resource does not exist. Today it loops on `409`. +- Deleting an application whose state archive contains a child resource in `Updating` and whose underlying cloud resource genuinely is still updating waits normally and does not falsely succeed. The reconciliation must observe the RP-reported state and leave the store unchanged. +- `rad startup` never fails because reconciliation could not reach a resource provider. The workflow log records the failure and startup returns success. +- The reconciliation pass runs no direct SQL against the resource-provider databases. Every state change goes through the RP, so state machines stay intact. +- No `--force` flag is added to `rad app delete` (or to any other command) as part of this feature. + +## Follow-up + +A separate issue is filed to verify that concurrent `rad app delete` against the same application from two terminals is handled correctly by a regular (persistent) Radius control plane. That case is not affected by hydration — a persistent control plane already tracks in-flight operations — but it needs an explicit test so a future change cannot regress it. See [radius-project/radius#12870](https://github.com/radius-project/radius/issues/12870). From d2d70990f3d4d2486de67f677373c352c35dcf26 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Fri, 28 Aug 2026 12:45:19 -0700 Subject: [PATCH 02/18] wip --- specs/006-state-restoration/spec.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 8772067ac6..7e36a7b92d 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -80,7 +80,7 @@ An application's child resources (containers, databases, gateways, etc.) each ha - No workflow-level change. [restore-state/action.yml](../../.github/extension/actions/restore-state/action.yml) already runs `rad startup`; the reconciliation is transparent. - No CLI change to `rad app delete`. Once reconciliation runs, the delete sees an accurate state store and proceeds normally. -### Why regular Radius is unaffected +### Why regular Radius is not as much affected A persistently running Radius control plane has resource providers that reconcile their world continuously — the async operation controller polls, the health controller polls, and stuck non-terminal states resolve within the RP's own polling interval. The archive-hydrate topology short-circuits that: the state is loaded from disk and immediately trusted. Reconciliation on hydrate closes that gap only for the ephemeral topology. @@ -90,7 +90,6 @@ A persistently running Radius control plane has resource providers that reconcil - Deleting an application whose state archive contains a child resource in `Updating` and whose underlying cloud resource genuinely is still updating waits normally and does not falsely succeed. The reconciliation must observe the RP-reported state and leave the store unchanged. - `rad startup` never fails because reconciliation could not reach a resource provider. The workflow log records the failure and startup returns success. - The reconciliation pass runs no direct SQL against the resource-provider databases. Every state change goes through the RP, so state machines stay intact. -- No `--force` flag is added to `rad app delete` (or to any other command) as part of this feature. ## Follow-up From 4f4af5de1e09a80ec863d24abb292799c3e16bcc Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Fri, 28 Aug 2026 13:13:15 -0700 Subject: [PATCH 03/18] Rewrite spec around app-scoped reconcile action; add plan Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/plan.md | 140 ++++++++++++++++++++++++++++ specs/006-state-restoration/spec.md | 128 +++++++++++++++++-------- 2 files changed, 229 insertions(+), 39 deletions(-) create mode 100644 specs/006-state-restoration/plan.md diff --git a/specs/006-state-restoration/plan.md b/specs/006-state-restoration/plan.md new file mode 100644 index 0000000000..e8a41e29f4 --- /dev/null +++ b/specs/006-state-restoration/plan.md @@ -0,0 +1,140 @@ +# Implementation Plan: Reconcile hydrated state against reality on `rad startup` + +**Branch**: `state-restoration` | **Date**: 2026-08-28 | **Spec**: [spec.md](spec.md) + +## Summary + +Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go)) plus a per-resource-type `reconcile` custom action registered on `Radius.Compute/containers` (prototype scope). `rad startup` gains a fifth stage, `ReconcileHydratedState`, that lists applications after `ScaleUp` and POSTs the app-scoped action for each. The corerp application handler fans out through the UCP proxy to per-resource-type handlers, which query reality (Kubernetes for containers) and rewrite the state store through the RP's normal write path. Best-effort throughout: individual failures never fail `rad startup`. + +## Technical Context + +**Language/Version**: Go 1.26.5 (per `go.mod`) +**Primary Dependencies**: no new external dependencies. Reuses `github.com/Azure/azure-sdk-for-go/sdk/azcore` (async operation), `k8s.io/client-go` (per-container reality check), and the internal `pkg/armrpc/builder` custom-action registration mechanism. +**Storage**: no schema changes. Reconciliation writes go through the RPs' existing state-store paths. +**Testing**: `go test` with `stretchr/testify`; table-driven unit tests for the corerp orchestrator and the containers handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. Existing `rad startup` tests get a new fake for `ReconcileHydratedState`. +**Target Platform**: Radius control plane (Linux server binary) and `rad` CLI (macOS/Linux/Windows). +**Project Type**: Single Go module `github.com/radius-project/radius`. +**Performance Goals**: Reconciliation for an application with ≤50 containers must complete within 30 s on the k3d control plane. No hot-path allocations in the containers handler (a k8s GET per container is the dominant cost). +**Constraints**: no direct SQL against RP databases; no boot-time reconciliation in the persistent control plane; opt-in per resource type. +**Scale/Scope**: prototype covers `Radius.Compute/containers` only. Ships as one PR; other types are follow-ups. + +## Constitution Check + +*GATE: Passed at plan authoring time.* + +| Principle | Verdict | Note | +| ---------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| I. API-First Design | ✅ | Wire change authored in TypeSpec (`typespec/Radius.Core/applications.tsp` and `typespec/Radius.Compute/containers.tsp`); Go models regenerated via `make generate`. | +| II. Idiomatic Code Standards | ✅ | `gofmt`, small exported surface, godoc on every exported symbol, table-driven tests. | +| III. Multi-Cloud Neutrality | ✅ | The application-scoped action is cloud-agnostic. Per-resource handlers query their own underlying provider (Kubernetes for the prototype; TF/Azure/AWS for follow-ups). No provider carve-out in the orchestrator. | +| IV. Testing Pyramid Discipline | ✅ | Unit tests for the orchestrator and the containers handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. | +| V. Collaboration-Centric Design | ✅ | Fixes an operator-visible failure (delete workflow loops forever) without new user-facing surface — the flag path deliberately not taken. | +| VI. Open Source and Community-First | ✅ | Spec and plan authored in the public repo; commits will carry `Signed-off-by`. | +| VII. Simplicity Over Cleverness | ✅ | Reuses the existing `Custom` action mechanism, the existing `getGraph` traversal, and the existing async-operation pattern. No new framework code. | +| VIII. Separation of Concerns | ✅ | Orchestrator in corerp, reality-check logic in the per-type handler, transport through UCP. Each layer owns what it already owns. | +| IX. Incremental Adoption & Backward Compatibility | ✅ | `Radius.Core/2025-08-01-preview` is preview; adding a `Custom` action is additive. `Applications.Core` is not touched. | +| XII / XIII (resource type / recipe standards) | N/A | No new resource types or recipes. | +| XVII. Polyglot Project Coherence | ✅ | TypeSpec is the single source of truth for the wire; Go generated code follows. | + +**No violations. Complexity Tracking section is empty.** + +## Project Structure + +### Documentation (this feature) + +```text +specs/006-state-restoration/ +├── plan.md # This file +└── spec.md # Feature spec +``` + +Additional artifacts (research/data-model/quickstart/tasks) are not required — the scope is small enough to plan directly. + +### Source Code (repository root) + +Additions and edits, all within the existing single Go module: + +```text +# TypeSpec — additive custom actions +typespec/Radius.Core/applications.tsp # add `reconcile` action on applications/{name} +typespec/Radius.Compute/containers.tsp # add `reconcile` action on containers/{name} + +# Regenerated Go models — via `make generate` +pkg/corerp/api/v20250801preview/zz_generated_*.go + +# New — application-scoped orchestrator (mirror of getgraph.go) +pkg/corerp/frontend/controller/applications/v20250801preview/ +├── reconcile.go +└── reconcile_test.go + +# New — per-resource-type reality check (prototype scope) +pkg/corerp/frontend/controller/containers/ +├── reconcile.go +└── reconcile_test.go + +# Edited — register the two new custom actions +pkg/corerp/setup/setup.go + +# Edited — new ReconcileHydratedState stage +pkg/cli/cmd/startup/ +├── startup.go # wire the stage +├── stateclient.go # add method to the StateRestoreClient interface +└── startup_test.go # add coverage for the new stage +``` + +## Phases + +### Phase 0 — Wire the app-scoped action end to end with a no-op handler + +Goal: prove the registration, routing, async-operation lifecycle, and `rad startup` invocation before we do any reality checking. + +- Add the `reconcile` custom action on `Radius.Core/applications/{name}` in TypeSpec; regenerate. +- Implement `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go` as a stub that returns an immediately-succeeded async operation with an empty report. +- Register the action in `pkg/corerp/setup/setup.go` beside `getGraph`. +- Add `ReconcileHydratedState(ctx, connection)` on `StateRestoreClient` in `pkg/cli/cmd/startup/stateclient.go`. Implementation lists applications, POSTs the action, polls to completion, logs the (empty) report. +- Wire the new stage in `pkg/cli/cmd/startup/startup.go` after `ScaleUp`. +- Unit tests: fake `StateRestoreClient` records the call; corerp handler test verifies the async-operation shape. + +**Exit criterion**: `rad startup` on a k3d cluster with one hydrated `Radius.Core/applications` succeeds and logs `reconciled 0 resources` for it. + +### Phase 1 — Implement the containers per-resource `reconcile` + +Goal: reality-check containers against Kubernetes and rewrite the state store. + +- Add the `reconcile` custom action on `Radius.Compute/containers/{name}` in TypeSpec; regenerate. +- Implement `pkg/corerp/frontend/controller/containers/reconcile.go`: + - Look up the container record. + - If `provisioningState` is terminal, return unchanged. + - Read the underlying Kubernetes object (Deployment) via the RP's existing k8s client. Interpret 404, terminal, non-terminal per the [reality table](spec.md#what-reality-is-for-each-resource). + - Write the outcome back through the container RP's normal PATCH path. + - Return the new state. +- Register the action in `pkg/corerp/setup/setup.go` under the `containers` resource block. +- Update the corerp orchestrator from Phase 0 to actually walk children, filter to non-terminal, POST `reconcile` to each container in parallel (bounded), collect responses, reconcile the application record from the aggregated child states, return a populated report. +- Unit tests: containers handler with a fake k8s client (three cases from the reality table); orchestrator with a fake UCP connection and multiple child responses. +- Integration test: mount the whole thing behind `httptest` — application with two containers, one 404 in k8s and one healthy — verify the app-scoped POST returns a report and the state store reflects reality. + +**Exit criterion**: the [Acceptance](spec.md#acceptance) prototype criterion holds — an application whose containers were hydrated in `Updating` but do not exist in Kubernetes is deletable after `rad startup`. + +### Phase 2 — Integration and functional coverage + +Goal: prove the end-to-end delete flow against a k3d cluster. + +- Extend `test/functional/` with a case that: + 1. Seeds a state archive with an application whose container is `Updating` and whose k8s Deployment does not exist. + 2. Runs `rad startup` against a fresh k3d cluster loaded with that archive. + 3. Runs `rad app delete --yes --preview` and asserts it succeeds (no 409 loop). +- Verify no direct SQL calls in the reconciler code path via a `grep_search`-style CI check (informational). + +**Exit criterion**: functional test passes in CI on every PR. + +## Rollout + +- Ship as one PR that lands Phase 0 + Phase 1 together (the two are cheap and separating them leaves a dead endpoint in the tree). Phase 2 is a follow-up PR because functional-test infra changes deserve their own review. +- No feature flag. The action is dormant unless called; only `rad startup` calls it; only Repo Radius runs `rad startup`. The persistent control plane never invokes it and does not care. +- No release-note user impact (behavior change is invisible to `rad app delete` callers — they just stop looping on 409). + +## Risks and open questions + +- **Child enumeration source of truth.** `getGraph` walks children by scanning UCP's `System.Resources/resourceProviders` and listing each type. If a resource type is registered but its RP is unresponsive, the corerp orchestrator must not hang. Timebox each per-child call to a bounded deadline and record the failure in the report. +- **PATCH vs re-PUT semantics on a "gone" resource.** When reality reports 404 and we want to remove the state-store entry, the cleanest path is to issue a synchronous DELETE against the RP through UCP, bypassing normal deletion semantics via a well-scoped flag/header understood only by the RP's `reconcile` handler. Alternative: PATCH `provisioningState` to `Failed` and let the next `rad app delete` clean up naturally. **Open.** Prototype implementation will PATCH to `Failed` (simpler, keeps the delete path in one place); if that produces user-visible weirdness (an `rad app show` reporting `Failed` on a resource that has been deleted from k8s), we revisit. +- **Preview-API sensitivity.** Adding a `Custom` action on the preview surface is additive, but downstream consumers of the generated Go client have to regenerate. There is no public preview SDK release cadence to worry about; internal callers regenerate on the next `make generate`. diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 7e36a7b92d..615b5bd398 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -11,86 +11,136 @@ Repo Radius (the ephemeral k3d control plane the GitHub workflows spin up on eve That is not sufficient when the previous run was interrupted while a resource was mid-operation. The archive can preserve a resource in a non-terminal state — for example `provisioningState: "Updating"` — that never actually completed. On the next run, the control plane accepts that state as authoritative, so every subsequent operation against the resource is blocked with `409 Conflict / target resource is in progress`. The delete workflow loops on that 409 forever and the application becomes undeletable through Radius. -This feature adds a reconciliation pass at the end of `rad startup`: for every resource the archive restored in a non-terminal `provisioningState`, query the resource's actual current state (through the resource provider that owns it) and update the state store to match reality — including marking the resource as gone when the underlying resource does not exist. +This feature adds a reconciliation pass triggered by `rad startup` and executed against the running control plane: for every application in the plane, an application-scoped `reconcile` action asks each resource's owning resource provider to check its actual current state and rewrite the state store to match reality — including removing entries when the underlying resource does not exist. -The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. It does not change the delete command, does not add a `--force` flag, and does not touch the control-plane runtime for regular (non-Repo) Radius, which already reconciles asynchronously through its own resource providers. +The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. It does not change the delete command, does not add a `--force` flag, and does not touch the regular (non-Repo) Radius runtime — the action exists but is never invoked there. ## Non-goals - **A `rad app delete --force` flag was considered and explicitly rejected.** With two concurrent deletes (for example, a user re-runs `rad app delete` after their terminal died), a force option that bypasses state can convert an in-progress happy-path delete into a broken one by overwriting the state store while the first delete is still driving to a terminal state. Fixing hydration removes the need for the flag. - **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it, do not warn. -- **Regular Radius (persistent control plane) is out of scope.** Its resource providers reconcile continuously; this bug is specific to the ephemeral archive-hydrated topology. +- **The persistent Radius control plane's async controllers are not changed.** They already reconcile continuously; the new action is dormant unless `rad startup` (or a test) invokes it. - **The concurrent-`rad app delete` behavior is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). ## Decisions -### Reconciliation runs at the end of `rad startup`, before it returns success +### The client-facing endpoint is an application-scoped custom action, mirroring `getGraph` -`rad startup` today performs three stages, in order: +Reconciliation is a per-application operation: walk the application's children, check each one's reality, roll the results back into the state store. That is the same shape as [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go) — an application-scoped custom action registered on `Radius.Core/applications` that walks children across resource providers. `reconcile` therefore reuses the exact pattern, up to and including the corerp orchestrator that already knows how to fan out across RPs through the UCP proxy. -1. [`ScaleDown`](../../pkg/cli/cmd/startup/stateclient.go) the resource-provider deployments so no live pgx connections hold the databases open. -2. [`RestoreDatabases`](../../pkg/cli/cmd/startup/stateclient.go) — load the PostgreSQL dumps. -3. [`RestoreTerraform`](../../pkg/cli/cmd/startup/stateclient.go) — recreate the Terraform recipe state Secrets. -4. [`ScaleUp`](../../pkg/cli/cmd/startup/stateclient.go) — bring the resource providers back online. +```text +POST /planes/radius/local/resourceGroups/{rg}/providers/Radius.Core/applications/{app}/reconcile?api-version=2025-08-01-preview +Content-Type: application/json +{} +``` -Reconciliation adds a fifth stage that runs after ScaleUp and after the resource providers have finished their readiness probes: enumerate every resource in the restored state store whose `provisioningState` is non-terminal (`Accepted`, `Provisioning`, `Updating`, `Deleting`, or any other state that is neither `Succeeded` nor `Failed`), query its owning resource provider for the resource's current state, and rewrite the state-store entry accordingly. The whole reconciliation completes before `rad startup` exits, so downstream steps in the workflow (deploy, delete, or plain `rad` commands) observe an accurate state store from their first request. +- Registered in [pkg/corerp/setup/setup.go](../../pkg/corerp/setup/setup.go) under the `Custom` map on the application resource, next to `getGraph`. +- Handler in `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go`. +- Response is the standard ARM-RPC async pattern: `202 Accepted` with a `Location` header. The client polls to completion. This matches every other write-shaped action in Radius and keeps `rad startup` from holding a synchronous connection open while UCP proxies to many RPs. -Running after ScaleUp is required because the reconciliation queries flow through the same UCP/RP path that regular clients use — the resource providers must be ready to answer. Reconciliation must not talk directly to the database; that would re-introduce the class of bug Nicole flagged, where a database write bypasses the resource provider's state machine. +Naming: `reconcile`, lowercase, matches the codebase convention (`getGraph`, `join`, `getmetadata`). Not `refresh`, not `reconcileStatus` — the action is exactly analogous to the RP-internal reconciliation the persistent control plane already does asynchronously. -### What "reality" is for each resource +### The corerp handler orchestrates; UCP is the proxy + +corerp's [`GetGraphv20250801preview`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go) already receives a `sdk.Connection` at construction, enumerates an application's children by walking resource types across resource providers, and issues per-resource GETs that UCP proxies to the owning RP. `reconcile` reuses that walk and issues a per-resource `reconcile` POST to each non-terminal child instead of a GET. + +Concretely, the handler: + +1. Loads the application record. +2. Traverses the same resource-type registration list `getGraph` uses (via UCP's `System.Resources/resourceProviders`) to build the child set. +3. Filters to children whose current `provisioningState` is non-terminal — anything other than `Succeeded` or `Failed`. Terminal-state children are left alone; the archive captured a settled state and the next user operation will refresh it through the normal path. +4. For each such child, issues: + + ```text + POST /planes/…/providers/{Namespace}/{resourceType}/{name}/reconcile?api-version=… + ``` + + in parallel (bounded fan-out), through the UCP-fronted connection the handler already has. +5. After every child response returns, reconciles the application record itself: if all children are now terminal, transition the application accordingly; if any child remains non-terminal, leave the application in its hydrated state. +6. Aggregates per-child outcomes into a report and completes the async operation. + +UCP is not a smart orchestrator here — it is the proxy layer that already routes `/planes/…/providers/{ns}/…` to the owning RP. That is enough. "UCP asks every RP" is satisfied by construction because every per-child call goes through UCP. + +### Per-resource-type `reconcile` custom action, opt-in -For each hydrated non-terminal resource, `rad startup` issues a GET against the resource-provider surface (UCP → RP) and interprets the response: +Each resource type that participates registers `reconcile` as a `Custom` action alongside the type's existing `Put`/`Patch`/`Delete` operations. Registration is the same three-line addition `getGraph` uses on the application resource; nothing new in the armrpc builder. -| Hydrated `provisioningState` | RP response | Action | -| ---------------------------- | -------------------- | -------------------------------------------------------------------------------------------------- | -| any non-terminal | `200`, terminal | Rewrite the state-store entry with the RP-reported terminal state (`Succeeded` / `Failed` / etc.). | -| any non-terminal | `200`, still non-terminal | Leave as-is. The hydrated state is accurate. | -| any non-terminal | `404` | Delete the state-store entry. The resource does not exist. | -| any non-terminal | error (network, 5xx) | Leave as-is; log a warning. Reconciliation is best-effort and never blocks startup. | +```go +Custom: map[string]builder.Operation[datamodel.ContainerResource]{ + "reconcile": { + APIController: func(opt apictrl.Options) (apictrl.Controller, error) { + return ctr_ctrl.NewReconcile(opt) + }, + }, +}, +``` -Terminal-state entries (`Succeeded`, `Failed`) are not reconciled. The archive captured a settled state; if it drifts, the next operation the user issues will refresh it through the normal path. +The per-resource handler is the only component that knows how to check reality for its type. It: + +- Queries the underlying provider (the Kubernetes API for containers/gateways/secretstores in corerp; the recipe engine's provider client for dynamic-rp types backed by Terraform / Azure / AWS). +- Interprets the response according to the table in [What "reality" is for each resource](#what-reality-is-for-each-resource). +- Writes the result back through the RP's normal state-store path — the same `PATCH` code path the async operation controller uses. **No direct SQL.** +- Returns the new state, or an error the corerp orchestrator will record in the report. + +A resource type that has not opted in is skipped by the corerp orchestrator with a warning in the report. Opt-in is per-type so we can ship the fix incrementally (see [Acceptance](#acceptance) — `Radius.Compute/containers` first). + +### What "reality" is for each resource + +| Hydrated `provisioningState` | Reality query result | Action | +| ---------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------- | +| any non-terminal | terminal (settled) | Rewrite the state-store entry with the observed terminal state (`Succeeded` / `Failed` / etc.). | +| any non-terminal | still non-terminal | Leave as-is. The hydrated state is accurate. | +| any non-terminal | not found | Delete the state-store entry. The resource does not exist. | +| any non-terminal | error (network, 5xx, etc.) | Leave as-is; record the error in the response. Reconciliation is best-effort and never fails. | + +Terminal-state entries are never reconciled by this action. ### Reconciliation is best-effort and does not fail `rad startup` -`rad startup` today already treats a failed archive open as fatal, because without the archive the control plane has nothing to serve. Reconciliation is different: a reconciliation failure on any single resource, or on the reconciliation pass as a whole, must not fail startup. The workflow's subsequent commands then run against the un-reconciled state, which is no worse than today's behavior. Every reconciliation outcome (skipped, unchanged, updated, deleted, failed to query) is written to the `rad startup` log so it is visible in the workflow log. +A failed per-resource `reconcile` does not fail the application's `reconcile`. A failed application `reconcile` does not fail `rad startup`. Every outcome (skipped, unchanged, updated, deleted, failed to query) is written to the `rad startup` log so it is visible in the workflow log. -This preserves the guarantee that a run can always at least *try* to make progress. It also means the fix is safe to ship without a fallback flag: at worst the reconciliation pass is a no-op. +This preserves the guarantee that a run can always at least *try* to make progress. It also means the change is safe to ship without a fallback flag: at worst the pass is a no-op. -### Scope of resources reconciled +### The client-side stage -The reconciler enumerates every resource in every restored resource-provider database. It does not filter by resource type, by application, or by environment. Filtering would require the reconciler to know which resource types can transition non-terminal states on their own (they all can) and would create carve-outs to keep in sync as new resource types land. +`rad startup` today performs four stages ([pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go)): `ScaleDown` → `RestoreDatabases` → `RestoreTerraform` → `ScaleUp`. A fifth stage, `ReconcileHydratedState`, is added after `ScaleUp` and after the resource-provider deployments are ready to serve. It: -Applications and environments are themselves tracked resources and are included. If an application is hydrated in `Updating`, its record is reconciled the same way as any container or database inside it. +1. Lists applications in the plane through UCP. +2. For each application, POSTs `.../applications/{app}/reconcile` and polls the async operation to completion (with a bounded timeout). +3. Logs the per-application report. +4. Always returns success. -### Reconciliation semantics for children +No workflow-level change. [restore-state/action.yml](../../.github/extension/actions/restore-state/action.yml) already runs `rad startup`; the reconciliation is transparent. -An application's child resources (containers, databases, gateways, etc.) each have their own `provisioningState` and are reconciled independently. The reconciler does not need to walk the application graph; it enumerates directly from the resource store. If an application is hydrated in `Succeeded` but a container inside it was hydrated in `Updating`, only the container is reconciled. +### Why the persistent control plane is not disrupted + +The `reconcile` action is dormant unless something calls it. Regular Radius never does — the persistent control plane's async operation controller and health controller reconcile continuously through their own polling loops, and stuck non-terminal states resolve within the RP's own interval. The archive-hydrate topology short-circuits that: state is loaded from disk and immediately trusted. `rad startup` calling `reconcile` closes that gap only for the ephemeral topology, without changing the runtime for the persistent one. ## System Context ### Where the bug manifests today - The GitHub delete workflow ([.github/extension/delete-azure.yml](../../.github/extension/delete-azure.yml), [.github/extension/delete-aws.yml](../../.github/extension/delete-aws.yml)) runs [`restore-state`](../../.github/extension/actions/restore-state/action.yml) which shells out to `rad startup`, then [`delete-resource`](../../.github/extension/actions/delete-resource/action.yml) which shells out to `rad app delete --yes --preview`. -- The failure mode: `rad app delete` retries a `409 Conflict / target resource is in progress` indefinitely because the hydrated state store reports a resource in a non-terminal state that never actually existed (or that has since settled underneath). The specific transcript case was an application whose deployment had failed, leaving nothing in the cloud, while the state store insisted the resource was `Updating`. +- The failure mode: `rad app delete` retries `409 Conflict / target resource is in progress` indefinitely because the hydrated state store reports a resource in a non-terminal state that never actually existed (or that has since settled underneath). The transcript's specific case was an application whose deployment had failed, leaving nothing in the cloud, while the state store insisted the resource was `Updating`. ### Where the change lives -- The `rad startup` command in [pkg/cli/cmd/startup/startup.go](../../pkg/cli/cmd/startup/startup.go) and its state client in [pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go). A new stage (call it `ReconcileHydrated`) is added there. -- The reconciler enumerates state through the standard UCP list endpoints and queries individual resources via the standard RP GET endpoints. It does not import the resource providers' internal packages. -- No workflow-level change. [restore-state/action.yml](../../.github/extension/actions/restore-state/action.yml) already runs `rad startup`; the reconciliation is transparent. -- No CLI change to `rad app delete`. Once reconciliation runs, the delete sees an accurate state store and proceeds normally. - -### Why regular Radius is not as much affected +- Client-side stage: [pkg/cli/cmd/startup/startup.go](../../pkg/cli/cmd/startup/startup.go) and [pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go). +- Application-scoped orchestrator: `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go` (new), registered in [pkg/corerp/setup/setup.go](../../pkg/corerp/setup/setup.go) beside `getGraph`. +- First per-resource handler (prototype scope): `pkg/corerp/frontend/controller/containers/reconcile.go` (new), registered on the `containers` resource type in the same `setup.go`. +- API surface: additive `reconcile` custom action on `Radius.Core/applications/{name}` and on the participating resource type(s), authored in [typespec/](../../typespec/) and regenerated via `make generate`. -A persistently running Radius control plane has resource providers that reconcile their world continuously — the async operation controller polls, the health controller polls, and stuck non-terminal states resolve within the RP's own polling interval. The archive-hydrate topology short-circuits that: the state is loaded from disk and immediately trusted. Reconciliation on hydrate closes that gap only for the ephemeral topology. +Nothing outside these files needs to change. ## Acceptance -- Deleting an application whose state archive contains at least one child resource in a non-terminal `provisioningState` succeeds when the underlying cloud resource does not exist. Today it loops on `409`. -- Deleting an application whose state archive contains a child resource in `Updating` and whose underlying cloud resource genuinely is still updating waits normally and does not falsely succeed. The reconciliation must observe the RP-reported state and leave the store unchanged. +- Deleting an application whose state archive contains at least one child resource in a non-terminal `provisioningState` succeeds when the underlying cloud/Kubernetes resource does not exist. Today it loops on `409`. +- Deleting an application whose state archive contains a child resource in `Updating` and whose underlying resource genuinely is still updating waits normally and does not falsely succeed. The reconciler must observe the reality-reported state and leave the store unchanged. - `rad startup` never fails because reconciliation could not reach a resource provider. The workflow log records the failure and startup returns success. -- The reconciliation pass runs no direct SQL against the resource-provider databases. Every state change goes through the RP, so state machines stay intact. +- The reconciler runs no direct SQL against any resource-provider database. Every state change goes through the RP's normal write path, so state machines stay intact. +- The prototype covers `Radius.Compute/containers` end-to-end: an application containing only containers is deletable after a `rad startup` from an archive that hydrated the containers in `Updating`, whether or not the container objects exist in Kubernetes. ## Follow-up -A separate issue is filed to verify that concurrent `rad app delete` against the same application from two terminals is handled correctly by a regular (persistent) Radius control plane. That case is not affected by hydration — a persistent control plane already tracks in-flight operations — but it needs an explicit test so a future change cannot regress it. See [radius-project/radius#12870](https://github.com/radius-project/radius/issues/12870). +- A separate issue tracks verifying that concurrent `rad app delete` against the same application from two terminals is handled correctly by a regular (persistent) Radius control plane. That case is not affected by hydration — a persistent control plane already tracks in-flight operations — but it needs an explicit test so a future change cannot regress it. See [radius-project/radius#12870](https://github.com/radius-project/radius/issues/12870). +- Per-resource `reconcile` handlers for the remaining resource types (`Radius.Compute/gateways`, `Radius.Compute/secretstores`, and the dynamic-rp Terraform-backed types) are follow-up work. Each is a new handler that registers the same custom action; no framework changes are required to add them. From 37324eae0952953b9a22f94defbd9ee32d1d4729 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Fri, 28 Aug 2026 13:20:58 -0700 Subject: [PATCH 04/18] Clarify 'resource is gone' handling in plan risks Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/006-state-restoration/plan.md b/specs/006-state-restoration/plan.md index e8a41e29f4..3ee6bd4079 100644 --- a/specs/006-state-restoration/plan.md +++ b/specs/006-state-restoration/plan.md @@ -136,5 +136,5 @@ Goal: prove the end-to-end delete flow against a k3d cluster. ## Risks and open questions - **Child enumeration source of truth.** `getGraph` walks children by scanning UCP's `System.Resources/resourceProviders` and listing each type. If a resource type is registered but its RP is unresponsive, the corerp orchestrator must not hang. Timebox each per-child call to a bounded deadline and record the failure in the report. -- **PATCH vs re-PUT semantics on a "gone" resource.** When reality reports 404 and we want to remove the state-store entry, the cleanest path is to issue a synchronous DELETE against the RP through UCP, bypassing normal deletion semantics via a well-scoped flag/header understood only by the RP's `reconcile` handler. Alternative: PATCH `provisioningState` to `Failed` and let the next `rad app delete` clean up naturally. **Open.** Prototype implementation will PATCH to `Failed` (simpler, keeps the delete path in one place); if that produces user-visible weirdness (an `rad app show` reporting `Failed` on a resource that has been deleted from k8s), we revisit. +- **What to write when the underlying resource is gone.** When the reality check returns 404 from Kubernetes, the reconcile handler moves `provisioningState` from `Updating` to `Failed`. It does **not** delete the state-store row. Rationale: once the row is in a terminal state, it no longer blocks the delete path with `409`, and the user's next `rad app delete` runs the normal delete workflow — which will call k8s, receive its own 404, treat it as "already gone", and remove the row. This keeps all cleanup on one code path. The alternative (removing the row here in the reconciler) requires a new "delete without running the delete workflow" bypass on the RP, which is exactly the kind of side-door around the RP state machine we agreed to avoid. Trade-off: between `rad startup` and the next `rad app delete`, `rad app show` will list the container as `Failed` even though there is no k8s object. If that shows up as a real UX problem we revisit; the k8s-based prototype is unlikely to hit it. - **Preview-API sensitivity.** Adding a `Custom` action on the preview surface is additive, but downstream consumers of the generated Go client have to regenerate. There is no public preview SDK release cadence to worry about; internal callers regenerate on the next `make generate`. From 644be73ac3c7f82dc7f7e35d380a293bbc3dfbb5 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 08:36:20 -0700 Subject: [PATCH 05/18] Reconcile dynamic-rp resources only; algorithm and layout updated Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/plan.md | 69 ++++++++++++++++------------- specs/006-state-restoration/spec.md | 40 +++++++---------- 2 files changed, 54 insertions(+), 55 deletions(-) diff --git a/specs/006-state-restoration/plan.md b/specs/006-state-restoration/plan.md index 3ee6bd4079..a9e4f4b837 100644 --- a/specs/006-state-restoration/plan.md +++ b/specs/006-state-restoration/plan.md @@ -4,19 +4,19 @@ ## Summary -Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go)) plus a per-resource-type `reconcile` custom action registered on `Radius.Compute/containers` (prototype scope). `rad startup` gains a fifth stage, `ReconcileHydratedState`, that lists applications after `ScaleUp` and POSTs the app-scoped action for each. The corerp application handler fans out through the UCP proxy to per-resource-type handlers, which query reality (Kubernetes for containers) and rewrite the state store through the RP's normal write path. Best-effort throughout: individual failures never fail `rad startup`. +Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go)) and a per-resource `reconcile` handler in [dynamic-rp](../../pkg/dynamicrp/) served for every dynamic resource type. Legacy `Applications.*` types are out of scope. `rad startup` gains a fifth stage, `ReconcileHydratedState`, that lists applications after `ScaleUp` and POSTs the app-scoped action for each. The corerp application handler fans out through the UCP proxy to dynamic-rp, which runs the CLI-equivalent reality check for each resource — read `properties.status.outputResources`, GET each Kubernetes object, PATCH `provisioningState` to match reality. Terraform-backed cloud outputs record `skipped` and are follow-ups. Best-effort throughout: individual failures never fail `rad startup`. ## Technical Context **Language/Version**: Go 1.26.5 (per `go.mod`) -**Primary Dependencies**: no new external dependencies. Reuses `github.com/Azure/azure-sdk-for-go/sdk/azcore` (async operation), `k8s.io/client-go` (per-container reality check), and the internal `pkg/armrpc/builder` custom-action registration mechanism. +**Primary Dependencies**: no new external dependencies. Reuses `github.com/Azure/azure-sdk-for-go/sdk/azcore` (async operation), `k8s.io/client-go` (dynamic-rp's per-output reality check), and dynamic-rp's existing routing scaffold plus the internal `pkg/armrpc/builder` custom-action mechanism (for the app-scoped orchestrator on corerp). **Storage**: no schema changes. Reconciliation writes go through the RPs' existing state-store paths. -**Testing**: `go test` with `stretchr/testify`; table-driven unit tests for the corerp orchestrator and the containers handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. Existing `rad startup` tests get a new fake for `ReconcileHydratedState`. +**Testing**: `go test` with `stretchr/testify`; table-driven unit tests for the corerp orchestrator and the dynamic-rp `reconcile` handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. Existing `rad startup` tests get a new fake for `ReconcileHydratedState`. **Target Platform**: Radius control plane (Linux server binary) and `rad` CLI (macOS/Linux/Windows). **Project Type**: Single Go module `github.com/radius-project/radius`. -**Performance Goals**: Reconciliation for an application with ≤50 containers must complete within 30 s on the k3d control plane. No hot-path allocations in the containers handler (a k8s GET per container is the dominant cost). -**Constraints**: no direct SQL against RP databases; no boot-time reconciliation in the persistent control plane; opt-in per resource type. -**Scale/Scope**: prototype covers `Radius.Compute/containers` only. Ships as one PR; other types are follow-ups. +**Performance Goals**: Reconciliation for an application with ≤50 resources must complete within 30 s on the k3d control plane. No hot-path allocations in the dynamic-rp handler (a k8s GET per output resource is the dominant cost). +**Constraints**: no direct SQL against RP databases; no boot-time reconciliation in the persistent control plane; legacy `Applications.*` types out of scope. +**Scale/Scope**: prototype covers dynamic-rp resources with Kubernetes `outputResources` (the transcript's failing case). Terraform-backed cloud outputs record `skipped` and are follow-ups. ## Constitution Check @@ -24,14 +24,14 @@ Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of | Principle | Verdict | Note | | ---------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| I. API-First Design | ✅ | Wire change authored in TypeSpec (`typespec/Radius.Core/applications.tsp` and `typespec/Radius.Compute/containers.tsp`); Go models regenerated via `make generate`. | +| I. API-First Design | ✅ | Wire change authored in TypeSpec on `typespec/Radius.Core/applications.tsp` (the app-scoped action). Dynamic types expose `reconcile` through dynamic-rp's routing without per-type TypeSpec. Go models regenerated via `make generate`. | | II. Idiomatic Code Standards | ✅ | `gofmt`, small exported surface, godoc on every exported symbol, table-driven tests. | -| III. Multi-Cloud Neutrality | ✅ | The application-scoped action is cloud-agnostic. Per-resource handlers query their own underlying provider (Kubernetes for the prototype; TF/Azure/AWS for follow-ups). No provider carve-out in the orchestrator. | -| IV. Testing Pyramid Discipline | ✅ | Unit tests for the orchestrator and the containers handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. | +| III. Multi-Cloud Neutrality | ✅ | The application-scoped action is cloud-agnostic. Dynamic-rp iterates each resource's `outputResources` and queries the recorded provider per output (Kubernetes for the prototype; TF/Azure/AWS branches for follow-ups). No provider carve-out in the orchestrator. | +| IV. Testing Pyramid Discipline | ✅ | Unit tests for the corerp orchestrator and the dynamic-rp `reconcile` handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. | | V. Collaboration-Centric Design | ✅ | Fixes an operator-visible failure (delete workflow loops forever) without new user-facing surface — the flag path deliberately not taken. | | VI. Open Source and Community-First | ✅ | Spec and plan authored in the public repo; commits will carry `Signed-off-by`. | | VII. Simplicity Over Cleverness | ✅ | Reuses the existing `Custom` action mechanism, the existing `getGraph` traversal, and the existing async-operation pattern. No new framework code. | -| VIII. Separation of Concerns | ✅ | Orchestrator in corerp, reality-check logic in the per-type handler, transport through UCP. Each layer owns what it already owns. | +| VIII. Separation of Concerns | ✅ | Orchestrator in corerp, reality-check logic in dynamic-rp (one handler, all dynamic types), transport through UCP. Each layer owns what it already owns. | | IX. Incremental Adoption & Backward Compatibility | ✅ | `Radius.Core/2025-08-01-preview` is preview; adding a `Custom` action is additive. `Applications.Core` is not touched. | | XII / XIII (resource type / recipe standards) | N/A | No new resource types or recipes. | | XVII. Polyglot Project Coherence | ✅ | TypeSpec is the single source of truth for the wire; Go generated code follows. | @@ -55,9 +55,8 @@ Additional artifacts (research/data-model/quickstart/tasks) are not required — Additions and edits, all within the existing single Go module: ```text -# TypeSpec — additive custom actions +# TypeSpec — additive app-scoped custom action typespec/Radius.Core/applications.tsp # add `reconcile` action on applications/{name} -typespec/Radius.Compute/containers.tsp # add `reconcile` action on containers/{name} # Regenerated Go models — via `make generate` pkg/corerp/api/v20250801preview/zz_generated_*.go @@ -67,13 +66,16 @@ pkg/corerp/frontend/controller/applications/v20250801preview/ ├── reconcile.go └── reconcile_test.go -# New — per-resource-type reality check (prototype scope) -pkg/corerp/frontend/controller/containers/ +# Edited — register the app-scoped orchestrator +pkg/corerp/setup/setup.go + +# New — per-resource reality-check handler for every dynamic type +pkg/dynamicrp/frontend/ ├── reconcile.go └── reconcile_test.go -# Edited — register the two new custom actions -pkg/corerp/setup/setup.go +# Edited — wire the reconcile route into dynamic-rp's router +pkg/dynamicrp/frontend/routes.go # or the equivalent registration site # Edited — new ReconcileHydratedState stage pkg/cli/cmd/startup/ @@ -97,30 +99,33 @@ Goal: prove the registration, routing, async-operation lifecycle, and `rad start **Exit criterion**: `rad startup` on a k3d cluster with one hydrated `Radius.Core/applications` succeeds and logs `reconciled 0 resources` for it. -### Phase 1 — Implement the containers per-resource `reconcile` +### Phase 1 — Implement the dynamic-rp `reconcile` handler -Goal: reality-check containers against Kubernetes and rewrite the state store. +Goal: reality-check every dynamic resource in the app against its `outputResources` and rewrite the state store. -- Add the `reconcile` custom action on `Radius.Compute/containers/{name}` in TypeSpec; regenerate. -- Implement `pkg/corerp/frontend/controller/containers/reconcile.go`: - - Look up the container record. +- Register a `reconcile` route in dynamic-rp for every resource type it serves. One handler, no per-type registration. +- Implement `pkg/dynamicrp/frontend/reconcile.go`: + - Look up the resource record. - If `provisioningState` is terminal, return unchanged. - - Read the underlying Kubernetes object (Deployment) via the RP's existing k8s client. Interpret 404, terminal, non-terminal per the [reality table](spec.md#what-reality-is-for-each-resource). - - Write the outcome back through the container RP's normal PATCH path. - - Return the new state. -- Register the action in `pkg/corerp/setup/setup.go` under the `containers` resource block. -- Update the corerp orchestrator from Phase 0 to actually walk children, filter to non-terminal, POST `reconcile` to each container in parallel (bounded), collect responses, reconcile the application record from the aggregated child states, return a populated report. -- Unit tests: containers handler with a fake k8s client (three cases from the reality table); orchestrator with a fake UCP connection and multiple child responses. -- Integration test: mount the whole thing behind `httptest` — application with two containers, one 404 in k8s and one healthy — verify the app-scoped POST returns a report and the state store reflects reality. - -**Exit criterion**: the [Acceptance](spec.md#acceptance) prototype criterion holds — an application whose containers were hydrated in `Updating` but do not exist in Kubernetes is deletable after `rad startup`. + - Read `properties.status.outputResources`. + - For each output: + - Kubernetes object → GET via the target-cluster Kubernetes client the RP already holds. 404 → gone; terminal → settled; non-terminal → still updating. + - Non-Kubernetes (Terraform-backed cloud output) → skip; record `skipped: cloud output not yet reality-checked` in the per-output report. + - Aggregate per the reality table: if every output is gone → PATCH `provisioningState=Failed`; if every output is settled OK → leave `Succeeded`; if any output is still transitioning → leave `provisioningState` unchanged. + - Write the outcome through dynamic-rp's normal PATCH path. + - Return the new state and the per-output report. +- Update the corerp orchestrator from Phase 0 to actually walk children (the same list-per-registered-type walk `getGraph` uses, restricted to dynamic resource providers), filter to non-terminal, POST `reconcile` to each resource in parallel (bounded), collect responses, reconcile the application record from the aggregated child states, return a populated report. +- Unit tests: dynamic-rp handler with a fake k8s client (three cases from the reality table plus a `skipped` case for a cloud output); orchestrator with a fake UCP connection and multiple child responses. +- Integration test: mount the whole thing behind `httptest` — application with two `Radius.Compute/containers`, one 404 in k8s and one healthy — verify the app-scoped POST returns a report and the state store reflects reality. + +**Exit criterion**: the [Acceptance](spec.md#acceptance) prototype criterion holds — an application whose dynamic-rp resources were hydrated in `Updating` but whose Kubernetes `outputResources` do not exist is deletable after `rad startup`. ### Phase 2 — Integration and functional coverage Goal: prove the end-to-end delete flow against a k3d cluster. - Extend `test/functional/` with a case that: - 1. Seeds a state archive with an application whose container is `Updating` and whose k8s Deployment does not exist. + 1. Seeds a state archive with an application whose `Radius.Compute/containers` resource is `Updating` and whose k8s Deployment does not exist. 2. Runs `rad startup` against a fresh k3d cluster loaded with that archive. 3. Runs `rad app delete --yes --preview` and asserts it succeeds (no 409 loop). - Verify no direct SQL calls in the reconciler code path via a `grep_search`-style CI check (informational). @@ -135,6 +140,6 @@ Goal: prove the end-to-end delete flow against a k3d cluster. ## Risks and open questions -- **Child enumeration source of truth.** `getGraph` walks children by scanning UCP's `System.Resources/resourceProviders` and listing each type. If a resource type is registered but its RP is unresponsive, the corerp orchestrator must not hang. Timebox each per-child call to a bounded deadline and record the failure in the report. +- **Child enumeration source of truth.** `getGraph` walks children by scanning UCP's `System.Resources/resourceProviders` and listing each type. Corerp's `reconcile` orchestrator does the same walk but restricts to dynamic-rp types (the only ones whose reality-check handler is implemented in Phase 1). If a resource type is registered but its RP is unresponsive, the orchestrator must not hang. Timebox each per-child call to a bounded deadline and record the failure in the report. - **What to write when the underlying resource is gone.** When the reality check returns 404 from Kubernetes, the reconcile handler moves `provisioningState` from `Updating` to `Failed`. It does **not** delete the state-store row. Rationale: once the row is in a terminal state, it no longer blocks the delete path with `409`, and the user's next `rad app delete` runs the normal delete workflow — which will call k8s, receive its own 404, treat it as "already gone", and remove the row. This keeps all cleanup on one code path. The alternative (removing the row here in the reconciler) requires a new "delete without running the delete workflow" bypass on the RP, which is exactly the kind of side-door around the RP state machine we agreed to avoid. Trade-off: between `rad startup` and the next `rad app delete`, `rad app show` will list the container as `Failed` even though there is no k8s object. If that shows up as a real UX problem we revisit; the k8s-based prototype is unlikely to hit it. - **Preview-API sensitivity.** Adding a `Custom` action on the preview surface is additive, but downstream consumers of the generated Go client have to regenerate. There is no public preview SDK release cadence to worry about; internal callers regenerate on the next `make generate`. diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 615b5bd398..2c7bdff6d1 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -61,28 +61,22 @@ Concretely, the handler: UCP is not a smart orchestrator here — it is the proxy layer that already routes `/planes/…/providers/{ns}/…` to the owning RP. That is enough. "UCP asks every RP" is satisfied by construction because every per-child call goes through UCP. -### Per-resource-type `reconcile` custom action, opt-in - -Each resource type that participates registers `reconcile` as a `Custom` action alongside the type's existing `Put`/`Patch`/`Delete` operations. Registration is the same three-line addition `getGraph` uses on the application resource; nothing new in the armrpc builder. - -```go -Custom: map[string]builder.Operation[datamodel.ContainerResource]{ - "reconcile": { - APIController: func(opt apictrl.Options) (apictrl.Controller, error) { - return ctr_ctrl.NewReconcile(opt) - }, - }, -}, -``` +### Per-resource `reconcile` handled by dynamic-rp + +Legacy `Applications.Core/*` / `Applications.Datastores/*` / `Applications.Dapr/*` / `Applications.Messaging/*` types are out of scope. Only the dynamic types the modern Radius application model uses are reconciled — `Radius.Compute/containers`, `Radius.Compute/gateways`, and every community-contributed type served by [dynamic-rp](../../pkg/dynamicrp/). -The per-resource handler is the only component that knows how to check reality for its type. It: +Dynamic-rp implements `reconcile` once and serves it for every dynamic type — no per-type registration, no per-type TypeSpec. The handler runs the algorithm you would otherwise run from the CLI (`rad resource list -a --preview`, then check each resource): list the app's resources, check each one's underlying provider, PATCH state to match. -- Queries the underlying provider (the Kubernetes API for containers/gateways/secretstores in corerp; the recipe engine's provider client for dynamic-rp types backed by Terraform / Azure / AWS). -- Interprets the response according to the table in [What "reality" is for each resource](#what-reality-is-for-each-resource). -- Writes the result back through the RP's normal state-store path — the same `PATCH` code path the async operation controller uses. **No direct SQL.** -- Returns the new state, or an error the corerp orchestrator will record in the report. +For a single resource, the dynamic-rp handler: -A resource type that has not opted in is skipped by the corerp orchestrator with a warning in the report. Opt-in is per-type so we can ship the fix incrementally (see [Acceptance](#acceptance) — `Radius.Compute/containers` first). +1. Loads the resource from dynamic-rp's own store. +2. If `provisioningState` is terminal, returns unchanged. +3. Reads the resource's `properties.status.outputResources` — the concrete backing objects the recipe engine recorded when the resource was deployed. +4. For each output resource, queries its underlying provider: + - Kubernetes objects → GET via the target-cluster Kubernetes client the RP already holds. + - Terraform-backed cloud outputs (Azure/AWS resource IDs) → **out of scope for the prototype**; record `skipped: cloud output not yet reality-checked` in the per-output report. Follow-up work adds the cloud-SDK branches inside the same handler. +5. Aggregates outcomes per the table in [What "reality" is for each resource](#what-reality-is-for-each-resource) and writes the result back through dynamic-rp's normal `PATCH` code path. **No direct SQL.** +6. Returns the new state, or an error the corerp orchestrator will record in the report. ### What "reality" is for each resource @@ -127,8 +121,8 @@ The `reconcile` action is dormant unless something calls it. Regular Radius neve - Client-side stage: [pkg/cli/cmd/startup/startup.go](../../pkg/cli/cmd/startup/startup.go) and [pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go). - Application-scoped orchestrator: `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go` (new), registered in [pkg/corerp/setup/setup.go](../../pkg/corerp/setup/setup.go) beside `getGraph`. -- First per-resource handler (prototype scope): `pkg/corerp/frontend/controller/containers/reconcile.go` (new), registered on the `containers` resource type in the same `setup.go`. -- API surface: additive `reconcile` custom action on `Radius.Core/applications/{name}` and on the participating resource type(s), authored in [typespec/](../../typespec/) and regenerated via `make generate`. +- Per-resource handler for every dynamic type: `pkg/dynamicrp/frontend/reconcile.go` (new), wired into dynamic-rp's existing router. One implementation serves every registered dynamic type. +- API surface: additive `reconcile` custom action on `Radius.Core/applications/{name}`, authored in [typespec/](../../typespec/) and regenerated via `make generate`. Dynamic types do not need per-type TypeSpec — dynamic-rp exposes the action once for every type it serves. Nothing outside these files needs to change. @@ -138,9 +132,9 @@ Nothing outside these files needs to change. - Deleting an application whose state archive contains a child resource in `Updating` and whose underlying resource genuinely is still updating waits normally and does not falsely succeed. The reconciler must observe the reality-reported state and leave the store unchanged. - `rad startup` never fails because reconciliation could not reach a resource provider. The workflow log records the failure and startup returns success. - The reconciler runs no direct SQL against any resource-provider database. Every state change goes through the RP's normal write path, so state machines stay intact. -- The prototype covers `Radius.Compute/containers` end-to-end: an application containing only containers is deletable after a `rad startup` from an archive that hydrated the containers in `Updating`, whether or not the container objects exist in Kubernetes. +- The prototype covers dynamic-rp resources with Kubernetes `outputResources` end-to-end (the transcript's failing case): an application whose containers/gateways/secretstores were hydrated in `Updating` is deletable after `rad startup`, whether or not the underlying Kubernetes objects still exist. Terraform-backed cloud outputs record `skipped` and are follow-ups. ## Follow-up - A separate issue tracks verifying that concurrent `rad app delete` against the same application from two terminals is handled correctly by a regular (persistent) Radius control plane. That case is not affected by hydration — a persistent control plane already tracks in-flight operations — but it needs an explicit test so a future change cannot regress it. See [radius-project/radius#12870](https://github.com/radius-project/radius/issues/12870). -- Per-resource `reconcile` handlers for the remaining resource types (`Radius.Compute/gateways`, `Radius.Compute/secretstores`, and the dynamic-rp Terraform-backed types) are follow-up work. Each is a new handler that registers the same custom action; no framework changes are required to add them. +- Reality-checking Terraform-backed cloud `outputResources` (Azure/AWS resources managed via recipes) is follow-up work. Adds cloud-SDK branches inside the same dynamic-rp `reconcile` handler; no framework changes. From a08e2f4f165610a1b71dde1492c964d27d038196 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 08:56:02 -0700 Subject: [PATCH 06/18] Trim spec purpose/non-goals; add 409 error example Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/spec.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 2c7bdff6d1..aee8705210 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -13,18 +13,28 @@ That is not sufficient when the previous run was interrupted while a resource wa This feature adds a reconciliation pass triggered by `rad startup` and executed against the running control plane: for every application in the plane, an application-scoped `reconcile` action asks each resource's owning resource provider to check its actual current state and rewrite the state store to match reality — including removing entries when the underlying resource does not exist. -The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. It does not change the delete command, does not add a `--force` flag, and does not touch the regular (non-Repo) Radius runtime — the action exists but is never invoked there. +The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. ## Non-goals -- **A `rad app delete --force` flag was considered and explicitly rejected.** With two concurrent deletes (for example, a user re-runs `rad app delete` after their terminal died), a force option that bypasses state can convert an in-progress happy-path delete into a broken one by overwriting the state store while the first delete is still driving to a terminal state. Fixing hydration removes the need for the flag. -- **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it, do not warn. +- **A `rad app delete --force` flag was considered and explicitly rejected.** A force option that bypasses state can convert an in-progress happy-path delete into a broken one by overwriting the state store while the first delete is still driving to a terminal state. Fixing hydration is the right approach. +- **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it. The users will continue to see the rror message we see today: +``` +RESPONSE 409: 409 Conflict +ERROR CODE: Conflict +{ + "error": { + "code": "Conflict", + "message": "The target resource is in progress state: Updating." + } +} +``` - **The persistent Radius control plane's async controllers are not changed.** They already reconcile continuously; the new action is dormant unless `rad startup` (or a test) invokes it. -- **The concurrent-`rad app delete` behavior is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). +- **The concurrent-`rad app delete` behavior evaluation as part of control plane is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). ## Decisions -### The client-facing endpoint is an application-scoped custom action, mirroring `getGraph` +### The client-facing endpoint is an application-scoped custom action Reconciliation is a per-application operation: walk the application's children, check each one's reality, roll the results back into the state store. That is the same shape as [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go) — an application-scoped custom action registered on `Radius.Core/applications` that walks children across resource providers. `reconcile` therefore reuses the exact pattern, up to and including the corerp orchestrator that already knows how to fan out across RPs through the UCP proxy. From 1af3ea0bb1a8a9cdfe895f4cbe60275220437ea3 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 08:58:00 -0700 Subject: [PATCH 07/18] Fix typo and reword concurrent-delete non-goal in spec Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/spec.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index aee8705210..e84e9517fa 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -18,7 +18,7 @@ The scope is deliberately narrow: reconcile hydrated state so operations that fo ## Non-goals - **A `rad app delete --force` flag was considered and explicitly rejected.** A force option that bypasses state can convert an in-progress happy-path delete into a broken one by overwriting the state store while the first delete is still driving to a terminal state. Fixing hydration is the right approach. -- **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it. The users will continue to see the rror message we see today: +- **No user-facing message when a resource is genuinely still updating.** If reconciliation finds the resource actually is in `Updating` state, the hydrated state is accurate — leave it. Users will continue to see the same error message we surface today: ``` RESPONSE 409: 409 Conflict ERROR CODE: Conflict @@ -30,7 +30,7 @@ ERROR CODE: Conflict } ``` - **The persistent Radius control plane's async controllers are not changed.** They already reconcile continuously; the new action is dormant unless `rad startup` (or a test) invokes it. -- **The concurrent-`rad app delete` behavior evaluation as part of control plane is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). +- **Concurrent `rad app delete` behavior against a regular (persistent) Radius control plane is out of scope** — tracked as a separate follow-up (see [Follow-up](#follow-up)). ## Decisions From 611d6f25cfd7564a7a84127f3c9212935e832d25 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 09:18:35 -0700 Subject: [PATCH 08/18] wip --- specs/006-state-restoration/plan.md | 12 ++++++------ specs/006-state-restoration/spec.md | 6 +++--- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/specs/006-state-restoration/plan.md b/specs/006-state-restoration/plan.md index a9e4f4b837..fe0a69a9c0 100644 --- a/specs/006-state-restoration/plan.md +++ b/specs/006-state-restoration/plan.md @@ -9,7 +9,7 @@ Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of ## Technical Context **Language/Version**: Go 1.26.5 (per `go.mod`) -**Primary Dependencies**: no new external dependencies. Reuses `github.com/Azure/azure-sdk-for-go/sdk/azcore` (async operation), `k8s.io/client-go` (dynamic-rp's per-output reality check), and dynamic-rp's existing routing scaffold plus the internal `pkg/armrpc/builder` custom-action mechanism (for the app-scoped orchestrator on corerp). +**Primary Dependencies**: no new external dependencies. Reuses `k8s.io/client-go` (dynamic-rp's per-output reality check), dynamic-rp's existing routing scaffold, and the internal `pkg/armrpc/builder` custom-action mechanism (for the app-scoped orchestrator on corerp). **Storage**: no schema changes. Reconciliation writes go through the RPs' existing state-store paths. **Testing**: `go test` with `stretchr/testify`; table-driven unit tests for the corerp orchestrator and the dynamic-rp `reconcile` handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. Existing `rad startup` tests get a new fake for `ReconcileHydratedState`. **Target Platform**: Radius control plane (Linux server binary) and `rad` CLI (macOS/Linux/Windows). @@ -30,7 +30,7 @@ Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of | IV. Testing Pyramid Discipline | ✅ | Unit tests for the corerp orchestrator and the dynamic-rp `reconcile` handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. | | V. Collaboration-Centric Design | ✅ | Fixes an operator-visible failure (delete workflow loops forever) without new user-facing surface — the flag path deliberately not taken. | | VI. Open Source and Community-First | ✅ | Spec and plan authored in the public repo; commits will carry `Signed-off-by`. | -| VII. Simplicity Over Cleverness | ✅ | Reuses the existing `Custom` action mechanism, the existing `getGraph` traversal, and the existing async-operation pattern. No new framework code. | +| VII. Simplicity Over Cleverness | ✅ | Reuses the existing `Custom` action mechanism, the existing `getGraph` traversal, and the existing sync custom-action pattern. No new framework code. | | VIII. Separation of Concerns | ✅ | Orchestrator in corerp, reality-check logic in dynamic-rp (one handler, all dynamic types), transport through UCP. Each layer owns what it already owns. | | IX. Incremental Adoption & Backward Compatibility | ✅ | `Radius.Core/2025-08-01-preview` is preview; adding a `Custom` action is additive. `Applications.Core` is not touched. | | XII / XIII (resource type / recipe standards) | N/A | No new resource types or recipes. | @@ -88,14 +88,14 @@ pkg/cli/cmd/startup/ ### Phase 0 — Wire the app-scoped action end to end with a no-op handler -Goal: prove the registration, routing, async-operation lifecycle, and `rad startup` invocation before we do any reality checking. +Goal: prove the registration, routing, sync custom-action response, and `rad startup` invocation before we do any reality checking. - Add the `reconcile` custom action on `Radius.Core/applications/{name}` in TypeSpec; regenerate. -- Implement `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go` as a stub that returns an immediately-succeeded async operation with an empty report. +- Implement `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go` as a stub that returns an empty report inline. - Register the action in `pkg/corerp/setup/setup.go` beside `getGraph`. -- Add `ReconcileHydratedState(ctx, connection)` on `StateRestoreClient` in `pkg/cli/cmd/startup/stateclient.go`. Implementation lists applications, POSTs the action, polls to completion, logs the (empty) report. +- Add `ReconcileHydratedState(ctx, connection)` on `StateRestoreClient` in `pkg/cli/cmd/startup/stateclient.go`. Implementation lists applications, POSTs the action, reads the (empty) report inline, logs it. - Wire the new stage in `pkg/cli/cmd/startup/startup.go` after `ScaleUp`. -- Unit tests: fake `StateRestoreClient` records the call; corerp handler test verifies the async-operation shape. +- Unit tests: fake `StateRestoreClient` records the call; corerp handler test verifies the sync response shape. **Exit criterion**: `rad startup` on a k3d cluster with one hydrated `Radius.Core/applications` succeeds and logs `reconciled 0 resources` for it. diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index e84e9517fa..29b8b779d1 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -46,7 +46,7 @@ Content-Type: application/json - Registered in [pkg/corerp/setup/setup.go](../../pkg/corerp/setup/setup.go) under the `Custom` map on the application resource, next to `getGraph`. - Handler in `pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go`. -- Response is the standard ARM-RPC async pattern: `202 Accepted` with a `Location` header. The client polls to completion. This matches every other write-shaped action in Radius and keeps `rad startup` from holding a synchronous connection open while UCP proxies to many RPs. +- Response is the standard ARM-RPC synchronous pattern: `200 OK` with the report inline. This matches every existing custom action in Radius (`getGraph`, `listSecrets`, `getMetadata`, `join`). The reconcile pass only *reads* underlying providers and writes updated state through the RP's normal PATCH path — it does not provision anything — so total wall time stays bounded and sync is sufficient. If it ever grows too slow for a synchronous connection, we can flip to `ArmResourceActionAsync` in a follow-up without changing the URL. Naming: `reconcile`, lowercase, matches the codebase convention (`getGraph`, `join`, `getmetadata`). Not `refresh`, not `reconcileStatus` — the action is exactly analogous to the RP-internal reconciliation the persistent control plane already does asynchronously. @@ -67,7 +67,7 @@ Concretely, the handler: in parallel (bounded fan-out), through the UCP-fronted connection the handler already has. 5. After every child response returns, reconciles the application record itself: if all children are now terminal, transition the application accordingly; if any child remains non-terminal, leave the application in its hydrated state. -6. Aggregates per-child outcomes into a report and completes the async operation. +6. Aggregates per-child outcomes into a report and returns it inline in the sync response. UCP is not a smart orchestrator here — it is the proxy layer that already routes `/planes/…/providers/{ns}/…` to the owning RP. That is enough. "UCP asks every RP" is satisfied by construction because every per-child call goes through UCP. @@ -110,7 +110,7 @@ This preserves the guarantee that a run can always at least *try* to make progre `rad startup` today performs four stages ([pkg/cli/cmd/startup/stateclient.go](../../pkg/cli/cmd/startup/stateclient.go)): `ScaleDown` → `RestoreDatabases` → `RestoreTerraform` → `ScaleUp`. A fifth stage, `ReconcileHydratedState`, is added after `ScaleUp` and after the resource-provider deployments are ready to serve. It: 1. Lists applications in the plane through UCP. -2. For each application, POSTs `.../applications/{app}/reconcile` and polls the async operation to completion (with a bounded timeout). +2. For each application, POSTs `.../applications/{app}/reconcile` (with a bounded per-application timeout) and reads the report inline from the response. 3. Logs the per-application report. 4. Always returns success. From 80a8e2ed863e9531c56e2d4478a1bcc79ee4327d Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 09:22:09 -0700 Subject: [PATCH 09/18] regen: refresh Radius.Core/v20250801preview clients from current TypeSpec toolchain No schema change; picks up emitter drift accumulated in typespec-go since the last regen. Isolated from the reconcile action addition that follows so each commit is independently reviewable. Verified with 'go build ./...' and existing corerp application-controller tests. Signed-off-by: Nithya Subramanian --- .../zz_generated_applications_client.go | 95 +++++---- .../zz_generated_bicepsettings_client.go | 83 ++++---- .../zz_generated_environments_client.go | 83 ++++---- .../v20250801preview/zz_generated_models.go | 14 +- .../zz_generated_models_serde.go | 188 +++++++++--------- .../zz_generated_operations_client.go | 38 ++-- .../zz_generated_recipepacks_client.go | 83 ++++---- .../zz_generated_responses.go | 42 ++-- .../zz_generated_terraformsettings_client.go | 83 ++++---- 9 files changed, 325 insertions(+), 384 deletions(-) diff --git a/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go b/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go index a0840bf957..658ff8ccde 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go @@ -56,7 +56,12 @@ func (client *ApplicationsClient) CreateOrUpdate(ctx context.Context, rootScope if err != nil { return ApplicationsClientCreateOrUpdateResponse{}, err } - return client.createOrUpdateHandleResponse(httpResp, http.StatusOK, http.StatusCreated) + if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusCreated) { + err = runtime.NewResponseError(httpResp) + return ApplicationsClientCreateOrUpdateResponse{}, err + } + resp, err := client.createOrUpdateHandleResponse(httpResp) + return resp, err } // createOrUpdateCreateRequest creates the CreateOrUpdate request. @@ -86,11 +91,8 @@ func (client *ApplicationsClient) createOrUpdateCreateRequest(ctx context.Contex } // createOrUpdateHandleResponse handles the CreateOrUpdate response. -func (client *ApplicationsClient) createOrUpdateHandleResponse(resp *http.Response, successCodes ...int) (ApplicationsClientCreateOrUpdateResponse, error) { +func (client *ApplicationsClient) createOrUpdateHandleResponse(resp *http.Response) (ApplicationsClientCreateOrUpdateResponse, error) { result := ApplicationsClientCreateOrUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.ApplicationResource); err != nil { return ApplicationsClientCreateOrUpdateResponse{}, err } @@ -115,7 +117,8 @@ func (client *ApplicationsClient) Delete(ctx context.Context, rootScope string, return ApplicationsClientDeleteResponse{}, err } if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusNoContent) { - return ApplicationsClientDeleteResponse{}, runtime.NewResponseError(httpResp) + err = runtime.NewResponseError(httpResp) + return ApplicationsClientDeleteResponse{}, err } return ApplicationsClientDeleteResponse{}, nil } @@ -158,7 +161,12 @@ func (client *ApplicationsClient) Get(ctx context.Context, rootScope string, app if err != nil { return ApplicationsClientGetResponse{}, err } - return client.getHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return ApplicationsClientGetResponse{}, err + } + resp, err := client.getHandleResponse(httpResp) + return resp, err } // getCreateRequest creates the Get request. @@ -184,11 +192,8 @@ func (client *ApplicationsClient) getCreateRequest(ctx context.Context, rootScop } // getHandleResponse handles the Get response. -func (client *ApplicationsClient) getHandleResponse(resp *http.Response, successCodes ...int) (ApplicationsClientGetResponse, error) { +func (client *ApplicationsClient) getHandleResponse(resp *http.Response) (ApplicationsClientGetResponse, error) { result := ApplicationsClientGetResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.ApplicationResource); err != nil { return ApplicationsClientGetResponse{}, err } @@ -213,7 +218,12 @@ func (client *ApplicationsClient) GetGraph(ctx context.Context, rootScope string if err != nil { return ApplicationsClientGetGraphResponse{}, err } - return client.getGraphHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return ApplicationsClientGetGraphResponse{}, err + } + resp, err := client.getGraphHandleResponse(httpResp) + return resp, err } // getGraphCreateRequest creates the GetGraph request. @@ -243,11 +253,8 @@ func (client *ApplicationsClient) getGraphCreateRequest(ctx context.Context, roo } // getGraphHandleResponse handles the GetGraph response. -func (client *ApplicationsClient) getGraphHandleResponse(resp *http.Response, successCodes ...int) (ApplicationsClientGetGraphResponse, error) { +func (client *ApplicationsClient) getGraphHandleResponse(resp *http.Response) (ApplicationsClientGetGraphResponse, error) { result := ApplicationsClientGetGraphResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.ApplicationGraphResponse); err != nil { return ApplicationsClientGetGraphResponse{}, err } @@ -270,52 +277,38 @@ func (client *ApplicationsClient) NewListByScopePager(rootScope string, options if page != nil { nextLink = *page.NextLink } - req, err := client.listByScopeCreateRequest(ctx, rootScope, nextLink, options) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listByScopeCreateRequest(ctx, rootScope, options) + }, nil) if err != nil { return ApplicationsClientListByScopeResponse{}, err } - resp, err := client.internal.Pipeline().Do(req) - if err != nil { - return ApplicationsClientListByScopeResponse{}, err - } - return client.listByScopeHandleResponse(resp, http.StatusOK) + return client.listByScopeHandleResponse(resp) }, }) } // listByScopeCreateRequest creates the ListByScope request. -func (client *ApplicationsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, nextLink string, _ *ApplicationsClientListByScopeOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/{rootScope}/providers/Radius.Core/applications" - if rootScope == "" { - return nil, errors.New("parameter rootScope cannot be empty") - } - urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) +func (client *ApplicationsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, _ *ApplicationsClientListByScopeOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/applications" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listByScopeHandleResponse handles the ListByScope response. -func (client *ApplicationsClient) listByScopeHandleResponse(resp *http.Response, successCodes ...int) (ApplicationsClientListByScopeResponse, error) { +func (client *ApplicationsClient) listByScopeHandleResponse(resp *http.Response) (ApplicationsClientListByScopeResponse, error) { result := ApplicationsClientListByScopeResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.ApplicationResourceListResult); err != nil { return ApplicationsClientListByScopeResponse{}, err } @@ -340,7 +333,12 @@ func (client *ApplicationsClient) Update(ctx context.Context, rootScope string, if err != nil { return ApplicationsClientUpdateResponse{}, err } - return client.updateHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return ApplicationsClientUpdateResponse{}, err + } + resp, err := client.updateHandleResponse(httpResp) + return resp, err } // updateCreateRequest creates the Update request. @@ -370,11 +368,8 @@ func (client *ApplicationsClient) updateCreateRequest(ctx context.Context, rootS } // updateHandleResponse handles the Update response. -func (client *ApplicationsClient) updateHandleResponse(resp *http.Response, successCodes ...int) (ApplicationsClientUpdateResponse, error) { +func (client *ApplicationsClient) updateHandleResponse(resp *http.Response) (ApplicationsClientUpdateResponse, error) { result := ApplicationsClientUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.ApplicationResource); err != nil { return ApplicationsClientUpdateResponse{}, err } diff --git a/pkg/corerp/api/v20250801preview/zz_generated_bicepsettings_client.go b/pkg/corerp/api/v20250801preview/zz_generated_bicepsettings_client.go index 59c3b4e056..aaa90d453b 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_bicepsettings_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_bicepsettings_client.go @@ -56,7 +56,12 @@ func (client *BicepSettingsClient) CreateOrUpdate(ctx context.Context, rootScope if err != nil { return BicepSettingsClientCreateOrUpdateResponse{}, err } - return client.createOrUpdateHandleResponse(httpResp, http.StatusOK, http.StatusCreated) + if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusCreated) { + err = runtime.NewResponseError(httpResp) + return BicepSettingsClientCreateOrUpdateResponse{}, err + } + resp, err := client.createOrUpdateHandleResponse(httpResp) + return resp, err } // createOrUpdateCreateRequest creates the CreateOrUpdate request. @@ -86,11 +91,8 @@ func (client *BicepSettingsClient) createOrUpdateCreateRequest(ctx context.Conte } // createOrUpdateHandleResponse handles the CreateOrUpdate response. -func (client *BicepSettingsClient) createOrUpdateHandleResponse(resp *http.Response, successCodes ...int) (BicepSettingsClientCreateOrUpdateResponse, error) { +func (client *BicepSettingsClient) createOrUpdateHandleResponse(resp *http.Response) (BicepSettingsClientCreateOrUpdateResponse, error) { result := BicepSettingsClientCreateOrUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.BicepSettingsResource); err != nil { return BicepSettingsClientCreateOrUpdateResponse{}, err } @@ -115,7 +117,8 @@ func (client *BicepSettingsClient) Delete(ctx context.Context, rootScope string, return BicepSettingsClientDeleteResponse{}, err } if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusNoContent) { - return BicepSettingsClientDeleteResponse{}, runtime.NewResponseError(httpResp) + err = runtime.NewResponseError(httpResp) + return BicepSettingsClientDeleteResponse{}, err } return BicepSettingsClientDeleteResponse{}, nil } @@ -158,7 +161,12 @@ func (client *BicepSettingsClient) Get(ctx context.Context, rootScope string, bi if err != nil { return BicepSettingsClientGetResponse{}, err } - return client.getHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return BicepSettingsClientGetResponse{}, err + } + resp, err := client.getHandleResponse(httpResp) + return resp, err } // getCreateRequest creates the Get request. @@ -184,11 +192,8 @@ func (client *BicepSettingsClient) getCreateRequest(ctx context.Context, rootSco } // getHandleResponse handles the Get response. -func (client *BicepSettingsClient) getHandleResponse(resp *http.Response, successCodes ...int) (BicepSettingsClientGetResponse, error) { +func (client *BicepSettingsClient) getHandleResponse(resp *http.Response) (BicepSettingsClientGetResponse, error) { result := BicepSettingsClientGetResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.BicepSettingsResource); err != nil { return BicepSettingsClientGetResponse{}, err } @@ -211,52 +216,38 @@ func (client *BicepSettingsClient) NewListByScopePager(rootScope string, options if page != nil { nextLink = *page.NextLink } - req, err := client.listByScopeCreateRequest(ctx, rootScope, nextLink, options) - if err != nil { - return BicepSettingsClientListByScopeResponse{}, err - } - resp, err := client.internal.Pipeline().Do(req) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listByScopeCreateRequest(ctx, rootScope, options) + }, nil) if err != nil { return BicepSettingsClientListByScopeResponse{}, err } - return client.listByScopeHandleResponse(resp, http.StatusOK) + return client.listByScopeHandleResponse(resp) }, }) } // listByScopeCreateRequest creates the ListByScope request. -func (client *BicepSettingsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, nextLink string, _ *BicepSettingsClientListByScopeOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/{rootScope}/providers/Radius.Core/bicepSettings" - if rootScope == "" { - return nil, errors.New("parameter rootScope cannot be empty") - } - urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) +func (client *BicepSettingsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, _ *BicepSettingsClientListByScopeOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/bicepSettings" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listByScopeHandleResponse handles the ListByScope response. -func (client *BicepSettingsClient) listByScopeHandleResponse(resp *http.Response, successCodes ...int) (BicepSettingsClientListByScopeResponse, error) { +func (client *BicepSettingsClient) listByScopeHandleResponse(resp *http.Response) (BicepSettingsClientListByScopeResponse, error) { result := BicepSettingsClientListByScopeResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.BicepSettingsResourceListResult); err != nil { return BicepSettingsClientListByScopeResponse{}, err } @@ -281,7 +272,12 @@ func (client *BicepSettingsClient) Update(ctx context.Context, rootScope string, if err != nil { return BicepSettingsClientUpdateResponse{}, err } - return client.updateHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return BicepSettingsClientUpdateResponse{}, err + } + resp, err := client.updateHandleResponse(httpResp) + return resp, err } // updateCreateRequest creates the Update request. @@ -311,11 +307,8 @@ func (client *BicepSettingsClient) updateCreateRequest(ctx context.Context, root } // updateHandleResponse handles the Update response. -func (client *BicepSettingsClient) updateHandleResponse(resp *http.Response, successCodes ...int) (BicepSettingsClientUpdateResponse, error) { +func (client *BicepSettingsClient) updateHandleResponse(resp *http.Response) (BicepSettingsClientUpdateResponse, error) { result := BicepSettingsClientUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.BicepSettingsResource); err != nil { return BicepSettingsClientUpdateResponse{}, err } diff --git a/pkg/corerp/api/v20250801preview/zz_generated_environments_client.go b/pkg/corerp/api/v20250801preview/zz_generated_environments_client.go index 2173a85456..506570c9c5 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_environments_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_environments_client.go @@ -56,7 +56,12 @@ func (client *EnvironmentsClient) CreateOrUpdate(ctx context.Context, rootScope if err != nil { return EnvironmentsClientCreateOrUpdateResponse{}, err } - return client.createOrUpdateHandleResponse(httpResp, http.StatusOK, http.StatusCreated) + if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusCreated) { + err = runtime.NewResponseError(httpResp) + return EnvironmentsClientCreateOrUpdateResponse{}, err + } + resp, err := client.createOrUpdateHandleResponse(httpResp) + return resp, err } // createOrUpdateCreateRequest creates the CreateOrUpdate request. @@ -86,11 +91,8 @@ func (client *EnvironmentsClient) createOrUpdateCreateRequest(ctx context.Contex } // createOrUpdateHandleResponse handles the CreateOrUpdate response. -func (client *EnvironmentsClient) createOrUpdateHandleResponse(resp *http.Response, successCodes ...int) (EnvironmentsClientCreateOrUpdateResponse, error) { +func (client *EnvironmentsClient) createOrUpdateHandleResponse(resp *http.Response) (EnvironmentsClientCreateOrUpdateResponse, error) { result := EnvironmentsClientCreateOrUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.EnvironmentResource); err != nil { return EnvironmentsClientCreateOrUpdateResponse{}, err } @@ -115,7 +117,8 @@ func (client *EnvironmentsClient) Delete(ctx context.Context, rootScope string, return EnvironmentsClientDeleteResponse{}, err } if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusNoContent) { - return EnvironmentsClientDeleteResponse{}, runtime.NewResponseError(httpResp) + err = runtime.NewResponseError(httpResp) + return EnvironmentsClientDeleteResponse{}, err } return EnvironmentsClientDeleteResponse{}, nil } @@ -158,7 +161,12 @@ func (client *EnvironmentsClient) Get(ctx context.Context, rootScope string, env if err != nil { return EnvironmentsClientGetResponse{}, err } - return client.getHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return EnvironmentsClientGetResponse{}, err + } + resp, err := client.getHandleResponse(httpResp) + return resp, err } // getCreateRequest creates the Get request. @@ -184,11 +192,8 @@ func (client *EnvironmentsClient) getCreateRequest(ctx context.Context, rootScop } // getHandleResponse handles the Get response. -func (client *EnvironmentsClient) getHandleResponse(resp *http.Response, successCodes ...int) (EnvironmentsClientGetResponse, error) { +func (client *EnvironmentsClient) getHandleResponse(resp *http.Response) (EnvironmentsClientGetResponse, error) { result := EnvironmentsClientGetResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.EnvironmentResource); err != nil { return EnvironmentsClientGetResponse{}, err } @@ -211,52 +216,38 @@ func (client *EnvironmentsClient) NewListByScopePager(rootScope string, options if page != nil { nextLink = *page.NextLink } - req, err := client.listByScopeCreateRequest(ctx, rootScope, nextLink, options) - if err != nil { - return EnvironmentsClientListByScopeResponse{}, err - } - resp, err := client.internal.Pipeline().Do(req) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listByScopeCreateRequest(ctx, rootScope, options) + }, nil) if err != nil { return EnvironmentsClientListByScopeResponse{}, err } - return client.listByScopeHandleResponse(resp, http.StatusOK) + return client.listByScopeHandleResponse(resp) }, }) } // listByScopeCreateRequest creates the ListByScope request. -func (client *EnvironmentsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, nextLink string, _ *EnvironmentsClientListByScopeOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/{rootScope}/providers/Radius.Core/environments" - if rootScope == "" { - return nil, errors.New("parameter rootScope cannot be empty") - } - urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) +func (client *EnvironmentsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, _ *EnvironmentsClientListByScopeOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/environments" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listByScopeHandleResponse handles the ListByScope response. -func (client *EnvironmentsClient) listByScopeHandleResponse(resp *http.Response, successCodes ...int) (EnvironmentsClientListByScopeResponse, error) { +func (client *EnvironmentsClient) listByScopeHandleResponse(resp *http.Response) (EnvironmentsClientListByScopeResponse, error) { result := EnvironmentsClientListByScopeResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.EnvironmentResourceListResult); err != nil { return EnvironmentsClientListByScopeResponse{}, err } @@ -281,7 +272,12 @@ func (client *EnvironmentsClient) Update(ctx context.Context, rootScope string, if err != nil { return EnvironmentsClientUpdateResponse{}, err } - return client.updateHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return EnvironmentsClientUpdateResponse{}, err + } + resp, err := client.updateHandleResponse(httpResp) + return resp, err } // updateCreateRequest creates the Update request. @@ -311,11 +307,8 @@ func (client *EnvironmentsClient) updateCreateRequest(ctx context.Context, rootS } // updateHandleResponse handles the Update response. -func (client *EnvironmentsClient) updateHandleResponse(resp *http.Response, successCodes ...int) (EnvironmentsClientUpdateResponse, error) { +func (client *EnvironmentsClient) updateHandleResponse(resp *http.Response) (EnvironmentsClientUpdateResponse, error) { result := EnvironmentsClientUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.EnvironmentResource); err != nil { return EnvironmentsClientUpdateResponse{}, err } diff --git a/pkg/corerp/api/v20250801preview/zz_generated_models.go b/pkg/corerp/api/v20250801preview/zz_generated_models.go index 1beef48056..1eede80cd5 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_models.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_models.go @@ -369,14 +369,12 @@ type EnvironmentProperties struct { // that platform engineers configure for their developers. Every Radius Application is deployed to an Environment through // its `environment` property. // An Environment defines three things for the Applications deployed to it: -// -// - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` -// property. -// - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, -// set through the `recipePacks` property. -// - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration -// applied when Recipes run. -// +// - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` +// property. +// - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, +// set through the `recipePacks` property. +// - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration +// applied when Recipes run. // ## Defining an Environment // The simplest Environment can be created directly with the `rad environment create` command, without a Bicep file: // ```bash diff --git a/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go b/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go index 5dacd4d50f..4fde33d786 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go @@ -25,7 +25,7 @@ func (a ApplicationGraphConnection) MarshalJSON() ([]byte, error) { func (a *ApplicationGraphConnection) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -41,7 +41,7 @@ func (a *ApplicationGraphConnection) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -61,7 +61,7 @@ func (a ApplicationGraphOutputResource) MarshalJSON() ([]byte, error) { func (a *ApplicationGraphOutputResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -80,7 +80,7 @@ func (a *ApplicationGraphOutputResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -105,7 +105,7 @@ func (a ApplicationGraphResource) MarshalJSON() ([]byte, error) { func (a *ApplicationGraphResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -139,7 +139,7 @@ func (a *ApplicationGraphResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -157,7 +157,7 @@ func (a ApplicationGraphResponse) MarshalJSON() ([]byte, error) { func (a *ApplicationGraphResponse) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -170,7 +170,7 @@ func (a *ApplicationGraphResponse) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -189,7 +189,7 @@ func (a ApplicationProperties) MarshalJSON() ([]byte, error) { func (a *ApplicationProperties) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -205,7 +205,7 @@ func (a *ApplicationProperties) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -228,7 +228,7 @@ func (a ApplicationResource) MarshalJSON() ([]byte, error) { func (a *ApplicationResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -256,7 +256,7 @@ func (a *ApplicationResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -274,7 +274,7 @@ func (a ApplicationResourceListResult) MarshalJSON() ([]byte, error) { func (a *ApplicationResourceListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -287,7 +287,7 @@ func (a *ApplicationResourceListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -307,7 +307,7 @@ func (a AzureContainerInstanceCompute) MarshalJSON() ([]byte, error) { func (a *AzureContainerInstanceCompute) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } for key, val := range rawMsg { var err error @@ -326,7 +326,7 @@ func (a *AzureContainerInstanceCompute) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", a, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", a, err) } } return nil @@ -347,7 +347,7 @@ func (b BicepRegistryAuthentication) MarshalJSON() ([]byte, error) { func (b *BicepRegistryAuthentication) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } for key, val := range rawMsg { var err error @@ -369,7 +369,7 @@ func (b *BicepRegistryAuthentication) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } } return nil @@ -388,7 +388,7 @@ func (b BicepSettingsProperties) MarshalJSON() ([]byte, error) { func (b *BicepSettingsProperties) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } for key, val := range rawMsg { var err error @@ -404,7 +404,7 @@ func (b *BicepSettingsProperties) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } } return nil @@ -427,7 +427,7 @@ func (b BicepSettingsResource) MarshalJSON() ([]byte, error) { func (b *BicepSettingsResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } for key, val := range rawMsg { var err error @@ -455,7 +455,7 @@ func (b *BicepSettingsResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } } return nil @@ -473,7 +473,7 @@ func (b BicepSettingsResourceListResult) MarshalJSON() ([]byte, error) { func (b *BicepSettingsResourceListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } for key, val := range rawMsg { var err error @@ -486,7 +486,7 @@ func (b *BicepSettingsResourceListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", b, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", b, err) } } return nil @@ -505,7 +505,7 @@ func (e EnvironmentCompute) MarshalJSON() ([]byte, error) { func (e *EnvironmentCompute) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } for key, val := range rawMsg { var err error @@ -521,7 +521,7 @@ func (e *EnvironmentCompute) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } } return nil @@ -544,7 +544,7 @@ func (e EnvironmentProperties) MarshalJSON() ([]byte, error) { func (e *EnvironmentProperties) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } for key, val := range rawMsg { var err error @@ -572,7 +572,7 @@ func (e *EnvironmentProperties) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } } return nil @@ -595,7 +595,7 @@ func (e EnvironmentResource) MarshalJSON() ([]byte, error) { func (e *EnvironmentResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } for key, val := range rawMsg { var err error @@ -623,7 +623,7 @@ func (e *EnvironmentResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } } return nil @@ -641,7 +641,7 @@ func (e EnvironmentResourceListResult) MarshalJSON() ([]byte, error) { func (e *EnvironmentResourceListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } for key, val := range rawMsg { var err error @@ -654,7 +654,7 @@ func (e *EnvironmentResourceListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", e, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", e, err) } } return nil @@ -672,7 +672,7 @@ func (g GetGraphRequest) MarshalJSON() ([]byte, error) { func (g *GetGraphRequest) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", g, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", g, err) } for key, val := range rawMsg { var err error @@ -685,7 +685,7 @@ func (g *GetGraphRequest) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", g, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", g, err) } } return nil @@ -705,7 +705,7 @@ func (i IdentitySettings) MarshalJSON() ([]byte, error) { func (i *IdentitySettings) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", i, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", i, err) } for key, val := range rawMsg { var err error @@ -724,7 +724,7 @@ func (i *IdentitySettings) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", i, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", i, err) } } return nil @@ -744,7 +744,7 @@ func (k KubernetesCompute) MarshalJSON() ([]byte, error) { func (k *KubernetesCompute) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", k, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", k, err) } for key, val := range rawMsg { var err error @@ -763,7 +763,7 @@ func (k *KubernetesCompute) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", k, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", k, err) } } return nil @@ -784,7 +784,7 @@ func (o Operation) MarshalJSON() ([]byte, error) { func (o *Operation) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } for key, val := range rawMsg { var err error @@ -806,7 +806,7 @@ func (o *Operation) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } } return nil @@ -826,7 +826,7 @@ func (o OperationDisplay) MarshalJSON() ([]byte, error) { func (o *OperationDisplay) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } for key, val := range rawMsg { var err error @@ -845,7 +845,7 @@ func (o *OperationDisplay) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } } return nil @@ -863,7 +863,7 @@ func (o OperationListResult) MarshalJSON() ([]byte, error) { func (o *OperationListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } for key, val := range rawMsg { var err error @@ -876,7 +876,7 @@ func (o *OperationListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } } return nil @@ -895,7 +895,7 @@ func (o OutputResource) MarshalJSON() ([]byte, error) { func (o *OutputResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } for key, val := range rawMsg { var err error @@ -911,7 +911,7 @@ func (o *OutputResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", o, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", o, err) } } return nil @@ -930,7 +930,7 @@ func (p Providers) MarshalJSON() ([]byte, error) { func (p *Providers) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } for key, val := range rawMsg { var err error @@ -946,7 +946,7 @@ func (p *Providers) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } } return nil @@ -964,7 +964,7 @@ func (p ProvidersAws) MarshalJSON() ([]byte, error) { func (p *ProvidersAws) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } for key, val := range rawMsg { var err error @@ -977,7 +977,7 @@ func (p *ProvidersAws) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } } return nil @@ -996,7 +996,7 @@ func (p ProvidersAzure) MarshalJSON() ([]byte, error) { func (p *ProvidersAzure) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } for key, val := range rawMsg { var err error @@ -1012,7 +1012,7 @@ func (p *ProvidersAzure) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } } return nil @@ -1029,7 +1029,7 @@ func (p ProvidersKubernetes) MarshalJSON() ([]byte, error) { func (p *ProvidersKubernetes) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } for key, val := range rawMsg { var err error @@ -1039,7 +1039,7 @@ func (p *ProvidersKubernetes) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", p, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", p, err) } } return nil @@ -1060,7 +1060,7 @@ func (r RecipeDefinition) MarshalJSON() ([]byte, error) { func (r *RecipeDefinition) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1082,7 +1082,7 @@ func (r *RecipeDefinition) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1101,7 +1101,7 @@ func (r RecipePackProperties) MarshalJSON() ([]byte, error) { func (r *RecipePackProperties) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1117,7 +1117,7 @@ func (r *RecipePackProperties) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1140,7 +1140,7 @@ func (r RecipePackResource) MarshalJSON() ([]byte, error) { func (r *RecipePackResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1168,7 +1168,7 @@ func (r *RecipePackResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1186,7 +1186,7 @@ func (r RecipePackResourceListResult) MarshalJSON() ([]byte, error) { func (r *RecipePackResourceListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1199,7 +1199,7 @@ func (r *RecipePackResourceListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1220,7 +1220,7 @@ func (r RecipeParameterValue) MarshalJSON() ([]byte, error) { func (r *RecipeParameterValue) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1237,7 +1237,7 @@ func (r *RecipeParameterValue) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1256,7 +1256,7 @@ func (r RecipeStatus) MarshalJSON() ([]byte, error) { func (r *RecipeStatus) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1272,7 +1272,7 @@ func (r *RecipeStatus) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1291,7 +1291,7 @@ func (r ResourceStatus) MarshalJSON() ([]byte, error) { func (r *ResourceStatus) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } for key, val := range rawMsg { var err error @@ -1307,7 +1307,7 @@ func (r *ResourceStatus) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", r, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", r, err) } } return nil @@ -1316,10 +1316,10 @@ func (r *ResourceStatus) UnmarshalJSON(data []byte) error { // MarshalJSON implements the json.Marshaller interface for type SystemData. func (s SystemData) MarshalJSON() ([]byte, error) { objectMap := make(map[string]any) - populateTime[datetime.RFC3339](objectMap, "createdAt", s.CreatedAt, true) + populateTime[datetime.RFC3339](objectMap, "createdAt", s.CreatedAt) populate(objectMap, "createdBy", s.CreatedBy) populate(objectMap, "createdByType", s.CreatedByType) - populateTime[datetime.RFC3339](objectMap, "lastModifiedAt", s.LastModifiedAt, true) + populateTime[datetime.RFC3339](objectMap, "lastModifiedAt", s.LastModifiedAt) populate(objectMap, "lastModifiedBy", s.LastModifiedBy) populate(objectMap, "lastModifiedByType", s.LastModifiedByType) return json.Marshal(objectMap) @@ -1329,7 +1329,7 @@ func (s SystemData) MarshalJSON() ([]byte, error) { func (s *SystemData) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", s, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", s, err) } for key, val := range rawMsg { var err error @@ -1354,7 +1354,7 @@ func (s *SystemData) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", s, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", s, err) } } return nil @@ -1371,7 +1371,7 @@ func (t TerraformCredentialConfig) MarshalJSON() ([]byte, error) { func (t *TerraformCredentialConfig) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1381,7 +1381,7 @@ func (t *TerraformCredentialConfig) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1399,7 +1399,7 @@ func (t TerraformProviderDirect) MarshalJSON() ([]byte, error) { func (t *TerraformProviderDirect) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1412,7 +1412,7 @@ func (t *TerraformProviderDirect) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1430,7 +1430,7 @@ func (t TerraformProviderInstallation) MarshalJSON() ([]byte, error) { func (t *TerraformProviderInstallation) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1443,7 +1443,7 @@ func (t *TerraformProviderInstallation) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1462,7 +1462,7 @@ func (t TerraformProviderMirror) MarshalJSON() ([]byte, error) { func (t *TerraformProviderMirror) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1478,7 +1478,7 @@ func (t *TerraformProviderMirror) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1498,7 +1498,7 @@ func (t TerraformSettingsProperties) MarshalJSON() ([]byte, error) { func (t *TerraformSettingsProperties) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1517,7 +1517,7 @@ func (t *TerraformSettingsProperties) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1540,7 +1540,7 @@ func (t TerraformSettingsResource) MarshalJSON() ([]byte, error) { func (t *TerraformSettingsResource) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1568,7 +1568,7 @@ func (t *TerraformSettingsResource) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1586,7 +1586,7 @@ func (t TerraformSettingsResourceListResult) MarshalJSON() ([]byte, error) { func (t *TerraformSettingsResourceListResult) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1599,7 +1599,7 @@ func (t *TerraformSettingsResourceListResult) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1617,7 +1617,7 @@ func (t TerraformrcConfig) MarshalJSON() ([]byte, error) { func (t *TerraformrcConfig) UnmarshalJSON(data []byte) error { var rawMsg map[string]json.RawMessage if err := json.Unmarshal(data, &rawMsg); err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } for key, val := range rawMsg { var err error @@ -1630,7 +1630,7 @@ func (t *TerraformrcConfig) UnmarshalJSON(data []byte) error { delete(rawMsg, key) } if err != nil { - return fmt.Errorf("unmarshalling type %T: %s", t, err.Error()) + return fmt.Errorf("unmarshalling type %T: %v", t, err) } } return nil @@ -1646,17 +1646,13 @@ func populate(m map[string]any, k string, v any) { } } -func populateTime[T dateTimeConstraints](m map[string]any, k string, t *time.Time, utc bool) { +func populateTime[T dateTimeConstraints](m map[string]any, k string, t *time.Time) { if t == nil { return } else if azcore.IsNullValue(t) { m[k] = nil } else if !reflect.ValueOf(t).IsNil() { - tt := *t - if utc { - tt = tt.UTC() - } - newTime := T(tt) + newTime := T(*t) m[k] = (*T)(&newTime) } } @@ -1666,7 +1662,7 @@ func unpopulate(data json.RawMessage, fn string, v any) error { return nil } if err := json.Unmarshal(data, v); err != nil { - return fmt.Errorf("struct field %s: %s", fn, err.Error()) + return fmt.Errorf("struct field %s: %v", fn, err) } return nil } @@ -1677,7 +1673,7 @@ func unpopulateTime[T dateTimeConstraints](data json.RawMessage, fn string, t ** } var aux T if err := json.Unmarshal(data, &aux); err != nil { - return fmt.Errorf("struct field %s: %s", fn, err.Error()) + return fmt.Errorf("struct field %s: %v", fn, err) } newTime := time.Time(aux) *t = &newTime diff --git a/pkg/corerp/api/v20250801preview/zz_generated_operations_client.go b/pkg/corerp/api/v20250801preview/zz_generated_operations_client.go index 6f203c0993..b7918a99f8 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_operations_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_operations_client.go @@ -48,48 +48,34 @@ func (client *OperationsClient) NewListPager(options *OperationsClientListOption if page != nil { nextLink = *page.NextLink } - req, err := client.listCreateRequest(ctx, nextLink, options) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listCreateRequest(ctx, options) + }, nil) if err != nil { return OperationsClientListResponse{}, err } - resp, err := client.internal.Pipeline().Do(req) - if err != nil { - return OperationsClientListResponse{}, err - } - return client.listHandleResponse(resp, http.StatusOK) + return client.listHandleResponse(resp) }, }) } // listCreateRequest creates the List request. -func (client *OperationsClient) listCreateRequest(ctx context.Context, nextLink string, _ *OperationsClientListOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/providers/Radius.Core/operations" - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) - } +func (client *OperationsClient) listCreateRequest(ctx context.Context, _ *OperationsClientListOptions) (*policy.Request, error) { + urlPath := "/providers/Radius.Core/operations" + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listHandleResponse handles the List response. -func (client *OperationsClient) listHandleResponse(resp *http.Response, successCodes ...int) (OperationsClientListResponse, error) { +func (client *OperationsClient) listHandleResponse(resp *http.Response) (OperationsClientListResponse, error) { result := OperationsClientListResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.OperationListResult); err != nil { return OperationsClientListResponse{}, err } diff --git a/pkg/corerp/api/v20250801preview/zz_generated_recipepacks_client.go b/pkg/corerp/api/v20250801preview/zz_generated_recipepacks_client.go index 434da296d9..79b7289247 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_recipepacks_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_recipepacks_client.go @@ -56,7 +56,12 @@ func (client *RecipePacksClient) CreateOrUpdate(ctx context.Context, rootScope s if err != nil { return RecipePacksClientCreateOrUpdateResponse{}, err } - return client.createOrUpdateHandleResponse(httpResp, http.StatusOK, http.StatusCreated) + if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusCreated) { + err = runtime.NewResponseError(httpResp) + return RecipePacksClientCreateOrUpdateResponse{}, err + } + resp, err := client.createOrUpdateHandleResponse(httpResp) + return resp, err } // createOrUpdateCreateRequest creates the CreateOrUpdate request. @@ -86,11 +91,8 @@ func (client *RecipePacksClient) createOrUpdateCreateRequest(ctx context.Context } // createOrUpdateHandleResponse handles the CreateOrUpdate response. -func (client *RecipePacksClient) createOrUpdateHandleResponse(resp *http.Response, successCodes ...int) (RecipePacksClientCreateOrUpdateResponse, error) { +func (client *RecipePacksClient) createOrUpdateHandleResponse(resp *http.Response) (RecipePacksClientCreateOrUpdateResponse, error) { result := RecipePacksClientCreateOrUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.RecipePackResource); err != nil { return RecipePacksClientCreateOrUpdateResponse{}, err } @@ -115,7 +117,8 @@ func (client *RecipePacksClient) Delete(ctx context.Context, rootScope string, r return RecipePacksClientDeleteResponse{}, err } if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusNoContent) { - return RecipePacksClientDeleteResponse{}, runtime.NewResponseError(httpResp) + err = runtime.NewResponseError(httpResp) + return RecipePacksClientDeleteResponse{}, err } return RecipePacksClientDeleteResponse{}, nil } @@ -158,7 +161,12 @@ func (client *RecipePacksClient) Get(ctx context.Context, rootScope string, reci if err != nil { return RecipePacksClientGetResponse{}, err } - return client.getHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return RecipePacksClientGetResponse{}, err + } + resp, err := client.getHandleResponse(httpResp) + return resp, err } // getCreateRequest creates the Get request. @@ -184,11 +192,8 @@ func (client *RecipePacksClient) getCreateRequest(ctx context.Context, rootScope } // getHandleResponse handles the Get response. -func (client *RecipePacksClient) getHandleResponse(resp *http.Response, successCodes ...int) (RecipePacksClientGetResponse, error) { +func (client *RecipePacksClient) getHandleResponse(resp *http.Response) (RecipePacksClientGetResponse, error) { result := RecipePacksClientGetResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.RecipePackResource); err != nil { return RecipePacksClientGetResponse{}, err } @@ -211,52 +216,38 @@ func (client *RecipePacksClient) NewListByScopePager(rootScope string, options * if page != nil { nextLink = *page.NextLink } - req, err := client.listByScopeCreateRequest(ctx, rootScope, nextLink, options) - if err != nil { - return RecipePacksClientListByScopeResponse{}, err - } - resp, err := client.internal.Pipeline().Do(req) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listByScopeCreateRequest(ctx, rootScope, options) + }, nil) if err != nil { return RecipePacksClientListByScopeResponse{}, err } - return client.listByScopeHandleResponse(resp, http.StatusOK) + return client.listByScopeHandleResponse(resp) }, }) } // listByScopeCreateRequest creates the ListByScope request. -func (client *RecipePacksClient) listByScopeCreateRequest(ctx context.Context, rootScope string, nextLink string, _ *RecipePacksClientListByScopeOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/{rootScope}/providers/Radius.Core/recipePacks" - if rootScope == "" { - return nil, errors.New("parameter rootScope cannot be empty") - } - urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) +func (client *RecipePacksClient) listByScopeCreateRequest(ctx context.Context, rootScope string, _ *RecipePacksClientListByScopeOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/recipePacks" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listByScopeHandleResponse handles the ListByScope response. -func (client *RecipePacksClient) listByScopeHandleResponse(resp *http.Response, successCodes ...int) (RecipePacksClientListByScopeResponse, error) { +func (client *RecipePacksClient) listByScopeHandleResponse(resp *http.Response) (RecipePacksClientListByScopeResponse, error) { result := RecipePacksClientListByScopeResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.RecipePackResourceListResult); err != nil { return RecipePacksClientListByScopeResponse{}, err } @@ -281,7 +272,12 @@ func (client *RecipePacksClient) Update(ctx context.Context, rootScope string, r if err != nil { return RecipePacksClientUpdateResponse{}, err } - return client.updateHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return RecipePacksClientUpdateResponse{}, err + } + resp, err := client.updateHandleResponse(httpResp) + return resp, err } // updateCreateRequest creates the Update request. @@ -311,11 +307,8 @@ func (client *RecipePacksClient) updateCreateRequest(ctx context.Context, rootSc } // updateHandleResponse handles the Update response. -func (client *RecipePacksClient) updateHandleResponse(resp *http.Response, successCodes ...int) (RecipePacksClientUpdateResponse, error) { +func (client *RecipePacksClient) updateHandleResponse(resp *http.Response) (RecipePacksClientUpdateResponse, error) { result := RecipePacksClientUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.RecipePackResource); err != nil { return RecipePacksClientUpdateResponse{}, err } diff --git a/pkg/corerp/api/v20250801preview/zz_generated_responses.go b/pkg/corerp/api/v20250801preview/zz_generated_responses.go index bd94780ff2..b99a305892 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_responses.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_responses.go @@ -363,14 +363,12 @@ type EnvironmentsClientCreateOrUpdateResponse struct { // The `Radius.Core/environments` Resource Type represents a Radius Environment: the deployment target that platform engineers // configure for their developers. Every Radius Application is deployed to an Environment through its `environment` property. // An Environment defines three things for the Applications deployed to it: - // - // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` - // property. - // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, - // set through the `recipePacks` property. - // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration - // applied when Recipes run. - // + // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` + // property. + // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, + // set through the `recipePacks` property. + // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration + // applied when Recipes run. // ## Defining an Environment // The simplest Environment can be created directly with the `rad environment create` command, without a Bicep file: // ```bash @@ -461,14 +459,12 @@ type EnvironmentsClientGetResponse struct { // The `Radius.Core/environments` Resource Type represents a Radius Environment: the deployment target that platform engineers // configure for their developers. Every Radius Application is deployed to an Environment through its `environment` property. // An Environment defines three things for the Applications deployed to it: - // - // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` - // property. - // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, - // set through the `recipePacks` property. - // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration - // applied when Recipes run. - // + // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` + // property. + // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, + // set through the `recipePacks` property. + // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration + // applied when Recipes run. // ## Defining an Environment // The simplest Environment can be created directly with the `rad environment create` command, without a Bicep file: // ```bash @@ -560,14 +556,12 @@ type EnvironmentsClientUpdateResponse struct { // The `Radius.Core/environments` Resource Type represents a Radius Environment: the deployment target that platform engineers // configure for their developers. Every Radius Application is deployed to an Environment through its `environment` property. // An Environment defines three things for the Applications deployed to it: - // - // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` - // property. - // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, - // set through the `recipePacks` property. - // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration - // applied when Recipes run. - // + // - **Where resources are deployed**: the target compute platform and cloud provider accounts, set through the `providers` + // property. + // - **Which Recipes are used**: the Recipe Packs whose Recipes provision the infrastructure backing application resources, + // set through the `recipePacks` property. + // - **Advanced Terraform and Bicep settings**: environment-wide Recipe parameters and Terraform or Bicep engine configuration + // applied when Recipes run. // ## Defining an Environment // The simplest Environment can be created directly with the `rad environment create` command, without a Bicep file: // ```bash diff --git a/pkg/corerp/api/v20250801preview/zz_generated_terraformsettings_client.go b/pkg/corerp/api/v20250801preview/zz_generated_terraformsettings_client.go index 9c79cb2092..eaacf9a01f 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_terraformsettings_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_terraformsettings_client.go @@ -56,7 +56,12 @@ func (client *TerraformSettingsClient) CreateOrUpdate(ctx context.Context, rootS if err != nil { return TerraformSettingsClientCreateOrUpdateResponse{}, err } - return client.createOrUpdateHandleResponse(httpResp, http.StatusOK, http.StatusCreated) + if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusCreated) { + err = runtime.NewResponseError(httpResp) + return TerraformSettingsClientCreateOrUpdateResponse{}, err + } + resp, err := client.createOrUpdateHandleResponse(httpResp) + return resp, err } // createOrUpdateCreateRequest creates the CreateOrUpdate request. @@ -86,11 +91,8 @@ func (client *TerraformSettingsClient) createOrUpdateCreateRequest(ctx context.C } // createOrUpdateHandleResponse handles the CreateOrUpdate response. -func (client *TerraformSettingsClient) createOrUpdateHandleResponse(resp *http.Response, successCodes ...int) (TerraformSettingsClientCreateOrUpdateResponse, error) { +func (client *TerraformSettingsClient) createOrUpdateHandleResponse(resp *http.Response) (TerraformSettingsClientCreateOrUpdateResponse, error) { result := TerraformSettingsClientCreateOrUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.TerraformSettingsResource); err != nil { return TerraformSettingsClientCreateOrUpdateResponse{}, err } @@ -116,7 +118,8 @@ func (client *TerraformSettingsClient) Delete(ctx context.Context, rootScope str return TerraformSettingsClientDeleteResponse{}, err } if !runtime.HasStatusCode(httpResp, http.StatusOK, http.StatusNoContent) { - return TerraformSettingsClientDeleteResponse{}, runtime.NewResponseError(httpResp) + err = runtime.NewResponseError(httpResp) + return TerraformSettingsClientDeleteResponse{}, err } return TerraformSettingsClientDeleteResponse{}, nil } @@ -159,7 +162,12 @@ func (client *TerraformSettingsClient) Get(ctx context.Context, rootScope string if err != nil { return TerraformSettingsClientGetResponse{}, err } - return client.getHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return TerraformSettingsClientGetResponse{}, err + } + resp, err := client.getHandleResponse(httpResp) + return resp, err } // getCreateRequest creates the Get request. @@ -185,11 +193,8 @@ func (client *TerraformSettingsClient) getCreateRequest(ctx context.Context, roo } // getHandleResponse handles the Get response. -func (client *TerraformSettingsClient) getHandleResponse(resp *http.Response, successCodes ...int) (TerraformSettingsClientGetResponse, error) { +func (client *TerraformSettingsClient) getHandleResponse(resp *http.Response) (TerraformSettingsClientGetResponse, error) { result := TerraformSettingsClientGetResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.TerraformSettingsResource); err != nil { return TerraformSettingsClientGetResponse{}, err } @@ -212,52 +217,38 @@ func (client *TerraformSettingsClient) NewListByScopePager(rootScope string, opt if page != nil { nextLink = *page.NextLink } - req, err := client.listByScopeCreateRequest(ctx, rootScope, nextLink, options) - if err != nil { - return TerraformSettingsClientListByScopeResponse{}, err - } - resp, err := client.internal.Pipeline().Do(req) + resp, err := runtime.FetcherForNextLink(ctx, client.internal.Pipeline(), nextLink, func(ctx context.Context) (*policy.Request, error) { + return client.listByScopeCreateRequest(ctx, rootScope, options) + }, nil) if err != nil { return TerraformSettingsClientListByScopeResponse{}, err } - return client.listByScopeHandleResponse(resp, http.StatusOK) + return client.listByScopeHandleResponse(resp) }, }) } // listByScopeCreateRequest creates the ListByScope request. -func (client *TerraformSettingsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, nextLink string, _ *TerraformSettingsClientListByScopeOptions) (*policy.Request, error) { - firstPage := nextLink == "" - var req *policy.Request - var err error - if firstPage { - urlPath := "/{rootScope}/providers/Radius.Core/terraformSettings" - if rootScope == "" { - return nil, errors.New("parameter rootScope cannot be empty") - } - urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) - req, err = runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) - } else { - req, err = runtime.NewRequestForNextLink(ctx, http.MethodGet, client.internal.Endpoint(), nextLink) +func (client *TerraformSettingsClient) listByScopeCreateRequest(ctx context.Context, rootScope string, _ *TerraformSettingsClientListByScopeOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/terraformSettings" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + req, err := runtime.NewRequest(ctx, http.MethodGet, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) if err != nil { return nil, err } - if firstPage { - reqQP := req.Raw().URL.Query() - reqQP.Set("api-version", version20250801Preview) - req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") - req.Raw().Header["Accept"] = []string{"application/json"} - } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} return req, nil } // listByScopeHandleResponse handles the ListByScope response. -func (client *TerraformSettingsClient) listByScopeHandleResponse(resp *http.Response, successCodes ...int) (TerraformSettingsClientListByScopeResponse, error) { +func (client *TerraformSettingsClient) listByScopeHandleResponse(resp *http.Response) (TerraformSettingsClientListByScopeResponse, error) { result := TerraformSettingsClientListByScopeResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.TerraformSettingsResourceListResult); err != nil { return TerraformSettingsClientListByScopeResponse{}, err } @@ -283,7 +274,12 @@ func (client *TerraformSettingsClient) Update(ctx context.Context, rootScope str if err != nil { return TerraformSettingsClientUpdateResponse{}, err } - return client.updateHandleResponse(httpResp, http.StatusOK) + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return TerraformSettingsClientUpdateResponse{}, err + } + resp, err := client.updateHandleResponse(httpResp) + return resp, err } // updateCreateRequest creates the Update request. @@ -313,11 +309,8 @@ func (client *TerraformSettingsClient) updateCreateRequest(ctx context.Context, } // updateHandleResponse handles the Update response. -func (client *TerraformSettingsClient) updateHandleResponse(resp *http.Response, successCodes ...int) (TerraformSettingsClientUpdateResponse, error) { +func (client *TerraformSettingsClient) updateHandleResponse(resp *http.Response) (TerraformSettingsClientUpdateResponse, error) { result := TerraformSettingsClientUpdateResponse{} - if !runtime.HasStatusCode(resp, successCodes...) { - return result, runtime.NewResponseError(resp) - } if err := runtime.UnmarshalAsJSON(resp, &result.TerraformSettingsResource); err != nil { return TerraformSettingsClientUpdateResponse{}, err } From b4a63e47d8e928e8bdf7c2b2e130cca53f822412 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 09:24:36 -0700 Subject: [PATCH 10/18] corerp: add reconcile custom action on Radius.Core/applications (typespec + regen) Adds a synchronous reconcile action to Radius.Core/applications/{name} mirroring the shape of getGraph: POST .../providers/Radius.Core/applications/{app}/reconcile Adds ReconcileRequest / ReconcileResponse / ReconcileResourceOutcome models to carry the per-resource outcome report. This commit is TypeSpec + emitter output only; the corerp handler and 'rad startup' wiring land in follow-up commits (Phase 0 of specs/006-state-restoration). No callers are registered yet, so the action is currently unreachable. Signed-off-by: Nithya Subramanian --- .../fake/zz_generated_applications_server.go | 43 ++++++++++++ .../zz_generated_applications_client.go | 64 ++++++++++++++++++ .../v20250801preview/zz_generated_models.go | 29 ++++++++ .../zz_generated_models_serde.go | 66 +++++++++++++++++++ .../v20250801preview/zz_generated_options.go | 5 ++ .../zz_generated_responses.go | 6 ++ typespec/Radius.Core/applications.tsp | 34 ++++++++++ 7 files changed, 247 insertions(+) diff --git a/pkg/corerp/api/v20250801preview/fake/zz_generated_applications_server.go b/pkg/corerp/api/v20250801preview/fake/zz_generated_applications_server.go index bb69c33b35..0af7e91a19 100644 --- a/pkg/corerp/api/v20250801preview/fake/zz_generated_applications_server.go +++ b/pkg/corerp/api/v20250801preview/fake/zz_generated_applications_server.go @@ -40,6 +40,10 @@ type ApplicationsServer struct { // HTTP status codes to indicate success: http.StatusOK NewListByScopePager func(rootScope string, options *v20250801preview.ApplicationsClientListByScopeOptions) (resp azfake.PagerResponder[v20250801preview.ApplicationsClientListByScopeResponse]) + // Reconcile is the fake for method ApplicationsClient.Reconcile + // HTTP status codes to indicate success: http.StatusOK + Reconcile func(ctx context.Context, rootScope string, applicationName string, body v20250801preview.ReconcileRequest, options *v20250801preview.ApplicationsClientReconcileOptions) (resp azfake.Responder[v20250801preview.ApplicationsClientReconcileResponse], errResp azfake.ErrorResponder) + // Update is the fake for method ApplicationsClient.Update // HTTP status codes to indicate success: http.StatusOK Update func(ctx context.Context, rootScope string, applicationName string, properties v20250801preview.ApplicationResource, options *v20250801preview.ApplicationsClientUpdateOptions) (resp azfake.Responder[v20250801preview.ApplicationsClientUpdateResponse], errResp azfake.ErrorResponder) @@ -93,6 +97,8 @@ func (a *ApplicationsServerTransport) dispatchToMethodFake(req *http.Request, me res.resp, res.err = a.dispatchGetGraph(req) case "ApplicationsClient.NewListByScopePager": res.resp, res.err = a.dispatchNewListByScopePager(req) + case "ApplicationsClient.Reconcile": + res.resp, res.err = a.dispatchReconcile(req) case "ApplicationsClient.Update": res.resp, res.err = a.dispatchUpdate(req) default: @@ -288,6 +294,43 @@ func (a *ApplicationsServerTransport) dispatchNewListByScopePager(req *http.Requ return resp, nil } +func (a *ApplicationsServerTransport) dispatchReconcile(req *http.Request) (*http.Response, error) { + if a.srv.Reconcile == nil { + return nil, &nonRetriableError{errors.New("fake for method Reconcile not implemented")} + } + const regexStr = `/(?P[!#&$-;=?-\[\]_a-zA-Z0-9~%@]+)/providers/Radius\.Core/applications/(?P[!#&$-;=?-\[\]_a-zA-Z0-9~%@]+)/reconcile` + regex := regexp.MustCompile(regexStr) + matches := regex.FindStringSubmatch(req.URL.EscapedPath()) + if len(matches) < 3 { + return nil, fmt.Errorf("failed to parse path %s", req.URL.Path) + } + body, err := server.UnmarshalRequestAsJSON[v20250801preview.ReconcileRequest](req) + if err != nil { + return nil, err + } + rootScopeParam, err := url.PathUnescape(matches[regex.SubexpIndex("rootScope")]) + if err != nil { + return nil, err + } + applicationNameParam, err := url.PathUnescape(matches[regex.SubexpIndex("applicationName")]) + if err != nil { + return nil, err + } + respr, errRespr := a.srv.Reconcile(req.Context(), rootScopeParam, applicationNameParam, body, nil) + if respErr := server.GetError(errRespr, req); respErr != nil { + return nil, respErr + } + respContent := server.GetResponseContent(respr) + if !slices.Contains([]int{http.StatusOK}, respContent.HTTPStatus) { + return nil, &nonRetriableError{fmt.Errorf("unexpected status code %d. acceptable values are http.StatusOK", respContent.HTTPStatus)} + } + resp, err := server.MarshalResponseAsJSON(respContent, server.GetResponse(respr).ReconcileResponse, req) + if err != nil { + return nil, err + } + return resp, nil +} + func (a *ApplicationsServerTransport) dispatchUpdate(req *http.Request) (*http.Response, error) { if a.srv.Update == nil { return nil, &nonRetriableError{errors.New("fake for method Update not implemented")} diff --git a/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go b/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go index 658ff8ccde..41eff5ac8a 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_applications_client.go @@ -315,6 +315,70 @@ func (client *ApplicationsClient) listByScopeHandleResponse(resp *http.Response) return result, nil } +// Reconcile - Reconciles the application's resources against their underlying providers. For every non-terminal child, dynamic-rp +// queries the recorded outputResources and updates provisioningState to match reality. Called by `rad startup` after the +// state archive is hydrated so a subsequent `rad app delete` is not blocked by 409s on resources whose real state has moved +// on. Returns a report of what was observed and rewritten. +// If the operation fails it returns an *azcore.ResponseError type. +// - rootScope - The scope in which the resource is present. UCP Scope is /planes/{planeType}/{planeName}/resourceGroup/{resourcegroupID} +// and Azure resource scope is /subscriptions/{subscriptionID}/resourceGroup/{resourcegroupID} +// - applicationName - The application name +// - body - The content of the action request +// - options - ApplicationsClientReconcileOptions contains the optional parameters for the ApplicationsClient.Reconcile method. +func (client *ApplicationsClient) Reconcile(ctx context.Context, rootScope string, applicationName string, body ReconcileRequest, options *ApplicationsClientReconcileOptions) (ApplicationsClientReconcileResponse, error) { + var err error + ctx = context.WithValue(ctx, runtime.CtxAPINameKey{}, "ApplicationsClient.Reconcile") + req, err := client.reconcileCreateRequest(ctx, rootScope, applicationName, body, options) + if err != nil { + return ApplicationsClientReconcileResponse{}, err + } + httpResp, err := client.internal.Pipeline().Do(req) + if err != nil { + return ApplicationsClientReconcileResponse{}, err + } + if !runtime.HasStatusCode(httpResp, http.StatusOK) { + err = runtime.NewResponseError(httpResp) + return ApplicationsClientReconcileResponse{}, err + } + resp, err := client.reconcileHandleResponse(httpResp) + return resp, err +} + +// reconcileCreateRequest creates the Reconcile request. +func (client *ApplicationsClient) reconcileCreateRequest(ctx context.Context, rootScope string, applicationName string, body ReconcileRequest, _ *ApplicationsClientReconcileOptions) (*policy.Request, error) { + urlPath := "/{rootScope}/providers/Radius.Core/applications/{applicationName}/reconcile" + if rootScope == "" { + return nil, errors.New("parameter rootScope cannot be empty") + } + urlPath = strings.ReplaceAll(urlPath, "{rootScope}", rootScope) + if applicationName == "" { + return nil, errors.New("parameter applicationName cannot be empty") + } + urlPath = strings.ReplaceAll(urlPath, "{applicationName}", url.PathEscape(applicationName)) + req, err := runtime.NewRequest(ctx, http.MethodPost, runtime.JoinPaths(client.internal.Endpoint(), urlPath)) + if err != nil { + return nil, err + } + reqQP := req.Raw().URL.Query() + reqQP.Set("api-version", version20250801Preview) + req.Raw().URL.RawQuery = strings.ReplaceAll(reqQP.Encode(), "+", "%20") + req.Raw().Header["Accept"] = []string{"application/json"} + req.Raw().Header["Content-Type"] = []string{"application/json"} + if err := runtime.MarshalAsJSON(req, body); err != nil { + return nil, err + } + return req, nil +} + +// reconcileHandleResponse handles the Reconcile response. +func (client *ApplicationsClient) reconcileHandleResponse(resp *http.Response) (ApplicationsClientReconcileResponse, error) { + result := ApplicationsClientReconcileResponse{} + if err := runtime.UnmarshalAsJSON(resp, &result.ReconcileResponse); err != nil { + return ApplicationsClientReconcileResponse{}, err + } + return result, nil +} + // Update - Update a ApplicationResource // If the operation fails it returns an *azcore.ResponseError type. // - rootScope - The scope in which the resource is present. UCP Scope is /planes/{planeType}/{planeName}/resourceGroup/{resourcegroupID} diff --git a/pkg/corerp/api/v20250801preview/zz_generated_models.go b/pkg/corerp/api/v20250801preview/zz_generated_models.go index 1eede80cd5..83be0ca71a 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_models.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_models.go @@ -803,6 +803,35 @@ type RecipeStatus struct { TemplateVersion *string } +// ReconcileRequest - Request body for the reconcile action. Currently empty; reserved for future filters (for example, a +// resource-type allowlist). +type ReconcileRequest struct { +} + +// ReconcileResourceOutcome - Per-resource outcome recorded by the reconcile action. +type ReconcileResourceOutcome struct { + // REQUIRED; The provisioningState observed on the resource before the reality check. + From *string + + // REQUIRED; The fully-qualified resource ID that was reconciled. + ResourceID *string + + // REQUIRED; The provisioningState written back after the reality check. Same as `from` when no change was needed (for example, + // when reality confirms the resource is still updating) or when the check could not run. + To *string + + // Short human-readable reason for the change or explanation of the outcome (for example, `underlying kubernetes object not + // found`, `still updating`, `provider query failed`). + Reason *string +} + +// ReconcileResponse - Response body for the reconcile action. +type ReconcileResponse struct { + // REQUIRED; Per-resource outcomes of the reconciliation pass. One entry per non-terminal child the orchestrator attempted + // to reconcile. Terminal-state children are skipped and do not appear. + Resources []*ReconcileResourceOutcome +} + // ResourceStatus - Status of a resource. type ResourceStatus struct { // The compute resource associated with the resource. diff --git a/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go b/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go index 4fde33d786..7510635ff3 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_models_serde.go @@ -1278,6 +1278,72 @@ func (r *RecipeStatus) UnmarshalJSON(data []byte) error { return nil } +// MarshalJSON implements the json.Marshaller interface for type ReconcileResourceOutcome. +func (r ReconcileResourceOutcome) MarshalJSON() ([]byte, error) { + objectMap := make(map[string]any) + populate(objectMap, "from", r.From) + populate(objectMap, "reason", r.Reason) + populate(objectMap, "resourceId", r.ResourceID) + populate(objectMap, "to", r.To) + return json.Marshal(objectMap) +} + +// UnmarshalJSON implements the json.Unmarshaller interface for type ReconcileResourceOutcome. +func (r *ReconcileResourceOutcome) UnmarshalJSON(data []byte) error { + var rawMsg map[string]json.RawMessage + if err := json.Unmarshal(data, &rawMsg); err != nil { + return fmt.Errorf("unmarshalling type %T: %v", r, err) + } + for key, val := range rawMsg { + var err error + switch key { + case "from": + err = unpopulate(val, "From", &r.From) + delete(rawMsg, key) + case "reason": + err = unpopulate(val, "Reason", &r.Reason) + delete(rawMsg, key) + case "resourceId": + err = unpopulate(val, "ResourceID", &r.ResourceID) + delete(rawMsg, key) + case "to": + err = unpopulate(val, "To", &r.To) + delete(rawMsg, key) + } + if err != nil { + return fmt.Errorf("unmarshalling type %T: %v", r, err) + } + } + return nil +} + +// MarshalJSON implements the json.Marshaller interface for type ReconcileResponse. +func (r ReconcileResponse) MarshalJSON() ([]byte, error) { + objectMap := make(map[string]any) + populate(objectMap, "resources", r.Resources) + return json.Marshal(objectMap) +} + +// UnmarshalJSON implements the json.Unmarshaller interface for type ReconcileResponse. +func (r *ReconcileResponse) UnmarshalJSON(data []byte) error { + var rawMsg map[string]json.RawMessage + if err := json.Unmarshal(data, &rawMsg); err != nil { + return fmt.Errorf("unmarshalling type %T: %v", r, err) + } + for key, val := range rawMsg { + var err error + switch key { + case "resources": + err = unpopulate(val, "Resources", &r.Resources) + delete(rawMsg, key) + } + if err != nil { + return fmt.Errorf("unmarshalling type %T: %v", r, err) + } + } + return nil +} + // MarshalJSON implements the json.Marshaller interface for type ResourceStatus. func (r ResourceStatus) MarshalJSON() ([]byte, error) { objectMap := make(map[string]any) diff --git a/pkg/corerp/api/v20250801preview/zz_generated_options.go b/pkg/corerp/api/v20250801preview/zz_generated_options.go index b8b3fb1f27..947fe0d664 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_options.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_options.go @@ -28,6 +28,11 @@ type ApplicationsClientListByScopeOptions struct { // placeholder for future optional parameters } +// ApplicationsClientReconcileOptions contains the optional parameters for the ApplicationsClient.Reconcile method. +type ApplicationsClientReconcileOptions struct { + // placeholder for future optional parameters +} + // ApplicationsClientUpdateOptions contains the optional parameters for the ApplicationsClient.Update method. type ApplicationsClientUpdateOptions struct { // placeholder for future optional parameters diff --git a/pkg/corerp/api/v20250801preview/zz_generated_responses.go b/pkg/corerp/api/v20250801preview/zz_generated_responses.go index b99a305892..2815f7e9bd 100644 --- a/pkg/corerp/api/v20250801preview/zz_generated_responses.go +++ b/pkg/corerp/api/v20250801preview/zz_generated_responses.go @@ -114,6 +114,12 @@ type ApplicationsClientListByScopeResponse struct { ApplicationResourceListResult } +// ApplicationsClientReconcileResponse contains the response from method ApplicationsClient.Reconcile. +type ApplicationsClientReconcileResponse struct { + // Response body for the reconcile action. + ReconcileResponse +} + // ApplicationsClientUpdateResponse contains the response from method ApplicationsClient.Update. type ApplicationsClientUpdateResponse struct { // The `Radius.Core/applications` Resource Type represents a Radius Application: a logical grouping of the resources that diff --git a/typespec/Radius.Core/applications.tsp b/typespec/Radius.Core/applications.tsp index 71a00bb01c..8ccb03df9b 100644 --- a/typespec/Radius.Core/applications.tsp +++ b/typespec/Radius.Core/applications.tsp @@ -205,6 +205,31 @@ model ApplicationGraphOutputResource { portalUrl?: string; } +@doc("Request body for the reconcile action. Currently empty; reserved for future filters (for example, a resource-type allowlist).") +model ReconcileRequest {} + +@doc("Response body for the reconcile action.") +model ReconcileResponse { + @doc("Per-resource outcomes of the reconciliation pass. One entry per non-terminal child the orchestrator attempted to reconcile. Terminal-state children are skipped and do not appear.") + @extension("x-ms-identifiers", #["resourceId"]) + resources: Array; +} + +@doc("Per-resource outcome recorded by the reconcile action.") +model ReconcileResourceOutcome { + @doc("The fully-qualified resource ID that was reconciled.") + resourceId: string; + + @doc("The provisioningState observed on the resource before the reality check.") + from: string; + + @doc("The provisioningState written back after the reality check. Same as `from` when no change was needed (for example, when reality confirms the resource is still updating) or when the check could not run.") + to: string; + + @doc("Short human-readable reason for the change or explanation of the outcome (for example, `underlying kubernetes object not found`, `still updating`, `provider query failed`).") + reason?: string; +} + #suppress "@azure-tools/typespec-azure-core/casing-style" @armResourceOperations interface Applications { @@ -244,4 +269,13 @@ interface Applications { ApplicationGraphResponse, UCPBaseParameters >; + + @doc("Reconciles the application's resources against their underlying providers. For every non-terminal child, dynamic-rp queries the recorded outputResources and updates provisioningState to match reality. Called by `rad startup` after the state archive is hydrated so a subsequent `rad app delete` is not blocked by 409s on resources whose real state has moved on. Returns a report of what was observed and rewritten.") + @action("reconcile") + reconcile is ArmResourceActionSync< + ApplicationResource, + ReconcileRequest, + ReconcileResponse, + UCPBaseParameters + >; } From 497c9451d87d424ac237710b1d6ff1d32bf79835 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 09:31:27 -0700 Subject: [PATCH 11/18] corerp: add Phase 0 stub reconcile handler on Radius.Core/applications Wires the reconcile custom action registered in the previous commit into a concrete handler and registers it alongside getGraph on the Radius.Core preview applications resource. The handler validates the application exists (404 if not) and returns an empty ReconcileResponse. Phase 0 of specs/006-state-restoration: this end-to-end path lets a client POST /planes/.../applications/{app}/reconcile without error, so 'rad startup' can be wired up next before the orchestrator that walks children and the dynamic-rp per-resource reality-check are added. The sdk.Connection is threaded through the constructor now so future commits that add the child walk (which fans out through UCP) do not need to change the handler's signature or its registration in setup.go. Tests cover the empty-report happy path, 404 when the application does not exist, and error-propagation on database failures. Signed-off-by: Nithya Subramanian --- .../v20250801preview/reconcile.go | 84 ++++++++++ .../v20250801preview/reconcile_test.go | 149 ++++++++++++++++++ pkg/corerp/setup/setup.go | 5 + 3 files changed, 238 insertions(+) create mode 100644 pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go create mode 100644 pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go diff --git a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go new file mode 100644 index 0000000000..83bcf2faed --- /dev/null +++ b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go @@ -0,0 +1,84 @@ +/* +Copyright 2023 The Radius Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v20250801preview + +import ( + "context" + "net/http" + + v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" + ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" + "github.com/radius-project/radius/pkg/armrpc/rest" + corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" + "github.com/radius-project/radius/pkg/corerp/datamodel" + "github.com/radius-project/radius/pkg/corerp/datamodel/converter" + "github.com/radius-project/radius/pkg/sdk" +) + +var _ ctrl.Controller = (*Reconcilev20250801preview)(nil) + +// Reconcilev20250801preview is the controller implementation for the reconcile custom action on +// Radius.Core/applications. It reconciles the hydrated state of every non-terminal child resource +// against its underlying provider (Kubernetes, cloud SDKs) and rewrites provisioningState in the +// state store to match reality. Called by `rad startup` after the state archive is loaded, so a +// subsequent `rad app delete` is not blocked by 409s on resources whose real state has moved on. +// See specs/006-state-restoration for the design. +// +// This is the Phase 0 stub: it validates the target application exists and returns an empty +// report. The corerp orchestrator that walks children and dispatches per-resource reconcile to +// dynamic-rp is added in a follow-up commit alongside the dynamic-rp handler that does the +// reality check. +type Reconcilev20250801preview struct { + ctrl.Operation[*datamodel.Application_v20250801preview, datamodel.Application_v20250801preview] + // connection is unused in the Phase 0 stub but wired now so the constructor signature stays + // stable when the orchestrator lands and needs to fan out through UCP. + connection sdk.Connection +} + +// NewReconcilev20250801preview creates a new instance of the Reconcilev20250801preview controller. +func NewReconcilev20250801preview(opts ctrl.Options, connection sdk.Connection) (ctrl.Controller, error) { + return &Reconcilev20250801preview{ + ctrl.NewOperation(opts, + ctrl.ResourceOptions[datamodel.Application_v20250801preview]{ + RequestConverter: converter.Application20250801DataModelFromVersioned, + ResponseConverter: converter.Application20250801DataModelToVersioned, + }, + ), + connection, + }, nil +} + +// Run handles the reconcile custom action for Radius.Core/applications. In this Phase 0 stub it +// looks up the application (404s if missing) and returns an empty ReconcileResponse. The child +// walk and per-resource dispatch land in a follow-up commit. +func (c *Reconcilev20250801preview) Run(ctx context.Context, w http.ResponseWriter, req *http.Request) (rest.Response, error) { + sCtx := v1.ARMRequestContextFromContext(ctx) + + // Route: /planes/radius/local/resourcegroups/{rg}/providers/Radius.Core/applications/{app}/reconcile + applicationID := sCtx.ResourceID.Truncate() + applicationResource, _, err := c.GetResource(ctx, applicationID) + if err != nil { + return nil, err + } + if applicationResource == nil { + return rest.NewNotFoundResponse(sCtx.ResourceID), nil + } + + return rest.NewOKResponse(&corerpv20250801preview.ReconcileResponse{ + Resources: []*corerpv20250801preview.ReconcileResourceOutcome{}, + }), nil +} diff --git a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go new file mode 100644 index 0000000000..40f816e3fa --- /dev/null +++ b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go @@ -0,0 +1,149 @@ +/* +Copyright 2023 The Radius Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v20250801preview + +import ( + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "testing" + + v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" + ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" + "github.com/radius-project/radius/pkg/armrpc/rpctest" + "github.com/radius-project/radius/pkg/components/database" + corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" + "github.com/radius-project/radius/pkg/corerp/datamodel" + rpv1 "github.com/radius-project/radius/pkg/rp/v1" + "github.com/radius-project/radius/pkg/sdk" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" +) + +const reconcileRoute = "http://localhost:8080/planes/radius/local/resourcegroups/default/providers/Radius.Core/applications/myapp/reconcile?api-version=2025-08-01-preview" + +func newReconcileTestConnection(t *testing.T) sdk.Connection { + t.Helper() + conn, err := sdk.NewDirectConnection("http://localhost:9000/apis/api.ucp.dev/v1alpha3") + require.NoError(t, err) + return conn +} + +func TestReconcileRun_ReturnsEmptyReport(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + req, err := rpctest.NewHTTPRequestWithContent( + t.Context(), + v1.OperationPost.HTTPMethod(), + reconcileRoute, nil, + ) + require.NoError(t, err) + + // Return an application resource so the stub proceeds past the existence check. The stub + // does not read any properties from the resource yet, so a minimal stored record is enough. + stored := &database.Object{ + Metadata: database.Metadata{ID: "/planes/radius/local/resourceGroups/default/providers/Radius.Core/applications/myapp"}, + Data: &datamodel.Application_v20250801preview{ + BaseResource: v1.BaseResource{ + TrackedResource: v1.TrackedResource{ + ID: "/planes/radius/local/resourceGroups/default/providers/Radius.Core/applications/myapp", + Name: "myapp", + Type: "Radius.Core/applications", + }, + InternalMetadata: v1.InternalMetadata{ + UpdatedAPIVersion: "2025-08-01-preview", + AsyncProvisioningState: v1.ProvisioningStateSucceeded, + }, + }, + Properties: datamodel.ApplicationProperties_v20250801preview{ + BasicResourceProperties: rpv1.BasicResourceProperties{ + Environment: "/planes/radius/local/resourceGroups/default/providers/Radius.Core/environments/env0", + }, + }, + }, + } + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(stored, nil) + + ctx := rpctest.NewARMRequestContext(req) + opts := ctrl.Options{DatabaseClient: databaseClient} + c, err := NewReconcilev20250801preview(opts, newReconcileTestConnection(t)) + require.NoError(t, err) + + w := httptest.NewRecorder() + resp, err := c.Run(ctx, w, req) + require.NoError(t, err) + require.NoError(t, resp.Apply(ctx, w, req)) + + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + var body corerpv20250801preview.ReconcileResponse + require.NoError(t, json.NewDecoder(w.Result().Body).Decode(&body)) + require.NotNil(t, body.Resources) + require.Empty(t, body.Resources, "Phase 0 stub must return an empty report") +} + +func TestReconcileRun_NotFound(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + req, err := rpctest.NewHTTPRequestWithContent( + t.Context(), + v1.OperationPost.HTTPMethod(), + reconcileRoute, nil, + ) + require.NoError(t, err) + + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(nil, &database.ErrNotFound{}) + + ctx := rpctest.NewARMRequestContext(req) + c, err := NewReconcilev20250801preview(ctrl.Options{DatabaseClient: databaseClient}, newReconcileTestConnection(t)) + require.NoError(t, err) + + w := httptest.NewRecorder() + resp, err := c.Run(ctx, w, req) + require.NoError(t, err) + require.NoError(t, resp.Apply(ctx, w, req)) + require.Equal(t, http.StatusNotFound, w.Result().StatusCode) +} + +func TestReconcileRun_DatabaseError(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + req, err := rpctest.NewHTTPRequestWithContent( + t.Context(), + v1.OperationPost.HTTPMethod(), + reconcileRoute, nil, + ) + require.NoError(t, err) + + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(nil, errors.New("boom")) + + ctx := rpctest.NewARMRequestContext(req) + c, err := NewReconcilev20250801preview(ctrl.Options{DatabaseClient: databaseClient}, newReconcileTestConnection(t)) + require.NoError(t, err) + + w := httptest.NewRecorder() + resp, actErr := c.Run(ctx, w, req) + require.Error(t, actErr) + require.Nil(t, resp) +} diff --git a/pkg/corerp/setup/setup.go b/pkg/corerp/setup/setup.go index 78eb570e89..2991677ab6 100644 --- a/pkg/corerp/setup/setup.go +++ b/pkg/corerp/setup/setup.go @@ -301,6 +301,11 @@ func SetupRadiusCoreNamespace(recipeControllerConfig *controllerconfig.RecipeCon return app_v20250801_ctrl.NewGetGraphv20250801preview(opt, *recipeControllerConfig.UCPConnection) }, }, + "reconcile": { + APIController: func(opt apictrl.Options) (apictrl.Controller, error) { + return app_v20250801_ctrl.NewReconcilev20250801preview(opt, *recipeControllerConfig.UCPConnection) + }, + }, }, }) From 3da0bcb072fc7189137cdbe62999533fb7155f3c Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 09:44:04 -0700 Subject: [PATCH 12/18] cli: add ReconcileHydratedState stage to 'rad startup' Adds a fifth stage to rad startup that runs after ScaleUp: lists every application in the workspace's plane and POSTs the Radius.Core/applications reconcile custom action on each. Aggregates per-application outcomes into a report that is logged to the workflow log. The stage is best-effort per the spec's acceptance criterion: * individual per-application reconcile failures are recorded in the report and do not halt the pass; * a failure to even begin the pass (unreachable workspace, LIST failure) is logged and rad startup still returns success. Since Phase 0's server-side handler returns an empty report, the log will show 'reconciled 0 resource(s)' per application until the child walk and the dynamic-rp per-resource handler land in Phase 1. Nothing else about the existing startup flow changes. Tests extend the fakeStateRestoreClient to record the reconcile call and add coverage for the happy path (order includes reconcile after scaleup), the best-effort contract (reconcile error does not fail startup), and workspace plumbing (the runner's active workspace is what the reconcile stage receives). Signed-off-by: Nithya Subramanian --- pkg/cli/cmd/startup/startup.go | 11 +++ pkg/cli/cmd/startup/startup_test.go | 61 +++++++++++++-- pkg/cli/cmd/startup/stateclient.go | 110 ++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 5 deletions(-) diff --git a/pkg/cli/cmd/startup/startup.go b/pkg/cli/cmd/startup/startup.go index c0cda15cc5..eacfbcbc7d 100644 --- a/pkg/cli/cmd/startup/startup.go +++ b/pkg/cli/cmd/startup/startup.go @@ -175,6 +175,17 @@ func (r *Runner) Run(ctx context.Context) error { } scaledBackUp = true + // ReconcileHydratedState is best-effort: it POSTs the reconcile custom action per application + // so the state store reflects reality before the next 'rad ...' command runs. Failures are + // logged and the workflow still succeeds; see specs/006-state-restoration. + r.Output.LogInfo("Reconciling hydrated state against reality...") + reports, err := r.StateClient.ReconcileHydratedState(ctx, r.Workspace) + if err != nil { + r.Output.LogInfo("Reconcile skipped: %v", err) + } else { + logReconcileReports(r.Output, reports) + } + r.Output.LogInfo("State restored successfully.") return nil } diff --git a/pkg/cli/cmd/startup/startup_test.go b/pkg/cli/cmd/startup/startup_test.go index e9b473d692..9a70804546 100644 --- a/pkg/cli/cmd/startup/startup_test.go +++ b/pkg/cli/cmd/startup/startup_test.go @@ -76,10 +76,15 @@ type fakeStateRestoreClient struct { waitErr error restoreDBErr error restoreTFErr error + reconcileErr error - waited bool - dbCalled bool - tfCalled bool + waited bool + dbCalled bool + tfCalled bool + reconcileCalled bool + + reconcileReports []ApplicationReconcileReport + reconcileArg *workspaces.Workspace order []string } @@ -102,6 +107,16 @@ func (f *fakeStateRestoreClient) RestoreTerraform(ctx context.Context, kubeConte return f.restoreTFErr } +func (f *fakeStateRestoreClient) ReconcileHydratedState(ctx context.Context, workspace *workspaces.Workspace) ([]ApplicationReconcileReport, error) { + f.reconcileCalled = true + f.reconcileArg = workspace + f.order = append(f.order, "reconcile") + if f.reconcileErr != nil { + return nil, f.reconcileErr + } + return f.reconcileReports, nil +} + // fakeScaler records scale operations and appends them to a shared order slice so tests can assert // that the control plane is scaled down before any restore and back up afterward. type fakeScaler struct { @@ -177,10 +192,46 @@ func Test_Run_RestoresInOrderWaitDatabaseTerraform(t *testing.T) { require.True(t, client.waited) require.True(t, client.dbCalled) require.True(t, client.tfCalled) + require.True(t, client.reconcileCalled) require.True(t, scaler.downCalled) require.True(t, scaler.upCalled) - require.Equal(t, []string{"scaledown", "wait", "db", "tf", "scaleup"}, client.order, - "must scale down, wait, restore databases, restore terraform, then scale up") + require.Equal(t, []string{"scaledown", "wait", "db", "tf", "scaleup", "reconcile"}, client.order, + "reconcile must run after scale up so the resource providers are ready to serve it") +} + +// Test_Run_ReconcileFailureDoesNotFailStartup verifies the best-effort contract of the reconcile +// stage: when ReconcileHydratedState returns an error, rad startup logs and still succeeds. Test +// case matches the spec's acceptance criterion "rad startup never fails because reconciliation +// could not reach a resource provider". +func Test_Run_ReconcileFailureDoesNotFailStartup(t *testing.T) { + ctrl := gomock.NewController(t) + defer ctrl.Finish() + + client := &fakeStateRestoreClient{reconcileErr: errors.New("ucp unreachable")} + r, _ := newTestRunner(t, ctrl, client) + + err := r.Run(t.Context()) + require.NoError(t, err, "reconcile failure must not fail rad startup") + require.True(t, client.reconcileCalled) +} + +// Test_Run_ReconcileReceivesWorkspace verifies that the workspace passed to the reconcile stage +// is the runner's active workspace, so the default client can build a connection to the right +// control plane. +func Test_Run_ReconcileReceivesWorkspace(t *testing.T) { + ctrl := gomock.NewController(t) + defer ctrl.Finish() + + client := &fakeStateRestoreClient{ + reconcileReports: []ApplicationReconcileReport{ + {Name: "cool-app", ResourceCount: 3}, + }, + } + r, _ := newTestRunner(t, ctrl, client) + + require.NoError(t, r.Run(t.Context())) + require.NotNil(t, client.reconcileArg) + require.Equal(t, r.Workspace, client.reconcileArg) } func Test_Run_ScaleDownFailureStopsBeforeRestore(t *testing.T) { diff --git a/pkg/cli/cmd/startup/stateclient.go b/pkg/cli/cmd/startup/stateclient.go index 3617ed2881..474a1c5874 100644 --- a/pkg/cli/cmd/startup/stateclient.go +++ b/pkg/cli/cmd/startup/stateclient.go @@ -18,10 +18,17 @@ package startup import ( "context" + "errors" + "fmt" + "github.com/radius-project/radius/pkg/azure/tokencredentials" "github.com/radius-project/radius/pkg/cli/controlplane" + "github.com/radius-project/radius/pkg/cli/output" "github.com/radius-project/radius/pkg/cli/pgbackup" "github.com/radius-project/radius/pkg/cli/tfstate" + "github.com/radius-project/radius/pkg/cli/workspaces" + corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" + "github.com/radius-project/radius/pkg/sdk" ) // ControlPlaneScaler scales the database-backed control-plane deployments to zero and back, so @@ -53,6 +60,31 @@ type StateRestoreClient interface { // RestoreTerraform re-creates the Terraform state Secrets from stateDir. RestoreTerraform(ctx context.Context, kubeContext, namespace, stateDir string) error + + // ReconcileHydratedState invokes the reconcile custom action on every application in the + // workspace's plane. It is best-effort: individual per-application failures are recorded in + // the returned reports but never propagate as a fatal error. Called by 'rad startup' after + // ScaleUp so subsequent commands see reality-checked state. + // + // A non-nil error is returned only when the pass could not begin at all (for example, the + // workspace's control plane is unreachable). Callers should treat such errors as advisory and + // still return success from the outer startup command. + ReconcileHydratedState(ctx context.Context, workspace *workspaces.Workspace) ([]ApplicationReconcileReport, error) +} + +// ApplicationReconcileReport captures the per-application outcome of ReconcileHydratedState. One +// entry is produced for every application the reconcile pass attempted, whether it succeeded or +// not. +type ApplicationReconcileReport struct { + // Name is the application resource name (not the fully-qualified resource ID). + Name string + // ResourceCount is the number of child resources the reconcile handler reported an outcome + // for. Zero when the reconcile handler is still a stub, when the application has no + // non-terminal children, or when the reconcile call itself failed. + ResourceCount int + // Err is set when the reconcile call for this application failed. The pass continues to the + // next application regardless. + Err error } // defaultStateRestoreClient is the production implementation. @@ -78,3 +110,81 @@ func (defaultStateRestoreClient) RestoreTerraform(ctx context.Context, kubeConte } return client.Restore(ctx, stateDir) } + +// ReconcileHydratedState lists every application in the workspace's plane and POSTs the +// Radius.Core/applications 'reconcile' custom action on each. Reports are aggregated across +// pagination and returned to the caller. +func (defaultStateRestoreClient) ReconcileHydratedState(ctx context.Context, workspace *workspaces.Workspace) ([]ApplicationReconcileReport, error) { + if workspace == nil { + return nil, errors.New("workspace is required") + } + + connection, err := workspace.Connect(ctx) + if err != nil { + return nil, fmt.Errorf("failed to connect to workspace: %w", err) + } + + clientOptions := sdk.NewClientOptions(connection) + factory, err := corerpv20250801preview.NewClientFactory(&tokencredentials.AnonymousCredential{}, clientOptions) + if err != nil { + return nil, fmt.Errorf("failed to build Radius.Core client factory: %w", err) + } + applications := factory.NewApplicationsClient() + + // Collect application names first so a stalled reconcile does not stall the LIST. + names, err := listApplicationNames(ctx, applications, workspace.Scope) + if err != nil { + return nil, fmt.Errorf("failed to list applications for reconcile: %w", err) + } + + reports := make([]ApplicationReconcileReport, 0, len(names)) + for _, name := range names { + report := ApplicationReconcileReport{Name: name} + resp, err := applications.Reconcile(ctx, workspace.Scope, name, corerpv20250801preview.ReconcileRequest{}, nil) + if err != nil { + report.Err = err + } else { + report.ResourceCount = len(resp.Resources) + } + reports = append(reports, report) + } + return reports, nil +} + +// listApplicationNames walks the paginated ListByScope response for `scope` and returns the +// application resource names. Nil entries and entries without a Name are skipped. +func listApplicationNames(ctx context.Context, client *corerpv20250801preview.ApplicationsClient, scope string) ([]string, error) { + pager := client.NewListByScopePager(scope, &corerpv20250801preview.ApplicationsClientListByScopeOptions{}) + var names []string + for pager.More() { + page, err := pager.NextPage(ctx) + if err != nil { + return nil, err + } + for _, app := range page.Value { + if app == nil || app.Name == nil { + continue + } + names = append(names, *app.Name) + } + } + return names, nil +} + +// logReconcileReports emits one line per application, and a summary line if any application +// failed. Used by rad startup's ReconcileHydratedState stage to surface outcomes in the workflow +// log. +func logReconcileReports(out output.Interface, reports []ApplicationReconcileReport) { + failed := 0 + for _, r := range reports { + if r.Err != nil { + failed++ + out.LogInfo(" reconcile %s: failed (%s)", r.Name, r.Err.Error()) + continue + } + out.LogInfo(" reconcile %s: reconciled %d resource(s)", r.Name, r.ResourceCount) + } + if failed > 0 { + out.LogInfo("Reconcile completed with %d/%d application failures; continuing.", failed, len(reports)) + } +} From 4fe9a673b68b3af4030851d1b640acfabc846bc0 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 10:00:07 -0700 Subject: [PATCH 13/18] dynamicrp: wire reconcile route with a stub handler Adds a POST /{resourceName}/reconcile route to the dynamic-rp resource-group scope and a stub controller that resolves the target resource (404 if missing) and returns an empty ReconcileResponse. Purpose: give the corerp app-scoped orchestrator a stable endpoint to dispatch to before the reality-check logic exists. Every dynamic resource type served by dynamic-rp gets the route automatically, since the router adds it once inside the resource-group / provider-namespace / resource-type subrouter shared by all dynamic types. No per-type registration needed. The reality-check that walks properties.status.outputResources, GETs each Kubernetes object, and PATCHes provisioningState to match reality lands in the next commit. The response shape (ReconcileResponse / ReconcileResourceOutcome) is finalised here so the orchestrator's response-decoding path does not need to change when the handler starts populating the array. Tests cover the empty-report happy path and the 404 for a missing resource. Signed-off-by: Nithya Subramanian --- pkg/dynamicrp/frontend/reconcile.go | 91 +++++++++++++++++++ pkg/dynamicrp/frontend/reconcile_test.go | 108 +++++++++++++++++++++++ pkg/dynamicrp/frontend/routes.go | 4 + 3 files changed, 203 insertions(+) create mode 100644 pkg/dynamicrp/frontend/reconcile.go create mode 100644 pkg/dynamicrp/frontend/reconcile_test.go diff --git a/pkg/dynamicrp/frontend/reconcile.go b/pkg/dynamicrp/frontend/reconcile.go new file mode 100644 index 0000000000..41aede9a98 --- /dev/null +++ b/pkg/dynamicrp/frontend/reconcile.go @@ -0,0 +1,91 @@ +/* +Copyright 2023 The Radius Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package frontend + +import ( + "context" + "net/http" + + v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" + ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" + "github.com/radius-project/radius/pkg/armrpc/rest" + "github.com/radius-project/radius/pkg/dynamicrp/datamodel" + "github.com/radius-project/radius/pkg/ucp/api/v20231001preview" +) + +// ReconcileResourceOutcome mirrors the wire shape defined in Radius.Core's TypeSpec so the +// corerp app-scoped orchestrator can aggregate reports across resource-provider namespaces +// without re-marshaling. Kept lowercase in JSON to match the client's expectations. +type ReconcileResourceOutcome struct { + ResourceID string `json:"resourceId"` + From string `json:"from"` + To string `json:"to"` + Reason string `json:"reason,omitempty"` +} + +// ReconcileResponse is the per-resource reconcile response emitted by dynamic-rp. The corerp +// orchestrator (which fans out per-application) collects one ReconcileResourceOutcome per +// resource; each dynamic-rp reconcile call returns a single-element resources array (or an empty +// array when the resource is already terminal and no work was needed). +type ReconcileResponse struct { + Resources []ReconcileResourceOutcome `json:"resources"` +} + +// Reconcile is the dynamic-rp handler for the reconcile custom action registered on every dynamic +// resource type. See specs/006-state-restoration: when 'rad startup' invokes the app-scoped +// reconcile on Radius.Core/applications, the corerp orchestrator will POST to this handler once +// per non-terminal child resource; the handler walks the resource's outputResources, checks each +// one against its underlying provider, and PATCHes provisioningState to reflect reality. +// +// This is the Phase 1 wiring commit: routing is in place and the handler returns an empty report, +// so the corerp orchestrator can start dispatching without breaking the build. The reality-check +// logic (Kubernetes GETs on outputResources, PATCH back through the RP's normal write path) lands +// in the next commit. +type Reconcile struct { + ctrl.Operation[*datamodel.DynamicResource, datamodel.DynamicResource] + resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource] + ucpClient *v20231001preview.ClientFactory +} + +// NewReconcile constructs the reconcile controller for a dynamic resource type. +func NewReconcile(opts ctrl.Options, resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource], ucpClient *v20231001preview.ClientFactory) (ctrl.Controller, error) { + return &Reconcile{ + Operation: ctrl.NewOperation(opts, resourceOptions), + resourceOptions: resourceOptions, + ucpClient: ucpClient, + }, nil +} + +// Run resolves the target resource (404 if missing) and returns an empty ReconcileResponse. The +// per-outputResource reality check that populates the response lands in a follow-up commit; this +// handler exists now so the corerp orchestrator has a stable endpoint to dispatch to while +// Phase 1 fills in. +func (c *Reconcile) Run(ctx context.Context, w http.ResponseWriter, req *http.Request) (rest.Response, error) { + sCtx := v1.ARMRequestContextFromContext(ctx) + + // Route: /planes/radius/{plane}/resourceGroups/{rg}/providers/{ns}/{type}/{name}/reconcile + resourceID := sCtx.ResourceID.Truncate() + resource, _, err := c.GetResource(ctx, resourceID) + if err != nil { + return nil, err + } + if resource == nil { + return rest.NewNotFoundResponse(sCtx.ResourceID), nil + } + + return rest.NewOKResponse(&ReconcileResponse{Resources: []ReconcileResourceOutcome{}}), nil +} diff --git a/pkg/dynamicrp/frontend/reconcile_test.go b/pkg/dynamicrp/frontend/reconcile_test.go new file mode 100644 index 0000000000..c82073581d --- /dev/null +++ b/pkg/dynamicrp/frontend/reconcile_test.go @@ -0,0 +1,108 @@ +/* +Copyright 2023 The Radius Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package frontend + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" + "github.com/radius-project/radius/pkg/armrpc/frontend/controller" + "github.com/radius-project/radius/pkg/armrpc/rpctest" + "github.com/radius-project/radius/pkg/components/database" + "github.com/radius-project/radius/pkg/dynamicrp/datamodel" + "github.com/radius-project/radius/pkg/dynamicrp/datamodel/converter" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" +) + +const reconcileTestURL = "/planes/radius/local/resourceGroups/test-group/providers/Applications.Test/testResources/myResource/reconcile?api-version=2023-10-01-preview" + +func newReconcileController(t *testing.T, databaseClient database.Client) controller.Controller { + t.Helper() + + ucpClient, err := testUCPClientFactoryWithSensitiveFields() + require.NoError(t, err) + + opts := controller.Options{DatabaseClient: databaseClient} + resourceOpts := controller.ResourceOptions[datamodel.DynamicResource]{ + RequestConverter: converter.DynamicResourceDataModelFromVersioned, + ResponseConverter: converter.DynamicResourceDataModelToVersioned, + } + + c, err := NewReconcile(opts, resourceOpts, ucpClient) + require.NoError(t, err) + return c +} + +func TestReconcile_ReturnsEmptyReport(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := &datamodel.DynamicResource{ + ID: testResourceID, + Name: "myResource", + Type: "Applications.Test/testResources", + UpdatedAPIVersion: testAPIVersion, + AsyncProvisioningState: v1.ProvisioningStateSucceeded, + Properties: map[string]any{}, + } + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + + c := newReconcileController(t, databaseClient) + + req, err := http.NewRequest(http.MethodPost, reconcileTestURL, nil) + require.NoError(t, err) + ctx := rpctest.NewARMRequestContext(req) + w := httptest.NewRecorder() + + resp, err := c.Run(ctx, w, req) + require.NoError(t, err) + require.NoError(t, resp.Apply(ctx, w, req)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + var body ReconcileResponse + require.NoError(t, json.NewDecoder(w.Result().Body).Decode(&body)) + require.NotNil(t, body.Resources) + require.Empty(t, body.Resources, "Phase 1 wiring stub must return an empty resources array") +} + +func TestReconcile_NotFound(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(nil, &database.ErrNotFound{}) + + c := newReconcileController(t, databaseClient) + + req, err := http.NewRequest(http.MethodPost, reconcileTestURL, nil) + require.NoError(t, err) + ctx := rpctest.NewARMRequestContext(req) + w := httptest.NewRecorder() + + resp, err := c.Run(ctx, w, req) + require.NoError(t, err) + require.NoError(t, resp.Apply(ctx, w, req)) + require.Equal(t, http.StatusNotFound, w.Result().StatusCode) +} diff --git a/pkg/dynamicrp/frontend/routes.go b/pkg/dynamicrp/frontend/routes.go index 23f6343c03..35fd83fb23 100644 --- a/pkg/dynamicrp/frontend/routes.go +++ b/pkg/dynamicrp/frontend/routes.go @@ -106,6 +106,10 @@ func (s *Service) registerRoutes( func(opts controller.Options) (controller.Controller, error) { return defaultoperation.NewDefaultAsyncDelete(opts, resourceOptions) })) + r.Post("/{resourceName}/reconcile", dynamicOperationHandler(v1.OperationPost, controllerOptions, + func(opts controller.Options) (controller.Controller, error) { + return NewReconcile(opts, resourceOptions, ucpClient) + })) }) }) From a197e91dc4496f6a65be55a68643b9594b869313 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 10:15:17 -0700 Subject: [PATCH 14/18] dynamicrp: reality-check outputResources in reconcile handler Replace the reconcile placeholder with the Phase 1 reality check. For a non-terminal dynamic resource the handler walks properties.status.outputResources, does a Kubernetes GET on each output via the runtime client the RP already holds, and aggregates the outcomes to decide the new provisioningState: * all outputs gone (404 in k8s) -> transition to Failed * all outputs settled (GET succeeded) -> transition to Succeeded * any output skipped (cloud output, unresolved API version, transient GET failure) -> leave provisioningState unchanged, because we refuse to lie about state we could not verify The new state is persisted through the frontend controller's SaveResource path (no direct database writes). Cloud outputs served by Terraform / cloud providers are reported as 'skipped: cloud output not yet reality-checked' so the caller sees exactly which resources were and were not verified. Full cloud output reality checking is a follow-up. The frontend service now fetches the Kubernetes runtime client and the discovery client from the KubernetesClientProvider and threads them into the reconcile controller. The runtime client rides in controller.Options alongside the other controllers; the discovery client is passed directly to NewReconcile because no other frontend controller needs it. The test host was updated to plant an empty fake DiscoveryClient so plane bring-up still succeeds without a real cluster. Unit tests cover the four aggregation branches: 404 -> Failed, present -> Succeeded, cloud output -> skipped + state unchanged, terminal state -> empty report. A separate test still asserts the 404-on-missing-resource behavior. The corerp orchestrator on Radius.Core/applications still returns an empty report; the child walk lands in the next commit. Signed-off-by: Nithya Subramanian --- pkg/dynamicrp/frontend/reconcile.go | 222 +++++++++++++++++++-- pkg/dynamicrp/frontend/reconcile_test.go | 233 ++++++++++++++++++++--- pkg/dynamicrp/frontend/routes.go | 4 +- pkg/dynamicrp/frontend/service.go | 19 +- pkg/dynamicrp/testhost/host.go | 10 + 5 files changed, 439 insertions(+), 49 deletions(-) diff --git a/pkg/dynamicrp/frontend/reconcile.go b/pkg/dynamicrp/frontend/reconcile.go index 41aede9a98..fd04c60b3d 100644 --- a/pkg/dynamicrp/frontend/reconcile.go +++ b/pkg/dynamicrp/frontend/reconcile.go @@ -18,13 +18,24 @@ package frontend import ( "context" + "fmt" "net/http" + "strings" v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" "github.com/radius-project/radius/pkg/armrpc/rest" "github.com/radius-project/radius/pkg/dynamicrp/datamodel" + rpv1 "github.com/radius-project/radius/pkg/rp/v1" "github.com/radius-project/radius/pkg/ucp/api/v20231001preview" + resources_kubernetes "github.com/radius-project/radius/pkg/ucp/resources/kubernetes" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/discovery" + runtimeclient "sigs.k8s.io/controller-runtime/pkg/client" ) // ReconcileResourceOutcome mirrors the wire shape defined in Radius.Core's TypeSpec so the @@ -47,39 +58,60 @@ type ReconcileResponse struct { // Reconcile is the dynamic-rp handler for the reconcile custom action registered on every dynamic // resource type. See specs/006-state-restoration: when 'rad startup' invokes the app-scoped -// reconcile on Radius.Core/applications, the corerp orchestrator will POST to this handler once -// per non-terminal child resource; the handler walks the resource's outputResources, checks each -// one against its underlying provider, and PATCHes provisioningState to reflect reality. +// reconcile on Radius.Core/applications, the corerp orchestrator POSTs to this handler once per +// non-terminal child resource. The handler walks the resource's outputResources, checks each one +// against its underlying provider, and updates provisioningState to reflect reality. // -// This is the Phase 1 wiring commit: routing is in place and the handler returns an empty report, -// so the corerp orchestrator can start dispatching without breaking the build. The reality-check -// logic (Kubernetes GETs on outputResources, PATCH back through the RP's normal write path) lands -// in the next commit. +// This is the Phase 1 reality-check implementation. For each output resource we do a Kubernetes +// GET via the runtime client the RP already holds and categorize the result as gone (404), +// settled (present), or skipped (unknown provider / transient error). We then aggregate: all +// outputs gone → Failed, all settled → Succeeded, otherwise the current state is retained. Cloud +// outputs (Terraform-backed Azure / AWS resources) are recorded as skipped without being touched +// — that lives in a follow-up commit. type Reconcile struct { ctrl.Operation[*datamodel.DynamicResource, datamodel.DynamicResource] resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource] ucpClient *v20231001preview.ClientFactory + discovery discovery.DiscoveryInterface } -// NewReconcile constructs the reconcile controller for a dynamic resource type. -func NewReconcile(opts ctrl.Options, resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource], ucpClient *v20231001preview.ClientFactory) (ctrl.Controller, error) { +// NewReconcile constructs the reconcile controller for a dynamic resource type. The runtime +// client is read from opts.KubeClient at request time so the same handler serves every dynamic +// type without per-type wiring. +func NewReconcile(opts ctrl.Options, resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource], ucpClient *v20231001preview.ClientFactory, discovery discovery.DiscoveryInterface) (ctrl.Controller, error) { return &Reconcile{ Operation: ctrl.NewOperation(opts, resourceOptions), resourceOptions: resourceOptions, ucpClient: ucpClient, + discovery: discovery, }, nil } -// Run resolves the target resource (404 if missing) and returns an empty ReconcileResponse. The -// per-outputResource reality check that populates the response lands in a follow-up commit; this -// handler exists now so the corerp orchestrator has a stable endpoint to dispatch to while -// Phase 1 fills in. +// outputStatus is the reality category assigned to a single outputResource. +type outputStatus string + +const ( + outputGone outputStatus = "gone" + outputSettled outputStatus = "settled" + outputSkipped outputStatus = "skipped" +) + +type outputCheck struct { + id string + status outputStatus + reason string +} + +// Run reconciles the target resource's provisioningState against the reality of its +// outputResources and returns a single-element report for the resource. If the resource is +// already in a terminal state, no work is done and an empty resources array is returned so the +// caller can distinguish "no-op" from "reconciled". func (c *Reconcile) Run(ctx context.Context, w http.ResponseWriter, req *http.Request) (rest.Response, error) { sCtx := v1.ARMRequestContextFromContext(ctx) // Route: /planes/radius/{plane}/resourceGroups/{rg}/providers/{ns}/{type}/{name}/reconcile resourceID := sCtx.ResourceID.Truncate() - resource, _, err := c.GetResource(ctx, resourceID) + resource, etag, err := c.GetResource(ctx, resourceID) if err != nil { return nil, err } @@ -87,5 +119,165 @@ func (c *Reconcile) Run(ctx context.Context, w http.ResponseWriter, req *http.Re return rest.NewNotFoundResponse(sCtx.ResourceID), nil } - return rest.NewOKResponse(&ReconcileResponse{Resources: []ReconcileResourceOutcome{}}), nil + fromState := resource.ProvisioningState() + if fromState.IsTerminal() { + return rest.NewOKResponse(&ReconcileResponse{Resources: []ReconcileResourceOutcome{}}), nil + } + + checks := make([]outputCheck, 0) + for _, out := range resource.OutputResources() { + checks = append(checks, c.checkOutput(ctx, out)) + } + + toState := aggregateReconcileState(fromState, checks) + outcome := ReconcileResourceOutcome{ + ResourceID: resourceID.String(), + From: string(fromState), + To: string(toState), + Reason: summarizeChecks(checks), + } + + if toState != fromState { + resource.SetProvisioningState(toState) + if _, err := c.SaveResource(ctx, resourceID.String(), resource, etag); err != nil { + return nil, err + } + } + + return rest.NewOKResponse(&ReconcileResponse{Resources: []ReconcileResourceOutcome{outcome}}), nil +} + +// checkOutput probes a single output resource against its underlying provider. +func (c *Reconcile) checkOutput(ctx context.Context, out rpv1.OutputResource) outputCheck { + idStr := out.ID.String() + + // Only Kubernetes outputs are reality-checked in this prototype. Terraform-backed cloud + // outputs (Azure, AWS) are reported as skipped so the caller can act on them separately. + scopes := out.ID.ScopeSegments() + if len(scopes) == 0 || !strings.EqualFold(scopes[0].Type, resources_kubernetes.PlaneTypeKubernetes) { + return outputCheck{id: idStr, status: outputSkipped, reason: "cloud output not yet reality-checked"} + } + + kubeClient := c.Options().KubeClient + if kubeClient == nil { + return outputCheck{id: idStr, status: outputSkipped, reason: "kubernetes runtime client not configured"} + } + + group, kind, namespace, name := resources_kubernetes.ToParts(out.ID) + + version, err := c.lookupKubernetesAPIVersion(group, kind, namespace != "") + if err != nil { + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("could not resolve Kubernetes API version for %s/%s: %v", group, kind, err)} + } + + obj := &unstructured.Unstructured{} + obj.SetGroupVersionKind(schema.GroupVersionKind{Group: group, Version: version, Kind: kind}) + + err = kubeClient.Get(ctx, runtimeclient.ObjectKey{Namespace: namespace, Name: name}, obj) + switch { + case apierrors.IsNotFound(err): + return outputCheck{id: idStr, status: outputGone, reason: "kubernetes object not found"} + case err != nil: + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("kubernetes GET failed: %v", err)} + default: + return outputCheck{id: idStr, status: outputSettled} + } +} + +// lookupKubernetesAPIVersion resolves the preferred API version for a group+kind via the +// discovery client. This mirrors the walk in the corerp kubernetes handler; keeping a copy here +// decouples the reconcile path from the deployment codepath. +func (c *Reconcile) lookupKubernetesAPIVersion(group, kind string, namespaced bool) (string, error) { + if c.discovery == nil { + return "", fmt.Errorf("discovery client is not configured") + } + + // resources_kubernetes.ToParts maps the "core" ProviderNamespace back to "" for the built-in + // group. Discovery reports the same group as "" — normalize before comparing. + normalizedGroup := group + if normalizedGroup == "core" { + normalizedGroup = "" + } + + var lists []*metav1.APIResourceList + var err error + if namespaced { + lists, err = c.discovery.ServerPreferredNamespacedResources() + } else { + lists, err = c.discovery.ServerPreferredResources() + } + if err != nil { + return "", err + } + + for _, list := range lists { + gv, parseErr := schema.ParseGroupVersion(list.GroupVersion) + if parseErr != nil { + continue + } + if !strings.EqualFold(gv.Group, normalizedGroup) { + continue + } + for _, r := range list.APIResources { + if strings.EqualFold(r.Kind, kind) { + return gv.Version, nil + } + } + } + + return "", fmt.Errorf("no preferred API version for %s/%s", group, kind) +} + +// aggregateReconcileState folds the per-output outcomes into a single provisioningState decision. +// +// Rules: +// - No outputs on record → leave state unchanged. We do not assume "gone" without evidence. +// - All outputs gone → move to Failed. +// - Any output skipped (cloud output, unresolved version, transient GET failure) → leave state +// unchanged. We refuse to lie about state we could not verify. +// - All outputs settled → move to Succeeded. +// - Otherwise (mix of settled and gone with no skipped) → leave state unchanged. +func aggregateReconcileState(from v1.ProvisioningState, checks []outputCheck) v1.ProvisioningState { + if len(checks) == 0 { + return from + } + hasSkipped := false + hasSettled := false + hasGone := false + for _, c := range checks { + switch c.status { + case outputSkipped: + hasSkipped = true + case outputSettled: + hasSettled = true + case outputGone: + hasGone = true + } + } + if hasSkipped { + return from + } + if hasGone && !hasSettled { + return v1.ProvisioningStateFailed + } + if hasSettled && !hasGone { + return v1.ProvisioningStateSucceeded + } + return from +} + +// summarizeChecks flattens the per-output outcomes into one human-readable reason string. +func summarizeChecks(checks []outputCheck) string { + if len(checks) == 0 { + return "no output resources on record" + } + parts := make([]string, 0, len(checks)) + for _, c := range checks { + s := fmt.Sprintf("%s: %s", c.id, c.status) + if c.reason != "" { + s += " (" + c.reason + ")" + } + parts = append(parts, s) + } + return strings.Join(parts, "; ") } diff --git a/pkg/dynamicrp/frontend/reconcile_test.go b/pkg/dynamicrp/frontend/reconcile_test.go index c82073581d..043762c908 100644 --- a/pkg/dynamicrp/frontend/reconcile_test.go +++ b/pkg/dynamicrp/frontend/reconcile_test.go @@ -17,6 +17,7 @@ limitations under the License. package frontend import ( + "context" "encoding/json" "net/http" "net/http/httptest" @@ -28,49 +29,103 @@ import ( "github.com/radius-project/radius/pkg/components/database" "github.com/radius-project/radius/pkg/dynamicrp/datamodel" "github.com/radius-project/radius/pkg/dynamicrp/datamodel/converter" + k8stest "github.com/radius-project/radius/test/k8sutil" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + runtimeclient "sigs.k8s.io/controller-runtime/pkg/client" ) const reconcileTestURL = "/planes/radius/local/resourceGroups/test-group/providers/Applications.Test/testResources/myResource/reconcile?api-version=2023-10-01-preview" -func newReconcileController(t *testing.T, databaseClient database.Client) controller.Controller { +const ( + deploymentOutputID = "/planes/kubernetes/local/namespaces/default/providers/apps/Deployment/my-deployment" + azureOutputID = "/planes/azure/mycloud/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.Storage/storageAccounts/mystorage" +) + +// scheme registers the built-in Kubernetes types the reconcile handler will look up. +func reconcileTestScheme(t *testing.T) *runtime.Scheme { t.Helper() + s := runtime.NewScheme() + require.NoError(t, appsv1.AddToScheme(s)) + require.NoError(t, corev1.AddToScheme(s)) + return s +} - ucpClient, err := testUCPClientFactoryWithSensitiveFields() - require.NoError(t, err) +// reconcileTestDiscovery returns a discovery client that reports the two built-in kinds the +// reconcile handler needs to resolve for these tests. +func reconcileTestDiscovery() *k8stest.DiscoveryClient { + return &k8stest.DiscoveryClient{ + Resources: []*metav1.APIResourceList{ + { + GroupVersion: "apps/v1", + APIResources: []metav1.APIResource{ + {Name: "deployments", Kind: "Deployment", Namespaced: true, Version: "v1"}, + }, + }, + { + GroupVersion: "v1", + APIResources: []metav1.APIResource{ + {Name: "services", Kind: "Service", Namespaced: true, Version: "v1"}, + }, + }, + }, + } +} - opts := controller.Options{DatabaseClient: databaseClient} +func newReconcileController(t *testing.T, databaseClient database.Client, kubeClient runtimeclient.Client) controller.Controller { + t.Helper() + + opts := controller.Options{ + DatabaseClient: databaseClient, + KubeClient: kubeClient, + } resourceOpts := controller.ResourceOptions[datamodel.DynamicResource]{ RequestConverter: converter.DynamicResourceDataModelFromVersioned, ResponseConverter: converter.DynamicResourceDataModelToVersioned, } - c, err := NewReconcile(opts, resourceOpts, ucpClient) + c, err := NewReconcile(opts, resourceOpts, nil, reconcileTestDiscovery()) require.NoError(t, err) return c } -func TestReconcile_ReturnsEmptyReport(t *testing.T) { - mctrl := gomock.NewController(t) - defer mctrl.Finish() +// newReconcileResource builds a DynamicResource with the given provisioningState and +// outputResources planted under properties.status. outputResources are stored as []map[string]any +// on disk; OutputResources() JSON-unmarshals them back into rpv1.OutputResource values. +func newReconcileResource(state v1.ProvisioningState, outputIDs ...string) *datamodel.DynamicResource { + outputs := make([]map[string]any, 0, len(outputIDs)) + radiusManaged := true + for _, id := range outputIDs { + outputs = append(outputs, map[string]any{ + "localID": "output-" + id, + "id": id, + "radiusManaged": radiusManaged, + }) + } - resource := &datamodel.DynamicResource{ + props := map[string]any{} + if len(outputs) > 0 { + props["status"] = map[string]any{ + "outputResources": outputs, + } + } + + return &datamodel.DynamicResource{ ID: testResourceID, Name: "myResource", Type: "Applications.Test/testResources", UpdatedAPIVersion: testAPIVersion, - AsyncProvisioningState: v1.ProvisioningStateSucceeded, - Properties: map[string]any{}, + AsyncProvisioningState: state, + Properties: props, } - storeObject := rpctest.FakeStoreObject(resource) - storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} - - databaseClient := database.NewMockClient(mctrl) - databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) - - c := newReconcileController(t, databaseClient) +} +func runReconcile(t *testing.T, c controller.Controller) *httptest.ResponseRecorder { + t.Helper() req, err := http.NewRequest(http.MethodPost, reconcileTestURL, nil) require.NoError(t, err) ctx := rpctest.NewARMRequestContext(req) @@ -79,12 +134,14 @@ func TestReconcile_ReturnsEmptyReport(t *testing.T) { resp, err := c.Run(ctx, w, req) require.NoError(t, err) require.NoError(t, resp.Apply(ctx, w, req)) - require.Equal(t, http.StatusOK, w.Result().StatusCode) + return w +} +func decodeReconcileResponse(t *testing.T, w *httptest.ResponseRecorder) ReconcileResponse { + t.Helper() var body ReconcileResponse require.NoError(t, json.NewDecoder(w.Result().Body).Decode(&body)) - require.NotNil(t, body.Resources) - require.Empty(t, body.Resources, "Phase 1 wiring stub must return an empty resources array") + return body } func TestReconcile_NotFound(t *testing.T) { @@ -94,15 +151,131 @@ func TestReconcile_NotFound(t *testing.T) { databaseClient := database.NewMockClient(mctrl) databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(nil, &database.ErrNotFound{}) - c := newReconcileController(t, databaseClient) - - req, err := http.NewRequest(http.MethodPost, reconcileTestURL, nil) - require.NoError(t, err) - ctx := rpctest.NewARMRequestContext(req) - w := httptest.NewRecorder() + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t))) - resp, err := c.Run(ctx, w, req) - require.NoError(t, err) - require.NoError(t, resp.Apply(ctx, w, req)) + w := runReconcile(t, c) require.Equal(t, http.StatusNotFound, w.Result().StatusCode) } + +// A resource already in a terminal state has nothing to reconcile: return 200 with an empty +// resources array so the orchestrator can distinguish "no-op" from "reconciled". +func TestReconcile_TerminalState_ReturnsEmpty(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateSucceeded, deploymentOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + // Save must not be called: no state transition when already terminal. + + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t))) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Empty(t, body.Resources) +} + +// The transcript's failing case: a resource is hydrated in Updating but its Kubernetes object no +// longer exists. Reality check reports "gone" for the single output, and the handler transitions +// provisioningState to Failed and persists it so subsequent deletes are unblocked. +func TestReconcile_KubernetesGone_TransitionsToFailed(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateUpdating, deploymentOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + databaseClient.EXPECT(). + Save(gomock.Any(), gomock.Any(), gomock.Any()). + DoAndReturn(func(_ context.Context, obj *database.Object, _ ...database.SaveOptions) error { + saved, ok := obj.Data.(*datamodel.DynamicResource) + require.True(t, ok, "saved payload must be a DynamicResource") + require.Equal(t, v1.ProvisioningStateFailed, saved.ProvisioningState()) + return nil + }) + + // Empty cluster: the Deployment does not exist. + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t))) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, testResourceID, body.Resources[0].ResourceID) + require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].From) + require.Equal(t, string(v1.ProvisioningStateFailed), body.Resources[0].To) + require.Contains(t, body.Resources[0].Reason, "gone") +} + +// A resource whose Kubernetes output exists gets promoted from Updating to Succeeded. +func TestReconcile_KubernetesSettled_TransitionsToSucceeded(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateUpdating, deploymentOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + databaseClient.EXPECT(). + Save(gomock.Any(), gomock.Any(), gomock.Any()). + DoAndReturn(func(_ context.Context, obj *database.Object, _ ...database.SaveOptions) error { + saved := obj.Data.(*datamodel.DynamicResource) + require.Equal(t, v1.ProvisioningStateSucceeded, saved.ProvisioningState()) + return nil + }) + + // Populate the cluster with the deployment that outputResources references. + deployment := &appsv1.Deployment{ + ObjectMeta: metav1.ObjectMeta{Name: "my-deployment", Namespace: "default"}, + } + kubeClient := k8stest.NewFakeKubeClient(reconcileTestScheme(t), deployment) + + c := newReconcileController(t, databaseClient, kubeClient) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].From) + require.Equal(t, string(v1.ProvisioningStateSucceeded), body.Resources[0].To) + require.Contains(t, body.Resources[0].Reason, "settled") +} + +// A Terraform-backed cloud output cannot be reality-checked in Phase 1: it is recorded as +// "skipped" and the resource's provisioningState is left unchanged. No Save must be issued. +func TestReconcile_CloudOutput_Skipped(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateUpdating, azureOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + // Save must not be called: skipped output → state unchanged. + + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t))) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].From) + require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].To) + require.Contains(t, body.Resources[0].Reason, "skipped") + require.Contains(t, body.Resources[0].Reason, "cloud output") +} diff --git a/pkg/dynamicrp/frontend/routes.go b/pkg/dynamicrp/frontend/routes.go index 35fd83fb23..6f97c2197d 100644 --- a/pkg/dynamicrp/frontend/routes.go +++ b/pkg/dynamicrp/frontend/routes.go @@ -29,6 +29,7 @@ import ( "github.com/radius-project/radius/pkg/dynamicrp/datamodel/converter" "github.com/radius-project/radius/pkg/ucp/api/v20231001preview" "github.com/radius-project/radius/pkg/validator" + "k8s.io/client-go/discovery" ) func (s *Service) registerRoutes( @@ -36,6 +37,7 @@ func (s *Service) registerRoutes( controllerOptions controller.Options, ucpClient *v20231001preview.ClientFactory, handler *encryption.SensitiveDataHandler, + discoveryClient discovery.DiscoveryInterface, ) error { // Return ARM errors for invalid requests. r.NotFound(validator.APINotFoundHandler()) @@ -108,7 +110,7 @@ func (s *Service) registerRoutes( })) r.Post("/{resourceName}/reconcile", dynamicOperationHandler(v1.OperationPost, controllerOptions, func(opts controller.Options) (controller.Controller, error) { - return NewReconcile(opts, resourceOptions, ucpClient) + return NewReconcile(opts, resourceOptions, ucpClient, discoveryClient) })) }) }) diff --git a/pkg/dynamicrp/frontend/service.go b/pkg/dynamicrp/frontend/service.go index 85d57c228b..27bdfde25c 100644 --- a/pkg/dynamicrp/frontend/service.go +++ b/pkg/dynamicrp/frontend/service.go @@ -79,17 +79,30 @@ func (s *Service) initialize(ctx context.Context) (*http.Server, error) { return nil, fmt.Errorf("failed to create sensitive data handler: %w", err) } + // The reconcile handler reality-checks each resource's outputResources against the target + // cluster. Both clients are cluster-scoped and shared across every dynamic type served by + // dynamic-rp; failing to acquire either here is fatal — the same failure mode as the + // encryption key we just loaded from a Kubernetes secret. + kubeClient, err := s.options.KubernetesProvider.RuntimeClient() + if err != nil { + return nil, fmt.Errorf("failed to get Kubernetes runtime client: %w", err) + } + discoveryClient, err := s.options.KubernetesProvider.DiscoveryClient() + if err != nil { + return nil, fmt.Errorf("failed to get Kubernetes discovery client: %w", err) + } + controllerOptions := controller.Options{ Address: s.options.Config.Server.Address(), PathBase: s.options.Config.Server.PathBase, DatabaseClient: databaseClient, StatusManager: s.options.StatusManager, - KubeClient: nil, // Unused by DynamicRP - ResourceType: "", // Set dynamically + KubeClient: kubeClient, + ResourceType: "", // Set dynamically } - err = s.registerRoutes(r, controllerOptions, ucpClient, sensitiveDataHandler) + err = s.registerRoutes(r, controllerOptions, ucpClient, sensitiveDataHandler, discoveryClient) if err != nil { return nil, fmt.Errorf("failed to register routes: %w", err) } diff --git a/pkg/dynamicrp/testhost/host.go b/pkg/dynamicrp/testhost/host.go index 6990eed7c1..974767312a 100644 --- a/pkg/dynamicrp/testhost/host.go +++ b/pkg/dynamicrp/testhost/host.go @@ -41,8 +41,11 @@ import ( ucptesthost "github.com/radius-project/radius/pkg/ucp/testhost" "github.com/stretchr/testify/require" corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "sigs.k8s.io/controller-runtime/pkg/client/fake" + + k8stest "github.com/radius-project/radius/test/k8sutil" ) // TestHostOptions supports configuring the dynamic-rp test host. @@ -199,4 +202,11 @@ func setupFakeKubernetesClient(t *testing.T, options *dynamicrp.Options) { // Set the runtime client on the Kubernetes provider options.KubernetesProvider.SetRuntimeClient(fakeClient) + + // The frontend service's reconcile route needs a discovery client to resolve API versions + // for outputResource GETs; wire an empty fake so plane bring-up succeeds. Individual tests + // that exercise reconcile can override this via SetDiscoveryClient. + options.KubernetesProvider.SetDiscoveryClient(&k8stest.DiscoveryClient{ + Resources: []*metav1.APIResourceList{}, + }) } From abf4868904d5aec65b65b41d92db7998c5deb717 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 10:29:20 -0700 Subject: [PATCH 15/18] corerp: implement application-scoped reconcile child walk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the Phase 0 stub on Radius.Core/applications/reconcile with the Phase 1 orchestrator. When 'rad startup' invokes this action it now: 1. Looks up the application (404 if missing). 2. Enumerates the resource-type registry via UCP, filters out the built-in namespaces that don't implement /reconcile (Applications.Core / Dapr / Datastores / Messaging, Radius.Core, Microsoft.Resources), and lists every remaining resource associated with the application. This mirrors the walk getGraph already uses so the two custom actions stay consistent. 3. Skips children whose provisioningState is already terminal. 4. Fans out reconcile POSTs concurrently through the shared UCP connection, bounded to 8 in flight and capped at 15 s per child so one unresponsive RP cannot hang the whole reconcile (spec §Risks). 5. Aggregates each RP's per-resource outcomes into one ReconcileResponse. A dispatch failure (timeout, transport error, non-2xx) does not fail the reconcile: it is recorded as a per-child skipped outcome with the provisioningState left unchanged, so the caller always gets a report. 6. Returns 404 or 405 from a child RP as 'skipped: RP does not implement reconcile', so the orchestrator stays forward-compatible with RPs that haven't wired reconcile yet. The child walk and per-child dispatch are exposed as injectable hooks on the controller so the orchestration logic can be unit-tested without standing up a live UCP. The public constructor keeps its existing signature and installs the UCP-backed defaults. Tests cover: 404 on missing application, database error on lookup, empty child list, terminal children skipped, multi-child fan-out with mixed outcomes, and per-child dispatch failure recorded as skipped without failing the reconcile. Phase 1 is now feature-complete. An httptest-backed integration test that exercises the end-to-end path lands in the next commit. Signed-off-by: Nithya Subramanian --- .../v20250801preview/reconcile.go | 261 ++++++++++++++-- .../v20250801preview/reconcile_test.go | 279 ++++++++++++++---- 2 files changed, 456 insertions(+), 84 deletions(-) diff --git a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go index 83bcf2faed..0bd2f7a8e1 100644 --- a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go +++ b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile.go @@ -17,54 +17,111 @@ limitations under the License. package v20250801preview import ( + "bytes" "context" + "encoding/json" + "fmt" "net/http" + "strings" + "sync" + "time" + + "golang.org/x/sync/errgroup" v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" "github.com/radius-project/radius/pkg/armrpc/rest" + "github.com/radius-project/radius/pkg/cli/clients" + "github.com/radius-project/radius/pkg/cli/clients_new/generated" corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" "github.com/radius-project/radius/pkg/corerp/datamodel" "github.com/radius-project/radius/pkg/corerp/datamodel/converter" "github.com/radius-project/radius/pkg/sdk" + "github.com/radius-project/radius/pkg/to" + "github.com/radius-project/radius/pkg/ucp/resources" + "github.com/radius-project/radius/pkg/ucp/ucplog" ) var _ ctrl.Controller = (*Reconcilev20250801preview)(nil) -// Reconcilev20250801preview is the controller implementation for the reconcile custom action on -// Radius.Core/applications. It reconciles the hydrated state of every non-terminal child resource -// against its underlying provider (Kubernetes, cloud SDKs) and rewrites provisioningState in the -// state store to match reality. Called by `rad startup` after the state archive is loaded, so a -// subsequent `rad app delete` is not blocked by 409s on resources whose real state has moved on. -// See specs/006-state-restoration for the design. -// -// This is the Phase 0 stub: it validates the target application exists and returns an empty -// report. The corerp orchestrator that walks children and dispatches per-resource reconcile to -// dynamic-rp is added in a follow-up commit alongside the dynamic-rp handler that does the -// reality check. +// staticResourceProviderNamespaces are the built-in resource providers whose resources are not +// reality-checked by the orchestrator (either because they're served by a static RP that does not +// implement /reconcile, or because they're the parent Radius.Core namespace itself). Everything +// else is assumed to be served by dynamic-rp, which registers /reconcile on every dynamic type. +var staticResourceProviderNamespaces = map[string]struct{}{ + "Applications.Core": {}, + "Applications.Dapr": {}, + "Applications.Datastores": {}, + "Applications.Messaging": {}, + "Radius.Core": {}, + "Microsoft.Resources": {}, +} + +const ( + // reconcileChildTimeout caps one child RP's reconcile call so a single unresponsive RP cannot + // hang the whole orchestrator (plan §Risks). + reconcileChildTimeout = 15 * time.Second + // reconcileChildConcurrency bounds fan-out so we don't stampede UCP or the target RPs. + reconcileChildConcurrency = 8 +) + +// Reconcilev20250801preview is the controller for the reconcile custom action on +// Radius.Core/applications. When `rad startup` invokes it after loading a state archive, it walks +// the application's dynamic children, POSTs the reconcile action to each one's RP (dynamic-rp +// today), and aggregates the per-resource outcomes into one response. Terminal children are +// skipped and do not appear in the response. See specs/006-state-restoration for the end-to-end +// design. type Reconcilev20250801preview struct { ctrl.Operation[*datamodel.Application_v20250801preview, datamodel.Application_v20250801preview] - // connection is unused in the Phase 0 stub but wired now so the constructor signature stays - // stable when the orchestrator lands and needs to fan out through UCP. connection sdk.Connection + + // listChildren enumerates the resources associated with the application, restricted to + // dynamic-rp-served namespaces. Injectable so unit tests can stub the child walk without + // standing up UCP. + listChildren func(ctx context.Context, applicationID resources.ID) ([]generated.GenericResource, error) + // reconcileChild POSTs the reconcile action to one child resource and returns its RP's + // per-resource outcomes. Also injectable. + reconcileChild func(ctx context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) } -// NewReconcilev20250801preview creates a new instance of the Reconcilev20250801preview controller. +// NewReconcilev20250801preview constructs the reconcile controller with production defaults +// (UCP-backed child walk and per-child dispatch). func NewReconcilev20250801preview(opts ctrl.Options, connection sdk.Connection) (ctrl.Controller, error) { - return &Reconcilev20250801preview{ - ctrl.NewOperation(opts, + return newReconcilev20250801preview(opts, connection, nil, nil), nil +} + +// newReconcilev20250801preview constructs a Reconcile controller with optional hook overrides. +// Passing nil for a hook installs the default UCP-backed implementation. Tests use this directly +// to inject fakes without standing up a UCP endpoint. +func newReconcilev20250801preview( + opts ctrl.Options, + connection sdk.Connection, + listChildren func(ctx context.Context, applicationID resources.ID) ([]generated.GenericResource, error), + reconcileChild func(ctx context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error), +) *Reconcilev20250801preview { + c := &Reconcilev20250801preview{ + Operation: ctrl.NewOperation(opts, ctrl.ResourceOptions[datamodel.Application_v20250801preview]{ RequestConverter: converter.Application20250801DataModelFromVersioned, ResponseConverter: converter.Application20250801DataModelToVersioned, }, ), - connection, - }, nil + connection: connection, + } + if listChildren != nil { + c.listChildren = listChildren + } else { + c.listChildren = c.defaultListChildren + } + if reconcileChild != nil { + c.reconcileChild = reconcileChild + } else { + c.reconcileChild = c.defaultReconcileChild + } + return c } -// Run handles the reconcile custom action for Radius.Core/applications. In this Phase 0 stub it -// looks up the application (404s if missing) and returns an empty ReconcileResponse. The child -// walk and per-resource dispatch land in a follow-up commit. +// Run handles the reconcile custom action for Radius.Core/applications. func (c *Reconcilev20250801preview) Run(ctx context.Context, w http.ResponseWriter, req *http.Request) (rest.Response, error) { sCtx := v1.ARMRequestContextFromContext(ctx) @@ -78,7 +135,163 @@ func (c *Reconcilev20250801preview) Run(ctx context.Context, w http.ResponseWrit return rest.NewNotFoundResponse(sCtx.ResourceID), nil } - return rest.NewOKResponse(&corerpv20250801preview.ReconcileResponse{ - Resources: []*corerpv20250801preview.ReconcileResourceOutcome{}, - }), nil + children, err := c.listChildren(ctx, applicationID) + if err != nil { + return nil, fmt.Errorf("failed to enumerate application children: %w", err) + } + + logger := ucplog.FromContextOrDiscard(ctx) + outcomes := make([]*corerpv20250801preview.ReconcileResourceOutcome, 0, len(children)) + var mu sync.Mutex + sem := make(chan struct{}, reconcileChildConcurrency) + g, gCtx := errgroup.WithContext(ctx) + + for _, child := range children { + child := child + + if isTerminalProvisioningState(child.Properties) { + continue + } + + g.Go(func() error { + sem <- struct{}{} + defer func() { <-sem }() + + perChildCtx, cancel := context.WithTimeout(gCtx, reconcileChildTimeout) + defer cancel() + + childOutcomes, err := c.reconcileChild(perChildCtx, child) + if err != nil { + // One unreachable RP does not fail the whole reconcile: record the failure as a + // per-child skipped outcome and move on. + logger.V(ucplog.LevelDebug).Info("reconcile dispatch failed", "id", to.String(child.ID), "error", err.Error()) + state := readProvisioningState(child.Properties) + childOutcomes = []*corerpv20250801preview.ReconcileResourceOutcome{{ + ResourceID: child.ID, + From: state, + To: state, + Reason: to.Ptr(fmt.Sprintf("reconcile dispatch failed: %v", err)), + }} + } + + mu.Lock() + outcomes = append(outcomes, childOutcomes...) + mu.Unlock() + return nil + }) + } + + if err := g.Wait(); err != nil { + return nil, err + } + + return rest.NewOKResponse(&corerpv20250801preview.ReconcileResponse{Resources: outcomes}), nil +} + +// defaultListChildren walks the resource-type registry restricted to dynamic-rp-served namespaces +// and returns every resource associated with the application. Reuses the same helpers getGraph +// uses so the two custom actions stay consistent. +func (c *Reconcilev20250801preview) defaultListChildren(ctx context.Context, applicationID resources.ID) ([]generated.GenericResource, error) { + clientOptions := sdk.NewClientOptions(c.connection) + + ucpMgmt := &clients.UCPApplicationsManagementClient{ + RootScope: radiusPlane + planeName, + ClientOptions: clientOptions, + } + + allTypes, err := ucpMgmt.ListAllResourceTypesNames(ctx, planeName) + if err != nil { + return nil, err + } + + dynamicTypes := make([]string, 0, len(allTypes)) + for _, t := range allTypes { + ns, _, ok := strings.Cut(t, "/") + if !ok { + continue + } + if _, static := staticResourceProviderNamespaces[ns]; static { + continue + } + dynamicTypes = append(dynamicTypes, t) + } + + return listAllResourcesByApplication(ctx, applicationID, dynamicTypes, clientOptions) +} + +// defaultReconcileChild POSTs the reconcile action to a single child through the shared UCP +// connection. A 404 or 405 from the RP means "no reconcile route" — recorded as skipped so the +// orchestrator remains forward-compatible with RPs that don't (yet) implement /reconcile. +func (c *Reconcilev20250801preview) defaultReconcileChild(ctx context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) { + if child.ID == nil || *child.ID == "" || child.Type == nil { + return nil, fmt.Errorf("child resource is missing id or type") + } + + clientOptions := sdk.NewClientOptions(c.connection) + apiVersion, err := getAPIVersionForResourceType(ctx, *child.Type, clientOptions) + if err != nil { + return nil, fmt.Errorf("failed to resolve api-version for %s: %w", *child.Type, err) + } + + endpoint := strings.TrimSuffix(c.connection.Endpoint(), "/") + url := fmt.Sprintf("%s%s/reconcile?api-version=%s", endpoint, *child.ID, apiVersion) + + req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader([]byte("{}"))) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Accept", "application/json") + + resp, err := c.connection.Client().Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + + if resp.StatusCode == http.StatusMethodNotAllowed || resp.StatusCode == http.StatusNotFound { + state := readProvisioningState(child.Properties) + return []*corerpv20250801preview.ReconcileResourceOutcome{{ + ResourceID: child.ID, + From: state, + To: state, + Reason: to.Ptr(fmt.Sprintf("skipped: RP does not implement reconcile (%d)", resp.StatusCode)), + }}, nil + } + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return nil, fmt.Errorf("reconcile returned status %d", resp.StatusCode) + } + + var body corerpv20250801preview.ReconcileResponse + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { + return nil, fmt.Errorf("failed to decode reconcile response: %w", err) + } + return body.Resources, nil +} + +// isTerminalProvisioningState returns true when the resource's recorded provisioningState is +// terminal. Used to skip children that don't need reconciliation. +func isTerminalProvisioningState(props map[string]any) bool { + state := readProvisioningState(props) + if state == nil { + return false + } + return v1.ProvisioningState(*state).IsTerminal() +} + +// readProvisioningState pulls provisioningState out of a raw properties map. Returns nil when the +// field is absent or not a string. +func readProvisioningState(props map[string]any) *string { + if props == nil { + return nil + } + raw, ok := props["provisioningState"] + if !ok { + return nil + } + s, ok := raw.(string) + if !ok { + return nil + } + return to.Ptr(s) } diff --git a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go index 40f816e3fa..f72dd93ff8 100644 --- a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go +++ b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_test.go @@ -17,53 +17,45 @@ limitations under the License. package v20250801preview import ( + "context" "encoding/json" "errors" "net/http" "net/http/httptest" + "sort" "testing" v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" "github.com/radius-project/radius/pkg/armrpc/rpctest" + "github.com/radius-project/radius/pkg/cli/clients_new/generated" "github.com/radius-project/radius/pkg/components/database" corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" "github.com/radius-project/radius/pkg/corerp/datamodel" rpv1 "github.com/radius-project/radius/pkg/rp/v1" - "github.com/radius-project/radius/pkg/sdk" + "github.com/radius-project/radius/pkg/to" + "github.com/radius-project/radius/pkg/ucp/resources" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" ) -const reconcileRoute = "http://localhost:8080/planes/radius/local/resourcegroups/default/providers/Radius.Core/applications/myapp/reconcile?api-version=2025-08-01-preview" - -func newReconcileTestConnection(t *testing.T) sdk.Connection { - t.Helper() - conn, err := sdk.NewDirectConnection("http://localhost:9000/apis/api.ucp.dev/v1alpha3") - require.NoError(t, err) - return conn -} - -func TestReconcileRun_ReturnsEmptyReport(t *testing.T) { - mctrl := gomock.NewController(t) - defer mctrl.Finish() - - databaseClient := database.NewMockClient(mctrl) - req, err := rpctest.NewHTTPRequestWithContent( - t.Context(), - v1.OperationPost.HTTPMethod(), - reconcileRoute, nil, - ) - require.NoError(t, err) +const ( + reconcileRoute = "http://localhost:8080/planes/radius/local/resourcegroups/default/providers/Radius.Core/applications/myapp/reconcile?api-version=2025-08-01-preview" + testApplicationID = "/planes/radius/local/resourceGroups/default/providers/Radius.Core/applications/myapp" + testContainerFrontend = "/planes/radius/local/resourceGroups/default/providers/Radius.Compute/containers/frontend" + testContainerBackend = "/planes/radius/local/resourceGroups/default/providers/Radius.Compute/containers/backend" +) - // Return an application resource so the stub proceeds past the existence check. The stub - // does not read any properties from the resource yet, so a minimal stored record is enough. - stored := &database.Object{ - Metadata: database.Metadata{ID: "/planes/radius/local/resourceGroups/default/providers/Radius.Core/applications/myapp"}, +// storedApplication returns a minimal application data-model stored under testApplicationID that +// the controller's GetResource can hand back. The properties are not read by the orchestrator, so +// a bare shell is enough. +func storedApplication() *database.Object { + return &database.Object{ + Metadata: database.Metadata{ID: testApplicationID}, Data: &datamodel.Application_v20250801preview{ BaseResource: v1.BaseResource{ TrackedResource: v1.TrackedResource{ - ID: "/planes/radius/local/resourceGroups/default/providers/Radius.Core/applications/myapp", + ID: testApplicationID, Name: "myapp", Type: "Radius.Core/applications", }, @@ -79,24 +71,65 @@ func TestReconcileRun_ReturnsEmptyReport(t *testing.T) { }, }, } - databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(stored, nil) +} - ctx := rpctest.NewARMRequestContext(req) - opts := ctrl.Options{DatabaseClient: databaseClient} - c, err := NewReconcilev20250801preview(opts, newReconcileTestConnection(t)) +// newTestReconcileController builds a controller with test-supplied child walk and per-child +// dispatch hooks so the orchestration logic can be exercised without a live UCP. +func newTestReconcileController( + t *testing.T, + databaseClient database.Client, + children []generated.GenericResource, + reconcileChild func(ctx context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error), +) ctrl.Controller { + t.Helper() + return newReconcilev20250801preview( + ctrl.Options{DatabaseClient: databaseClient}, + nil, + func(ctx context.Context, _ resources.ID) ([]generated.GenericResource, error) { + return children, nil + }, + reconcileChild, + ) +} + +func makeReconcileRequest(t *testing.T) *http.Request { + t.Helper() + req, err := rpctest.NewHTTPRequestWithContent( + t.Context(), + v1.OperationPost.HTTPMethod(), + reconcileRoute, nil, + ) require.NoError(t, err) + return req +} +func runReconcile(t *testing.T, c ctrl.Controller, req *http.Request) *httptest.ResponseRecorder { + t.Helper() + ctx := rpctest.NewARMRequestContext(req) w := httptest.NewRecorder() resp, err := c.Run(ctx, w, req) require.NoError(t, err) require.NoError(t, resp.Apply(ctx, w, req)) + return w +} - require.Equal(t, http.StatusOK, w.Result().StatusCode) - +func decodeReconcileResponse(t *testing.T, w *httptest.ResponseRecorder) corerpv20250801preview.ReconcileResponse { + t.Helper() var body corerpv20250801preview.ReconcileResponse require.NoError(t, json.NewDecoder(w.Result().Body).Decode(&body)) - require.NotNil(t, body.Resources) - require.Empty(t, body.Resources, "Phase 0 stub must return an empty report") + return body +} + +// makeChild builds a GenericResource with the given ID, type, and provisioningState. +func makeChild(id, resourceType, state string) generated.GenericResource { + return generated.GenericResource{ + ID: to.Ptr(id), + Name: to.Ptr(resources.MustParse(id).Name()), + Type: to.Ptr(resourceType), + Properties: map[string]any{ + "provisioningState": state, + }, + } } func TestReconcileRun_NotFound(t *testing.T) { @@ -104,23 +137,11 @@ func TestReconcileRun_NotFound(t *testing.T) { defer mctrl.Finish() databaseClient := database.NewMockClient(mctrl) - req, err := rpctest.NewHTTPRequestWithContent( - t.Context(), - v1.OperationPost.HTTPMethod(), - reconcileRoute, nil, - ) - require.NoError(t, err) - databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(nil, &database.ErrNotFound{}) - ctx := rpctest.NewARMRequestContext(req) - c, err := NewReconcilev20250801preview(ctrl.Options{DatabaseClient: databaseClient}, newReconcileTestConnection(t)) - require.NoError(t, err) + c := newTestReconcileController(t, databaseClient, nil, nil) - w := httptest.NewRecorder() - resp, err := c.Run(ctx, w, req) - require.NoError(t, err) - require.NoError(t, resp.Apply(ctx, w, req)) + w := runReconcile(t, c, makeReconcileRequest(t)) require.Equal(t, http.StatusNotFound, w.Result().StatusCode) } @@ -129,21 +150,159 @@ func TestReconcileRun_DatabaseError(t *testing.T) { defer mctrl.Finish() databaseClient := database.NewMockClient(mctrl) - req, err := rpctest.NewHTTPRequestWithContent( - t.Context(), - v1.OperationPost.HTTPMethod(), - reconcileRoute, nil, - ) - require.NoError(t, err) - databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(nil, errors.New("boom")) - ctx := rpctest.NewARMRequestContext(req) - c, err := NewReconcilev20250801preview(ctrl.Options{DatabaseClient: databaseClient}, newReconcileTestConnection(t)) - require.NoError(t, err) + c := newTestReconcileController(t, databaseClient, nil, nil) + req := makeReconcileRequest(t) + ctx := rpctest.NewARMRequestContext(req) w := httptest.NewRecorder() - resp, actErr := c.Run(ctx, w, req) - require.Error(t, actErr) + resp, err := c.Run(ctx, w, req) + require.Error(t, err) require.Nil(t, resp) } + +// With no children, the report is empty. Application-scope succeeds without any child dispatch. +func TestReconcileRun_NoChildren(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + dispatched := 0 + c := newTestReconcileController(t, databaseClient, []generated.GenericResource{}, + func(_ context.Context, _ generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) { + dispatched++ + return nil, nil + }) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + body := decodeReconcileResponse(t, w) + require.Empty(t, body.Resources) + require.Equal(t, 0, dispatched, "no dispatch should occur when there are no children") +} + +// Terminal children are filtered out before dispatch; only non-terminal children are POSTed to +// their RP and appear in the aggregated report. +func TestReconcileRun_SkipsTerminalChildren(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + children := []generated.GenericResource{ + makeChild(testContainerFrontend, "Radius.Compute/containers", string(v1.ProvisioningStateSucceeded)), + makeChild(testContainerBackend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + } + + var dispatchedIDs []string + c := newTestReconcileController(t, databaseClient, children, + func(_ context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) { + dispatchedIDs = append(dispatchedIDs, to.String(child.ID)) + return []*corerpv20250801preview.ReconcileResourceOutcome{{ + ResourceID: child.ID, + From: to.Ptr(string(v1.ProvisioningStateUpdating)), + To: to.Ptr(string(v1.ProvisioningStateFailed)), + Reason: to.Ptr("kubernetes object not found"), + }}, nil + }) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, testContainerBackend, to.String(body.Resources[0].ResourceID)) + require.Equal(t, []string{testContainerBackend}, dispatchedIDs) +} + +// Fan-out aggregates each child's outcomes into the app-level response. Order-independent because +// the orchestrator dispatches concurrently. +func TestReconcileRun_AggregatesMultipleChildren(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + children := []generated.GenericResource{ + makeChild(testContainerFrontend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + makeChild(testContainerBackend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + } + + c := newTestReconcileController(t, databaseClient, children, + func(_ context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) { + toState := string(v1.ProvisioningStateSucceeded) + if to.String(child.ID) == testContainerFrontend { + toState = string(v1.ProvisioningStateFailed) + } + return []*corerpv20250801preview.ReconcileResourceOutcome{{ + ResourceID: child.ID, + From: to.Ptr(string(v1.ProvisioningStateUpdating)), + To: to.Ptr(toState), + }}, nil + }) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 2) + + byID := map[string]string{} + for _, o := range body.Resources { + byID[to.String(o.ResourceID)] = to.String(o.To) + } + require.Equal(t, string(v1.ProvisioningStateFailed), byID[testContainerFrontend]) + require.Equal(t, string(v1.ProvisioningStateSucceeded), byID[testContainerBackend]) +} + +// A single unreachable RP is recorded as a per-child skipped outcome; the reconcile as a whole +// still succeeds and reports outcomes for the healthy siblings. +func TestReconcileRun_DispatchFailureIsSkipped(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + children := []generated.GenericResource{ + makeChild(testContainerFrontend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + makeChild(testContainerBackend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + } + + c := newTestReconcileController(t, databaseClient, children, + func(_ context.Context, child generated.GenericResource) ([]*corerpv20250801preview.ReconcileResourceOutcome, error) { + if to.String(child.ID) == testContainerBackend { + return nil, errors.New("connection refused") + } + return []*corerpv20250801preview.ReconcileResourceOutcome{{ + ResourceID: child.ID, + From: to.Ptr(string(v1.ProvisioningStateUpdating)), + To: to.Ptr(string(v1.ProvisioningStateFailed)), + }}, nil + }) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 2) + + ids := []string{} + byID := map[string]*corerpv20250801preview.ReconcileResourceOutcome{} + for _, o := range body.Resources { + ids = append(ids, to.String(o.ResourceID)) + byID[to.String(o.ResourceID)] = o + } + sort.Strings(ids) + require.Equal(t, []string{testContainerBackend, testContainerFrontend}, ids) + + require.Equal(t, string(v1.ProvisioningStateFailed), to.String(byID[testContainerFrontend].To)) + require.Equal(t, string(v1.ProvisioningStateUpdating), to.String(byID[testContainerBackend].To), + "failed dispatch must not move the child out of Updating") + require.Contains(t, to.String(byID[testContainerBackend].Reason), "reconcile dispatch failed") +} From 9276b994dafa971a5abf990a2fc961894c604df7 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 11:04:37 -0700 Subject: [PATCH 16/18] corerp: add httptest end-to-end test for reconcile orchestrator MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds an integration test that exercises the reconcile orchestrator against a real HTTP server that impersonates UCP + downstream RPs. This covers the parts pure unit tests cannot: URL construction for each child's /reconcile endpoint, the resource-provider summary lookup that resolves each child's API version, request headers, request body, and JSON response decoding into the aggregated ReconcileResponse. The child walk is still stubbed with a static list — standing up a fake UCP resource-listing surface would balloon the test without adding coverage of anything that isn't already unit-tested. Two scenarios: * end-to-end: two-container application, one child's RP reports Failed ('kubernetes object not found') and the other reports Succeeded. The orchestrator must aggregate both outcomes verbatim and dispatch to each child's specific /reconcile path. * forward-compat: a child's RP returns 404 for /reconcile (the route is not registered). The orchestrator must record a skipped outcome with provisioningState unchanged, not fail the reconcile. Phase 1 is now feature-complete and covered end to end. Phase 2 (a functional test against a real k3d cluster in test/functional/) remains out of scope for this PR per the plan. Signed-off-by: Nithya Subramanian --- .../reconcile_integration_test.go | 211 ++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_integration_test.go diff --git a/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_integration_test.go b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_integration_test.go new file mode 100644 index 0000000000..756c1e18b8 --- /dev/null +++ b/pkg/corerp/frontend/controller/applications/v20250801preview/reconcile_integration_test.go @@ -0,0 +1,211 @@ +/* +Copyright 2023 The Radius Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v20250801preview + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + + v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" + ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" + "github.com/radius-project/radius/pkg/cli/clients_new/generated" + "github.com/radius-project/radius/pkg/components/database" + corerpv20250801preview "github.com/radius-project/radius/pkg/corerp/api/v20250801preview" + "github.com/radius-project/radius/pkg/sdk" + "github.com/radius-project/radius/pkg/to" + "github.com/radius-project/radius/pkg/ucp/resources" + "github.com/stretchr/testify/require" + "go.uber.org/mock/gomock" +) + +// TestReconcile_Integration_EndToEnd exercises the reconcile orchestrator against a real HTTP +// server that impersonates UCP + downstream RPs. It verifies the parts that pure unit tests +// cannot: URL construction for each child's /reconcile endpoint, the resource-provider summary +// call that resolves each child's API version, request headers, and JSON response decoding into +// the aggregated ReconcileResponse. The child walk is still stubbed with a static list because +// standing up a fake UCP resource-listing surface would obscure what this test actually protects. +// +// Scenario mirrors the plan's exit criterion: a two-container application where one k8s Deployment +// has vanished (dispatched RP reports Failed) and the other is healthy (dispatched RP reports +// Succeeded). The orchestrator must return both outcomes verbatim. +func TestReconcile_Integration_EndToEnd(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + // Track every request the fake server sees so the assertions can prove the orchestrator dispatched + // to each child's specific /reconcile path. + var ( + mu sync.Mutex + reconcileHit = map[string]bool{} + ) + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + defer mu.Unlock() + + switch { + // getAPIVersionForResourceType issues this GET to resolve the API version for the child's + // resource type before POSTing the reconcile action. + case r.Method == http.MethodGet && strings.HasSuffix(r.URL.Path, "/providers/Radius.Compute"): + require.Equal(t, "2023-10-01-preview", r.URL.Query().Get("api-version")) + w.Header().Set("Content-Type", "application/json") + require.NoError(t, json.NewEncoder(w).Encode(corerpv20250801PreviewProviderSummary())) + + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/containers/frontend/reconcile"): + reconcileHit["frontend"] = true + require.Equal(t, "2023-10-01-preview", r.URL.Query().Get("api-version")) + require.Equal(t, "application/json", r.Header.Get("Content-Type")) + // Verify the orchestrator sent an empty JSON object as the ReconcileRequest body. + body, err := io.ReadAll(r.Body) + require.NoError(t, err) + require.JSONEq(t, `{}`, string(body)) + + w.Header().Set("Content-Type", "application/json") + require.NoError(t, json.NewEncoder(w).Encode(map[string]any{ + "resources": []map[string]any{{ + "resourceId": testContainerFrontend, + "from": string(v1.ProvisioningStateUpdating), + "to": string(v1.ProvisioningStateFailed), + "reason": "kubernetes object not found", + }}, + })) + + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/containers/backend/reconcile"): + reconcileHit["backend"] = true + w.Header().Set("Content-Type", "application/json") + require.NoError(t, json.NewEncoder(w).Encode(map[string]any{ + "resources": []map[string]any{{ + "resourceId": testContainerBackend, + "from": string(v1.ProvisioningStateUpdating), + "to": string(v1.ProvisioningStateSucceeded), + }}, + })) + + default: + t.Errorf("unexpected request: %s %s", r.Method, r.URL.String()) + http.Error(w, "not found", http.StatusNotFound) + } + })) + t.Cleanup(server.Close) + + connection, err := sdk.NewDirectConnection(server.URL) + require.NoError(t, err) + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + children := []generated.GenericResource{ + makeChild(testContainerFrontend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + makeChild(testContainerBackend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + } + + c := newReconcilev20250801preview( + ctrl.Options{DatabaseClient: databaseClient}, + connection, + func(context.Context, resources.ID) ([]generated.GenericResource, error) { return children, nil }, + nil, // real defaultReconcileChild — this is what we're testing. + ) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 2, "orchestrator must aggregate both children into the report") + + byID := map[string]*corerpv20250801preview.ReconcileResourceOutcome{} + for _, o := range body.Resources { + byID[to.String(o.ResourceID)] = o + } + require.Equal(t, string(v1.ProvisioningStateFailed), to.String(byID[testContainerFrontend].To)) + require.Equal(t, "kubernetes object not found", to.String(byID[testContainerFrontend].Reason)) + require.Equal(t, string(v1.ProvisioningStateSucceeded), to.String(byID[testContainerBackend].To)) + + require.True(t, reconcileHit["frontend"], "orchestrator must POST /reconcile for the frontend container") + require.True(t, reconcileHit["backend"], "orchestrator must POST /reconcile for the backend container") +} + +// TestReconcile_Integration_RPWithoutReconcileIsSkipped verifies the forward-compat path: when a +// child's RP returns 404 or 405 for /reconcile (the route is not registered on that RP), the +// orchestrator records a skipped outcome instead of failing the whole reconcile. +func TestReconcile_Integration_RPWithoutReconcileIsSkipped(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.Method == http.MethodGet && strings.HasSuffix(r.URL.Path, "/providers/Radius.Compute"): + w.Header().Set("Content-Type", "application/json") + require.NoError(t, json.NewEncoder(w).Encode(corerpv20250801PreviewProviderSummary())) + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/reconcile"): + // Simulate an older RP that does not advertise /reconcile. + http.Error(w, "not found", http.StatusNotFound) + default: + t.Errorf("unexpected request: %s %s", r.Method, r.URL.String()) + } + })) + t.Cleanup(server.Close) + + connection, err := sdk.NewDirectConnection(server.URL) + require.NoError(t, err) + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), gomock.Any()).Return(storedApplication(), nil) + + children := []generated.GenericResource{ + makeChild(testContainerFrontend, "Radius.Compute/containers", string(v1.ProvisioningStateUpdating)), + } + + c := newReconcilev20250801preview( + ctrl.Options{DatabaseClient: databaseClient}, + connection, + func(context.Context, resources.ID) ([]generated.GenericResource, error) { return children, nil }, + nil, + ) + + w := runReconcile(t, c, makeReconcileRequest(t)) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, string(v1.ProvisioningStateUpdating), to.String(body.Resources[0].To), + "provisioningState must be left unchanged when the RP has no reconcile route") + require.Contains(t, to.String(body.Resources[0].Reason), "RP does not implement reconcile") +} + +// corerpv20250801PreviewProviderSummary builds a minimal ResourceProviderSummary payload that +// advertises Radius.Compute/containers with a default API version, which is all +// getAPIVersionForResourceType needs to resolve. +func corerpv20250801PreviewProviderSummary() map[string]any { + return map[string]any{ + "name": "Radius.Compute", + "locations": map[string]any{"global": map[string]any{}}, + "resourceTypes": map[string]any{ + "containers": map[string]any{ + "defaultApiVersion": "2023-10-01-preview", + "apiVersions": map[string]any{ + "2023-10-01-preview": map[string]any{}, + }, + }, + }, + } +} From a6407e3d3aadb1890ef7647cf684bc4b00e26611 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 11:14:46 -0700 Subject: [PATCH 17/18] specs: address PR review comments on 006-state-restoration Fixes the copilot-pull-request-reviewer feedback that pointed out drift between the spec doc and the plan/implementation actually shipped: * Purpose paragraph (spec.md): the reconciler does not remove state-store entries when the underlying resource is gone. It PATCHes provisioningState to Failed and keeps the row so the next normal 'rad app delete' cleans it up through the standard state machine. Updated wording to match. * Reality table (spec.md): the 'not found' row now describes the same 'PATCH Failed, keep row' action as the plan / implementation, not 'delete the state-store entry'. * Terminal set (spec.md): 'terminal' is not just Succeeded or Failed -- v1.ProvisioningState.IsTerminal() also treats Canceled and the empty string (synchronous resources that already settled) as terminal. Spec now references the helper so it cannot drift again. * Endpoint casing (spec.md): the example route now uses lowercase 'resourcegroups' to match the canonical casing in the getGraph route comment and elsewhere in the codebase. * Go version (plan.md): updated 1.26.5 to 1.27.0 to match go.mod. The tables in both files are already well-formed markdown -- the reviewer flagged them as having a leading '||' but they use standard '|' cells and render correctly on GitHub. No table syntax change needed. No implementation change: plan and code already match the corrected spec. Signed-off-by: Nithya Subramanian --- specs/006-state-restoration/plan.md | 2 +- specs/006-state-restoration/spec.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/006-state-restoration/plan.md b/specs/006-state-restoration/plan.md index fe0a69a9c0..973f1bdcbc 100644 --- a/specs/006-state-restoration/plan.md +++ b/specs/006-state-restoration/plan.md @@ -8,7 +8,7 @@ Add a `reconcile` custom action to `Radius.Core/applications/{name}` (mirror of ## Technical Context -**Language/Version**: Go 1.26.5 (per `go.mod`) +**Language/Version**: Go 1.27.0 (per `go.mod`) **Primary Dependencies**: no new external dependencies. Reuses `k8s.io/client-go` (dynamic-rp's per-output reality check), dynamic-rp's existing routing scaffold, and the internal `pkg/armrpc/builder` custom-action mechanism (for the app-scoped orchestrator on corerp). **Storage**: no schema changes. Reconciliation writes go through the RPs' existing state-store paths. **Testing**: `go test` with `stretchr/testify`; table-driven unit tests for the corerp orchestrator and the dynamic-rp `reconcile` handler; a `httptest`-backed integration test that mounts the whole custom-action flow end to end. Existing `rad startup` tests get a new fake for `ReconcileHydratedState`. diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 29b8b779d1..5c4a6c7ad2 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -11,7 +11,7 @@ Repo Radius (the ephemeral k3d control plane the GitHub workflows spin up on eve That is not sufficient when the previous run was interrupted while a resource was mid-operation. The archive can preserve a resource in a non-terminal state — for example `provisioningState: "Updating"` — that never actually completed. On the next run, the control plane accepts that state as authoritative, so every subsequent operation against the resource is blocked with `409 Conflict / target resource is in progress`. The delete workflow loops on that 409 forever and the application becomes undeletable through Radius. -This feature adds a reconciliation pass triggered by `rad startup` and executed against the running control plane: for every application in the plane, an application-scoped `reconcile` action asks each resource's owning resource provider to check its actual current state and rewrite the state store to match reality — including removing entries when the underlying resource does not exist. +This feature adds a reconciliation pass triggered by `rad startup` and executed against the running control plane: for every application in the plane, an application-scoped `reconcile` action asks each resource's owning resource provider to check its actual current state and rewrite the state store to match reality — including marking entries as `Failed` when the underlying resource does not exist, so the next normal `rad app delete` cleans them up through the standard state machine. The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. @@ -39,7 +39,7 @@ ERROR CODE: Conflict Reconciliation is a per-application operation: walk the application's children, check each one's reality, roll the results back into the state store. That is the same shape as [`getGraph`](../../pkg/corerp/frontend/controller/applications/v20250801preview/getgraph.go) — an application-scoped custom action registered on `Radius.Core/applications` that walks children across resource providers. `reconcile` therefore reuses the exact pattern, up to and including the corerp orchestrator that already knows how to fan out across RPs through the UCP proxy. ```text -POST /planes/radius/local/resourceGroups/{rg}/providers/Radius.Core/applications/{app}/reconcile?api-version=2025-08-01-preview +POST /planes/radius/local/resourcegroups/{rg}/providers/Radius.Core/applications/{app}/reconcile?api-version=2025-08-01-preview Content-Type: application/json {} ``` @@ -58,7 +58,7 @@ Concretely, the handler: 1. Loads the application record. 2. Traverses the same resource-type registration list `getGraph` uses (via UCP's `System.Resources/resourceProviders`) to build the child set. -3. Filters to children whose current `provisioningState` is non-terminal — anything other than `Succeeded` or `Failed`. Terminal-state children are left alone; the archive captured a settled state and the next user operation will refresh it through the normal path. +3. Filters to children whose current `provisioningState` is non-terminal. "Terminal" here matches [`v1.ProvisioningState.IsTerminal()`](../../pkg/armrpc/api/v1/types.go): `Succeeded`, `Failed`, `Canceled`, or the empty string (which the RP writes for synchronous resources that have already settled). Terminal-state children are left alone; the archive captured a settled state and the next user operation will refresh it through the normal path. 4. For each such child, issues: ```text @@ -94,7 +94,7 @@ For a single resource, the dynamic-rp handler: | ---------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------- | | any non-terminal | terminal (settled) | Rewrite the state-store entry with the observed terminal state (`Succeeded` / `Failed` / etc.). | | any non-terminal | still non-terminal | Leave as-is. The hydrated state is accurate. | -| any non-terminal | not found | Delete the state-store entry. The resource does not exist. | +| any non-terminal | not found | PATCH `provisioningState` to `Failed` and keep the state-store row so the next normal `rad app delete` can clean it up through the standard state machine. | | any non-terminal | error (network, 5xx, etc.) | Leave as-is; record the error in the response. Reconciliation is best-effort and never fails. | Terminal-state entries are never reconciled by this action. From d562f3e17c04af6be44f2c3111821fba09e785c8 Mon Sep 17 00:00:00 2001 From: Nithya Subramanian Date: Mon, 31 Aug 2026 15:48:10 -0700 Subject: [PATCH 18/18] wip --- pkg/dynamicrp/frontend/reconcile.go | 124 ++++++++++++++++++--- pkg/dynamicrp/frontend/reconcile_test.go | 130 +++++++++++++++++++++-- pkg/dynamicrp/frontend/routes.go | 2 +- specs/006-state-restoration/spec.md | 31 +++--- 4 files changed, 251 insertions(+), 36 deletions(-) diff --git a/pkg/dynamicrp/frontend/reconcile.go b/pkg/dynamicrp/frontend/reconcile.go index fd04c60b3d..4def25befc 100644 --- a/pkg/dynamicrp/frontend/reconcile.go +++ b/pkg/dynamicrp/frontend/reconcile.go @@ -22,12 +22,20 @@ import ( "net/http" "strings" + "github.com/Azure/azure-sdk-for-go/sdk/azcore/arm" + "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources/v3" v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" ctrl "github.com/radius-project/radius/pkg/armrpc/frontend/controller" "github.com/radius-project/radius/pkg/armrpc/rest" + "github.com/radius-project/radius/pkg/azure/clientv2" + aztoken "github.com/radius-project/radius/pkg/azure/tokencredentials" + "github.com/radius-project/radius/pkg/cli/clients" "github.com/radius-project/radius/pkg/dynamicrp/datamodel" rpv1 "github.com/radius-project/radius/pkg/rp/v1" - "github.com/radius-project/radius/pkg/ucp/api/v20231001preview" + "github.com/radius-project/radius/pkg/sdk" + ucp_credentials "github.com/radius-project/radius/pkg/ucp/credentials" + "github.com/radius-project/radius/pkg/ucp/resources" + resources_azure "github.com/radius-project/radius/pkg/ucp/resources/azure" resources_kubernetes "github.com/radius-project/radius/pkg/ucp/resources/kubernetes" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -62,27 +70,24 @@ type ReconcileResponse struct { // non-terminal child resource. The handler walks the resource's outputResources, checks each one // against its underlying provider, and updates provisioningState to reflect reality. // -// This is the Phase 1 reality-check implementation. For each output resource we do a Kubernetes -// GET via the runtime client the RP already holds and categorize the result as gone (404), -// settled (present), or skipped (unknown provider / transient error). We then aggregate: all -// outputs gone → Failed, all settled → Succeeded, otherwise the current state is retained. Cloud -// outputs (Terraform-backed Azure / AWS resources) are recorded as skipped without being touched -// — that lives in a follow-up commit. +// For each output resource we query its underlying provider and categorize the result as gone +// (404), settled (present), or skipped (unknown provider / transient error). We then aggregate: +// all outputs gone → Failed, all settled → Succeeded, otherwise the current state is retained. type Reconcile struct { ctrl.Operation[*datamodel.DynamicResource, datamodel.DynamicResource] resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource] - ucpClient *v20231001preview.ClientFactory + ucpConnection sdk.Connection discovery discovery.DiscoveryInterface } // NewReconcile constructs the reconcile controller for a dynamic resource type. The runtime // client is read from opts.KubeClient at request time so the same handler serves every dynamic // type without per-type wiring. -func NewReconcile(opts ctrl.Options, resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource], ucpClient *v20231001preview.ClientFactory, discovery discovery.DiscoveryInterface) (ctrl.Controller, error) { +func NewReconcile(opts ctrl.Options, resourceOptions ctrl.ResourceOptions[datamodel.DynamicResource], ucpConnection sdk.Connection, discovery discovery.DiscoveryInterface) (ctrl.Controller, error) { return &Reconcile{ Operation: ctrl.NewOperation(opts, resourceOptions), resourceOptions: resourceOptions, - ucpClient: ucpClient, + ucpConnection: ucpConnection, discovery: discovery, }, nil } @@ -151,11 +156,13 @@ func (c *Reconcile) Run(ctx context.Context, w http.ResponseWriter, req *http.Re func (c *Reconcile) checkOutput(ctx context.Context, out rpv1.OutputResource) outputCheck { idStr := out.ID.String() - // Only Kubernetes outputs are reality-checked in this prototype. Terraform-backed cloud - // outputs (Azure, AWS) are reported as skipped so the caller can act on them separately. + if resources_azure.IsAzureResource(out.ID) { + return c.checkAzureOutput(ctx, out.ID) + } + scopes := out.ID.ScopeSegments() if len(scopes) == 0 || !strings.EqualFold(scopes[0].Type, resources_kubernetes.PlaneTypeKubernetes) { - return outputCheck{id: idStr, status: outputSkipped, reason: "cloud output not yet reality-checked"} + return outputCheck{id: idStr, status: outputSkipped, reason: "unsupported output resource provider"} } kubeClient := c.Options().KubeClient @@ -184,6 +191,97 @@ func (c *Reconcile) checkOutput(ctx context.Context, out rpv1.OutputResource) ou } } +func (c *Reconcile) checkAzureOutput(ctx context.Context, id resources.ID) outputCheck { + idStr := id.String() + if c.ucpConnection == nil { + return outputCheck{id: idStr, status: outputSkipped, reason: "UCP connection not configured"} + } + + armID := id + azurePlaneScope := "/planes/azure/" + ucp_credentials.AzureCloud + if id.IsUCPQualified() { + var err error + armID, err = resources.ParseResource(resources.MakeRelativeID(id.ScopeSegments()[1:], id.TypeSegments(), id.ExtensionSegments())) + if err != nil { + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("could not normalize Azure resource ID: %v", err)} + } + azurePlaneScope = id.PlaneScope() + } + + clientOptions := sdk.NewClientOptions(&endpointConnection{ + Connection: c.ucpConnection, + endpoint: strings.TrimRight(c.ucpConnection.Endpoint(), "/") + azurePlaneScope, + }) + apiVersion, err := lookupAzureAPIVersion(ctx, armID, clientOptions) + if err != nil { + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("could not resolve Azure API version: %v", err)} + } + + client, err := clientv2.NewGenericResourceClient( + armID.FindScope(resources_azure.ScopeSubscriptions), + &clientv2.Options{Cred: &aztoken.AnonymousCredential{}}, + clientOptions, + ) + if err != nil { + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("could not create Azure resource client: %v", err)} + } + + _, err = client.GetByID(ctx, armID.String(), apiVersion, &armresources.ClientGetByIDOptions{}) + switch { + case clients.Is404Error(err): + return outputCheck{id: idStr, status: outputGone, reason: "Azure resource not found"} + case err != nil: + return outputCheck{id: idStr, status: outputSkipped, reason: fmt.Sprintf("Azure GET failed: %v", err)} + default: + return outputCheck{id: idStr, status: outputSettled} + } +} + +func lookupAzureAPIVersion(ctx context.Context, id resources.ID, clientOptions *arm.ClientOptions) (string, error) { + client, err := clientv2.NewProvidersClient( + id.FindScope(resources_azure.ScopeSubscriptions), + &clientv2.Options{Cred: &aztoken.AnonymousCredential{}}, + clientOptions, + ) + if err != nil { + return "", err + } + + provider, err := client.Get(ctx, id.ProviderNamespace(), nil) + if err != nil { + return "", err + } + + segments := id.TypeSegments() + if len(id.ExtensionSegments()) > 0 { + segments = id.ExtensionSegments() + } + shortType := strings.TrimPrefix(segments[0].Type, id.ProviderNamespace()+"/") + for _, resourceType := range provider.ResourceTypes { + if resourceType.ResourceType == nil || !strings.EqualFold(shortType, *resourceType.ResourceType) { + continue + } + if resourceType.DefaultAPIVersion != nil && *resourceType.DefaultAPIVersion != "" { + return *resourceType.DefaultAPIVersion, nil + } + if len(resourceType.APIVersions) > 0 && resourceType.APIVersions[0] != nil { + return *resourceType.APIVersions[0], nil + } + return "", fmt.Errorf("no supported API versions for type %q", id.Type()) + } + + return "", fmt.Errorf("resource type %q was not found", id.Type()) +} + +type endpointConnection struct { + sdk.Connection + endpoint string +} + +func (c *endpointConnection) Endpoint() string { + return c.endpoint +} + // lookupKubernetesAPIVersion resolves the preferred API version for a group+kind via the // discovery client. This mirrors the walk in the corerp kubernetes handler; keeping a copy here // decouples the reconcile path from the deployment codepath. diff --git a/pkg/dynamicrp/frontend/reconcile_test.go b/pkg/dynamicrp/frontend/reconcile_test.go index 043762c908..c78e2813a3 100644 --- a/pkg/dynamicrp/frontend/reconcile_test.go +++ b/pkg/dynamicrp/frontend/reconcile_test.go @@ -23,12 +23,16 @@ import ( "net/http/httptest" "testing" + "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources/v3" v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" "github.com/radius-project/radius/pkg/armrpc/frontend/controller" "github.com/radius-project/radius/pkg/armrpc/rpctest" "github.com/radius-project/radius/pkg/components/database" "github.com/radius-project/radius/pkg/dynamicrp/datamodel" "github.com/radius-project/radius/pkg/dynamicrp/datamodel/converter" + rpv1 "github.com/radius-project/radius/pkg/rp/v1" + "github.com/radius-project/radius/pkg/sdk" + "github.com/radius-project/radius/pkg/ucp/resources" k8stest "github.com/radius-project/radius/test/k8sutil" "github.com/stretchr/testify/require" "go.uber.org/mock/gomock" @@ -44,6 +48,7 @@ const reconcileTestURL = "/planes/radius/local/resourceGroups/test-group/provide const ( deploymentOutputID = "/planes/kubernetes/local/namespaces/default/providers/apps/Deployment/my-deployment" azureOutputID = "/planes/azure/mycloud/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.Storage/storageAccounts/mystorage" + azureRelativeID = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.Storage/storageAccounts/mystorage" ) // scheme registers the built-in Kubernetes types the reconcile handler will look up. @@ -76,8 +81,12 @@ func reconcileTestDiscovery() *k8stest.DiscoveryClient { } } -func newReconcileController(t *testing.T, databaseClient database.Client, kubeClient runtimeclient.Client) controller.Controller { +func newReconcileController(t *testing.T, databaseClient database.Client, kubeClient runtimeclient.Client, connections ...sdk.Connection) controller.Controller { t.Helper() + var connection sdk.Connection + if len(connections) > 0 { + connection = connections[0] + } opts := controller.Options{ DatabaseClient: databaseClient, @@ -88,7 +97,7 @@ func newReconcileController(t *testing.T, databaseClient database.Client, kubeCl ResponseConverter: converter.DynamicResourceDataModelToVersioned, } - c, err := NewReconcile(opts, resourceOpts, nil, reconcileTestDiscovery()) + c, err := NewReconcile(opts, resourceOpts, connection, reconcileTestDiscovery()) require.NoError(t, err) return c } @@ -144,6 +153,50 @@ func decodeReconcileResponse(t *testing.T, w *httptest.ResponseRecorder) Reconci return body } +func newAzureReconcileTestConnection(t *testing.T, resourceStatus int) (sdk.Connection, func()) { + t.Helper() + mux := http.NewServeMux() + for _, planeName := range []string{"mycloud", "azurecloud"} { + planePrefix := "/planes/azure/" + planeName + mux.HandleFunc(planePrefix+"/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Storage", func(w http.ResponseWriter, _ *http.Request) { + require.NoError(t, json.NewEncoder(w).Encode(armresources.Provider{ + Namespace: new("Microsoft.Storage"), + ResourceTypes: []*armresources.ProviderResourceType{{ + ResourceType: new("storageAccounts"), + DefaultAPIVersion: new("2023-05-01"), + }}, + })) + }) + mux.HandleFunc(planePrefix+"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg1/providers/Microsoft.Storage/storageAccounts/mystorage", func(w http.ResponseWriter, req *http.Request) { + require.Equal(t, "2023-05-01", req.URL.Query().Get("api-version")) + w.WriteHeader(resourceStatus) + if resourceStatus == http.StatusOK { + require.NoError(t, json.NewEncoder(w).Encode(armresources.GenericResource{})) + } + }) + } + server := httptest.NewServer(mux) + connection, err := sdk.NewDirectConnection(server.URL) + require.NoError(t, err) + return connection, server.Close +} + +func TestReconcile_IdentifiesAzureResourceIDs(t *testing.T) { + connection, closeServer := newAzureReconcileTestConnection(t, http.StatusOK) + defer closeServer() + controller := &Reconcile{ucpConnection: connection} + + for _, resourceID := range []string{azureOutputID, azureRelativeID} { + t.Run(resourceID, func(t *testing.T) { + id, err := resources.ParseResource(resourceID) + require.NoError(t, err) + + check := controller.checkOutput(t.Context(), rpv1.OutputResource{ID: id}) + require.Equal(t, outputSettled, check.status) + }) + } +} + func TestReconcile_NotFound(t *testing.T) { mctrl := gomock.NewController(t) defer mctrl.Finish() @@ -253,9 +306,7 @@ func TestReconcile_KubernetesSettled_TransitionsToSucceeded(t *testing.T) { require.Contains(t, body.Resources[0].Reason, "settled") } -// A Terraform-backed cloud output cannot be reality-checked in Phase 1: it is recorded as -// "skipped" and the resource's provisioningState is left unchanged. No Save must be issued. -func TestReconcile_CloudOutput_Skipped(t *testing.T) { +func TestReconcile_AzureGone_TransitionsToFailed(t *testing.T) { mctrl := gomock.NewController(t) defer mctrl.Finish() @@ -265,9 +316,18 @@ func TestReconcile_CloudOutput_Skipped(t *testing.T) { databaseClient := database.NewMockClient(mctrl) databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) - // Save must not be called: skipped output → state unchanged. + databaseClient.EXPECT(). + Save(gomock.Any(), gomock.Any(), gomock.Any()). + DoAndReturn(func(_ context.Context, obj *database.Object, _ ...database.SaveOptions) error { + saved := obj.Data.(*datamodel.DynamicResource) + require.Equal(t, v1.ProvisioningStateFailed, saved.ProvisioningState()) + return nil + }) - c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t))) + connection, closeServer := newAzureReconcileTestConnection(t, http.StatusNotFound) + defer closeServer() + + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t)), connection) w := runReconcile(t, c) require.Equal(t, http.StatusOK, w.Result().StatusCode) @@ -275,7 +335,61 @@ func TestReconcile_CloudOutput_Skipped(t *testing.T) { body := decodeReconcileResponse(t, w) require.Len(t, body.Resources, 1) require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].From) + require.Equal(t, string(v1.ProvisioningStateFailed), body.Resources[0].To) + require.Contains(t, body.Resources[0].Reason, "gone") + require.Contains(t, body.Resources[0].Reason, "Azure resource not found") +} + +func TestReconcile_AzureSettled_TransitionsToSucceeded(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateUpdating, azureOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + databaseClient.EXPECT(). + Save(gomock.Any(), gomock.Any(), gomock.Any()). + DoAndReturn(func(_ context.Context, obj *database.Object, _ ...database.SaveOptions) error { + saved := obj.Data.(*datamodel.DynamicResource) + require.Equal(t, v1.ProvisioningStateSucceeded, saved.ProvisioningState()) + return nil + }) + + connection, closeServer := newAzureReconcileTestConnection(t, http.StatusOK) + defer closeServer() + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t)), connection) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) + require.Equal(t, string(v1.ProvisioningStateSucceeded), body.Resources[0].To) + require.Contains(t, body.Resources[0].Reason, "settled") +} + +func TestReconcile_AzureGetFailure_LeavesStateUnchanged(t *testing.T) { + mctrl := gomock.NewController(t) + defer mctrl.Finish() + + resource := newReconcileResource(v1.ProvisioningStateUpdating, azureOutputID) + storeObject := rpctest.FakeStoreObject(resource) + storeObject.Metadata = database.Metadata{ID: testResourceID, ETag: "etag-1"} + + databaseClient := database.NewMockClient(mctrl) + databaseClient.EXPECT().Get(gomock.Any(), testResourceID).Return(storeObject, nil) + + connection, closeServer := newAzureReconcileTestConnection(t, http.StatusInternalServerError) + defer closeServer() + c := newReconcileController(t, databaseClient, k8stest.NewFakeKubeClient(reconcileTestScheme(t)), connection) + + w := runReconcile(t, c) + require.Equal(t, http.StatusOK, w.Result().StatusCode) + body := decodeReconcileResponse(t, w) + require.Len(t, body.Resources, 1) require.Equal(t, string(v1.ProvisioningStateUpdating), body.Resources[0].To) require.Contains(t, body.Resources[0].Reason, "skipped") - require.Contains(t, body.Resources[0].Reason, "cloud output") + require.Contains(t, body.Resources[0].Reason, "Azure GET failed") } diff --git a/pkg/dynamicrp/frontend/routes.go b/pkg/dynamicrp/frontend/routes.go index 6f97c2197d..07bdbd3e4e 100644 --- a/pkg/dynamicrp/frontend/routes.go +++ b/pkg/dynamicrp/frontend/routes.go @@ -110,7 +110,7 @@ func (s *Service) registerRoutes( })) r.Post("/{resourceName}/reconcile", dynamicOperationHandler(v1.OperationPost, controllerOptions, func(opts controller.Options) (controller.Controller, error) { - return NewReconcile(opts, resourceOptions, ucpClient, discoveryClient) + return NewReconcile(opts, resourceOptions, s.options.UCP, discoveryClient) })) }) }) diff --git a/specs/006-state-restoration/spec.md b/specs/006-state-restoration/spec.md index 5c4a6c7ad2..0bbc264354 100644 --- a/specs/006-state-restoration/spec.md +++ b/specs/006-state-restoration/spec.md @@ -13,6 +13,11 @@ That is not sufficient when the previous run was interrupted while a resource wa This feature adds a reconciliation pass triggered by `rad startup` and executed against the running control plane: for every application in the plane, an application-scoped `reconcile` action asks each resource's owning resource provider to check its actual current state and rewrite the state store to match reality — including marking entries as `Failed` when the underlying resource does not exist, so the next normal `rad app delete` cleans them up through the standard state machine. +Throughout this specification, provisioning-state classification follows [`v1.ProvisioningState.IsTerminal()`](../../pkg/armrpc/api/v1/types.go) exactly: + +- **Terminal:** `Succeeded`, `Failed`, `Canceled`, and the empty string (`""`). The empty string is terminal because synchronous resources may settle without storing an explicit provisioning state. +- **Non-terminal:** every other value, including `None`, `Updating`, `Deleting`, `Accepted`, `Provisioning`, `Provisioned`, and `NotSpecified`. Reconciliation treats these values as in progress even when a name such as `Provisioned` might sound complete; it does not infer semantics beyond `IsTerminal()`. + The scope is deliberately narrow: reconcile hydrated state so operations that follow see reality. ## Non-goals @@ -58,7 +63,7 @@ Concretely, the handler: 1. Loads the application record. 2. Traverses the same resource-type registration list `getGraph` uses (via UCP's `System.Resources/resourceProviders`) to build the child set. -3. Filters to children whose current `provisioningState` is non-terminal. "Terminal" here matches [`v1.ProvisioningState.IsTerminal()`](../../pkg/armrpc/api/v1/types.go): `Succeeded`, `Failed`, `Canceled`, or the empty string (which the RP writes for synchronous resources that have already settled). Terminal-state children are left alone; the archive captured a settled state and the next user operation will refresh it through the normal path. +3. Filters to children whose current `provisioningState` is non-terminal according to the definition above (`None`, `Updating`, `Deleting`, `Accepted`, `Provisioning`, `Provisioned`, `NotSpecified`, or any other non-empty value not recognized as terminal). Terminal-state children (`Succeeded`, `Failed`, `Canceled`, or `""`) are left alone; the archive captured a settled state and the next user operation will refresh it through the normal path. 4. For each such child, issues: ```text @@ -80,24 +85,22 @@ Dynamic-rp implements `reconcile` once and serves it for every dynamic type — For a single resource, the dynamic-rp handler: 1. Loads the resource from dynamic-rp's own store. -2. If `provisioningState` is terminal, returns unchanged. +2. If `provisioningState` is terminal (`Succeeded`, `Failed`, `Canceled`, or `""`), returns unchanged. Otherwise, including for `None`, `Updating`, `Deleting`, `Accepted`, `Provisioning`, `Provisioned`, and `NotSpecified`, continues reconciliation. 3. Reads the resource's `properties.status.outputResources` — the concrete backing objects the recipe engine recorded when the resource was deployed. -4. For each output resource, queries its underlying provider: - - Kubernetes objects → GET via the target-cluster Kubernetes client the RP already holds. - - Terraform-backed cloud outputs (Azure/AWS resource IDs) → **out of scope for the prototype**; record `skipped: cloud output not yet reality-checked` in the per-output report. Follow-up work adds the cloud-SDK branches inside the same handler. +4. For each output resource, queries its underlying provider. Kubernetes objects use GET via the target-cluster Kubernetes client the RP already holds. Azure resources produced by any recipe driver, including Bicep and Terraform, use the same UCP/ARM ID classification as the application graph; the handler resolves the resource type's API version from Azure provider metadata and GETs the resource through its UCP Azure plane. AWS resources are **out of scope for the prototype** and are recorded as skipped; follow-up work adds the AWS SDK branch inside the same handler. 5. Aggregates outcomes per the table in [What "reality" is for each resource](#what-reality-is-for-each-resource) and writes the result back through dynamic-rp's normal `PATCH` code path. **No direct SQL.** 6. Returns the new state, or an error the corerp orchestrator will record in the report. ### What "reality" is for each resource -| Hydrated `provisioningState` | Reality query result | Action | -| ---------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------- | -| any non-terminal | terminal (settled) | Rewrite the state-store entry with the observed terminal state (`Succeeded` / `Failed` / etc.). | -| any non-terminal | still non-terminal | Leave as-is. The hydrated state is accurate. | -| any non-terminal | not found | PATCH `provisioningState` to `Failed` and keep the state-store row so the next normal `rad app delete` can clean it up through the standard state machine. | -| any non-terminal | error (network, 5xx, etc.) | Leave as-is; record the error in the response. Reconciliation is best-effort and never fails. | +| Hydrated `provisioningState` | Reality GET result | Action | +|------------------------------|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| +| terminal | not queried | Leave unchanged. Terminal means `Succeeded`, `Failed`, `Canceled`, or `""`. | +| non-terminal | present (`2xx`) | Set `provisioningState` to `Succeeded`. Non-terminal includes `None`, `Updating`, `Deleting`, `Accepted`, `Provisioning`, `Provisioned`, and `NotSpecified`. | +| non-terminal | not found (`404`) | Set `provisioningState` to `Failed` and keep the state-store row so the next normal `rad app delete` can clean it up through the standard state machine. | +| non-terminal | error (network, `5xx`, etc.) | Leave unchanged and record the error in the response. Reconciliation is best-effort and never fails `rad startup`. | -Terminal-state entries are never reconciled by this action. +The underlying Kubernetes and Azure GETs establish whether each output resource is present; they do not interpret provider-specific provisioning-state fields. For a Radius resource with multiple outputs, all present means `Succeeded`, all missing means `Failed`, and any skipped/error or a mix of present and missing leaves the hydrated state unchanged. ### Reconciliation is best-effort and does not fail `rad startup` @@ -142,9 +145,9 @@ Nothing outside these files needs to change. - Deleting an application whose state archive contains a child resource in `Updating` and whose underlying resource genuinely is still updating waits normally and does not falsely succeed. The reconciler must observe the reality-reported state and leave the store unchanged. - `rad startup` never fails because reconciliation could not reach a resource provider. The workflow log records the failure and startup returns success. - The reconciler runs no direct SQL against any resource-provider database. Every state change goes through the RP's normal write path, so state machines stay intact. -- The prototype covers dynamic-rp resources with Kubernetes `outputResources` end-to-end (the transcript's failing case): an application whose containers/gateways/secretstores were hydrated in `Updating` is deletable after `rad startup`, whether or not the underlying Kubernetes objects still exist. Terraform-backed cloud outputs record `skipped` and are follow-ups. +- The prototype covers dynamic-rp resources with Kubernetes and Azure `outputResources` end-to-end (the transcript's failing case): an application whose resources were hydrated in `Updating` is deletable after `rad startup` when the underlying Kubernetes or Azure resources no longer exist. Azure resources produced by Bicep and Terraform recipes are both covered, using either UCP-qualified Azure IDs or legacy relative ARM IDs. AWS outputs record `skipped` and remain follow-up work. ## Follow-up - A separate issue tracks verifying that concurrent `rad app delete` against the same application from two terminals is handled correctly by a regular (persistent) Radius control plane. That case is not affected by hydration — a persistent control plane already tracks in-flight operations — but it needs an explicit test so a future change cannot regress it. See [radius-project/radius#12870](https://github.com/radius-project/radius/issues/12870). -- Reality-checking Terraform-backed cloud `outputResources` (Azure/AWS resources managed via recipes) is follow-up work. Adds cloud-SDK branches inside the same dynamic-rp `reconcile` handler; no framework changes. +- Reality-checking AWS `outputResources` is follow-up work. It adds an AWS SDK branch inside the same dynamic-rp `reconcile` handler; no framework changes.