FEATURE: Add Autumn subscriptions and usage tracking - #38
bmdavis419 wants to merge 7 commits into
Conversation
|
Warning Review limit reached
On-demand reviews are free for the next 9 days. After that, they cost $0.25 per reviewed file. Or wait 33 minutes for your next included review. View limit detailsLimit details: You’ve used all 6 included reviews currently available. Your 49 included PR review attempts over the past 7 days set your current allowance at 6 reviews per hour. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (66)
Comment |
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
|
|
||
| yield* completePurge(sql, org.id, fileId); | ||
| forgetTagListCache(org.id); | ||
| yield* sendUsageSync; |
There was a problem hiding this comment.
🟠 High files/purge.ts:164
When sendUsageSync fails after completePurge, Autumn is never notified that the file's storage was released, so the org's billing usage remains stale indefinitely if no later mutation occurs. sendUsageSync uses jobs.trySend, which swallows enqueue failures, and the completed purge row is no longer available for sweepPurges to rediscover; use a durable usage-sync reconciliation or propagate and retry the enqueue.
Also found in 2 other location(s)
apps/web/src/lib/server/services/files/internals.ts:118
sendUsageSyncusesjobs.trySend, which deliberately swallows a queue-send outage. Unlike indexing, purge, and site cleanup, the lifecycle sweep has no usage-sync reconciliation and there is no persisted usage job state; if this enqueue fails and the org makes no later mutation, its updated storage/AI counters are never sent to Autumn, leaving billing permanently stale.
apps/web/src/lib/server/services/files/upload.ts:113
sendUsageSyncusesjobs.trySend, which deliberately logs and suppresses enqueue failures, but there is no periodic reconciliation that enqueuesusage-syncfrom the durableorg_usagecounters. If the queue is unavailable for an upload (and the org performs no later metered action), its storage balance and pending AI operations are never sent to Autumn; subsequent provider-side AI checks use stale usage and can continue allowing work beyond the plan limit.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/services/files/purge.ts around line 164:
When `sendUsageSync` fails after `completePurge`, Autumn is never notified that the file's storage was released, so the org's billing usage remains stale indefinitely if no later mutation occurs. `sendUsageSync` uses `jobs.trySend`, which swallows enqueue failures, and the completed purge row is no longer available for `sweepPurges` to rediscover; use a durable usage-sync reconciliation or propagate and retry the enqueue.
Also found in 2 other location(s):
- apps/web/src/lib/server/services/files/internals.ts:118 -- `sendUsageSync` uses `jobs.trySend`, which deliberately swallows a queue-send outage. Unlike indexing, purge, and site cleanup, the lifecycle sweep has no usage-sync reconciliation and there is no persisted usage job state; if this enqueue fails and the org makes no later mutation, its updated storage/AI counters are never sent to Autumn, leaving billing permanently stale.
- apps/web/src/lib/server/services/files/upload.ts:113 -- `sendUsageSync` uses `jobs.trySend`, which deliberately logs and suppresses enqueue failures, but there is no periodic reconciliation that enqueues `usage-sync` from the durable `org_usage` counters. If the queue is unavailable for an upload (and the org performs no later metered action), its storage balance and pending AI operations are never sent to Autumn; subsequent provider-side AI checks use stale usage and can continue allowing work beyond the plan limit.
| @@ -109,6 +110,7 @@ export const uploadOps = ( | |||
| forgetTagListCache(org.id); | |||
| yield* sendIndexJob(id, 1); | |||
There was a problem hiding this comment.
🟠 High files/upload.ts:111
Concurrent sendIndexJob(id, 1) jobs can both pass the AI quota check and embed before either usage is recorded, allowing their combined chunk count to exceed the monthly limit. The quota path performs a read-only Autumn check and only tracks after embedding commits; use Autumn’s atomic check-and-reserve flow before embedding.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/services/files/upload.ts around line 111:
Concurrent `sendIndexJob(id, 1)` jobs can both pass the AI quota check and embed before either usage is recorded, allowing their combined chunk count to exceed the monthly limit. The quota path performs a read-only Autumn `check` and only `track`s after embedding commits; use Autumn’s atomic check-and-reserve flow before embedding.
| BillingGates, | ||
| Effect.map(AutumnClient, (autumn) => | ||
| BillingGates.of({ | ||
| canShare: (orgId) => |
There was a problem hiding this comment.
🟡 Medium services/billing-gates.ts:24
Verified free-plan organizations can publish files or sites publicly, so the public_sharing paid-plan gate is not enforced. canShare is never consumed by the publish authorization path: requirePublishAllowed still checks only canPublish(trust), which explicitly allows every verified organization. Wire canShare into requirePublishAllowed (and deny when it returns false) before permitting publication.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/services/billing-gates.ts around line 24:
Verified free-plan organizations can publish files or sites publicly, so the `public_sharing` paid-plan gate is not enforced. `canShare` is never consumed by the publish authorization path: `requirePublishAllowed` still checks only `canPublish(trust)`, which explicitly allows every verified organization. Wire `canShare` into `requirePublishAllowed` (and deny when it returns `false`) before permitting publication.
| forgetTagListCache(org.id); | ||
| yield* sendIndexJob(id, 1); | ||
| if (visibility.public) yield* sendScanJob(id, 1); | ||
| yield* sendUsageSync; |
There was a problem hiding this comment.
🟠 High files/upload.ts:113
Concurrent sendUsageSync jobs can charge the same aiOpsPending value multiple times: syncUsage reads and sends the value to Autumn.track before settling it, so overlapping jobs both report the full amount while the local counter is only reduced to zero afterward. Add per-org serialization or an atomic claim/idempotency key before tracking usage.
Also found in 3 other location(s)
apps/web/src/lib/server/jobs/consumer.ts:111
usageSynchas no per-org claim or idempotency key, although every storage mutation and indexing completion can enqueue another identical job. If two queued sync deliveries for the same org overlap, both read the same positiveaiOpsPending, both callautumn.trackfor it, and only afterward subtract it locally; the provider is charged twice for the same embedded chunks.
apps/web/src/lib/server/services/billing.ts:88
syncUsagereadsaiOpsPending, sends it to Autumn, and only then subtracts it with a non-atomic update. Twousage-syncjobs for the same org can both read the same positive pending value before either reachessettleAiOps; both calls then invokeautumn.trackfor that full value, while the twoGREATEST(0, ... - value)updates merely leave the local counter at zero. This over-reports AI usage and can overcharge the customer whenever multiple queued syncs overlap.
apps/web/src/lib/server/services/files/thumbnails.ts:121
The new
usage-syncenqueue can run concurrently with the other storage/indexing sync jobs for the same org. Each worker'sBilling.syncUsagereads the sameaiOpsPending, callsAutumn.trackfor that full value, and only then subtracts it; two workers therefore both report the same AI operations while the local counter is clamped back to zero. A thumbnail upload overlapping another queued sync can consequently overcharge the customer in Autumn.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/services/files/upload.ts around line 113:
Concurrent `sendUsageSync` jobs can charge the same `aiOpsPending` value multiple times: `syncUsage` reads and sends the value to `Autumn.track` before settling it, so overlapping jobs both report the full amount while the local counter is only reduced to zero afterward. Add per-org serialization or an atomic claim/idempotency key before tracking usage.
Also found in 3 other location(s):
- apps/web/src/lib/server/jobs/consumer.ts:111 -- `usageSync` has no per-org claim or idempotency key, although every storage mutation and indexing completion can enqueue another identical job. If two queued sync deliveries for the same org overlap, both read the same positive `aiOpsPending`, both call `autumn.track` for it, and only afterward subtract it locally; the provider is charged twice for the same embedded chunks.
- apps/web/src/lib/server/services/billing.ts:88 -- `syncUsage` reads `aiOpsPending`, sends it to Autumn, and only then subtracts it with a non-atomic update. Two `usage-sync` jobs for the same org can both read the same positive pending value before either reaches `settleAiOps`; both calls then invoke `autumn.track` for that full value, while the two `GREATEST(0, ... - value)` updates merely leave the local counter at zero. This over-reports AI usage and can overcharge the customer whenever multiple queued syncs overlap.
- apps/web/src/lib/server/services/files/thumbnails.ts:121 -- The new `usage-sync` enqueue can run concurrently with the other storage/indexing sync jobs for the same org. Each worker's `Billing.syncUsage` reads the same `aiOpsPending`, calls `Autumn.track` for that full value, and only then subtracts it; two workers therefore both report the same AI operations while the local counter is clamped back to zero. A thumbnail upload overlapping another queued sync can consequently overcharge the customer in Autumn.
| siteCleanup: (job) => | ||
| sites.cleanupSession(job.sessionId).pipe(Effect.as('done')) | ||
| sites.cleanupSession(job.sessionId).pipe(Effect.as('done')), | ||
| usageSync: () => billing.syncUsage.pipe(Effect.as('done')) |
There was a problem hiding this comment.
🟠 High jobs/consumer.ts:111
usageSync redelivers after billing.syncUsage fails, so a successful Autumn.track followed by a failed settleAiOps causes the unchanged aiOpsPending value to be tracked again without an idempotency key. This permanently double-counts usage and can overcharge the organization; make the sync idempotent or prevent redelivery after the provider call succeeds.
Also found in 1 other location(s)
apps/web/src/lib/server/services/files/upload.ts:113
A
usage-syncinitiated here can be retried afterAutumn.tracksucceeds but before its local pending counter is settled (for example, a transient Postgres failure). The retry reads the unchangedai_ops_pendingand callstrackagain; Autumn documents that eachtrackrecords/decrements usage, and this code supplies no idempotency key. This permanently double-charges the same AI operations.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/jobs/consumer.ts around line 111:
`usageSync` redelivers after `billing.syncUsage` fails, so a successful `Autumn.track` followed by a failed `settleAiOps` causes the unchanged `aiOpsPending` value to be tracked again without an idempotency key. This permanently double-counts usage and can overcharge the organization; make the sync idempotent or prevent redelivery after the provider call succeeds.
Also found in 1 other location(s):
- apps/web/src/lib/server/services/files/upload.ts:113 -- A `usage-sync` initiated here can be retried after `Autumn.track` succeeds but before its local pending counter is settled (for example, a transient Postgres failure). The retry reads the unchanged `ai_ops_pending` and calls `track` again; Autumn documents that each `track` records/decrements usage, and this code supplies no idempotency key. This permanently double-charges the same AI operations.
| free: { storedBytes: 2 * GIB }, | ||
| pro: { storedBytes: 100 * GIB } | ||
| free: { storedBytes: 2 * GIB, aiOpsPerMonth: 500 }, | ||
| pro: { storedBytes: 100 * GIB, aiOpsPerMonth: 10_000 } |
There was a problem hiding this comment.
🟡 Medium server/plans.ts:8
After a month rollover, syncUsage sends prior-month ai_ops_pending through Autumn.track as new-cycle usage, so delayed operations consume the customer's new-month aiOpsPerMonth allowance while the dashboard reports zero prior usage. Reconcile or clear ai_ops_pending when readOrgUsage advances aiOpsMonth so only current-month operations are tracked.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/plans.ts around line 8:
After a month rollover, `syncUsage` sends prior-month `ai_ops_pending` through `Autumn.track` as new-cycle usage, so delayed operations consume the customer's new-month `aiOpsPerMonth` allowance while the dashboard reports zero prior usage. Reconcile or clear `ai_ops_pending` when `readOrgUsage` advances `aiOpsMonth` so only current-month operations are tracked.
e5622b1 to
3da061a
Compare
3da061a to
a8491a7
Compare
|
Reviewed the actual Macroscope findings against the revised source. The missing webhook configuration guard was valid and is fixed: real billing keys now require a nonblank webhook secret in development and production; disabled billing and development fakes remain supported. All20 configuration tests passed and targeted independent review is clean. Other findings are resolved by the reviewed implementation: local quota reservations, bounded raw webhook bodies, durable usage recovery, serialized authoritative subscription reconciliation, absolute current-month balance synchronization, and filtering billing.updated events. Additive track/ai_ops_pending is removed. Free public sharing remains intentional for verified organizations and is represented in the Autumn plan. These comments originated on the original billing commit even where GitHub now associates their positions with the updated head. Provider sandbox checkout and production verification remain outstanding launch checks. |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| yield* holdUsage(sql, orgId); | ||
| const rows = yield* sql<{ value: number }>` | ||
| DELETE FROM ai_usage_reservations | ||
| WHERE org_id = ${orgId} AND token = ${token} AND expires_at > clock_timestamp() | ||
| RETURNING value`.pipe(Effect.mapError(storage('commit AI reservation'))); | ||
| const row = rows.at(0); | ||
| if (!row) | ||
| return yield* new StorageError({ | ||
| operation: 'commit AI reservation', | ||
| cause: 'The AI reservation expired before indexing completed' | ||
| }); | ||
| yield* recordAiOps(sql, orgId, row.value); |
There was a problem hiding this comment.
A reservation made while an organization is on Pro can be committed after the organization has been downgraded to Free. This code removes the reservation and adds its full value without checking the current plan limit, so a Free organization can retain more than its allowed monthly AI usage. In-flight indexing work can therefore consume AI quota that the downgraded plan no longer permits.
How this was verified: A Postgres-backed check reserved 600 operations on Pro, changed the plan to Free, and committed a Free balance of 600 against the 500-operation allowance.
Artifacts
Plan-downgrade quota validation source
- The executed test reserves AI usage on Pro, applies a Free-plan downgrade, and asserts that the committed balance remains within the Free allowance.
- The test output shows a Free organization with 600 committed AI operations against a 500-operation allowance, proving the quota can be exceeded.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/src/lib/server/usage.ts
Line: 147-158
Comment:
**Downgraded quota can be exceeded**
A reservation made while an organization is on Pro can be committed after the organization has been downgraded to Free. This code removes the reservation and adds its full value without checking the current plan limit, so a Free organization can retain more than its allowed monthly AI usage. In-flight indexing work can therefore consume AI quota that the downgraded plan no longer permits.
**How this was verified:** A Postgres-backed check reserved 600 operations on Pro, changed the plan to Free, and committed a Free balance of 600 against the 500-operation allowance.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
autumn.config.ts declares the storage_bytes, ai_ops, and public_sharing features and the free and pro plans; plans.ts mirrors the limits and a unit test imports the config to keep them in step. AutumnClient wraps autumn-js behind a service with a real client when AUTUMN_SECRET_KEY is set, a fail-open null client that logs once otherwise, and an in-memory fake for tests. First sign-in creates the Autumn customer for the org. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A usage-sync job reports the org's stored bytes as its storage balance and tracks the embedded chunks recorded since the last sync; it is sent after every upload, version commit, site commit, thumbnail, and purge, and reads the org_usage row when it runs so a burst of sends costs one report. Migration 0008 adds the monthly and pending AI counters. Before embedding, indexing asks BillingGates whether the plan allows the chunks; when it does not, the file finishes keyword-only with index_error 'AI quota exhausted' and is offered again a day later. BillingGates.canShare answers the public_sharing feature for the trust policy to combine with the org's trust. Every gate fails open; the local org_usage reservation stays the hard stop. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
/settings/billing shows the org's plan with a storage bar and an AI operations bar read from org_usage, an Upgrade button that follows the Autumn checkout URL from POST /api/billing/checkout, and a Manage billing button that follows the customer portal URL from POST /api/billing/portal. GET /api/billing serves the summary; the two POSTs require an owner with a write credential. Without Autumn configured the buttons are disabled and the summary still renders from local counters. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
POST /api/webhooks/autumn verifies the Svix headers (svix-id, svix-timestamp, svix-signature) against AUTUMN_WEBHOOK_SECRET with a WebCrypto HMAC-SHA256 and a five minute timestamp window, logs every event by type, and on a billing.updated plan list sets orgs.plan to pro when a pro subscription is in effect and back to free otherwise, so the quota and trust gates read the plan locally. Unknown customers and other event types are acknowledged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
a8491a7 to
9df460d
Compare
| const rows = yield* sql<{ id: string }>` | ||
| SELECT id FROM orgs WHERE id = ${orgId} FOR UPDATE`; | ||
| if (rows.length === 0) return null; | ||
| const plan = yield* autumn.getPlan({ customerId: orgId }); |
There was a problem hiding this comment.
🟠 High server/billing-webhook.ts:42
When a customer is in a Pro trial, billing.updated causes reconcileOrgPlan to write free to orgs.plan, removing the customer's Pro limits and access. autumn.getPlan excludes the documented trialing subscription status, so the reconciliation must treat trialing as an eligible subscription status.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/billing-webhook.ts around line 42:
When a customer is in a Pro trial, `billing.updated` causes `reconcileOrgPlan` to write `free` to `orgs.plan`, removing the customer's Pro limits and access. `autumn.getPlan` excludes the documented `trialing` subscription status, so the reconciliation must treat `trialing` as an eligible subscription status.
| yield* holdUsage(sql, orgId); | ||
| const rows = yield* sql<{ value: number }>` | ||
| DELETE FROM ai_usage_reservations | ||
| WHERE org_id = ${orgId} AND token = ${token} AND expires_at > clock_timestamp() | ||
| RETURNING value`.pipe(Effect.mapError(storage('commit AI reservation'))); | ||
| const row = rows.at(0); | ||
| if (!row) | ||
| return yield* new StorageError({ | ||
| operation: 'commit AI reservation', | ||
| cause: 'The AI reservation expired before indexing completed' | ||
| }); | ||
| yield* recordAiOps(sql, orgId, row.value); |
There was a problem hiding this comment.
🟠 High server/usage.ts:147
commitAiOps records a reservation using the current plan without rechecking its allowance, so work reserved before a downgrade can commit afterward and leave ai_ops_month above the downgraded plan's limit. Compare the reservation plus current usage against planLimits(usage.plan).aiOpsPerMonth and fail the transaction before recording it.
- yield* holdUsage(sql, orgId);
+ const usage = yield* holdUsage(sql, orgId);
const rows = yield* sql<{ value: number }>`
DELETE FROM ai_usage_reservations
WHERE org_id = ${orgId} AND token = ${token} AND expires_at > clock_timestamp()
RETURNING value`.pipe(Effect.mapError(storage('commit AI reservation')));
const row = rows.at(0);
if (!row)
return yield* new StorageError({
operation: 'commit AI reservation',
cause: 'The AI reservation expired before indexing completed'
});
+ if (usage.used + row.value > planLimits(usage.plan).aiOpsPerMonth)
+ return yield* new StorageError({
+ operation: 'commit AI reservation',
+ cause: 'The AI reservation exceeds the current plan allowance'
+ });
yield* recordAiOps(sql, orgId, row.value);🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @apps/web/src/lib/server/usage.ts around lines 147-158:
`commitAiOps` records a reservation using the current plan without rechecking its allowance, so work reserved before a downgrade can commit afterward and leave `ai_ops_month` above the downgraded plan's limit. Compare the reservation plus current usage against `planLimits(usage.plan).aiOpsPerMonth` and fail the transaction before recording it.
1 participant
Add Free and Pro subscriptions, checkout, billing management, and local storage/AI limits. Verified Free organizations retain public sharing. Billing controls are owner-only, show pending state, and recover from bounded request failures.
Indexing reserves local AI quota before provider work and commits successful indexing with usage. Queue delivery synchronizes absolute storage and UTC-calendar-month AI balances, with serialized writes and durable recovery. Signed Autumn webhooks reconcile authoritative subscriptions under an organization lock. Real billing keys require a webhook secret.
Billing customers use the stored organization name and a deterministic known owner’s email. An ownerless organization defers customer creation; lookup/provider failures preserve sign-in and retry on a later login.
Validation:
Stack layer 9/11: depends on #37; followed by #39. No merge or deployment.
Note
Add Autumn subscriptions and usage tracking
BillingandBillingGatesservices for plan summaries, checkout/portal URLs, and AI quota enforcement during indexing in billing.ts and billing-gates.ts.org_usage.Macroscope summarized 9df460d.
Not safe to merge until the outstanding AI quota enforcement issue is fixed.
Fix with agent prompt
Summary
Reviews (2) · Last reviewed commit: "Use organization owner details when crea..."