Repository navigation
1.9.79: the cron job sweeps lapsed LTV subscriptions hourly and runs the rest of LTV maintenance daily - #285
Merged
Conversation
LTV webhooks were queued by every /ltv write and sent by nothing a
default install runs: 202-cronjobs/ltv_webhooks.php was the only
dispatcher, and every default scheduler (the installer's cron line,
install.sh, both Docker cron services, AutoCron) fetches index.php
alone. EveryCronJobIsScheduledByDefaultTest listed it as NOT_SCHEDULED
("opt-in, predates the measurement rewrite": history, not a reason). The
Landing Page Optimizer's pairing webhook waited with the rest.
The dispatch moves into Prosper202\Ltv\WebhookDispatcher, and two entry
points run it: the minutely tier of index.php (last, with a 15-second
budget) and ltv_webhooks.php on its own (--budget=N, default 50), which
the Coolify worker service now runs too. Both take one MySQL named
lock, as the attribution worker and the Android intake do, and each
delivery is still claimed by compare-and-set before it is sent, so runs
the lock does not see cannot send it twice either.
- The deadline is asked before each delivery, at least one a run, so a
run ends at most one delivery past it.
- A webhook whose delivery took 2 s or more to fail is not sent to again
in that run; its other deliveries stay due, untried, and a cursor on
(next_attempt_at, delivery_id) steps past them. A receiver that times
out costs one timeout a run instead of the budget every webhook on the
install shares. A failure that comes back at once is retried at its own
pace, as before.
- Retries and backoff are unchanged (recordAttempt(): 2^n minutes, failed
after 6, the webhook marked dead). A refusal, a failed lookup or a send
that throws fails its own delivery and the run goes on (#24).
- A delivery still unsent 7 days after it was queued is failed with
"expired: ..." as its last response, never sent, its webhook left
active: an upgraded install sends the last week of its backlog, oldest
first, rather than months of stale events. Expired 1,000 a statement;
a run whose budget ends first sends nothing, since what is left to
expire is the oldest due.
- The response body is stored as UTF-8 cut at a character boundary: a
body that is not UTF-8 was refused under a strict sql_mode, the outcome
went unrecorded and the delivery would have been sent again.
- post() builds every option through requestOptions(), the guard's set
winning over the POST's, and refuses to send when curl refuses one.
Tests: WebhookDispatchIntegrationTest (a real server, a php -S receiver,
two child processes for the overlapping runs, under the lock and
without it), WebhookDispatcherTest, LtvWebhooksAreSentEveryMinuteTest
(the minutely tier, last, with its budget; the Coolify worker), and the
scheduling, guard-parity and LTV server-failure tests updated. Each
removal planted was caught by a named test. tests/live/ltv-webhooks.sh
is the pass end to end against an https receiver.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
1.9.79 gained #262. The only conflict was 202-config/changelogs.json, where both sides added 1.9.79 lines: 1.9.79's lines are kept and this branch's follow them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Brings in what 1.9.79 gained since this branch last merged it. The conflicts were only in the changelogs, where both sides added 1.9.79 lines: 1.9.79's lines are kept as it has them and this branch's follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
…the rest of LTV maintenance daily 202-cronjobs/ltv_maintenance.php was the only thing that moved a subscription no renewal reached to past_due and then canceled, and no default deployment scheduled it (EveryCronJobIsScheduledByDefaultTest had it as "opt-in, predates the measurement rewrite"). So on every default install a subscription that stopped renewing stayed active for good: in MRR (GET /ltv/mrr, the customers' rollups, GET /ltv/summary) and never in churn. The jobs move into Prosper202\Ltv\LtvMaintenance, which both entry points run under one MySQL named lock (the dispatcher's pattern, #273): - index.php's hourly tier runs the sweep (20 s budget); the daily tier runs the reconcile, the next-offer transitions, company linking and the personalization purge (60 s). ltv_maintenance.php runs all of them (--full, --batch-size=N, --budget=N, default 300 s) and refuses an option it does not know. - The sweep reads 500 subscriptions at a time, oldest period end first, by the period_end_sweep index, and writes each batch and its customers' rollups in one transaction, asking each row again so a renewal that commits meanwhile wins. A deadline before each batch bounds every job, so an upgraded install's backlog drains over successive runs. - A canceled subscription is dated when the rule cancels it (period end + grace + 30 days), not when the run got to it: dated now, the first run on a seeded install put 29,404 long-lapsed subscriptions into the 90-day churn window (18.4% monthly churn; 5.1% dated by the rule). An active subscription past the cancel point goes straight to canceled. - The reconcile asked the ledger for a customer id range with no index on customer_id, reading all of it for every 500 ids: 59.6 s for the 3,349 customers active in 48 hours out of 100,000. It now reads each account's rows by (user_id, customer_id) and writes only rows that differ: 0.7 s. Both rollup statements tie the ledger and subscriptions to the customer's account (CLAUDE.md #27). Measured on 100,000 customers, 60,000 subscriptions (30,148 lapsed) and 550,717 revenue events: sweep 3.2 s for the backlog, 0.01 s after; the reconcile 0.7 s, 0.2 s after; transitions 3.9 s; company linking 6.9 s for 4,959 unattached customers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
…e-scheduled The base now carries 1.9.79 through #282. Conflicts: changelogs.json and the upgrade guide's 1.9.79 list, the base's lines kept and this branch's added after them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
Contributor
This was referenced Oct 28, 2026
Merged
Brings in 1.9.79's new commits. The conflicts were in 202-config/changelogs.json, changelogs.txt, tests/live/README.md, where both sides added lines: the incoming lines are kept as they are and this branch's follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Brings in ltv-webhooks-scheduled (this branch's stacked base), and through it 1.9.79's new commits. The conflicts were in 202-config/changelogs.json, where both sides added lines: the incoming lines are kept as they are and this branch's follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #273 (QSB2,
1.9.79-ltv-webhooks-scheduled): review and merge this after #273. Until #273 merges, this diff includes #273's commits; the change of its own is commit dfb9783, followed by a merge of #273's branch as it now stands (272f792, with 1.9.79 through #282 in it).Backlog: 9BA0
What was wrong
202-cronjobs/ltv_maintenance.phpwas the only code that moved an LTV subscription no renewal reached topast_due, and then tocanceled. No default deployment scheduled it.EveryCronJobIsScheduledByDefaultTestlisted it as NOT_SCHEDULED ("opt-in, predates the measurement rewrite"). So on every default install a subscription that stopped renewing stayedactivefor good:GET /ltv/mrr, and the customer rollups thatGET /ltv/summarysums);at_risksegment.The same file also runs four other jobs that never ran either: the customer rollup reconcile, the next-offer transitions rebuild, company linking, and the personalization token purge.
What changed
Prosper202\Ltv\LtvMaintenanceholds the five jobs. Both entry points run it under one MySQL named lock (p202_ltv_maintenance:+ sha1 of the database), the same pattern as 1.9.79: the minutely cron sends LTV webhooks, within a budget #273's dispatcher.HOURLY_SECONDS).DAILY_SECONDS) and run before the day's run is recorded.ltv_maintenance.phpruns all five on its own. It takes--full,--batch-size=N(50–5000) and--budget=N(1–86400 s, default 300). It now refuses an option it does not know, where before it ran with defaults (--batch-size=abcwas read as 50). It exits 1 when a job failed, and answers 500 over HTTP.period_end_sweepindex (a sargablecurrent_period_end < nowcomes first). It uses a cursor, so a run never reads a row twice.mrr/active_subscription_countgo in one transaction. Before, the owners were refreshed after the sweep, and a crash in between left them stale.status = …and the lapse condition), so a renewal that commits while the sweep waits on the row wins. There is a test for this race.activesubscription already past the cancel point goes straight tocanceled, in the same batch.202_revenue_eventsforcustomer_id BETWEEN x AND y, and no index leads withcustomer_id. So it read the whole ledger for every 500 ids in the range spanned by the customers active in the last 48 hours. It now selects that window's customers 500 at a time by primary key and reads each account's ledger and subscriptions by(user_id, customer_id). It writes only rows that differ, so "N corrected" means drift was found, not thatupdated_atwas touched.user_id = ?in each derived table). TheAccountScopedJoinTestentries for the old file's three untied statements are gone.linkUnlinkedCustomers()while the budget lasts, instead of 2,000 per account per run.purgeExpired()takes aLIMIT. Each job's failure is reported and logged, and the jobs after it run, as the old file's isolated sections did.Decision: the schedule (measured)
I measured on the live instance's database, seeded with 100,000 customers, 60,000 subscriptions (30,148 of them active but lapsed 4–700 days ago, the backlog an upgraded install has) and 550,717 revenue events (116,057 of them conversions). MariaDB 10.11, PHP 8.3.6.
at_risksegment read, so it should lag the clock by at most an hour. In steady state it is one empty index probe. A backlog clears at about 9,000 subscriptions a second here, so a 20 s run clears roughly 180,000, and a larger one drains over the next hours.Decision: when a lapsed subscription was canceled
MysqlLtvRepository::mrr()counts churn as subscriptions withcanceled_atin the last 90 days. The old sweep setcanceled_at = now. On an upgraded install, its first run would therefore have dated every subscription that lapsed in the last two years as canceled this quarter.Measured on the seed:
canceled_in_window29,404, monthly churn 18.4%;The sweep now writes
canceled_at = current_period_end + (grace_days + 30) × 86400. That is the moment the rule cancels the subscription, and where an hourly sweep that had always run would have put it. It is also what the API'scancelevent does: it dates byoccurred_at, not by when the request arrived. An upgraded install therefore reads the same churn as one that ran the sweep all along.Tests
tests/Ltv/LtvMaintenanceIntegrationTest.php(@group integration, scratch database, strictsql_mode), 10 tests:canceled_atdated by the rule;mrr();--full, another account's ledger row ignored, and a row already right not rewritten;tests/Ltv/fixtures/maintenance-runner.php) runs the sweep, held at its write until it is seen inLOCK WAIT. The renewal wins.tests/Cron/LtvMaintenanceRunsHourlyAndDailyTest.php:RunLtvMaintenance([SWEEP], HOURLY_SECONDS)after its other jobs and before recording the hour;RunLtvMaintenance(DAILY_JOBS, DAILY_SECONDS)before recording the day;time() + $secondsand the jobs torunExclusive(), inside a try that logs;RunLtvMaintenance(.tests/Ltv/LtvMaintenanceSummaryTest.php: the printed line, and when index.php prints one.EveryCronJobIsScheduledByDefaultTest:ltv_maintenance.phpmoves to SCHEDULED (LtvMaintenance::runExclusive), and NOT_SCHEDULED shrinks by one;AccountScopedJoinTest: the three stale entries are gone;LtvFieldErrorsTest: the two lock failures are listed;phpstan-baseline.neon: the old file's 10 directbind_param/executeentries are gone.cmpand restored. Every one was caught by a named test failing. They ran before the base merge and the PSR-12 line wrapping, neither of which touched the planted code.updated_atdid not change. The test now backdatesupdated_atbetween the runs.RELEASE_LOCKbeforeclose().The 22 plants
testALapsedSubscriptionIsPastDueThenCanceledDatedByTheRule,testChurnCountsALapseByWhenItLapsedNotWhenTheSweepRantestEveryAccountIsSweptAndARollupCountsOnlyItsAccountsSubscriptionstestALapsed…,testEveryAccount…,testABacklogIsSweptInBatches…testABacklogIsSweptInBatchesAndADeadlineLeavesTheRestForTheNextRuntestABacklogIsSweptInBatches…testALapsed…testARenewalBetweenTheSweepsReadAndItsWriteWinstestTheReconcileCorrectsTheCustomersActiveInItsWindowtestTheReconcile…testTheReconcile…testTheLockKeepsASecondRunOuttestAJobThatFailsIsReportedAndTheJobsAfterItRuntestCompaniesAreLinkedAndExpiredTokensPurgedtestAJobThatFails…,testCompanies…testAJobNotOnTheListIsRefusedBeforeAnythingRunsLtvMaintenanceRunsHourlyAndDailyTest::testTheHourlyTierSweepsAndTheDailyTierRunsTheResttestTheHelperRunsTheLockedMaintenanceWithTheTiersDeadlinetestTheHelper…,EveryCronJobIsScheduledByDefaultTest::testIndexPhpRunsTheEntryPointOfEveryScheduledJobtestBetweenTheTiersEveryJobRunsAndTheSweepOnlyHourlyEveryCronJobIsScheduledByDefaultTest::testIndexPhpRunsTheEntryPointOfEveryScheduledJobLive pass
tests/live/ltv-subscription-lapse.sh(new) ran on an instance installed from this branch: PHP 8.3.6php -S, MariaDB 10.11, port 8240. The cron runs as the installer's cron line runs it,curl …/202-cronjobs/index.php. Before each run the script deletes this hour's and this minute's202_cronjobsrecords, which is what the clock passing does.POST /ltv/subscriptions. MRR went up by 100, and active subscriptions by 4.LTV maintenance: subscriptions 1 to past_due, 2 to canceled, 3 customer rollups refreshed. After it:GET /ltv/subscriptionsreadspast_duefor the 5-day one andcanceledfor the other two, withcanceled_at= period end + 33 days;GET /ltv/mrr: MRR is down to the one current subscription, past_due +1, canceled +2, andcanceled_in_window+1 (the one canceled 27 days ago, not the one canceled 367 days ago);GET /ltv/summary's MRR, from the customer rollups, equalsGET /ltv/mrr's;GET /ltv/customers/{id}: each lapsed customer has 0 MRR and 0 active subscriptions, and the current one keeps 30/1;p202 ltv mrrandp202 ltv subscriptions --status past_due(Go), andbin/p202 ltv:mrr --json(PHP).POST /ltv/subscriptions/{ref}/events) made the past_due one active again, a run left it so, and it was back in MRR.php 202-cronjobs/ltv_maintenance.php --budget=abcexits 1, naming the argument.--budget=30exits 0 and prints its line.Results:
p202 ltv mrrexits 6). The PHP CLI has no licence gate and ran.The measurements above were taken against the same instance's database, through
ltv_maintenance.phpand the class.Verification
.claude/skills/p202-verify/scripts/verify.sh --since origin/1.9.79-ltv-webhooks-scheduled --changed, on the merged branch (e2d4680), PHP 8.3.6. The unit tier ran 6,120 tests with 5 skipped: the 4 documented "User class requires DB singleton" skips, andCronEntryPointsFailLoudlyTest's executed case, which skips while the worktree holds the live instance's202-config.php.Beyond the script:
LtvMaintenanceIntegrationTestagainst a fresh scratch MariaDB 10.11 database (p202_9ba0_it), after the merge: OK (10 tests, 500 assertions).shellcheck --severity=warningon the new live script: clean.Not run:
tests/run-integration-suites.sh(only the new suite ran);For the reviewer to check
canceled_atdated by the rule. This is a deliberate change to what the sweep writes; the reasons are above.out of time, next run: reconcile, andltv_maintenance.php --budget=Ndoes the whole set. I did not persist a cursor for that.ltv_maintenance.phprefuses--batch-sizebelow 50 or above 5000. Before, it clamped to 50.Found outside this item (not fixed here)
subscription.changedwebhook event, while every API status change does (upsert, renewal, cancel). A CRM or the LPO never hears that a subscription lapsed. Emitting it would also send an upgraded install's whole backlog as events.trialingsubscription whose trial ends with no renewal is never swept: no rule moves it, and it staystrialingforever.pausedlikewise.GET /ltv/customers/{id}answers money and counts as mysqli strings ("mrr": "0.00000","active_subscription_count": "0"). Other routes answer numbers (CLAUDE.md [Update] Prevent warnings in subid deletion #28).MysqlCompanyRepository::linkUnlinkedCustomers()reads 2,000 pending rows with no order and without skipping whitespace-only company strings. More than 2,000 such rows in one account hold back linking for every other row in it (pre-existing).Generated by Claude Code