Repository navigation
1.9.79: the same-IP fallback is an account's setting, consent-aware and off for new accounts - #283
Merged
Merged
Conversation
A conversion row said what sent it (source) but not how its click was
found, so a pixel that guessed the click from the visitor's address read
exactly like a postback that named it. Every conversion now stores a
match_method (Prosper202\Conversion\Ledger\MatchMethod):
- click_id: the request named the click (a subid or sid on gpx, upx, gpb
and pb; ClickBank's tracking code; POST /conversions and POST /events; an
Update Subids or revenue upload line; an app install's signed token, and
the goals its events reach);
- cookie: a tracking cookie named it (gpx and upx with no subid: the
campaign's tracking202subid_a_<cid> cookie or tracking202subid; px);
- visitor_id: p202.track()'s first-party visitor id (event.php);
- same_ip: nothing named a click, so the account's latest click from the
visitor's address (LastClickFromAddress in gpx, upx and px);
- unknown: before 1.9.79, the legacy baseline row, and goals reached by an
event stored before then. It is the column's default and the one
spelling every reader uses for "not recorded".
The method is a required, non-nullable MatchMethod parameter of every
writer entry point (MysqlConversionRepository::record(),
recordInTransaction(), create(); GoalEngine::ingest();
p202RecordConversion(), p202RecordLegacyConversion(), p202RecordWebEvent()),
so no default can stand in for a path that did not say.
ConversionWritersNameTheMatchMethodTest holds the signatures, every call
in the served tree, and the two tables' inserts to it. A goal row takes the
method of the event that reached it, stored on 202_goal_events, so a replay
or a re-evaluation writes rows for earlier events with their own methods.
A row written before the upgrade added the column (the tracking endpoints
do not ask whether the database is upgraded) is stored without it, logged,
and reads unknown.
Reads: GET /conversions and /conversions/{id} serve match_method and filter
on it; GET /clicks/{id}/conversions adds match_method and its label; the
new GET /conversions/match-methods counts each campaign's conversions,
counted conversions and income by method (masked for a role without
campaign data); Group Overview gets a Match method level beside Goal /
source; the Visitors and Spy breakdown gets a "Click found by" column;
/capabilities lists the methods. Go CLI: conversion list --match-method,
conversion match-methods, matched_by in click conversions. PHP CLI: the
same as conversion:list --match-method, conversion:match-methods and
click:conversions.
Schema: 202_conversion_logs.match_method and 202_goal_events.match_method,
varchar(16) NOT NULL DEFAULT 'unknown', last, in the installer's
definitions and the 1.9.78 -> 1.9.79 rung (the two definitions added to
its reconcile; the conversion ledger step adds the column with the other
ledger columns and skips its backfill walk once dedupe_key is NOT NULL).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
# Conflicts: # 202-config/changelogs.json
Found in the live pass: `p202 conversion match-methods --aff-campaign-id 2` headed its totals "Every campaign, by match method", and the API docs said `totals` was over every campaign, while the controller sums the rows it lists, so with campaign_id they are that campaign's. The heading in both CLIs is "Totals by match method" and the docs say what is summed. The pre-upgrade log line named its table with a trailing space before the closing parenthesis; it is trimmed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
The rung now reconciles 202_conversion_logs and 202_goal_events beside the re-click tables, and its line for a failed reconcile still named only the re-click tables, so an operator reading the log after a failed match_method column would have looked in the wrong place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Agent Evals failed on this branch: breakdown-003, the case this branch
added, expects a command that reads the match methods, and the reference
agent had no branch for its ask, so it fell through to `report summary`
("runs_one_of: none of the 1 captured commands contains any of
[conversion match-methods, conversion list --match-method]"). The case
encodes the intended shape; the reference agent lacked the behaviour.
It now reads the campaign's split with `p202 conversion match-methods
--aff-campaign-id` and replies with the same_ip count beside the matched
and unknown ones.
Reproduced on an instance from this branch with the committed agent
(the same failure), then all 52 cases needs_judge with none failing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
1.9.79 gained #262. The only conflict was 202-config/changelogs.json, where both sides added 1.9.79 lines: 1.9.79's lines are kept and this branch's follow them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
…ounts
When a pixel (gpx.php, upx.php, px.php) named no click, neither by a subid
nor by a cookie, it credited the latest click from the visitor's address in
the last 30 days, for every account, on by default. That guess is often
wrong on iPhones (iCloud Private Relay) and carrier NAT, it was made for a
visitor the privacy setting held back (the one click from the masked /24),
and gpx.php and upx.php searched the install's first account's clicks for
every account's pixel, so another account's pixel never got it and could
convert the first account's click.
- 202_users_pref.user_pref_same_ip_fallback (tinyint, DEFAULT 0): new
accounts and new installs start off. The 1.9.78 -> 1.9.79 rung reconciles
UserTables::usersPref(), and _upgrade_measurement_tables() runs
_upgrade_same_ip_fallback() before the reconciler meets that table: the
column is added DEFAULT 1, so every account already there keeps the
fallback in the statement that adds it, and the DEFAULT is then set to 0.
It runs wherever the table is first reconciled (the 1.9.76 step for an
install older than 1.9.77, which would otherwise add the column off for
everyone), and is probed so a re-run finishes what is left.
- p202SameIpFallbackClick() is the one guess the pixels make. It asks, in
order: whose pixel (SameIpFallback::pixelAccount(): the campaign gpx.php's
or upx.php's cid names, or the install's one account with campaigns when
no cid is sent; ofCampaign() for px.php's acip), that account's privacy in
force, p202MayRecogniseVisitor() (the decision p202ClickIdentity() now
gives or withholds the p202vid cookie by: the privacy setting, p202_consent
=0, the campaign's identity capture), and SameIpFallback::enabledFor().
A failed or unreadable setting is off; before the upgrade has added the
column it is on, which is what the upgrade will store.
- Settings: Personal settings > Advanced > Match conversions by IP address;
PUT /users/{id}/preferences (PreferenceRules), p202 user prefs update
--user-pref-same-ip-fallback, bin/p202 user:prefs:update
--user-pref-same-ip-fallback.
- Tests: SameIpFallbackIntegrationTest (the setting, the account, the guess
under each hold-back, against a server), SameIpFallbackUpgradeIntegrationTest
(from 1.9.78 and from 1.9.76, and a half-finished run),
SameIpFallbackReadsTheSettingTest (structural: every guess goes through the
function, handed the pixel's own account, which asks before it looks), and
SameIpFallbackInstanceTest (two accounts over HTTP). The instance suite for
match methods and the breakdown-003 eval case turn the fallback on for the
account they guess for and name the campaign.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
The base took origin/1.9.79 (#262). changelogs.json: the base's lines first, this branch's after them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
The base now reads a p202_consent other than 1 as consent withheld (RequestSignals::consentGiven()); the docs and the release note say so, the held-back cases include p202_consent=false, and the visitor-identity page keeps its paragraph on the landing-page id together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Brings in what 1.9.79 gained since this branch last merged it. The conflicts were only in the changelogs, where both sides added 1.9.79 lines: 1.9.79's lines are kept as it has them and this branch's follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
The base took origin/1.9.79 (#270, #280, #281, #282). #270 classifies every 202_users_pref column: user_pref_same_ip_fallback is served as stored (ServedPreferences::SERVED), and PreferenceSecretCoverageTest reads that list. changelogs.json: the base's lines first, this branch's after them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
Contributor
Brings in 1.9.79's new commits. The 1.9.78 -> 1.9.79 rung conflicted where both sides added tables to the step's list and reworded its failure message: the list keeps 1.9.79's entries (202_google, and 202_users_pref with its privacy default) and then this branch's (202_conversion_logs, 202_goal_events), and the one message names every table the step reconciles and the privacy default. The changelogs conflicted where both sides added lines: 1.9.79's are kept as they are and this branch's follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
Brings in 1.9.79-conversion-match-method (this branch's stacked base), and through it 1.9.79's new commits. 202_users_pref's definition and its OpenAPI preferences schema conflicted where both sides documented a new default: user_pref_privacy's (1.9.79) and user_pref_same_ip_fallback's (this branch); both paragraphs and both properties are kept. In the 1.9.78 -> 1.9.79 rung 1.9.79 already reconciles 202_users_pref, now with the privacy default, so this branch's own entry for it is dropped and the step's one failure message names every table it reconciles. The changelog conflicted where both sides added lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VRoS2SViuxaucFaKWsFnLr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #269 (
1.9.79-conversion-match-method): review and merge it after #269. Until #269 merges, this diff includes #269's commits; this PR's own work is 9ab2898 and f8ddf36, plus the two merge commits that bring in #269's base.Backlog: 438B ("Make the same-IP fallback configurable, consent-aware and off for new installs", P1)
What changed and why
When a pixel (
gpx.php,upx.php,px.php) named no click, neither by a subid nor by a tracking cookie, it credited the latest click from the visitor's address in the last 30 days. This was on for every account. That guess is often wrong on iPhones (iCloud Private Relay) and behind carrier NAT. It was also made for a visitor the privacy setting held back (since 1.9.77, the one click from the masked /24). Andgpx.phpandupx.phpsearched the install's first account's clicks for every account's pixel ($mysql['user_id'] = 1). Another account's pixel therefore never got the fallback, and could convert the first account's click.The setting.
202_users_pref.user_pref_same_ip_fallback,tinyint(1) NOT NULL DEFAULT '0':PUT /users/{id}/preferences(PreferenceRules:0/1),p202 user prefs update --user-pref-same-ip-fallback, andbin/p202 user:prefs:update --user-pref-same-ip-fallback.GETserves it as stored: it is onServedPreferences::SERVED, because 1.9.79: GET /users/{id}/preferences serves no integration secret #270 merged before this PR.The upgrade (the 1.9.78 → 1.9.79 rung).
_upgrade_measurement_tables([...])list gainsUserTables::usersPref()._upgrade_measurement_tables()runs_upgrade_same_ip_fallback()before the reconciler meets that table. The ledger pre-step works the same way. The helper adds the column withDEFAULT '1', so every existing row is on in the one statement that adds it. It then runsALTER COLUMN … SET DEFAULT '0'.UPDATEin the rung for one reason. An install older than 1.9.77 reconciles202_users_prefin the 1.9.76 step, which would add the column with the definition's DEFAULT, so every account would be off.SHOW TABLES,SHOW COLUMNS). A column that is already there keeps every value it holds, and a re-run after a failure only finishes the DEFAULT.UpgradeLadderTestrequires one_upgrade_measurement_tables()call per rung, assigned alone. That is why the change is a pre-step and not a second call in the rung.The one guess.
p202SameIpFallbackClick()(static-endpoint-helpers.php) is the only way a pixel guesses. It asks, in this order, and returns before the lookup at each refusal:SameIpFallback::pixelAccount()reads the account of the campaigngpx.php's orupx.php'scidnames. With nocid, it takes the install's one account that has campaigns (not deleted, campaigns not deleted).SameIpFallback::ofCampaign()readspx.php'sacipcampaign. The account always comes from the pixel's own data, never from account 1.p202ApplyOwnerPrivacy()).p202MayRecogniseVisitor(). This is the decisionp202ClickIdentity()now gives or withholds thep202vidcookie by: the privacy setting in force, ap202_consentother than1(the base'sRequestSignals::consentGiven()), or a campaign whose identity capture is off. It is one function for both, not a copy. A visitor held back fromp202vidis never matched by address.SameIpFallback::enabledFor().1is on.0, no row, a value that is not0/1, or a read that fails is off. The last two are logged.Every match is stored with match method
same_ip, as #269 does.Verification
verify.sh --since origin/1.9.79-conversion-match-method --changedon 2ba838f (this branch with #269's base at 4f2bf21, which includes 1.9.79 at 915a367),P202_TEST_DB_*pointed at a scratch database, PHP 8.3.6, MariaDB 10.11:unit: 6119 tests, 5 skipped. Four are the documented
UserTestDB-singleton skips. The fifth isCronEntryPointsFailLoudlyTest, which skips because the worktree holds the live instance's202-config.php.New tests
tests/Conversion/SameIpFallbackIntegrationTest.phpruns against a server, with connect2.php's privacy functions extracted asOwnerPrivacyTestdoes. It covers:pixelAccount(): bycid;cidvalues that name nothing or are not ids; nocidwith one account, two accounts, a deleted account or a deleted campaign, and no campaigns at all;ofCampaign();all, the installall, an unplaceable visitor undereu,p202_consent=0,p202_consent=false, and the campaign's capture off.tests/Upgrade/SameIpFallbackUpgradeIntegrationTest.phpruns the ladder as the upgrade page does:tests/Conversion/SameIpFallbackReadsTheSettingTest.phpis the structural test the brief asks for. It holds that:LastClickFromAddress::find()in the served tree is the fallback's own, or listed with why (off.php and rtr.php: redirects, not conversions);MatchMethod::SAME_IPcallsp202SameIpFallbackClick();pixelAccount(…, $_GET['cid'] ?? null, …)orofCampaign($aff_campaign_row, …)) and$_GET;p202ClickIdentity()asksp202MayRecogniseVisitor()and nevertrackingEnabled()itself.tests/Conversion/SameIpFallbackInstanceTest.php(@group instance) runs over HTTP with two accounts. It covers by subid, by cookie, address-only with the setting on and off, a visitor held back by B's own privacy, B's pixel crediting B's click while A's click from the same address is later, and a pixel naming no campaign while two accounts have campaigns.MatchMethodEndpointsInstanceTestturns user 1's fallback on for its run and names the campaign (cid), then puts the settings back. A fresh install's account now starts off.breakdown-003eval case's setup does the same.tests/live/account-pages.shcovers the new select: a refused value, a saved On shown selected, and Off saved.StoredVisitorIpSourceTest,OwnerPrivacyTest,OwnerPrivacyAppliedFirstTest,StoredAddressWritesTest/AddressWriteScanandSnakeCaseOptionsStillWorkTestare adjusted.AddressWriteScangets aNOT_ADDRESSESlist: the column hasipas a word but holds 0/1, and the test holds the listed column totinyint(1) NOT NULL.End-to-end pass on a live instance (
php -Sthrough the CI router, port 8200, two accounts, real pixel URLs, rows read from202_conversion_logsandGET /conversions). 39 passed, 0 failed, 1 skip (below). Covered:gpx.php?subid=andupx.php?subid=→click_id.cookie.gpx?cid=,upx?cid=(200) andpx?acip=→same_ip;p202_consent=0→ nothing.bin/p202 user:prefs:update 1 --user-pref-same-ip-fallback 0, stored 0):POST /users:same_ip, and A's click is untouched.cidwith two accounts: nothing, and the log says why.all: the click set no cookie and stored127.0.0.0; px, gpx and upx credit nothing.tests/live/account-pages.sh161 passed, 0 failed (10/23, before the latest merge).SameIpFallbackInstanceTestOK (6 tests) andMatchMethodEndpointsInstanceTestOK (3 tests).Plants. Each was planted in the tree, backed up and compared with
cmpbefore restore, and counted only when a named test failed:p202MayRecogniseVisitorgate removedtestTheFallbackAsksBeforeItLooks,testAVisitorHeldBackFromTheVisitorCookieIsNotMatchedByAddress(all 5 cases after the fix below),testAVisitorThePrivacySettingHoldsBackIsNotMatchedByAddress(instance)enabledForgate removedtestTheFallbackAsksBeforeItLooks,testAnAccountWithTheFallbackOffGetsNone,testAnAccountWithTheFallbackOffHasNoPixelGuess,testASecondAccountsPixelGuessesAmongItsOwnClickstestEveryFallbackIsHandedThePixelsOwnAccount,testASecondAccountsPixelGuessesAmongItsOwnClicks,testAPixelThatNamesNoCampaignGuessesNothingWhileTwoAccountsHaveCampaignstestEveryFallbackIsHandedThePixelsOwnAccountcidtakes the first owner of severaltestAPixelNamingNoCampaignIsTheOnlyAccountWithCampaigns, the instance no-cid testtestBeforeTheUpgradeAddsTheColumnTheAccountKeepsTheFallbacktestTheSettingIsReadAsStoredAndAnythingElseIsOff,testAnAccountWithTheFallbackOffGetsNonetestACampaignRowNamesItsAccountAndCaptureconsentGivendropped from the shared decisiontestTheVisitorCookieAsksTheSameDecision, the held-back integration casesp202ClickIdentitydeciding bytrackingEnabled()itselftestTheVisitorCookieAsksTheSameDecisionoffer()withoutidentity_signals(px could never guess)SameIpFallbackInstanceTest(2),MatchMethodEndpointsInstanceTestSameIpFallbackUpgradeIntegrationTest(2-3 tests each)TestPrefFlagsMatchThePreferenceRules(Go), the instance suites (422)tests/live/account-pages.sh"an unknown IP matching value is refused"The first run of the consent-gate plant was caught by only 2 of the 5 integration cases. The privacy cases passed because the seeded clicks were stored unmasked: the masked lookup found nothing, gate or no gate. A click stored masked, as a held-back visitor's is, was added, with the assertion above, and the plant is now caught by all 5.
Could not run here
my.tracking202.comstub's certificate expired on Oct 25, and the machine restart removed its/etc/hostsentry. I did not replace the trusted certificate (a system trust-store change, refused here). Without the stub:INSTALL::install_databases(),InstallAccount::create(), and the REST key insert) without the licence form.p202 user prefs update --helpand the CLI's refusal of--user-pref-same-ip-fallback yesdid run.user prefs get/update --user-pref-same-ip-fallbackdid run against a stubbed instance on 10/23, before the latest merges: 39/39 then.PreferenceRulesbyTestPrefFlagsMatchThePreferenceRules, which ran in the ladder.tests/live/upgrade-equals-install.sh. Its fresh-install phase needs the stub.SameIpFallbackUpgradeIntegrationTestcompares the upgraded202_users_prefwith the installed one from 1.9.78 and from 1.9.76.MatchMethodUpgradeIntegrationTest,LastClickFromAddressIntegrationTestandClickPathReadsIntegrationTest.UpgradeLadderTestran in the unit tier.Decisions to check
gpx.phppixel, and the universal pixel's script, which always sendscid=) belongs to the install's one account that has campaigns. With more than one such account it guesses nothing and logs why. Strictly, the brief's rule ("the pixel's own data, never account 1") leaves such a pixel with no account. Without this rule, every single-account install using the simple pixel would lose the fallback overnight. On a multi-account install, account 1's cid-less pixels stop guessing: the pixel cannot say it is account 1's. The changelog says to use the advanced pixel.identity_signals = 0) also withholds the guess, because it withholdsp202vid. The found click's own campaign is not checked.cid, the guess still searches the whole account's clicks, not only that campaign's, as px.php always has. Restricting it to the campaign would be cleaner data, and it is a separate change.0can be written only after the column exists.202_users_pref. It touches 1.9.79: record how each conversion's click was found, and report on it #269's line, a one-line merge with any other PR that edits it.Found outside this item
/etc/hostsentry was lost in the restart, and it is not running. Everyinstall-instance.shrun fails at the licence step until it is restored.bin/p202 user:prefs:update's--user-daily-emailhelp says "on/off". The column takes an hour,00-23, or"".Generated by Claude Code