A paged LDAP search has one overall deadline, and the group cap counts entries, not pages (BACKLOG #3696 items 3 and 4)

[wshallwshall]

Vault BACKLOG #3696, items 3 and 4: the fix forward the Lander decided for engine PR 2347 (#3010) on vault PR 2528 ("each must page or be bounded"). That is a Lander decision under the owner's delegation, not an owner ruling. Item 1 (a deleted search base pages nobody) is NOT here: draft engine PR 2387 (b257-dtail, the MISCONFIGURED outcome) builds it. Item 2 is already on main.

What changes

Item 3: one overall deadline across the pages of a paged LDAP search. Before this change the worst case was 20 x ad_receive_timeout, while docs/CONNECTIONS.md called the AD login single-shot.

• The budget is _ldap3_receive_timeout(ad_receive_timeout): the setting rounded up to whole seconds, 10 s by default. There is no new setting.
• Each page's socket timeout is capped, never lengthened, to what is left on a monotonic clock. No page is requested once the budget is spent.
• When a later page's read or send times out against that cap, the search is refused as the existing LdapUnansweredError, code_name "deadline spent", which callers already treat as an outage. That refusal warns once.
• No abandon is sent past the deadline; inside the deadline the abandon is capped to what is left. The socket timeout is restored afterwards.

Item 4: the group cap counts entries, not pages.

• _MAX_PAGED_ENTRIES = 10_000 holds at any page size, including a single unpaged page. Before this change a directory with MaxPageSize = 100 refused an account in more than 2,000 groups.
• _MAX_PAGES = _MAX_PAGED_ENTRIES // 10 + 1 (1,001) stays as a backstop. It cannot bind before the entry cap on pages of 10 or more.
• A refusal at the cap warns at WARNING, once per process, like every other paging refusal. The message names the operation, never an account or DN.

Files changed:

• messagefoundry/auth/ldap.py
• tests/test_ldap_search_result_codes.py
• messagefoundry/config/settings.py: the ad_receive_timeout comment and its loosening risk text.
• docs/CONNECTIONS.md, docs/CONFIGURATION.md, docs/SECURITY-LOOSENING.md
• changelog.d/3696.changed.md: new.
• changelog.d/3010.security.md: one sentence corrected. It said "20 pages", and both fragments ship in the same release.

Landing order: draft PR 2387 also edits auth/ldap.py. This branch touches only the paging code: the module docstring, the constants, _search_paged, the new _SocketTimeoutCap, _abandon_paging, _paging_refused, and the _search_paged call in _resolve_groups. It does not touch the referral or MISCONFIGURED logic, _search, _refuse_unanswered, or the reconciler.

Merge danger

Reversible: yes. No schema change. Blast radius: AD login group resolution and the directory reconciler's paged searches. A slow directory with a very large account may now be refused at 10 s where it used to get up to 200 s.

Checks (Builder, on the final head)

• ruff check and format: clean (2,095 files).
• mypy: linux 361, win32 342, tests 1,383, all clean.
• pytest over 137 files (every test_ldap_*, test_ad_* and test_kerberos_* file, plus every test naming auth.ldap, ad_receive_timeout, those three docs, security_loosenings or changelog.d): 8,536 passed, 365 skipped, 0 failed. The changelog check: 401 well-formed.
• Mutation: each of 8 mutants turns at least one test red. Main's ldap.py run against the new tests gives 19 red, 3 green.
• Skipped: the full suite outside those files, and the hosted-only legs.
• git merge-tree --write-tree origin/main 97e72fda2 exited 0 at main 501bfdc.

Open findings (code review at xhigh, 2 rounds; the Lander grades them)

1. There is no hard wall-clock bound inside one page. A page that arrives over several reads can take up to the number of reads times what was left. This is documented in CONNECTIONS.md; the fix would be a watchdog that shuts the socket at the deadline.
2. One knob sets both the per-read stall and the overall deadline, so giving a large account more time means raising ad_receive_timeout, which is a named loosening.
3. A 10 s total may refuse a very large account on a slow directory. The reconciler then reads that account as UNAVAILABLE every pass, so it is neither revoked nor re-diffed. Documented.
4. The entry cap is checked after ldap3 has buffered the page; size_limit is not sent, because AD's paged semantics are unverified.
5. The real-socket test relies on thread timing, with a 0.4 s margin.
6. The out_of_time flag and its closure could be simplified.
7. The entry-cap branch picks the cookie to abandon by mutating newest.
8. The cap figure is stated in CONNECTIONS.md, the code and the changelog.
9. A single-page refusal at the entry cap changes #3010's "the whole answer". It is documented in the _search_paged docstring.

Proposed banner

PARTIAL -- items 3 and 4 BUILT: one overall deadline (ad_receive_timeout) across a paged LDAP search, and a 10,000-entry group cap independent of page size (BACKLOG #3696). Item 1 rides in engine PR 2387.

[wshallwshall]

QA for head 97e72fd (the Builder's line, posted by the Manager):

QA -- korus roles/BUILDER.md step 11 / Level: xhigh, from the brief. Tag: none printed (both runs returned a JSON findings array only). / Rounds: 2. Findings: round 1 raised 14, round 2 raised 15; fixed included gating and capping the abandon on the deadline, naming the deadline refusal, rounding the budget like every other read, not reporting a first-page timeout as the deadline, mapping send timeouts, and correcting the docs and the loosening text; 9 open, listed in the PR body.

[wshallwshall]

Lander, 2026-10-18: SENT BACK at 97e72fd. Do not enqueue this head. This is a Lander decision under the owner's Driver delegation.

REQUIRED, because together findings 3 and 9 weaken offboarding against main:

• Finding 3 (with 2): a 10 s total deadline reads a very large account on a slow directory as UNAVAILABLE on every pass, so it is never revoked. On main the same search eventually completes and the revoke happens. Fix: give the overall deadline its own setting, separate from the per-read stall ad_receive_timeout, with a default well above it, and document both.
• Findings 3 and 9: an account that reads UNAVAILABLE because the deadline or the entry cap refused its search must PAGE through the AlertSink (once per account per state change, not every pass), naming the cause, so a revoke that cannot run is never silent. That is what vault #3696 asks: "each must page or be bounded". Add a test for each cause.

Findings 1, 4, 5, 6, 7 and 8 go to the ledger as rows in the Lander's next pass. None of them changes behaviour or weakens a control against main.

Post the new head with a QA line naming it.

[wshallwshall]

QA (Lander finding 1 fix, docs/PHI.md section 7 stream 8, the ad_reconcile_aborted entry): head 1d3a87c8247f3ed636e7cff987b2b5fae18fe1a9, a fast-forward of a7a8e99cc touching only that one line.

Change: the entry now names the fourth source, directory-reconciler-paging:<username>, keyed on the operator account's username, with reasons group_search_deadline / group_search_cap, the probed count and a fixed cause sentence; "never a username" is scoped by name to the breaker, referral and probe fault. The inverse note now says the paging refusal, like the probe fault, has no inverse (checked against _RECONCILE_CLEAR_RESOLVES, which has no paging key).

code-review: level xhigh, on git diff a7a8e99cc..HEAD with docs/REVIEW-STANDARDS.md applied; tag: none printed. 9 findings, none changes behaviour or weakens a control:

1. FIXED: "raised as the account enters that state" understated it; now adds "and again after a restart while it stands" (the latch is process-local).
2. FIXED: the paging clause contradicted the entry's "left signed-in accounts unrevoked"; it now says a role or scope change does not revoke the sessions, though a disable or deletion still does.
3. FIXED: the inverse note's trailing "and ad_reconcile_held" read as a third no-inverse item; reworded.
4. FIXED: "for the first three" scoped by position; now names the breaker, a referral and a probe fault.
5. FIXED in the commit message: it said section 8; the row is section 7, stream 8.
6. NOT FIXED, a code defect to file: on SQL Server alert_instance.connection and users.username are both NVARCHAR(256), so a username over 228 characters overflows the paging key; the upsert fails, is logged only as "alert-instance state write failed", and no instance row is written.
7. NOT FIXED, out of scope: the username sits in the plaintext connection column (as for ad_session_revoked) until an operator resolves the alert; section 2's at-rest inventory lists only alert_instance.reason.
8. NOT FIXED, out of scope: the row restates paging behaviour without linking CONNECTIONS.md's Active Directory rows (SDS-3.5).
9. NOT FIXED, out of scope: tests/test_ad_reconcile_paging_refusal.py's module docstring says the reconciler "never revokes it while it lasts", which contradicts _note_paging_refusals (a disable or deletion still revokes).

Tests: pytest tests/test_ad_reconcile_paging_refusal.py tests/test_phi_at_rest_inventory.py tests/test_no_store_phi_coverage.py tests/test_security_doc_drift.py tests/test_phi_logging_inventory.py -> 190 passed in 33.93s. No test pins this entry's text.

[wshallwshall]

Manager: QA for the send-back repair. It covers a7a8e99, which is 08ce204 (the setting and the page), 0c59ccd (the audit row and the prune), 897dcb3 (review round 1) and a7a8e99 (review round 2). The current head 1d3a87c adds only the docs/PHI.md line covered by the 03:07Z QA comment above. The Builder's git merge-tree --write-tree origin/main a7a8e99cc exited 0 at main 6b3a5b6.

What changed:

• The setting. New [auth].ad_group_search_deadline, default 120 s: one deadline shared by all pages of the nested group search. ad_receive_timeout goes back to bounding each read only, as on main. A value above 120 s is a named loosening, listed only while a paged group search runs. The first page is never capped, so a dead directory still reads as an outage.
• The page. A refusal by the deadline, the entry cap or the page backstop still reads UNAVAILABLE, but now carries a cause. The first such pass for an account writes auth.ad_reconcile_paging_refused (the ASVS 8.3.2 parity), then latches and pages ad_reconcile_aborted with reason group_search_deadline or group_search_cap, source directory-reconciler-paging:<username>.
• The latch. Only an account the pass actually judged re-arms it; an outage, a referral or a probe fault does not. It is pruned with reconcile.prune_ledger.
• Correction to the earlier wording. Such an account is not "never revoked": the user search runs first, so a disable or deletion still revokes it. What stops is the re-diff of its roles and scope. The docs and the alert text say so.

Tests: 14 cases in the new tests/test_ad_reconcile_paging_refusal.py, plus the setting, the first-page, real-socket, timeout, posture and inventory tests. Mutation: all 13 mutants turn tests red, from budget reverted to latch set before the row.

Checks: ruff clean; mypy linux 361, win32 342, tests 1,384, all clean; pytest over 502 files: 25,873 passed, 807 skipped, 95 xfailed, 0 failed. Read windows-service-smoke and the other hosted legs on the PR.

QA -- korus roles/BUILDER.md step 11 / Level: xhigh, from the brief. Tag: none printed (both runs returned a JSON findings array only). / Rounds: 2. Findings: round 1 raised 15 and round 2 raised 13. Fixed: the latch now set after the audit row; page ordering; no re-arm on a referral or fault; a change of cause is one state; the false "never revoked" wording; the backstop text; the scrubbed username and the detail kept under 200 characters; an unknown-cause fallback; the loosening entry; the first page uncapped; the loosening gated on the nested group search; re-arm keyed on plan.outcomes; the CONNECTIONS row. 9 open, listed below.

Open findings, for the Lander to grade:

1. A paging-refused probe is still a plain UNAVAILABLE, so it counts as an outage. If it is the only candidate, every pass logs "ALL n probes failed", writes auth.ad_reconcile_skipped, and can mark no clear. The fix is a dedicated ProbeOutcome, which collides with draft PR 2387's work in plan_pass.
2. A sink that raises after the latch is set can cost a page. The audit row stands.
3. A directory that stalls late in a search, with less than ad_receive_timeout left, is refused as "deadline spent" and pages instead of reading as an outage. Documented.
4. A long pass: about 200 x 120 s in the worst case, still below main's 200 s per account.
5. The deadline is soft within one page. Documented.
6. One page and one open instance per account, with no inverse alert.
7. Four vocabularies (ldap code_names, reconcile causes, the service map, the app text) are kept in step by hand.
8. The prefixed alert source leaves 228 characters for a username under SQL Server's NVARCHAR(256).
9. SDS-3.5: the page-once rule is described in several places.

[wshallwshall]

Lander: the sql server (store + connector) 2025 leg failed once at 1d3a87c on tests/test_stage_dispatcher.py::test_adr0070_8_correlated_outage_single_reload_rearms_all[sqlserver] (1 failed, 255 passed). This PR touches messagefoundry/pipeline/alerts.py only in a docstring, and nothing else under pipeline/ or store/, so it cannot reach that test. The failed jobs are re-run as the one rerun; a second red on that leg is a finding. Enqueue once green: the review proof covers 1d3a87c (QA 6258769913, with the Lander's xhigh review of a7a8e99), and the open findings are on vault #3869 (merged in vault PR 2535).

[wshallwshall]

Lander: enqueued pinned at 1d3a87c. All required contexts completed successfully on this head after one rerun; the sql server 2025 leg's stage-dispatcher red did not repeat. Review proof: the Manager's QA 6259204445 for the send-back fix (xhigh rounds on 0c59ccd and 897dcb3), the Lander's own xhigh review of a7a8e99 (nothing weaker than main; the alert path is stronger), and QA 6258769913 for 1d3a87c, the PHI.md fix to that review's finding 1. The open findings are on vault #3869 (merged in vault PRs 2533 and 2535). None changes behaviour beyond the design the send-back asked for, or weakens a control: a Lander decision under the owner's Driver delegation.