Skip to content

fix(pagination): follow meta.next so auto-pagination survives cursor mode - #319

Merged
Narayana Shanbhog Plivo (narayana-plivo) merged 2 commits into
masterfrom
fix/cursor-pagination
Sep 25, 2026
Merged

Narayana Shanbhog Plivo (narayana-plivo) merged 2 commits into
masterfrom
fix/cursor-pagination

Conversation

@anshagrawal-plivo

Copy link
Copy Markdown
Contributor

The defect

PlivoResourceInterface.__iter__ computed its own offsets (offset += limit)
and terminated only when a page came back empty. Under
force_cursor_pagination a client-supplied offset is ignored and page 1 is
returned instead — HTTP 200, well-formed body, wrong rows — so that condition
never fires. Iteration loops forever re-yielding the same 20 records. Calls
and Recordings both inherit this.

Separately, neither Calls.list nor Recordings.list accepted a cursor
argument. validate_args rejected the keyword outright via
inspect.getcallargs, before to_param_dict was ever reached — so a caller
could not opt out manually either.

Reproduced on live QA, not just in mocks

QA trace us-west-1 was flipped through all three deployment states
(plivo/voice-consul-cfg#1411, #1412) and the SDK run against each:

Mode Resource master this PR
CURSOR force=true Call 20 distinct / 240 — 220 dupes, non-terminating 240 / 240, 0 dupes
Recording — 240 / 240, 0 dupes
HYBRID enabled=true Call 240 / 240, 0 dupes —
OFFSET both false Call 240 / 240, 0 dupes 240 / 240, 0 dupes

In cursor mode master issued offset=0,20,40…240 and received the identical 20
rows every time. Capped at 13 requests for the test; uncapped it never returns.

Note the hybrid column: offset is still applied there, so iteration looks
perfectly healthy
and QA could not have caught this before the flag was
flipped.

The fix

__iter__ now follows meta.next, reading whichever of cursor= or offset=
the query string carries:

/v1/.../Call/?limit=20&cursor=<opaque>   cursor / hybrid
/v1/.../Call/?limit=20&offset=20         offset (legacy)

One code path stays correct in all three modes without the SDK knowing which is
live. Responses carrying no meta fall back to the offset walk, so resources
outside this rollout are unaffected.

When following a cursor, no offset is sent. Calls.list defaults
offset=0, and under hybrid the server still applies a supplied offset — so
offset=0 alongside a cursor would pin the walk to page 1, reintroducing the
same non-termination on the mode QA runs today. A guard also stops the walk if
the next page's params match the ones just used, which closes the failure class
even against a server that ignores paging params entirely.

Adds cursor to Calls.list and Recordings.list as a trailing optional
argument.

Backwards compatibility

Existing list(offset=...) calls are untouched — offset keeps its 0
default and plain list() still sends offset=0, as
tests/resources/test_calls.py asserts. cursor is appended last in both
signatures, so positional callers are unaffected.

Tests

tests/resources/test_pagination.py covers all three modes plus the meta-less
fallback, mocking list() rather than hitting a server — a runaway walk trips a
request cap and fails instead of hanging CI.

Full suite: 277 passed. The 4 test_jwt failures are pre-existing on Python
3.12 (assertRaisesRegexp, removed in 3.12) and fail identically on master.

Out of scope

Under offset pagination the Recordings walk duplicates rows on a live table —
65–73 per 240 in QA, which writes recordings continuously. That is offset
paging's inherent drift (head inserts shift the window), affects master and
this branch equally, and is not fixable client-side. Cursor mode eliminated it
on the same table: 0 duplicates. Worth noting as an argument for the
rollout.

🤖 Generated with Claude Code

…mode

PlivoResourceInterface.__iter__ computed its own offsets (offset +=
limit) and stopped only when a page came back empty. Under the
force_cursor_pagination switch a client-supplied offset is ignored and
page 1 is returned instead, so that condition never fires: iteration
loops forever re-yielding the same 20 records. Calls and Recordings
both inherit this.

__iter__ now follows meta.next, reading whichever of cursor= or offset=
the query string carries, so one code path stays correct across offset,
hybrid and cursor deployments without the SDK knowing which is live.
Responses with no meta fall back to the offset walk.

When following a cursor the walk sends no offset. Calls.list defaults
offset=0, and under hybrid the server still applies a supplied offset,
so offset=0 alongside a cursor would pin the walk to page 1 -- the same
non-termination, on the mode QA runs today. A guard also stops the walk
if the next page's params match the ones just used.

Adds a cursor param to Calls.list and Recordings.list; validate_args
rejected the keyword outright before this. Existing list(offset=...)
calls are unaffected.

Tests cover all three modes plus the meta-less fallback, mocking list()
rather than hitting a server: QA is on hybrid, where offset is still
applied and iteration looks healthy, so it cannot reproduce the defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@narayana-plivo
Narayana Shanbhog Plivo (narayana-plivo) merged commit af02c2a into master Sep 25, 2026
7 of 12 checks passed
@narayana-plivo
Narayana Shanbhog Plivo (narayana-plivo) deleted the fix/cursor-pagination branch September 25, 2026 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants