Repository navigation
podcasts 0.1.2: keep RSS parsing inside the Luau CPU budget - #1133
Conversation
The browser panel never listed episodes for large feeds. Noctalia runs plugin code in a Luau sandbox with a per-callback CPU budget, and lib/rss.luau spends that budget on pattern work for every episode, so the async http callback is killed before it can fill the list. The panel then gets disabled after repeated timeouts. Measured on a 666 KB feed with 557 items (search_limit = episodes per feed): stock limit 7 -> 21-25 ms, intermittently killed stock limit 10 -> killed (default setting) stock limit 12 -> killed patched limit 10 -> 11 ms patched limit 30 -> 24 ms Each helper now returns early when its pattern cannot match: - stripCdata: plain find for '<![CDATA[' before the anchored match - unescape: plain find for '&' before the eight gsub passes - cleanText: collapse '%s+' only when a run of whitespace exists - stripTags: run '<[^>]+>' only when a tag is present No behaviour change: every early return is the same string the pattern would have returned unchanged.
|
CC @Fel-2: this pull request was automatically moved to draft until you have had a chance to look at it. It will be marked ready for review automatically once you reply here. |
|
Looks good thanks. |
Fel-2
left a comment
There was a problem hiding this comment.
Not blocking — one clarification for future readers.
needsCollapse is the only guard here whose correctness isn't obvious from its name: it is exhaustive only because %s is exactly space \t \n \r \f \v in Luau. Two lines above the function listing those classes (and a "keep in sync if %s ever changes") would make that invariant explicit in the file rather than only in this thread. The other three guards (stripCdata, unescape, stripTags) are self-evident and read well as they are.
Also worth one line in the description on how the "parsed output is identical" claim was checked (stock vs patched on the same feed at a few maxItems values). The timing side is well documented with the measurements table; the equivalence side is currently an assertion, and that is the half a reviewer cannot re-derive from the table.
For the record, I checked the diff at both ends: the four guards are behaviour-preserving on a hand-built corpus of CDATA/entity/tag/whitespace edge cases, on 20k fuzz inputs, and the full parseFeed output is byte-identical on three real feeds (12 KB / 1.35 MB / 6.3 MB) at maxItems 7/10/20/30, while parsing 20–43 % faster. Diagnosis also confirms: the host's budget is ≈26 ms per callback (measured), which matches your 21–25 ms → killed and 11 ms → passes numbers.
Everything else is clear as-is. Nice work.
%s is exactly space, tab, LF, CR, FF and VT in Luau, so the six
plain finds cover every character gsub("%s+") would touch. Says so in
the file now, next to the guard.
|
Fair point — added the invariant to the file itself, right above the guard: -- The list below is exhaustive because `%s` is exactly space \t \n \r \f \v in
-- Luau. Keep it in sync if `%s` ever changes.
local function needsCollapse(s)Commit For the record, I checked the class empirically rather than trusting memory: iterating |
Plugin
fel/podcastsplugin.toml)What it does
Fixes the episode list never appearing for feeds above a certain size. Users
with a large subscription see the panel open with the subscription listed, no
episodes, and no error message — and after a few attempts the panel is disabled
for the rest of the session.
Noctalia runs plugin code in a Luau sandbox with a CPU budget per callback.
lib/rss.luauspends that budget on pattern work per episode, so the asyncnoctalia.httpcallback that callsrss.parseFeedis killed part-way through:Because the callback dies before
epCache[feed] = epsruns,episodesstaysempty and
errorMsgstays empty too, so the panel renders as if the feed hadno episodes.
Measurements
Feed: 666 KB, 557 items, 30 show-note descriptions per page load. The number of
episodes parsed per feed is
search_limit(clamped 3–30, default 10).search_limitThe margin is thin because the budget is per callback, not per parse: the same
input alternates between passing and failing run to run. That is why the
default value of 10 already fails for this feed.
Change
Four helpers now return early when their pattern cannot match:
stripCdata— plainfindfor<![CDATA[before the anchored match, whichotherwise scans the whole string with a pattern on every call
unescape— plainfindfor&before the eightgsubpassescleanText— run%s+only when a run of whitespace is actually presentstripTags— run<[^>]+>only when the string contains a tagEvery early return yields exactly the string the pattern would have returned
unchanged, so parsed output is identical. No API, setting, or file format
changes.
External dependencies
None added. Existing:
mpv(playback),socat(mpv IPC).Testing
Exercised:
noctalia msg panel-toggle fel/podcasts:browseratsearch_limit10, 20 and 30; episode rows render with title, date andduration.
exceeded its CPU budgetentries, panel not disabled.
noctalia plugins lint ./podcasts— 0 errors, 0 warnings.python3 .github/workflows/scripts/validate-plugins.py— 280 manifests valid.noctalia msg plugin fel/podcasts:player all playstillstarts the episode (the player path does not touch
rss.luau).Screenshots / Videos
Episode list rendering with
search_limit = 20after the change:Checklist
idafter the/inplugin.tomlexactly.plugin.toml,README.md,thumbnail.webp, andtranslations/en.json.README.mdfollows the README template, documents every entry id and dependency, and includes exact panel IPC commands and launcher prefixes where applicable.thumbnail.webpis present and relevant; for a new plugin I created it with the thumbnail generator, and for an update I regenerated it with the generator if the visual identity or user-facing appearance changed. (Visual identity unchanged.)versionfollows semver and is bumped in this PR;plugin_apiis the oldest API level this plugin requires.catalog.toml; CI generates it.Code review attestation
licensedeclared inplugin.toml.