Repository navigation
store: paginate catalogue sync, enable progressive UI population, and add 1-hour cache - #1022
Open
mubashardev wants to merge 6 commits into
Open
mubashardev wants to merge 6 commits into
mubashardev wants to merge 6 commits into
Conversation
mubashardev
force-pushed
the
feat/store-paginated-sync
branch
from
October 20, 2026 09:37
d344aee to
647a8f3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background and Endpoint Status
The current store implementation queries the monolithic catalogue at:
https://modules.lsposed.org/modules.jsonThis endpoint now returns HTTP 403 Forbidden. As a result, the client permanently falls back to:
https://backup.modules.lsposed.org/modules.jsonHowever, this backup host serves an archived, static snapshot of the catalogue. It does not reflect new module submissions or new module versions.
The live LSPosed repository infrastructure exposes two active endpoints:
GET https://modules.lsposed.org/module-list/{page}.jsonServes the complete module catalogue in chunks of ~30 modules per page (currently 40 pages, 1,180+ active modules), including fields for
page,pageCount, andtotal.GET https://modules.lsposed.org/module/{packageName}.jsonServes full release notes, README HTML, and historical releases for an individual module.
Technical Changes
1. Endpoint Architecture and Mirror Routing (
RepoRepository.kt)https://modules.lsposed.org/module-list/{page}.json.DETAIL_MIRRORSsohttps://modules.lsposed.org/takes precedence overhttps://backup.modules.lsposed.org/, ensuring release changelogs and download assets are served from the primary host.https://backup.modules.lsposed.org/modules.jsonas an offline/legacy fallback if page 1 network requests fail.2. Sequential Background Sync Pipeline
pageCountin the background. Sequenced requests prevent mirror rate-limiting and ensure deterministic catalog accumulation.JsonParser.parseReaderinto structuredOnlineModuleentries, passing throughusable()to filter hidden entries and deduplicate across pages.3. Progressive UI Reactivity (
RepoViewModel.kt&RepoScreen.kt)_catalog.valueis emitted with updatedmodules, currentpage, andsyncProgress.RepoViewModelreacts todataSource.catalogvia Kotlin Coroutinescombine, mapping toallEntriesand applying search query filtering and sorting off the main thread (Dispatchers.Default). Newly ingested modules appear live in the list as each page arrives.4. UI Indicators and Interaction Locking
LinearProgressIndicator) and status label above the bottom navigation bar showing sync progress percentage and page count whilecatalog.isSyncingis active.enabled = !isSyncing) to prevent users from sorting partial datasets before all pages arrive.5. 1-Hour In-Memory Cache and Tab Navigation
lastFullSyncTimeMillisupon completion of full catalog sync.RepoRepository.refresh(), ifforce == falseand the existing catalogue is fully synced withinCATALOG_MAX_AGE_MINUTES(60 minutes), the network sync is skipped.force == true) bypasses the cache, enforcesCacheControl.FORCE_NETWORK, and resets the 1-hour window.