From 61a12cf8a1d817ea1efb06899610aa7771e9364a Mon Sep 17 00:00:00 2001 From: Karolis Strazdas Date: Sat, 3 Oct 2026 22:17:24 +0300 Subject: [PATCH 1/5] chore: update reader integration and server tooling references --- client | 2 +- reader | 2 +- server | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/client b/client index 513d177..57d670a 160000 --- a/client +++ b/client @@ -1 +1 @@ -Subproject commit 513d1774670be1a617affa5249fc06a65d98b1ce +Subproject commit 57d670a4e734a0312debf9b3439e0efcdbfaf54b diff --git a/reader b/reader index 22b08f2..16611b6 160000 --- a/reader +++ b/reader @@ -1 +1 @@ -Subproject commit 22b08f2a61b1f33969ee9797b6e5acddbde0fc00 +Subproject commit 16611b646ee93ee12a2c009f0bd6e49061c72eab diff --git a/server b/server index 71b21c6..b199394 160000 --- a/server +++ b/server @@ -1 +1 @@ -Subproject commit 71b21c65833891826427a568a6d990db964b5962 +Subproject commit b19939482fcac9fba708e7aa49d2904f5bf882ec From 13dfabe933134cf67db321e820907f4a54f9b7a5 Mon Sep 17 00:00:00 2001 From: Karolis Strazdas Date: Sat, 3 Oct 2026 22:17:24 +0300 Subject: [PATCH 2/5] docs: plan repository cleanup and bounded restructuring --- CLEANUP_PLAN.md | 260 ++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 3 + 2 files changed, 263 insertions(+) create mode 100644 CLEANUP_PLAN.md diff --git a/CLEANUP_PLAN.md b/CLEANUP_PLAN.md new file mode 100644 index 0000000..6897f27 --- /dev/null +++ b/CLEANUP_PLAN.md @@ -0,0 +1,260 @@ +# Repository cleanup plan + +Status: proposed; audited 2026-10-03. This change records the plan and current +reader integration. Branch deletion, source removal, dependency removal, and +reorganization are future work, to be delivered in the separate PRs below. + +## Scope and baseline + +Keep core reading, offline library reliability, and account/server isolation as +the product priorities. Start with maintenance that reduces misleading guidance +and duplicate implementations; preserve user data and public contracts. + +| Repository | Audited checkout | Default branch at audit | +| --- | --- | --- | +| Workspace | `7ca8bdc`, before this plan | `7ca8bdc` | +| Reader | `16611b6` (`feat/reader-core`) | `f8286e1` | +| Client | `57d670a` (`feat/reader-integration`) | `fca3438` | +| Server | `b199394` | `b199394` | +| Docs | `57048e7`, detached workspace pin | `14ab148` | +| Website | `50e7561`, detached workspace pin | `50e7561` | + +The reader release is [reader PR #3](https://github.com/PapyrusReader/reader/pull/3), +followed by [client PR #29](https://github.com/PapyrusReader/client/pull/29), then +the workspace reference update. Its client dependency is immutable Git revision +`16611b646ee93ee12a2c009f0bd6e49061c72eab`. Keep these branches while their PRs +are open. The server reference advances to the already merged tooling revision; +the docs and website checkouts remain unchanged in this release. + +Audit evidence: tracked files, import/export/part reachability (including Dart +conditional imports), symbol/caller searches, workflows, Git ancestry and patch +equivalence, live GitHub branch/PR inventory, and docs Pages settings. Reachability +identifies candidates, not proof of removability. No Python dead-code conclusion +or comprehensive dependency/license audit is claimed. + +## Delivery order + +| Order / proposed PR | Owner | Concrete outcome | Completion gate | +| --- | --- | --- | --- | +| 1. Branch inventory and retirement | Each repository | Record retained SHAs and remove verified obsolete remote heads; prune stale tracking refs afterward | Re-fetch, verify worktrees/open PRs and branch rules; use the decision table below | +| 2. Current documentation and CI ownership | Workspace, client, server, docs, website | Fix misleading setup/features/links, archive completed plans, remove client-owned server workflows | Validate links and each affected workflow; build Sphinx/site; verify no required check or release depends on retired workflows | +| 3. Unused client presentation, assets and dependencies | Client | Remove caller-verified obsolete UI in small batches, with its obsolete tests | Analyzer, focused behavior tests, full offline suite, web build; native checks for plugin changes | +| 4. Test organization and coverage review | Client, reader, server | Map retained tests to current contracts; separate opt-in service tests from fast local checks | No lost persistence/reader contract coverage; deterministic local lane and documented service lane | +| 5. Reader presentation extraction | Reader, then client pin | Split shell controls/settings/navigation without changing engine/public API behavior | Reader suite, example checks, browser regressions, host integration and platform CI | +| 6. Client and server bounded restructuring | Owning component, one feature at a time | Extract library/acquisition responsibilities and isolate production/demo data | Feature and contract tests; persistence behavior unchanged; no bundled schema changes | +| 7. Prevent recurrence | Workspace and component CI | Explicit check/dependency scope, reproducible artifacts, docs freshness and pin consistency | CLI regression checks and a clean-checkout run; documented fast/full commands agree with CI | + +Prioritize 1–3 before moving directories. Each PR should name removed callers, +the replacement behavior, relevant verification, and its rollback commit. Record +before/after CI duration, tracked-file count, and dependency/build size where +affected; do not promise a performance gain from line counts alone. + +## Branch decisions + +These are actual GitHub heads, not merely local `origin/*` references. Refresh +this inventory immediately before executing retirement. + +| Repository / branch / tip | Evidence | Proposed action | +| --- | --- | --- | +| Client `feature/124c0xc0kbv-slack-community` / `6e78c8d` | [PR #27](https://github.com/PapyrusReader/client/pull/27) merged; both commits patch-equivalent to master | Retire after checking no dependent open PR/worktree | +| Reader `chore/refresh-example-lockfile` / `b3fc338` | [PR #2](https://github.com/PapyrusReader/reader/pull/2) merged; tip is an ancestor of master | Retire | +| Server `chore/development-tooling` / `71b21c6` | [PR #6](https://github.com/PapyrusReader/server/pull/6) merged; patch-equivalent to master | Retire | +| Docs `codex/kar-5-sphinx-readme` / `fd4c61c` | [PR #1](https://github.com/PapyrusReader/docs/pull/1) merged; tip is an ancestor of master | Retire | +| Workspace `feature/opds-support` / `f540e85` | [PR #3](https://github.com/PapyrusReader/papyrus/pull/3) merged an earlier head `fb95d18`; four subsequent commits only update client/reader gitlinks | Conditional retirement: confirm every referenced component revision is preserved/covered by current component master; retain the tip in the retirement record | +| Client `copilot/fix-running-task-attempts` / `a5294b2` | [PR #10](https://github.com/PapyrusReader/client/pull/10) closed unmerged; one unique “Initial plan” commit, but its tree is identical to its parent | Retirement candidate after checking abandoned-task/dependent-PR references; distinguish it from merged work | +| Docs `gh-pages` / `069a780` | Unique publication-history commit; Pages currently uses Actions, with source master, and Deploy uploads/deploys artifacts | Hold until deployment references, rollback needs and branch rules are checked; archive publication history before deciding | +| Active reader/client/workspace release branches; all `master` branches | Current PRs/default branches | Keep | +| Website | Only master exists remotely | No branch cleanup needed | + +Some local tracking refs already outlive their remote heads, including workspace +and client `chore/development-tooling` and several older client feature branches. +Use `git fetch --prune` only after recording the live inventory. Review local +branches separately: workspace tooling is merged, while local reader/docs master +branches are behind. Do not reset detached submodules or delete a checked-out +branch. `git cherry` can miss a combined squash; check PR head/merge SHAs and the +actual patch rather than treating every non-ancestor as unmerged work. + +Before any remote deletion, record the full tip SHA and PR URL, check tags, +worktrees, protected/required branches, deployments and dependent PRs. Delete only +the reviewed head, never force-purge history. A preserved tip can restore the head +with `git push origin :refs/heads/`. + +## Documentation and automation repairs + +| Location | Verified problem | Planned repair | +| --- | --- | --- | +| Workspace `DEVELOPMENT.md` | Claims reader lockfile is ignored; records an old reader pin and pending GitHub sign-in | Describe committed reader lockfile/worker reproducibility; replace transient workstation status with reproducible commands. Keep the dated baseline as history and label newer results separately | +| Workspace `tools/papyrus`, `AGENTS.md`, VS Code tasks | `check all` covers tooling/client/server, not reader; reader dependency setup does not enforce its committed lockfile | Define “all” explicitly; add reader checks/locked dependency setup and an opt-in example/browser lane. Update help, tasks, docs and CLI regressions together | +| Client `README.md` | Format/feature lists exceed the EPUB/PDF reader adapter; old coverage/project links | Separate reading support from metadata import and roadmap; verify each storage/annotation/statistics claim against active code | +| Client `.github/workflows/server-ci.yml`, `server-release.yml` | Both target nonexistent `server/` inside the client repo; typecheck nonexistent `src/`; server has its own CI | Confirm no repository rules/tag-release consumers require them, then delete these two workflows. Keep Flutter CI/release workflows | +| Server `README.md` | Missing `docs/flutter-auth-integration.md`, `auth-testing.md`, `powersync-sandbox.md`, `acquisition-downloads.md` | Restore concise current runbooks or replace links with real sources. Use `.env.example`, routes, tests and dev pages; do not invent missing guide contents | +| Server `AGENTS.md` | Infrastructure map names `papyrus/core` for configuration, while config is `papyrus/config.py` | Correct ownership paths while preserving migration/test rules | +| Docs `design/server-architecture.rst`, `_static/openapi.yaml` | Manually maintained API paths/contracts; architecture uses `/api/v1`, while `.env.example` configures `/v1` | Make API prefix configurable in prose; generate public schema from `create_app()` using safe deterministic test settings; exclude private dev surfaces and verify schema freshness in CI | +| Docs `implementation/technologies.rst` | Lists Supabase and Redis; current server dependencies/Compose instead use PostgreSQL, FastAPI and PowerSync | Describe the deployed implementation and clearly distinguish proposed options | +| Client `docs/catalogs-ui-redesign.md`, `docs/superpowers/plans/` | Completed plans/verification still describe unmerged working-tree state; historical agent execution instructions mixed with current guidance | Archive the four dated OPDS/catalog/library plans with status and PR links; retain useful current decisions/runbooks. Keep regenerable local screenshot commands; do not treat ignored build images as missing source | +| Docs workspace pin | `57048e7` predates already merged docs README improvements at `14ab148` | Build/check newer docs in a separate branch, then update the workspace pin in its own PR | +| Client `.gitignore`, local override guidance | Guidance calls `app/pubspec_overrides.yaml` ignored, but `git check-ignore` finds no rule | Explicitly ignore local overrides and document locked Git-pin restoration; never publish a local-path lockfile | +| Website `README.md`, `index.html` | Old `Eoic/Papyrus` docs/releases/project links; README uses `npm install` despite committed lockfile | Check final public destinations, update links and use reproducible `npm ci`; verify public capability claims against released behavior | +| Docs CI | Deploy builds only on master/dispatch; no PR documentation build | Add PR Sphinx validation, warnings policy and link/schema checks; preserve requirement IDs and publication behavior | + +Source ownership: workspace owns setup/coordination; each component README owns +its current commands and capabilities; reader `docs/integration.md` owns the +host/package contract; server schema/routes own HTTP contracts; docs owns the +published narrative; completed plans belong in dated archives with a current +index. Avoid maintaining copied setup commands and API tables in multiple places. + +## Client source and file candidates + +The tracked Dart graph rooted at `app/lib/main.dart` found 21 unreachable source +files. Symbol checks confirmed apparent matches such as private `_RegisterForm` +in `register_page.dart` are a different implementation. Generic `Book`, `Search` +and `SearchField` names also require import-aware review, not plain name counts. +Re-run the graph after preceding PRs; include tool/test/platform entry points. + +### First removal batch: obsolete presentation + +The following 14 files have no Dart importers in the audited source/tests: + +```text +app/lib/forms/login_form.dart +app/lib/forms/register_form.dart +app/lib/pages/books_page.dart +app/lib/widgets/book_details/eink_book_details_tab_bar.dart +app/lib/widgets/filter/active_filter_bar.dart +app/lib/widgets/goals/add_goal_card.dart +app/lib/widgets/heading.dart +app/lib/widgets/input/text_input.dart +app/lib/widgets/profile/profile_stats_card.dart +app/lib/widgets/profile_button.dart +app/lib/widgets/search_settings.dart +app/lib/widgets/shared/eink_page_header.dart +app/lib/widgets/shared/quick_filter_chips.dart +app/lib/widgets/shared/view_mode_toggle.dart +``` + +`login_form.dart` is entirely commented-out code. The router uses the current +library/auth pages. `app/lib/widgets/book/book.dart` and +`app/lib/widgets/search.dart` form a further two-file chain imported only by the +obsolete `books_page.dart`; remove together if the final caller check confirms +that chain. Keep `app/lib/models/book.dart`, the active library widgets, and +current auth forms inside their pages. + +### Second removal batch: test-only implementations + +| Candidate | Existing test consumers | Required review | +| --- | --- | --- | +| `app/lib/models/active_filter.dart` | `test/models/active_filter_test.dart`, plus the obsolete active filter bar | Compare against current `LibraryFilters` and retained serialized data | +| `app/lib/models/search_filter.dart`, `app/lib/utils/search_query_parser.dart` | `test/models/search_filter_test.dart`, `test/utils/search_query_parser_test.dart` | Check whether advanced-query behavior is a product requirement; preserve useful semantics in active search before retiring implementation/tests | +| `app/lib/widgets/add_book/digital_book_import_sheet.dart` | `test/widgets/add_book/digital_book_import_sheet_test.dart` | Compare selection, cancellation and file-picker contracts with active `BookImportSheet` | +| `app/lib/widgets/add_book/book_import_results_sheet.dart` | `test/widgets/add_book/book_import_results_sheet_test.dart` | Preserve retry, commit, duplicate handling and temporary-file cleanup coverage in active import flow | + +Delete a test only when its sole implementation is retired and its useful +behavior is covered by the replacement. Do not keep dead UI solely to keep its +tests green, or remove behavior tests solely to reduce suite size. + +### Assets, dependencies and generated files + +- `app/assets/images/auth-illustration-25.png` and `auth-illustration-3.png` have + no tracked text references and are absent from the asset manifest: first asset + removal candidates. Check native references and documentation previews first. +- `book_placeholder_2.jpg` and `profile.png` are declared but have no other + filename references. Investigate dynamic paths before removing declarations + and files. `book_placeholder.jpg` is used by a layout test; review separately. +- `collection`, `cupertino_icons` and `google_fonts` have no direct package URI + imports in client `app/lib`; inspect tests, generated code, fonts, tooling and + platform registration before proposing dependency removal. Resolve lockfiles + and measure build impact in that PR. +- Keep `epub_pro`, `syncfusion_flutter_pdf`, `dart_mobi` and `unrar_file` for now: + `file_metadata_service.dart` actively imports them. Reader support for EPUB/PDF + does not mean broader metadata-import dependencies are unused. +- Keep committed lockfiles, reader `assets/epub_worker.js`, its generator and + third-party notices, theme-generated source, native runner files and Podfiles. + Generated-but-packaged files need a reproducible generator, not blanket deletion. +- Build/cache directories, `.local/` screenshots, local media/databases and + credentials are separate from tracked source. Inventory disk usage separately; + do not use `git clean -xfd`, the workspace Purge task, or database resets as + housekeeping shortcuts. No local user-data purge is part of this plan. + +## Test retention and execution lanes + +Retain regressions for profile isolation, offline upload queues/tombstones, +media retry/cleanup, schema migrations, auth ownership, EPUB chapter order and +reflow offsets, PDF modes/refitting, theme popup contrast, focus-mode keyboard +navigation, session identity and version-1 locator restoration. Legacy CFI and +old metadata envelopes are compatibility data; old sync-route rejection tests +also protect the current API. “Legacy” is not a removal criterion. + +Current optional tests are intentional: client OPDS network smoke tests require +`OPDS_SMOKE_URL`; live sync requires `PAPYRUS_LIVE_SYNC`/integration configuration; +server auth smoke tests require provider/SMTP configuration; local env-file +tests can skip when `.env` is absent. Document conditions and run these in a +separate configured lane rather than deleting them as stale tests. + +For each feature extraction, list covered contracts and duplicate assertions. +Keep a fast deterministic unit/widget lane, native/browser reader transport and +layout lanes, and an explicitly configured network/provider/device lane. Run +server database tests serially against a distinct test database: shared fixtures +drop/recreate tables. This audit did not run destructive database tests. + +## Bounded structure improvements + +Keep the independent repositories and submodule coordination initially. A +monorepo conversion would change release/CI ownership and is not justified by +this audit. Improve reproducible pins and check orchestration first. + +1. **Reader:** `lib/src/presentation/papyrus_reader.dart` is 1,872 lines. Extract + private settings/contents panels, toolbar/progress controls and command/focus + coordination within presentation. Keep `lib/papyrus_reader.dart` exports, + controller/engine interfaces and stable viewport keys. Preserve native/web + worker boundaries and theme ownership. The reader public-export/worker graph + has no orphan runtime files; exported `EpubPaginator` and custom renderers are + API, not automatic deletion candidates. Re-run chapter-boundary arrow keys, + control visibility/resume, opposite host/reader themes, PDF layouts and phone + resizing after each extraction. +2. **Client:** start with the library page (1,066 lines), advanced filter sheet + (1,059), and acquisition downloads provider (1,058). Extract responsibilities + into their existing feature directories before changing imports broadly. + Then pilot `lib/features//{presentation,application}` for one feature + if it makes ownership clearer; keep shared persistence, PowerSync, media, + auth/platform contracts stable. Align tests with final feature boundaries. + Avoid moving all `pages/widgets/providers` in one mechanical PR. +3. **Demo data:** `data/sample_data.dart` is 949 lines and still called as a + fallback by `BookDetailsProvider.loadBook`. It is live code. Plan explicit + demo/test injection and a product decision for missing-book behavior before + moving samples out of production; add a focused missing-book regression. +4. **Server:** `services/acquisition.py` is 1,310 lines; monitor 562, acquisition + routes 413. Follow the existing auth service package pattern: propose + `services/acquisition/` with provider adapters, job orchestration and shared + types, retaining a stable facade. Keep routes thin and transaction/owner + boundaries explicit. Preserve qBittorrent compatibility, retry/cancellation, + sync atomicity and post-commit media deletion. Keep schema/migration changes + in separate behavior PRs, not bundled into moves. + +## Validation and prevention + +The accepted reader snapshot passed 140 reader tests, example tests/analysis/web +build, and Chrome worker/layout/focus/theme/resume checks. The client passed 23 +focused reader tests, formatting/analysis, four bootstrap tests and web release +build against the published Git dependency. Reader platform CI and client CI +also passed for the release heads. These are the release baseline, not proof +that proposed cleanup has been implemented or validated on physical devices. + +During execution, use the pinned workspace SDK and owning component checks. +Reader worker edits require regeneration and drift checks; platform/plugin edits +require the affected native and web targets; shared client composition changes +require the full offline suite. Docs changes require a Sphinx build and internal +links; website changes require its locked build and visible-link review. + +Add prevention only for observed drift: dead-link checks, safe OpenAPI freshness, +reader asset reproducibility, an import-graph report with explicit public/tool +entry points, and Git dependency/workspace-pin consistency. Root tooling CI +currently does not trigger on gitlink-only updates; add a lightweight consistency +check that accepts intentional release commits and states the merge order. +Do not add blanket unused-code rules that mistake platform implementations or +public APIs for dead code. + +Completion means reviewed obsolete heads are retired, current documentation +matches source, selected dead implementations and their redundant tests are +removed without losing contracts, each extraction has clear ownership and green +checks, and follow-up candidates have explicit decisions. Retain deferred items +in this plan with reasons rather than silently expanding the cleanup scope. diff --git a/README.md b/README.md index 6df3bd0..66e0c20 100644 --- a/README.md +++ b/README.md @@ -91,3 +91,6 @@ The same checks and test suites are available as VS Code tasks. See [development tooling](DEVELOPMENT.md) for the project skills, agent roles, Dart MCP integration, setup instructions, and verified baseline. + +See the [repository cleanup plan](CLEANUP_PLAN.md) for the audited branch, +documentation, unused-code and structural maintenance backlog. From e24850a58cd1162ba8b6d58ae7a85f880de34473 Mon Sep 17 00:00:00 2001 From: Karolis Strazdas Date: Sat, 3 Oct 2026 22:58:18 +0300 Subject: [PATCH 3/5] chore: reference reader review corrections --- client | 2 +- reader | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/client b/client index 57d670a..cbfc15b 160000 --- a/client +++ b/client @@ -1 +1 @@ -Subproject commit 57d670a4e734a0312debf9b3439e0efcdbfaf54b +Subproject commit cbfc15b6e0f2315a81be4d1c5a289cacdfc49751 diff --git a/reader b/reader index 16611b6..e9f91aa 160000 --- a/reader +++ b/reader @@ -1 +1 @@ -Subproject commit 16611b646ee93ee12a2c009f0bd6e49061c72eab +Subproject commit e9f91aa1a6fd6a93494e7aeba057f6f65c6e1c18 From 0de9d0b4bfc0d731901197727d70c437a5d65767 Mon Sep 17 00:00:00 2001 From: Karolis Strazdas Date: Sun, 4 Oct 2026 00:27:50 +0300 Subject: [PATCH 4/5] chore: integrate repository cleanup and enforce reader reference consistency --- .github/workflows/tooling.yml | 10 ++ .vscode/tasks.json | 51 +++++++ AGENTS.md | 6 +- CLEANUP_PLAN.md | 260 --------------------------------- DEVELOPMENT.md | 71 +++------ README.md | 9 +- client | 2 +- docs | 2 +- reader | 2 +- server | 2 +- tools/check_references.py | 35 +++++ tools/papyrus | 24 ++- tools/tests/test_papyrus.py | 21 +++ tools/tests/test_references.py | 107 ++++++++++++++ website | 2 +- 15 files changed, 273 insertions(+), 331 deletions(-) delete mode 100644 CLEANUP_PLAN.md create mode 100644 tools/check_references.py create mode 100644 tools/tests/test_references.py diff --git a/.github/workflows/tooling.yml b/.github/workflows/tooling.yml index 0fb1a3a..29f0f89 100644 --- a/.github/workflows/tooling.yml +++ b/.github/workflows/tooling.yml @@ -6,11 +6,17 @@ on: - 'tools/**' - '.vscode/setup.sh' - '.github/workflows/tooling.yml' + - 'client' + - 'reader' + - '.gitmodules' pull_request: paths: - 'tools/**' - '.vscode/setup.sh' - '.github/workflows/tooling.yml' + - 'client' + - 'reader' + - '.gitmodules' workflow_dispatch: permissions: @@ -21,6 +27,8 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + with: + submodules: recursive - uses: actions/setup-python@v5 with: python-version: '3.12' @@ -32,3 +40,5 @@ jobs: run: tools/papyrus check tooling - name: Check setup script run: shellcheck .vscode/setup.sh + - name: Check committed reader references + run: tools/papyrus check references diff --git a/.vscode/tasks.json b/.vscode/tasks.json index 1988185..13464d7 100644 --- a/.vscode/tasks.json +++ b/.vscode/tasks.json @@ -185,6 +185,57 @@ "panel": "dedicated", "reveal": "always" } + }, + { + "label": "Check reader", + "type": "process", + "command": "${workspaceFolder}/tools/papyrus", + "args": [ + "check", + "reader" + ], + "options": { + "cwd": "${workspaceFolder}" + }, + "problemMatcher": [], + "presentation": { + "panel": "dedicated", + "reveal": "always" + } + }, + { + "label": "Test reader", + "type": "process", + "command": "${workspaceFolder}/tools/papyrus", + "args": [ + "test", + "reader" + ], + "options": { + "cwd": "${workspaceFolder}" + }, + "problemMatcher": [], + "presentation": { + "panel": "dedicated", + "reveal": "always" + } + }, + { + "label": "Check all", + "type": "process", + "command": "${workspaceFolder}/tools/papyrus", + "args": [ + "check", + "all" + ], + "options": { + "cwd": "${workspaceFolder}" + }, + "problemMatcher": [], + "presentation": { + "panel": "dedicated", + "reveal": "always" + } } ], "inputs": [ diff --git a/AGENTS.md b/AGENTS.md index 5446d16..1a94993 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,8 +27,8 @@ Flutter is pinned in `.fvmrc` to the client CI version; use `tools/flutter` and `tools/dart` rather than the machine's potentially newer SDK. - `tools/papyrus doctor`: tools, SDK, local configuration, submodules, GitHub auth. -- `tools/papyrus deps client|server`: locked dependency setup. -- `tools/papyrus check client|server|tooling`: non-mutating quality checks. +- `tools/papyrus deps client|reader|server|all`: locked dependency setup. +- `tools/papyrus check client|reader|server|references|tooling|all`: non-mutating quality checks. - `tools/papyrus test client -- test/path_test.dart`: focused Flutter test. - `tools/papyrus test server -- tests/services/test_sync.py`: focused pytest run. - `tools/papyrus run client|server`: development processes. @@ -60,4 +60,4 @@ contract decisions with both implementers and run database tests serially. Follow the existing design tokens and e-ink motion preferences. Library operations must work offline and remain isolated across guest, user, and server profiles. Treat stored reading positions and user media as durable data. See -`DEVELOPMENT.md` for setup, measured baseline and known gaps. +`DEVELOPMENT.md` for setup, checks and integration workflow. diff --git a/CLEANUP_PLAN.md b/CLEANUP_PLAN.md deleted file mode 100644 index 6897f27..0000000 --- a/CLEANUP_PLAN.md +++ /dev/null @@ -1,260 +0,0 @@ -# Repository cleanup plan - -Status: proposed; audited 2026-10-03. This change records the plan and current -reader integration. Branch deletion, source removal, dependency removal, and -reorganization are future work, to be delivered in the separate PRs below. - -## Scope and baseline - -Keep core reading, offline library reliability, and account/server isolation as -the product priorities. Start with maintenance that reduces misleading guidance -and duplicate implementations; preserve user data and public contracts. - -| Repository | Audited checkout | Default branch at audit | -| --- | --- | --- | -| Workspace | `7ca8bdc`, before this plan | `7ca8bdc` | -| Reader | `16611b6` (`feat/reader-core`) | `f8286e1` | -| Client | `57d670a` (`feat/reader-integration`) | `fca3438` | -| Server | `b199394` | `b199394` | -| Docs | `57048e7`, detached workspace pin | `14ab148` | -| Website | `50e7561`, detached workspace pin | `50e7561` | - -The reader release is [reader PR #3](https://github.com/PapyrusReader/reader/pull/3), -followed by [client PR #29](https://github.com/PapyrusReader/client/pull/29), then -the workspace reference update. Its client dependency is immutable Git revision -`16611b646ee93ee12a2c009f0bd6e49061c72eab`. Keep these branches while their PRs -are open. The server reference advances to the already merged tooling revision; -the docs and website checkouts remain unchanged in this release. - -Audit evidence: tracked files, import/export/part reachability (including Dart -conditional imports), symbol/caller searches, workflows, Git ancestry and patch -equivalence, live GitHub branch/PR inventory, and docs Pages settings. Reachability -identifies candidates, not proof of removability. No Python dead-code conclusion -or comprehensive dependency/license audit is claimed. - -## Delivery order - -| Order / proposed PR | Owner | Concrete outcome | Completion gate | -| --- | --- | --- | --- | -| 1. Branch inventory and retirement | Each repository | Record retained SHAs and remove verified obsolete remote heads; prune stale tracking refs afterward | Re-fetch, verify worktrees/open PRs and branch rules; use the decision table below | -| 2. Current documentation and CI ownership | Workspace, client, server, docs, website | Fix misleading setup/features/links, archive completed plans, remove client-owned server workflows | Validate links and each affected workflow; build Sphinx/site; verify no required check or release depends on retired workflows | -| 3. Unused client presentation, assets and dependencies | Client | Remove caller-verified obsolete UI in small batches, with its obsolete tests | Analyzer, focused behavior tests, full offline suite, web build; native checks for plugin changes | -| 4. Test organization and coverage review | Client, reader, server | Map retained tests to current contracts; separate opt-in service tests from fast local checks | No lost persistence/reader contract coverage; deterministic local lane and documented service lane | -| 5. Reader presentation extraction | Reader, then client pin | Split shell controls/settings/navigation without changing engine/public API behavior | Reader suite, example checks, browser regressions, host integration and platform CI | -| 6. Client and server bounded restructuring | Owning component, one feature at a time | Extract library/acquisition responsibilities and isolate production/demo data | Feature and contract tests; persistence behavior unchanged; no bundled schema changes | -| 7. Prevent recurrence | Workspace and component CI | Explicit check/dependency scope, reproducible artifacts, docs freshness and pin consistency | CLI regression checks and a clean-checkout run; documented fast/full commands agree with CI | - -Prioritize 1–3 before moving directories. Each PR should name removed callers, -the replacement behavior, relevant verification, and its rollback commit. Record -before/after CI duration, tracked-file count, and dependency/build size where -affected; do not promise a performance gain from line counts alone. - -## Branch decisions - -These are actual GitHub heads, not merely local `origin/*` references. Refresh -this inventory immediately before executing retirement. - -| Repository / branch / tip | Evidence | Proposed action | -| --- | --- | --- | -| Client `feature/124c0xc0kbv-slack-community` / `6e78c8d` | [PR #27](https://github.com/PapyrusReader/client/pull/27) merged; both commits patch-equivalent to master | Retire after checking no dependent open PR/worktree | -| Reader `chore/refresh-example-lockfile` / `b3fc338` | [PR #2](https://github.com/PapyrusReader/reader/pull/2) merged; tip is an ancestor of master | Retire | -| Server `chore/development-tooling` / `71b21c6` | [PR #6](https://github.com/PapyrusReader/server/pull/6) merged; patch-equivalent to master | Retire | -| Docs `codex/kar-5-sphinx-readme` / `fd4c61c` | [PR #1](https://github.com/PapyrusReader/docs/pull/1) merged; tip is an ancestor of master | Retire | -| Workspace `feature/opds-support` / `f540e85` | [PR #3](https://github.com/PapyrusReader/papyrus/pull/3) merged an earlier head `fb95d18`; four subsequent commits only update client/reader gitlinks | Conditional retirement: confirm every referenced component revision is preserved/covered by current component master; retain the tip in the retirement record | -| Client `copilot/fix-running-task-attempts` / `a5294b2` | [PR #10](https://github.com/PapyrusReader/client/pull/10) closed unmerged; one unique “Initial plan” commit, but its tree is identical to its parent | Retirement candidate after checking abandoned-task/dependent-PR references; distinguish it from merged work | -| Docs `gh-pages` / `069a780` | Unique publication-history commit; Pages currently uses Actions, with source master, and Deploy uploads/deploys artifacts | Hold until deployment references, rollback needs and branch rules are checked; archive publication history before deciding | -| Active reader/client/workspace release branches; all `master` branches | Current PRs/default branches | Keep | -| Website | Only master exists remotely | No branch cleanup needed | - -Some local tracking refs already outlive their remote heads, including workspace -and client `chore/development-tooling` and several older client feature branches. -Use `git fetch --prune` only after recording the live inventory. Review local -branches separately: workspace tooling is merged, while local reader/docs master -branches are behind. Do not reset detached submodules or delete a checked-out -branch. `git cherry` can miss a combined squash; check PR head/merge SHAs and the -actual patch rather than treating every non-ancestor as unmerged work. - -Before any remote deletion, record the full tip SHA and PR URL, check tags, -worktrees, protected/required branches, deployments and dependent PRs. Delete only -the reviewed head, never force-purge history. A preserved tip can restore the head -with `git push origin :refs/heads/`. - -## Documentation and automation repairs - -| Location | Verified problem | Planned repair | -| --- | --- | --- | -| Workspace `DEVELOPMENT.md` | Claims reader lockfile is ignored; records an old reader pin and pending GitHub sign-in | Describe committed reader lockfile/worker reproducibility; replace transient workstation status with reproducible commands. Keep the dated baseline as history and label newer results separately | -| Workspace `tools/papyrus`, `AGENTS.md`, VS Code tasks | `check all` covers tooling/client/server, not reader; reader dependency setup does not enforce its committed lockfile | Define “all” explicitly; add reader checks/locked dependency setup and an opt-in example/browser lane. Update help, tasks, docs and CLI regressions together | -| Client `README.md` | Format/feature lists exceed the EPUB/PDF reader adapter; old coverage/project links | Separate reading support from metadata import and roadmap; verify each storage/annotation/statistics claim against active code | -| Client `.github/workflows/server-ci.yml`, `server-release.yml` | Both target nonexistent `server/` inside the client repo; typecheck nonexistent `src/`; server has its own CI | Confirm no repository rules/tag-release consumers require them, then delete these two workflows. Keep Flutter CI/release workflows | -| Server `README.md` | Missing `docs/flutter-auth-integration.md`, `auth-testing.md`, `powersync-sandbox.md`, `acquisition-downloads.md` | Restore concise current runbooks or replace links with real sources. Use `.env.example`, routes, tests and dev pages; do not invent missing guide contents | -| Server `AGENTS.md` | Infrastructure map names `papyrus/core` for configuration, while config is `papyrus/config.py` | Correct ownership paths while preserving migration/test rules | -| Docs `design/server-architecture.rst`, `_static/openapi.yaml` | Manually maintained API paths/contracts; architecture uses `/api/v1`, while `.env.example` configures `/v1` | Make API prefix configurable in prose; generate public schema from `create_app()` using safe deterministic test settings; exclude private dev surfaces and verify schema freshness in CI | -| Docs `implementation/technologies.rst` | Lists Supabase and Redis; current server dependencies/Compose instead use PostgreSQL, FastAPI and PowerSync | Describe the deployed implementation and clearly distinguish proposed options | -| Client `docs/catalogs-ui-redesign.md`, `docs/superpowers/plans/` | Completed plans/verification still describe unmerged working-tree state; historical agent execution instructions mixed with current guidance | Archive the four dated OPDS/catalog/library plans with status and PR links; retain useful current decisions/runbooks. Keep regenerable local screenshot commands; do not treat ignored build images as missing source | -| Docs workspace pin | `57048e7` predates already merged docs README improvements at `14ab148` | Build/check newer docs in a separate branch, then update the workspace pin in its own PR | -| Client `.gitignore`, local override guidance | Guidance calls `app/pubspec_overrides.yaml` ignored, but `git check-ignore` finds no rule | Explicitly ignore local overrides and document locked Git-pin restoration; never publish a local-path lockfile | -| Website `README.md`, `index.html` | Old `Eoic/Papyrus` docs/releases/project links; README uses `npm install` despite committed lockfile | Check final public destinations, update links and use reproducible `npm ci`; verify public capability claims against released behavior | -| Docs CI | Deploy builds only on master/dispatch; no PR documentation build | Add PR Sphinx validation, warnings policy and link/schema checks; preserve requirement IDs and publication behavior | - -Source ownership: workspace owns setup/coordination; each component README owns -its current commands and capabilities; reader `docs/integration.md` owns the -host/package contract; server schema/routes own HTTP contracts; docs owns the -published narrative; completed plans belong in dated archives with a current -index. Avoid maintaining copied setup commands and API tables in multiple places. - -## Client source and file candidates - -The tracked Dart graph rooted at `app/lib/main.dart` found 21 unreachable source -files. Symbol checks confirmed apparent matches such as private `_RegisterForm` -in `register_page.dart` are a different implementation. Generic `Book`, `Search` -and `SearchField` names also require import-aware review, not plain name counts. -Re-run the graph after preceding PRs; include tool/test/platform entry points. - -### First removal batch: obsolete presentation - -The following 14 files have no Dart importers in the audited source/tests: - -```text -app/lib/forms/login_form.dart -app/lib/forms/register_form.dart -app/lib/pages/books_page.dart -app/lib/widgets/book_details/eink_book_details_tab_bar.dart -app/lib/widgets/filter/active_filter_bar.dart -app/lib/widgets/goals/add_goal_card.dart -app/lib/widgets/heading.dart -app/lib/widgets/input/text_input.dart -app/lib/widgets/profile/profile_stats_card.dart -app/lib/widgets/profile_button.dart -app/lib/widgets/search_settings.dart -app/lib/widgets/shared/eink_page_header.dart -app/lib/widgets/shared/quick_filter_chips.dart -app/lib/widgets/shared/view_mode_toggle.dart -``` - -`login_form.dart` is entirely commented-out code. The router uses the current -library/auth pages. `app/lib/widgets/book/book.dart` and -`app/lib/widgets/search.dart` form a further two-file chain imported only by the -obsolete `books_page.dart`; remove together if the final caller check confirms -that chain. Keep `app/lib/models/book.dart`, the active library widgets, and -current auth forms inside their pages. - -### Second removal batch: test-only implementations - -| Candidate | Existing test consumers | Required review | -| --- | --- | --- | -| `app/lib/models/active_filter.dart` | `test/models/active_filter_test.dart`, plus the obsolete active filter bar | Compare against current `LibraryFilters` and retained serialized data | -| `app/lib/models/search_filter.dart`, `app/lib/utils/search_query_parser.dart` | `test/models/search_filter_test.dart`, `test/utils/search_query_parser_test.dart` | Check whether advanced-query behavior is a product requirement; preserve useful semantics in active search before retiring implementation/tests | -| `app/lib/widgets/add_book/digital_book_import_sheet.dart` | `test/widgets/add_book/digital_book_import_sheet_test.dart` | Compare selection, cancellation and file-picker contracts with active `BookImportSheet` | -| `app/lib/widgets/add_book/book_import_results_sheet.dart` | `test/widgets/add_book/book_import_results_sheet_test.dart` | Preserve retry, commit, duplicate handling and temporary-file cleanup coverage in active import flow | - -Delete a test only when its sole implementation is retired and its useful -behavior is covered by the replacement. Do not keep dead UI solely to keep its -tests green, or remove behavior tests solely to reduce suite size. - -### Assets, dependencies and generated files - -- `app/assets/images/auth-illustration-25.png` and `auth-illustration-3.png` have - no tracked text references and are absent from the asset manifest: first asset - removal candidates. Check native references and documentation previews first. -- `book_placeholder_2.jpg` and `profile.png` are declared but have no other - filename references. Investigate dynamic paths before removing declarations - and files. `book_placeholder.jpg` is used by a layout test; review separately. -- `collection`, `cupertino_icons` and `google_fonts` have no direct package URI - imports in client `app/lib`; inspect tests, generated code, fonts, tooling and - platform registration before proposing dependency removal. Resolve lockfiles - and measure build impact in that PR. -- Keep `epub_pro`, `syncfusion_flutter_pdf`, `dart_mobi` and `unrar_file` for now: - `file_metadata_service.dart` actively imports them. Reader support for EPUB/PDF - does not mean broader metadata-import dependencies are unused. -- Keep committed lockfiles, reader `assets/epub_worker.js`, its generator and - third-party notices, theme-generated source, native runner files and Podfiles. - Generated-but-packaged files need a reproducible generator, not blanket deletion. -- Build/cache directories, `.local/` screenshots, local media/databases and - credentials are separate from tracked source. Inventory disk usage separately; - do not use `git clean -xfd`, the workspace Purge task, or database resets as - housekeeping shortcuts. No local user-data purge is part of this plan. - -## Test retention and execution lanes - -Retain regressions for profile isolation, offline upload queues/tombstones, -media retry/cleanup, schema migrations, auth ownership, EPUB chapter order and -reflow offsets, PDF modes/refitting, theme popup contrast, focus-mode keyboard -navigation, session identity and version-1 locator restoration. Legacy CFI and -old metadata envelopes are compatibility data; old sync-route rejection tests -also protect the current API. “Legacy” is not a removal criterion. - -Current optional tests are intentional: client OPDS network smoke tests require -`OPDS_SMOKE_URL`; live sync requires `PAPYRUS_LIVE_SYNC`/integration configuration; -server auth smoke tests require provider/SMTP configuration; local env-file -tests can skip when `.env` is absent. Document conditions and run these in a -separate configured lane rather than deleting them as stale tests. - -For each feature extraction, list covered contracts and duplicate assertions. -Keep a fast deterministic unit/widget lane, native/browser reader transport and -layout lanes, and an explicitly configured network/provider/device lane. Run -server database tests serially against a distinct test database: shared fixtures -drop/recreate tables. This audit did not run destructive database tests. - -## Bounded structure improvements - -Keep the independent repositories and submodule coordination initially. A -monorepo conversion would change release/CI ownership and is not justified by -this audit. Improve reproducible pins and check orchestration first. - -1. **Reader:** `lib/src/presentation/papyrus_reader.dart` is 1,872 lines. Extract - private settings/contents panels, toolbar/progress controls and command/focus - coordination within presentation. Keep `lib/papyrus_reader.dart` exports, - controller/engine interfaces and stable viewport keys. Preserve native/web - worker boundaries and theme ownership. The reader public-export/worker graph - has no orphan runtime files; exported `EpubPaginator` and custom renderers are - API, not automatic deletion candidates. Re-run chapter-boundary arrow keys, - control visibility/resume, opposite host/reader themes, PDF layouts and phone - resizing after each extraction. -2. **Client:** start with the library page (1,066 lines), advanced filter sheet - (1,059), and acquisition downloads provider (1,058). Extract responsibilities - into their existing feature directories before changing imports broadly. - Then pilot `lib/features//{presentation,application}` for one feature - if it makes ownership clearer; keep shared persistence, PowerSync, media, - auth/platform contracts stable. Align tests with final feature boundaries. - Avoid moving all `pages/widgets/providers` in one mechanical PR. -3. **Demo data:** `data/sample_data.dart` is 949 lines and still called as a - fallback by `BookDetailsProvider.loadBook`. It is live code. Plan explicit - demo/test injection and a product decision for missing-book behavior before - moving samples out of production; add a focused missing-book regression. -4. **Server:** `services/acquisition.py` is 1,310 lines; monitor 562, acquisition - routes 413. Follow the existing auth service package pattern: propose - `services/acquisition/` with provider adapters, job orchestration and shared - types, retaining a stable facade. Keep routes thin and transaction/owner - boundaries explicit. Preserve qBittorrent compatibility, retry/cancellation, - sync atomicity and post-commit media deletion. Keep schema/migration changes - in separate behavior PRs, not bundled into moves. - -## Validation and prevention - -The accepted reader snapshot passed 140 reader tests, example tests/analysis/web -build, and Chrome worker/layout/focus/theme/resume checks. The client passed 23 -focused reader tests, formatting/analysis, four bootstrap tests and web release -build against the published Git dependency. Reader platform CI and client CI -also passed for the release heads. These are the release baseline, not proof -that proposed cleanup has been implemented or validated on physical devices. - -During execution, use the pinned workspace SDK and owning component checks. -Reader worker edits require regeneration and drift checks; platform/plugin edits -require the affected native and web targets; shared client composition changes -require the full offline suite. Docs changes require a Sphinx build and internal -links; website changes require its locked build and visible-link review. - -Add prevention only for observed drift: dead-link checks, safe OpenAPI freshness, -reader asset reproducibility, an import-graph report with explicit public/tool -entry points, and Git dependency/workspace-pin consistency. Root tooling CI -currently does not trigger on gitlink-only updates; add a lightweight consistency -check that accepts intentional release commits and states the merge order. -Do not add blanket unused-code rules that mistake platform implementations or -public APIs for dead code. - -Completion means reviewed obsolete heads are retired, current documentation -matches source, selected dead implementations and their redundant tests are -removed without losing contracts, each extraction has clear ownership and green -checks, and follow-up candidates have explicit decisions. Retain deferred items -in this plan with reasons rather than silently expanding the cleanup scope. diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 17cb304..e544644 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -3,8 +3,8 @@ Papyrus is a workspace of independent Git submodules. The client is Flutter with Provider, SQLite/PowerSync, platform adapters and a Git-pinned EPUB/PDF reader. The server is FastAPI with async SQLAlchemy, Pydantic, Alembic, PostgreSQL and a -self-hosted PowerSync service. Inspect the source for implemented capabilities; -some README feature lists and setup links are ahead of the current checkout. +self-hosted PowerSync service. Product requirements describe planned capabilities; component READMEs and source +describe current behavior. ## Setup @@ -49,11 +49,14 @@ directory. All subprocesses use the appropriate component directory. | `tools/papyrus doctor` | Inspect tools, config, SDK, repos, Docker and GitHub authentication | | `tools/papyrus sdk` | Install/link the FVM version from `.fvmrc` | | `tools/papyrus deps client` | Install client dependencies with its committed lock | +| `tools/papyrus deps reader` | Install reader dependencies with its committed lock | | `tools/papyrus deps server` | Install server/dev dependencies with the uv lock | | `tools/papyrus check client` | Non-writing Dart format check, Flutter analysis, four web-bootstrap tests | | `tools/papyrus check server` | Ruff lint, Ruff format check, strict Mypy | -| `tools/papyrus check tooling` | Eight CLI regression tests and ShellCheck | -| `tools/papyrus check all` | All three check groups above; reports independent failures | +| `tools/papyrus check reader` | Formatting, worker asset drift, library and example analysis | +| `tools/papyrus check references` | Match committed workspace reader and client dependency revisions | +| `tools/papyrus check tooling` | CLI regression tests and ShellCheck | +| `tools/papyrus check all` | Tooling, reference consistency, client, reader and server; reports independent failures | | `tools/papyrus test client -- test/auth/token_store_test.dart` | Focused Flutter tests; pass options after `--` | | `tools/papyrus test client -- --coverage` | Full client suite and coverage | | `tools/papyrus test server -- tests/services/test_sync.py` | Focused backend tests | @@ -63,9 +66,18 @@ directory. All subprocesses use the appropriate component directory. Use `tools/flutter` and `tools/dart` when a command is not covered by the CLI. These wrappers fail if the pinned SDK is missing. They do not silently use global Flutter. -Reader `deps`, `check`, and `test` commands are also supported; its library lockfile -is ignored, so reader dependency setup resolves its declared constraints normally. -Website and Sphinx checks remain in their own repos. +Reader `deps`, `check`, and `test` commands also use the pinned SDK and committed +lockfile. Run the reader browser suite after UI, layout, focus, or worker changes +as described in `reader/docs/validation.md`. + +Website checks use `npm ci && npm run build` in `website/`. Documentation checks +use `uv sync --locked --extra dev && make build` in `docs/` (Graphviz required). + +For a joint reader/client change, validate a local reader using an ignored +`client/app/pubspec_overrides.yaml`, then remove the override and pin the published +reader commit in `client/app/pubspec.yaml`. Regenerate the client lock and update +the workspace's reader and client gitlinks together. The reference check compares +committed gitlinks and committed dependency files, independently of local overrides. Server pytest fixtures drop and recreate test tables. The CLI checks for a separate local database named `*_test` or `test_*` and locks overlapping CLI server test runs. @@ -104,8 +116,7 @@ in disjoint files, then use the contract reviewer. Run database tests serially. `.codex/config.toml` starts the official Dart tooling MCP through `tools/dart-mcp`, which uses the pinned SDK. The launcher finds the workspace from the root or a -nested component directory. Protocol initialization and enumeration of **25 tools** -were verified from both the root and `client/app`. These include analysis, tests, +nested component directory. It provides analysis, tests, hot reload, widget inspection and runtime-error inspection. Restart the Codex session to load new project MCP/agent configuration if needed. @@ -119,45 +130,3 @@ The layout follows [Codex skills](https://learn.chatgpt.com/docs/build-skills), and [Dart MCP setup](https://docs.flutter.dev/ai/get-started). FVM's [project configuration](https://fvm.app/documentation/getting-started/configuration) keeps SDK selection separate from the global toolchain. - -## Verified baseline — 2026-10-03 - -| Scope | Result | -| --- | --- | -| Client lockfile | Five transitive versions normalized to the CI SDK; locked installation succeeds | -| Client formatting | 449 Dart files checked, no changes required | -| Client analysis | No issues | -| Client tests | 1,387 passed; 19 skipped | -| Web bootstrap | Four tests passed | -| Web build | Release compilation and Wasm dry run succeeded | -| Reader tests | 93 passed; 21 skipped in the sibling checkout | -| Server lint/types | Ruff lint and Mypy pass (138 source files) | -| Server tests | 340 passed; two provider smoke tests excluded | -| Server formatting | Five pre-existing files need formatting; check correctly fails | -| Tooling | Eight regression tests, ShellCheck and skill frontmatter validation pass | -| Dart MCP | Initialized successfully; 25 tools enumerated from root and nested cwd | -| Local platforms | Flutter doctor finds Android SDK, Xcode, Chrome and macOS target | -| GitHub CLI | Installed; account sign-in remains pending | - -The five server formatting files are `papyrus/models/powersync_demo.py`, -`tests/api/routes/test_auth_sandbox.py`, `test_powersync_sandbox.py`, -`tests/integration/test_auth_smoke.py`, and `tests/services/test_auth.py`. -They were not reformatted as part of tooling setup. Local verification logs are -ignored under `.local/tooling/`. - -Live cross-device PowerSync and external OAuth/SMTP smoke tests were not run. -Native release builds and Windows/Linux builds were not run. The global Flutter -SDK remains newer; Flutter doctor may report that PATH mismatch while project -commands and VS Code use FVM correctly. - -The client CI now enforces its lockfile and checks formatting without writing. -The workspace tooling workflow runs the CLI regressions and ShellCheck on relevant -pushes and pull requests; it does not require the application submodules or services. - -The server README links to auth, acquisition and PowerSync guides absent from -this checkout. Its existing `.env.example`, tests, services, and -`docs/opds-relay.md` are usable sources. The client reader dependency is pinned to -`08a5161b9d00eb73581f74ce087b9ad6c1568ca7`, while the sibling reader checkout is at a -different revision. Editing it alone does not change client behavior. For a joint -reader change, use an ignored `client/app/pubspec_overrides.yaml` with a local path -override during validation, then coordinate a deliberate revision update. diff --git a/README.md b/README.md index 66e0c20..1d458c2 100644 --- a/README.md +++ b/README.md @@ -80,8 +80,8 @@ Use the workspace CLI from the repository root: ```bash tools/papyrus doctor -tools/papyrus check client -tools/papyrus check server +tools/papyrus deps all +tools/papyrus check all tools/papyrus test client -- test/auth/token_store_test.dart tools/papyrus test server -- tests/services/test_sync.py ``` @@ -90,7 +90,4 @@ It uses the Flutter version pinned in `.fvmrc` and the server's uv lockfile. The same checks and test suites are available as VS Code tasks. See [development tooling](DEVELOPMENT.md) for the project skills, agent roles, -Dart MCP integration, setup instructions, and verified baseline. - -See the [repository cleanup plan](CLEANUP_PLAN.md) for the audited branch, -documentation, unused-code and structural maintenance backlog. +Dart MCP integration, and setup instructions. diff --git a/client b/client index cbfc15b..f31e41b 160000 --- a/client +++ b/client @@ -1 +1 @@ -Subproject commit cbfc15b6e0f2315a81be4d1c5a289cacdfc49751 +Subproject commit f31e41b9505ef6376fd206ef2bf77eea10a5f961 diff --git a/docs b/docs index 57048e7..a2d3fee 160000 --- a/docs +++ b/docs @@ -1 +1 @@ -Subproject commit 57048e7d60c2aaa2bc12557bd9c1e22c2a206f19 +Subproject commit a2d3fee4b813a1099373ae577e6195708ddaa808 diff --git a/reader b/reader index e9f91aa..c870bf4 160000 --- a/reader +++ b/reader @@ -1 +1 @@ -Subproject commit e9f91aa1a6fd6a93494e7aeba057f6f65c6e1c18 +Subproject commit c870bf4df98b01530c6b6e6ae5cd59bd5ea58cc3 diff --git a/server b/server index b199394..d01f219 160000 --- a/server +++ b/server @@ -1 +1 @@ -Subproject commit b19939482fcac9fba708e7aa49d2904f5bf882ec +Subproject commit d01f2196c006824dd4be494b1ed43ff3223cd700 diff --git a/tools/check_references.py b/tools/check_references.py new file mode 100644 index 0000000..1363cbf --- /dev/null +++ b/tools/check_references.py @@ -0,0 +1,35 @@ +"""Check the committed workspace reader pin against the committed client lock.""" + +import re +import subprocess +from pathlib import Path + + +def git(root: Path, *args: str) -> str: + return subprocess.check_output(["git", "-C", str(root), *args], text=True).strip() + + +def check(root: Path) -> None: + client = git(root, "ls-tree", "HEAD", "client").split()[2] + reader = git(root, "ls-tree", "HEAD", "reader").split()[2] + manifest = git(root / "client", "show", f"{client}:app/pubspec.yaml") + lock = git(root / "client", "show", f"{client}:app/pubspec.lock") + for source, field in [(manifest, "ref"), (lock, "resolved-ref")]: + block = re.search(r"(?m)^ papyrus_reader:\n((?: .*\n?)+)", source) + pin = ( + re.search(rf'{field}:\s*["\x27]?([a-f0-9]{{40}})\b', block[1]) + if block + else None + ) + if pin is None or pin[1] != reader: + raise ValueError( + f"Committed client {field} must match workspace reader {reader}" + ) + print(f"Committed reader references agree: {reader}") + + +if __name__ == "__main__": + try: + check(Path(__file__).resolve().parent.parent) + except (IndexError, ValueError, subprocess.CalledProcessError) as error: + raise SystemExit(f"Reader reference check failed: {error}") from error diff --git a/tools/papyrus b/tools/papyrus index 608e8e3..5bf20d7 100755 --- a/tools/papyrus +++ b/tools/papyrus @@ -12,6 +12,7 @@ from pathlib import Path ROOT = Path(__file__).resolve().parent.parent CLIENT = ROOT / "client/app" SERVER = ROOT / "server" +READER = ROOT / "reader" def run(args, cwd=ROOT, env=None): @@ -77,11 +78,18 @@ def checks(component): "--set-exit-if-changed", "lib", "test", + "tool", + "example/lib", + "example/test", ], - ROOT / "reader", + READER, ), - ([sdk_command("flutter"), "analyze", "--no-pub"], ROOT / "reader"), + (["bash", "tool/build_epub_worker.sh", "--check"], READER), + ([sdk_command("flutter"), "analyze", "--no-pub"], READER), + ([sdk_command("flutter"), "analyze", "--no-pub"], READER / "example"), ] + if component == "references": + return [([sys.executable, "tools/check_references.py"], ROOT)] return [ ( [sys.executable, "-m", "unittest", "discover", "-s", "tools/tests", "-v"], @@ -92,7 +100,11 @@ def checks(component): def check(component): - components = ["tooling", "client", "server"] if component == "all" else [component] + components = ( + ["tooling", "references", "client", "reader", "server"] + if component == "all" + else [component] + ) failed = [] for item in components: for args, cwd in checks(item): @@ -226,7 +238,7 @@ def main(argv=None): ) quality.add_argument( "component", - choices=["client", "server", "reader", "tooling", "all"], + choices=["client", "server", "reader", "tooling", "references", "all"], default="all", nargs="?", ) @@ -252,7 +264,7 @@ def main(argv=None): return test(options.component, args) if options.command == "deps": selected = ( - ["client", "server"] + ["client", "reader", "server"] if options.component == "all" else [options.component] ) @@ -270,7 +282,7 @@ def main(argv=None): sdk_command("flutter"), "pub", "get", - *(["--enforce-lockfile"] if item == "client" else []), + "--enforce-lockfile", ], directory, ) diff --git a/tools/tests/test_papyrus.py b/tools/tests/test_papyrus.py index ffb779a..a52f897 100644 --- a/tools/tests/test_papyrus.py +++ b/tools/tests/test_papyrus.py @@ -78,6 +78,27 @@ def test_client_format_check_does_not_write_files(self): self.assertIn("--set-exit-if-changed", args) self.assertEqual(cwd, cli.CLIENT) + def test_all_checks_include_reader_and_reference_gate(self): + with patch.object(cli, "checks", return_value=[]) as checks: + self.assertEqual(cli.check("all"), 0) + self.assertEqual( + [call.args[0] for call in checks.call_args_list], + ["tooling", "references", "client", "reader", "server"], + ) + + def test_all_flutter_dependencies_enforce_locks(self): + with ( + patch.object(cli, "sdk_command", return_value="flutter"), + patch.object(cli, "run", return_value=0) as run, + ): + self.assertEqual(cli.main(["deps", "all"]), 0) + self.assertEqual( + [call.args[1] for call in run.call_args_list], + [cli.CLIENT, cli.READER, cli.SERVER], + ) + for call in run.call_args_list[:2]: + self.assertIn("--enforce-lockfile", call.args[0]) + def test_sdk_absent_does_not_fall_back_to_global_flutter(self): with ( tempfile.TemporaryDirectory() as directory, diff --git a/tools/tests/test_references.py b/tools/tests/test_references.py new file mode 100644 index 0000000..187da6a --- /dev/null +++ b/tools/tests/test_references.py @@ -0,0 +1,107 @@ +"""Validate recorded dependency pins independently of dirty working trees.""" + +import importlib.util +import subprocess +import tempfile +import unittest +from pathlib import Path + +spec = importlib.util.spec_from_file_location( + "references", Path(__file__).resolve().parents[1] / "check_references.py" +) +references = importlib.util.module_from_spec(spec) +spec.loader.exec_module(references) + + +class CommittedReferencesTest(unittest.TestCase): + def setUp(self): + self.directory = tempfile.TemporaryDirectory() + self.addCleanup(self.directory.cleanup) + self.root = Path(self.directory.name) + self.command(self.root, "init", "-q") + for name in ("client", "reader"): + self.command(self.root / name, "init", "-q") + (self.root / "reader/example").mkdir() + (self.root / "reader/example/fixture").write_text("reader") + self.commit(self.root / "reader") + self.reader = self.command(self.root / "reader", "rev-parse", "HEAD") + (self.root / "client/app").mkdir() + self.pin(self.reader) + self.commit(self.root / "client") + self.record() + + def command(self, root, *args): + root.mkdir(parents=True, exist_ok=True) + return subprocess.check_output( + ["git", "-C", str(root), *args], text=True, stderr=subprocess.DEVNULL + ).strip() + + def commit(self, root): + self.command(root, "add", ".") + self.command( + root, + "-c", + "user.name=Test", + "-c", + "user.email=test@example.com", + "-c", + "commit.gpgsign=false", + "commit", + "-qm", + "fixture", + ) + + def pin(self, revision): + (self.root / "client/app/pubspec.yaml").write_text( + f"dependencies:\n papyrus_reader:\n git:\n ref: {revision}\n" + ) + (self.root / "client/app/pubspec.lock").write_text( + f'packages:\n papyrus_reader:\n description:\n resolved-ref: "{revision}"\n' + ) + + def record(self): + for name in ("client", "reader"): + revision = self.command(self.root / name, "rev-parse", "HEAD") + self.command( + self.root, + "update-index", + "--add", + "--cacheinfo", + "160000", + revision, + name, + ) + self.command( + self.root, + "-c", + "user.name=Test", + "-c", + "user.email=test@example.com", + "-c", + "commit.gpgsign=false", + "commit", + "-qm", + "workspace", + ) + + def test_dirty_dependency_files_do_not_override_committed_agreement(self): + self.pin("f" * 40) + references.check(self.root) + + def test_new_committed_client_requires_matching_reader_pin(self): + self.pin("f" * 40) + self.commit(self.root / "client") + self.record() + with self.assertRaisesRegex(ValueError, "must match workspace reader"): + references.check(self.root) + + def test_lock_drift_is_rejected_even_when_manifest_agrees(self): + (self.root / "client/app/pubspec.lock").write_text( + 'packages:\n papyrus_reader:\n description:\n resolved-ref: "' + + "f" * 40 + + '"\n' + ) + self.commit(self.root / "client") + self.record() + with self.assertRaisesRegex(ValueError, "resolved-ref must match"): + references.check(self.root) diff --git a/website b/website index 50e7561..d402479 160000 --- a/website +++ b/website @@ -1 +1 @@ -Subproject commit 50e75611c5ea79fb98a6acea77b6f2fd4efa454b +Subproject commit d402479c21f55c0e1b87900de3de6878e584877e From 0a4538e148dd8f363e353b3e29b2752b595be45e Mon Sep 17 00:00:00 2001 From: Karolis Strazdas Date: Sun, 4 Oct 2026 00:29:08 +0300 Subject: [PATCH 5/5] chore: pin documentation deployment dependency fix --- docs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs b/docs index a2d3fee..ca816d3 160000 --- a/docs +++ b/docs @@ -1 +1 @@ -Subproject commit a2d3fee4b813a1099373ae577e6195708ddaa808 +Subproject commit ca816d3bddadf6859047f8583877b8d3df9cf841