Repository navigation
Conversation
Opening Map and Nodes on a region of a few thousand nodes could spend the whole 300-request minute and leave the map half loaded. 600 still sits under the fail2ban flood jail's 900, so the server's 429 answers first.
* fix(db): type id cursors as bigint * chore: bump version to 2.0.8
#234) * perf(routes): keep hop_count indexes out of route search order * perf(stats): count node types without node_iatas when unfiltered * perf(db): drop the unused idx_known_routes_iata_last_seen * fix(db): bound boot migration lock waits and name the blockers * test(db): pin the lock-retry rebuild and tidy comments
…p-free reconfirm (#235) * feat(db): background_state table and the (hop_count, iata) route index * perf(db): one /routes statement per filter shape, hop pair on custom plans * feat(db): keyset route retention and grace slices at exact cutoffs * feat(background): hourly route retention with a bounded grace walk and watermark * fix(background): bound the grace walk at microsecond cutoffs * feat(db): stamp-free reconfirm sweep over the route primary key * feat(background): pace route reconfirm at routes.reconfirm_rows_per_hour, no stamps * test(routes): drive route retention end to end and tidy comments * fix(profiling): sample during the hourly route retention run
* perf(db): drop three route indexes no statement reads * docs(db): reword the retention residual comment
… (rest of #237) (#238) * feat(db): five pgx pools with fixed session settings and Prometheus tracing * feat(guards): per-family observe/enforce guards with client slots, deadlines and a shadow budget * fix(guards): attribute only guard deadlines and classify like the handlers * feat(db): route API statements by sqlc name; class C always unnamed * feat(admin): statement statistics by sqlc name; create pg_stat_statements where allowed * fix(db): skip pg_stat_statements when no schema is creatable * feat(background): index health against post-rebuild baselines, Postgres activity, job and ingest metrics * feat: separate API, ingest and maintenance stores; guards on the public API; metrics listener * fix(db): keep ingest writes waiting, size the ingest pool to the host, tidy * test(db): give the statements probe a query id of its own
* perf(db): drop small-table indexes that block HOT updates * perf(db): touch known channels by id instead of upserting * feat: preload channel ids at boot for ingest and maintenance
…d-only mode (#240) * feat(planguard): read sqlc statement text and load a strict plan corpus * fix(planguard): fail on unextracted statements, keep fragments mode-neutral * feat(planguard): plan assertions, scoped sort exemptions and narrow retired-index verdicts * fix(planguard): an errored run always fails * fix(planguard): keep subplan conds off their parent scan; fail on unanalyzed budgets * feat(planguard): read-only plan runner, reports and the run command * fix(planguard): validate identifiers in emitted scripts; prove read-only runs * feat(planguard): end-state corpus, planted regressions and the generated cached-plan policy * fix(planguard): derive observer cases from data; mark legacy generic plans * fix(planguard): bound the dense-region route resolvers; test policy refusals * fix(planguard): re-measure the dense-region route cases; cover every policy refusal * feat(planguard): synthetic fixture built through the migrations, checked in CI * ci(planguard): build the fixture once; guard fixture data for identifiers * fix(planguard): hash every fixture generator input; harden the fixture guard * feat(planguard): snapshot diff over every column, named result classes and cursor walks * fix(planguard): bound the relaxed diff classes; derive the cursor tie from data * fix(planguard): refuse multi-statement text in the corpus runner * feat: read-only server mode and an HTTP page diff between two builds * fix(planguard): refuse escape strings and psql meta-commands in corpus text * fix: storediff follows each build's own cursor; prove read-only gates via the log * fix(planguard): match the PostgreSQL lexer on high-bit bytes; check every emitted line * feat(admin): daily read-only plan check on the maintenance pool, its metric and report * feat(planguard): daily prod check behind planguard.daily_check * fix(planguard): cancel fresh runs at shutdown; expose a never-ran check * ci(planguard): pin the fixture server and collation; refresh baselines * fix(planguard): calibrate route budgets on one dataset; skip entries whose index is absent
… nodes and observers (2.1.0) (#241) * feat(db): packet type domains, row-cursor indexes and the inverted-heard index * feat(api): exact page cursors for packets, nodes and observers * feat(db): normalise packet list filters before choosing a statement * feat(db): one packets statement per filter shape with an exact row cursor * fix(db): let the window statements decide since after until * feat(db): exact keyset cursors for nodes and observers; observers page by last_seen * feat(api): pageCursor on packets, nodes and observers; it wins over cursor * test(planguard): packets statements and node/observer keysets leave pending; diff and storediff cases * chore: bump version to 2.1.0 * test(db): turn off JIT in the keyset cursor tests
#243) * feat(db): observer adverts on a partial index once observations carry their payload type * fix(db): keep today's adverts statement unguarded and rate-limit the gate's read warning * feat(background): batched, resumable payload-type backfill for untyped observations * fix(background): cap each payload-type backfill statement at 50,000 rows * fix(db): reject a non-positive backfill row cap * perf(db): probe rollups per hour and IATA for region-sized stats lists * fix(db): keep the stats probes inside each rollup table's hour range * test(db): push the far-future until to year 9999 so a missing path-stats end bound times out * perf(db): row-comparison trace cursor and a since-only telemetry statement * test(db): pin the empty trace cursor tag and run the trace pagination tests in CI * test(planguard): stats, adverts, traces and telemetry statements leave pending; diff cases * test(planguard): window-bounded advert roles, short quiet page, per-observer telemetry cursors
… observation is untyped (#244)
…ages while trusted (#246) * feat(db): channel_message_iatas table and indexes * feat(ingest): record channel message sites at insert and on every later hearing * feat(db): channel message site backfill and two-range reconcile * feat(db): serve region and scope message pages from the site read model when trusted * feat: boot repair, periodic reconcile and first-time verification for the message site read model * test(planguard): channel message site statements, diff and storediff cases * feat(background): log when a read-only server trusts the message site read model * test(planguard): storediff vars in query values, and message cases read from the data * fix(ingest): record a stored hearing's channel message site right after the observation insert * fix: bounded message dispatch for filtered all-channel pages, fresher read-only trust, earlier invalidation * fix: record a hearing's site after its path is marked and skip the backstop when the message was found; read-only trust ends at the writer's deadline
…ted (#247) * feat(db): packet_sites table and indexes * feat(ingest): keep each packet's site row in the observation insert * feat(db): packet_sites backfill, reconcile and audit * feat(db): region packet pages from packet_sites when trusted, with an exact page cursor * feat(db): packet_sites front sweeper with plan and liveness metrics * feat: boot repair, reconcile, audit, sweep and first-time verification for packet_sites * fix(db): read no site range when no packet was first heard in the window * fix(db): read the window guard's maxima through indexes * test(planguard): packet_sites statements active, data-derived cases, diff and storediff * docs(api): pageCursor applies to region packet pages * fix(db): drop packet_sites trust again after the unsited fallback write * fix(db): the packet_sites audit refills a pair at its newest hearing * fix(db): bounded region reads fill the page within the request, else today's statement answers * fix(planguard): tie_cut needs both pages at the limit and members of other identities * fix: packet_sites sweep without JIT, one verification VACUUM, read-only checks to the newest id * docs(api): cursor and pageCursor on region pages served without nextPageCursor * test: drop local baselines of entries the copy skips; plain race schedule names * test(planguard): storediff walks of the bounded region shapes; local baselines after the overlay rebuild * fix(db): drop message-site trust again after the unseeded insert * fix(db): a region page cursor's fallback to today's statement continues from its exact position
… headroom (#248) * feat(db): drop six lookup-table indexes that no plan reads * feat(db): drop known_routes.last_reconfirmed_at, which nothing reads * feat(metrics): report how much of each database sequence is used * refactor(db): retire the legacy API pool; slow region and message statements get their own two connections * feat(config): database.slow_pool_size sizes the slow pool * refactor(db): route the two former read-model fallbacks to the slow pool * fix(metrics): sequence headroom reads any schema name and marks cycling sequences * feat(metrics): count the 503s pages answer, by reason and model; warn about a busy slow pool once a minute * refactor(db): expose the adverts page mapping and the original channel message pages for oracles * refactor(db): name the read-model fallbacks the slow pool runs; count only what this tree refuses * feat(config): the slow pool defaults to 8 connections while it also serves untrusted read-model pages
… model is ready, instead of falling back (#249)
…rts index (#250) * perf(db): drop idx_observations_observer_id; advert pages read the partial adverts index * test(planguard): retired-index rule, end-state overlay for the local tier, budgets kept on degraded entries
…disk (#257) * Pace the one-off backfills by duty cycle so they cannot starve a small server's disk * Plan the read-model fills with custom plans and analyze new read-model tables * Analyze only never-analyzed read-model tables, bound the fill pause, and reset fill progress
* Read each observer's receptions on their own in the observer comparison * Tighten the observer comparison plan checks
…ores (#264) Read-only servers now restore the saved MeshMapper scope catalogues (a read-only step), so region scope lists are no longer empty. A statement that hits the pool's statement_timeout answers 503 with Retry-After on route evidence and the read-model pages instead of 500. Read-model trust logs when it comes back after a drop, and when repair runs drop it.
This was referenced Oct 21, 2026
Draft
Add optional attributed repeater telemetry and availability filtering
MeshCore-Beacon/beacon-web#184
Draft
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The backend fixes and the Topology/TRACE integration were spread across several draft branches. This candidate combines current dev with #253, #255, #259 and #256, preserving the latest observer-comparison, read-model restore, retryable timeout and Web 2.1 requirements. Collected telemetry and mobile intake are deliberately a separate dependent draft stack.
Backend performance and reliability
Validation and operating boundaries
Local Go build/vet/unit suite, targeted PostgreSQL 16.15 regressions and all eight added plan cases passed. Native CI, vulnerability checks, runtime image checks and paired preview qualification now pass. The PostgreSQL fixture uses C collation. Result tests now isolate their own schema instead of relying on public tables left by another test.
No schema migration beyond upstream 001-053 is added;
channel_historyuses the existing background-state table. Retention and raw evidence are preserved. Operators must check writable mount ownership for the nonroot image and the documented schema-compatible rollback before deployment. Database-role separation and production soak/restore gates remain explicit operational work; this PR does not silently revoke privileges or touch live.The paired web change adds basic Atlas and performance-enhanced Topology without a telemetry dependency. Operator documentation is in the companion docs draft. Current runtime measurements and remaining capacity gates are recorded below.
Refs #265, #252, #254, #258, #223, #224, #226. This consolidates the prior focused drafts; leave them recoverable until the candidate is accepted. No review request or production deployment is made by opening this draft.
This PR presents the same core head against
dev1. Its larger diff includes current upstream database/client changes missing from that branch. It is a paired migration candidate, not an image-only update of the stopped historical dev1 stack.The Topology follow-up streams adjacent array values, preserves NULL gaps and non-one array bounds, and caches the exact minute snapshot with canonical region keys. PostgreSQL result-parity, nonstandard-bound regressions and all six Topology plan cases pass; no migration is added.
Current qualification and promotion gates - 23 October
The full project roadmap, issue/PR disposition and staged promotion plan are in docs #43, tracked by docs #45. Core server #266/#267 and web #182/#183 are one paired candidate; optional server #268/#269, web #184/#185, docs #44 and Flutter #11 remain a separate rollout.
Current application heads pass native CI. Canadaverse now runs server
054e08de7d7fd581e55544e0326ca92596b9a990and web8dcb7f2f233c8ac4086c1ebe02f81d348630321b(2.5.1-n30nex.1), including preview extras. Its complete native frontend suite passed 1,560 tests in 172 files. Public verification matched all 34 assets and both source archives, with both MQTT brokers connected, six retained telemetry histories, and invalid-signature/unenrolled submission rejection. Preview artifacts are local builds of exact clean native-CI-qualified source; this is not a stable release.On a restored snapshot, the committed global Topology query median improved from 1.3691 s to 0.9836 s (28.2%), with the same 9,975 edges. All six unique-traffic raster cases retained identical pixels. These are query/CPU-raster checks, not browser/GPU FPS or production throughput measurements. The later public 24h sample returned the same 9,346 links in 5.619 s cold and 0.191 s warm; earlier severe shared-host I/O pressure caused timeouts. Cold-load capacity, target-browser/device checks and production-shaped migration/recovery soak remain acceptance gates.
Production/dev/dev1 were inspected read-only. Dev1 services are stopped. This preview update changed no schema, retention, intake/worker, credentials or radio ownership. Prior restored-database qualification is recorded separately; no fresh browser or physical mobile/background qualification is claimed here. Dependency advisories and the separate Vite #186 candidate still need their own resolution and qualification. No org merge or release approval is implied.