[szibis]

The old quality comment reduced API quality to three small smoke scores and labelled synthetic handler timing/allocation counters as CPU and memory. It also showed a missing base snapshot as zero. Those signals hid specific regressions and the remaining contract gaps.

The report now leads with actionable changes from existing CI: named failing/skipped/removed/recovered tests, package coverage drops, declared contract-state changes, real workload A/B slowdowns and Loki answer differences, visual failures and missing or stale evidence. Collapsed sections contain the supporting tables, original run links and explicitly scoped resource/cache measurements.

API quality has two distinct views:

• Base/PR comparisons of package-qualified Go outcomes, statement coverage and checked-in read-endpoint/behavior/translation/gap inventories. Failed runs retain observed outcomes and their exit code; incomplete populations cannot imply removed tests or improvements.
• Candidate execution scopes from existing API, semantic, E2E and Grafana job logs, verified against the tested revision. Parent tests, subtests, skips, flakes, interrupted cases and separate Playwright invocations remain visible.

The small smoke components are candidate-only. Both quality collectors can share the candidate image, so their scores are not a revision runtime comparison. The existing A/B job builds separate base/PR binaries and queries Loki for that comparison.

The completion-triggered publisher updates the same comment using trusted default-branch code and bounded data. It starts no tests, benchmarks, profiling or compose stacks. Source head/base/merge/attempt checks exclude stale evidence and run again before publication. The new publisher activates after merge; exact outcome comparisons populate from the next existing quality collections.

Resource labels retain their measured scope: synthetic elapsed time and allocation traffic, fleet process CPU, sampled RSS, backend container statistics and logical cache occupancy. Candidate fleet evidence does not establish main/PR savings. Physical SSD I/O and unavailable resource/layer deltas remain unmeasured.

Validation: 411 CI-script tests passed, including executable collector/renderer/publication fixtures, 12,000 named cases, partial failed runs, package identity, stale provenance, multiple Playwright runs and malformed artifacts. Pinned actionlint and Zizmor passed; independent review concerns were fixed and rechecked. Runtime proxy code and measurement workloads are unchanged.

Retained proof on the preceding revision: Loki pinned CI reports 15 top-level tests + 64 subtests passing, not 79 independent API endpoints. Its runtime stack versions remain distinct from the declared registry surface.

Read endpoint scope: two push routes and five explicitly unsupported deprecated read aliases are excluded. The /api/prom/rules* routes remain in scope for vmalert. Optional deletion management is separate; POST query transports remain reads. The current read inventory is 24/27 routed, 18/27 with differential tests and 11/27 declared proven. Two already-routed rule-group paths were misclassified by exact-template extraction; their subtree mapping is now recognized narrowly. These scope/extraction corrections do not establish new runtime compatibility or speedups. The native-share denominator was rebaselined for those additional routed endpoints, without changing native execution.

/loki/api/v1/index/shards remains a counted compatibility gap, explicitly optional and planner-only. It ranks after normal and last-priority work and does not block the pinned Grafana/Drilldown or peer-cache paths. Regression tests for exclusion guards, subtree matching and optional priority run in the existing CI-wired test module. Full conformance regeneration/check passes.

The README, API reference, Loki compatibility, Known Issues, roadmap, coverage map and related performance/matrix pages now agree on read scope and current implementation. Four curated API-status blocks and the README summary are regenerated and checked by the existing conformance gate. They distinguish routed endpoints, linked differential evidence and declared proof, list current missing routes, and keep optional deletion outside the read score. Canonical registry totals remain unchanged for existing CI consumers; the new read_api subsummary supplies public read totals. Historical latency/cache claims are marked or replaced with links to current evidence. Open PR support is not presented as shipped.

Additional validation: production Docusaurus build passed after checking comment-marker rendering and links; both new documentation regressions failed before the generator fix and pass now. The new tests run in the existing scope module. No proxy behavior or new measurement workload is introduced.

Release audit after v2.54.0: the released changelog is unchanged byte-for-byte and pending notes remain under Unreleased. The implicit-string-concatenation finding is addressed with explicit construction; report output and the conformance check are unchanged. Current main still has the POST metadata cache collision, so Known Issues documents its GET workaround until #949 lands instead of presenting the pending fix as shipped.

[szibis]

Fixed the CI security findings in signed commit c841f31642c031f780ef363525acf62f6941bae7.

• Gitleaks matched ordinary browser-proof prose in historical commit 0f60186536e2e11147670f9ed9e7999a48e573c3, reachable through the S2 branch during full-history scanning. The exception covers only that exact commit/file/rule/line fingerprint; no path or rule is excluded.
• Zizmor's generic workflow_run warning was reviewed against the publisher: trusted default-branch checkout, no persisted checkout credentials, bounded in-memory artifact data, no extraction/execution of PR artifacts, and PR-comment permissions without repository-write permissions. The trigger has a scoped audit annotation with that rationale; other audits remain enabled.

Local verification: pinned Gitleaks 8.28 scanned 72 reachable commits/48.41 MB with zero findings after reproducing the original single finding. A control scan of the same prose as a new file still reports it, proving the historical exception is narrow. Pinned Zizmor 1.30.1 and actionlint 1.7.7 pass; 61 relevant regression/workflow tests pass. Security rationale is recorded in the changelog.

The new Security / static run now passes on this commit, including Gitleaks, Gosec, Trivy, actionlint, Zizmor, hadolint and the Scorecard guardrails. The current PR merge ref has no open code-scanning alerts. Other CI jobs are still running; this does not claim the entire PR is green.

[github-actions]

⚪ Performance A/B: no change beyond noise

8 shape×range — 8 ⚪ same · results vs Loki: 7 same, 0 new difference(s), 1 pre-existing

Loki profiles compared (like for like):

• aligned: not run (no build supports -align-queries-with-step, so every aligned row would be not like-for-like)
• loki-defaults: 8 shape×range; proxy flag base off, pr off

Every shape is within noise of the base build (±30% and 100 ms).

8 unchanged shape×range (within noise)

	shape	range	Loki profile	base p50	PR p50	change	Loki p50	PR ÷ Loki	result vs Loki
⚪	drilldown service landing	1h	defaults	10 ms	17 ms	⚪ +70%	227 ms	0.07×	✅ same
⚪	instant ungrouped rate	instant	defaults	13 ms	13 ms	⚪ +0%	169 ms	0.08×	✅ same
⚪	json rate by level (no filter, no drop)	1h	defaults	400	400	⚪ -33%	400	—	✅ same
⚪	json volume no filters	1h	defaults	85 ms	88 ms	⚪ +4%	348 ms	0.25×	✅ same
⚪	logs plain	1h	defaults	107 ms	86 ms	⚪ -20%	459 ms	0.19×	✅ same
⚪	logs | json	1h	defaults	71 ms	87 ms	⚪ +23%	1.15 s	0.08×	✅ same
⚪	stream-label count by app	1h	defaults	6 ms	5 ms	⚪ -17%	237 ms	0.02×	✅ same
⚪	topk rate by app	1h	defaults	20 ms	15 ms	⚪ -25%	364 ms	0.04×	➖ differs (as base)

Cold (first-run) timings — what a user opening a panel sees

shape	range	Loki profile	base cold	PR cold	Loki cold	PR ÷ Loki cold
drilldown service landing	1h	defaults	36 ms	36 ms	876 ms	0.04×
instant ungrouped rate	instant	defaults	18 ms	12 ms	172 ms	0.07×
json rate by level (no filter, no drop)	1h	defaults	400	400	400	—
json volume no filters	1h	defaults	152 ms	148 ms	1.15 s	0.13×
logs plain	1h	defaults	251 ms	59 ms	643 ms	0.09×
logs | json	1h	defaults	125 ms	162 ms	1.61 s	0.10×
stream-label count by app	1h	defaults	19 ms	28 ms	904 ms	0.03×
topk rate by app	1h	defaults	19 ms	10 ms	1.08 s	0.01×

Why these shapes

• control smoke → 8: control/drilldown service landing, control/instant ungrouped rate, control/json rate by level (no filter, no drop), control/json volume no filters, control/logs plain, control/logs | json, control/stream-label count by app, control/topk rate by app

_{base 28a5a2f699 → PR 24da6e5cf1 · 4 runs (first is cold), warm p50 · ranges 1h · noise ±30% and 100 ms · VictoriaLogs restarts 0→0 · 1.7 min · phases stack_up 40s, seed_and_build 22s, loki-defaults:loki_profile 0s, loki-defaults:proxies 2s, loki-defaults:matrix 23s. Same runner, one fresh stack, targets interleaved per request. How to read and reproduce: bench/ab/README.md.}

[github-actions]

✅ Visual smoke: passed

Base 28a5a2f699 vs PR 3d99029487, Grafana through the proxy builds and Loki on one fresh stack; run.
Core set: 9 capture(s) at 1h. Detailed set: 0 entries (nothing visual touched). 9 capture(s), 15m/1h/6h, 24h and 7d stay out of PR runs.

page	range	set	base = PR (data)	vs Loki (PR)	pixel diff	result
dd-fields	1h	core	✅ identical	identical (4 explained, 1 history-dependent)	0.11%	✅
dd-landing	1h	core	✅ identical	identical (3 explained)	0.14%	✅
dd-service-logs	1h	core	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	✅
explore-logs-plain	1h	core	✅ identical	1 difference(s), 1 on base too	0.07%	✅
explore-metric-plain	1h	core	✅ identical	1 difference(s), 1 on base too	0.04%	✅
explore-sum-by-over-binary	1h	core	✅ identical	1 difference(s), 1 on base too	0.04%	✅
explore-topk-by-per-step	1h	core	✅ identical	1 difference(s), 1 on base too	0.04%	✅
explore-unwrap-sum	1h	core	✅ identical	1 difference(s), 1 on base too	0.04%	✅
lvl-explore-rate-errors-by-service	1h	core	✅ identical	2 difference(s), 2 on base too	0.04%	✅

Gate: a base-vs-PR data difference passes when the PR is closer to Loki, and fails when the PR diverges from Loki or Loki cannot decide it (at a range Loki does not hold, a difference passes when the same page proved that query to answer like Loki at a shorter range; otherwise label the pull request visual-change-expected to accept an intended change). Also failing: a panel empty on the PR but not on the base, an error banner, panel error, error answer or new "No data" panel on the PR, a PR side that never settled, a value difference that did not reproduce on the recapture. A delayed first-pass request and pixel differences only warn.

Core set montages (9): base | PR | Loki

dd-fields-1h

dd-landing-1h

dd-service-logs-1h

explore-logs-plain-1h

explore-metric-plain-1h

explore-sum-by-over-binary-1h

explore-topk-by-per-step-1h

explore-unwrap-sum-1h

lvl-explore-rate-errors-by-service-1h

Published inline montages: pr-visuals/pr-948. Full montage set: artifact.

[github-actions]

PR Quality Report

Compared with the PR base commit; measurements below are scoped to their linked run.

Focus signals:

• Declared API inventory: 24/27 read endpoints implemented, 18 linked to Loki comparison tests, 11 declared proven; 67 open compatibility gaps. These are registry counts, not a runtime pass percentage.

Main-versus-PR whole-process CPU, peak RSS, physical SSD throughput/IOPS and per-layer evictions are not measured here. Synthetic allocations and fleet cache occupancy do not substitute for them.

Coverage and tests

Signal	Base	PR	Delta
Test count	21349	21349	0
Coverage	89.6%	89.6%	0.0% (stable)

Test count includes passing Go test/subtest events, not independent scenarios.

API contract coverage and regression changes

Exact Go case outcomes from the existing quality test run. Top-level tests and subtests are separate populations; package + full test name defines identity.

Observed case outcomes	Base	PR
top-level pass	3101	3101
top-level fail	0	0
top-level skip	2	2
subtests pass	18248	18248
subtests fail	0	0
subtests skip	63	63

Showing 6/6 rows.

Per-package covered/total instrumented statements; missing packages or snapshots have no coverage delta. Coverage changes also reflect changes in the statement denominator.

Package	Base covered/total	PR covered/total	Change
github.com/ReliablyObserve/Loki-VL-proxy/cmd/configdoc	0/26 (0.00%)	0/26 (0.00%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/cmd/healthcheck	0/9 (0.00%)	0/9 (0.00%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/cmd/proxy	891/993 (89.73%)	891/993 (89.73%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/cmd/rules-migrate	37/42 (88.10%)	37/42 (88.10%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/cmd/validate_check	0/19 (0.00%)	0/19 (0.00%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/cache	2278/2557 (89.09%)	2278/2557 (89.09%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/config	206/232 (88.79%)	206/232 (88.79%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/logql	1786/1951 (91.54%)	1786/1951 (91.54%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/logsql	1134/1415 (80.14%)	1134/1415 (80.14%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/memlimit	30/49 (61.22%)	30/49 (61.22%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/metrics	2174/2300 (94.52%)	2174/2300 (94.52%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/middleware	415/454 (91.41%)	415/454 (91.41%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/observability	128/141 (90.78%)	128/141 (90.78%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/rulesmigrate	83/89 (93.26%)	83/89 (93.26%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/translator	3052/3285 (92.91%)	3052/3285 (92.91%)	+0.00 pp
github.com/ReliablyObserve/Loki-VL-proxy/internal/proxy	24677/27594 (89.43%)	24681/27594 (89.44%)	+0.01 pp

Showing 16/16 rows.

Declared conformance inventory, not full runtime compatibility. Registry states describe the checked-in contract and proof links; these counts do not certify that every endpoint, version or workload ran successfully.

Loki surface version: v3.7.7 → v3.7.7.

Endpoint scopes differ: loki-read-api → loki-read-api; deprecated read aliases excluded: 0 → 5; endpoint count deltas and trends are unavailable until both snapshots use the same scope.

Read API scope: write endpoints are excluded and deletion management is reported separately. POST query transports remain reads. Mixed rule endpoints retain their read methods after registry exclusions. Deprecated read aliases are excluded by owner scope decision; this changes the denominator without proving a compatibility gain.

Declared endpoint inventory	Base (read API)	PR (read API)	Change
Read endpoints: declared total	32	27	unavailable
Read endpoints: declared implemented	22	24	unavailable
Read endpoints: linked differential tests	18	18	unavailable
Read endpoints: declared proven	11	11	unavailable

Showing 4/4 rows.

Outside the read endpoint score	Base	PR	Change
Write-only endpoints excluded	2	2	+0
Deprecated read aliases excluded	0	5	+5
Deletion management: declared total	1	1	+0
Deletion management: declared implemented	1	1	+0
Deletion management: linked differential tests	0	0	+0
Deletion management: declared proven	0	0	+0

Showing 6/6 rows.

Declared track states	Base	PR	Change
behaviours: gap	3	3	+0
behaviours: not-applicable	0	0	+0
behaviours: partial	17	17	+0
behaviours: proven	6	6	+0
behaviours: unknown	0	0	+0
behaviours: waived	0	0	+0
translations: gap	0	0	+0
translations: not-applicable	0	0	+0
translations: partial	2	2	+0
translations: proven	0	0	+0
translations: unknown	0	0	+0
translations: waived	0	0	+0

Showing 12/12 rows.

Declared gaps / error layer	Base	PR	Change
Declared gaps: open	67	67	+0
Declared gaps: fixed	57	57	+0
Declared gaps: documented	12	12	+0
Declared gaps: owner-kept	6	6	+0
Declared gaps: upstream	2	2	+0
Known error-layer leak cells	57	57	+0
Known error-layer leak links	74	74	+0

Showing 7/7 rows.

Declared item	Transition	Base state	PR state
endpoint:api_prom_label	removed	gap	not present
endpoint:api_prom_label_name_values	removed	gap	not present
endpoint:api_prom_query	removed	gap	not present
endpoint:api_prom_series	removed	gap	not present
endpoint:api_prom_tail	removed	gap	not present
endpoint:api_prom_rules_namespace_groupname	changed	gap	partial
endpoint:loki_api_v1_rules_namespace_groupname	changed	gap	partial

Showing 7/7 rows.

Selected smoke component checks against the candidate backend only. Both quality snapshot invocations share that backend image, so these checks provide no runtime comparison between revisions. Use the real workload A/B evidence for separately built revisions. This is a small contract sample, not the full declared inventory or version matrix.

Track	Component	PR passing
drilldown	detected_fields	11/11 (100.00%)
drilldown	label_values	1/1 (100.00%)
drilldown	level_volume	2/2 (100.00%)
drilldown	patterns	1/1 (100.00%)
drilldown	service_logs	1/1 (100.00%)
drilldown	service_selection	1/1 (100.00%)
loki	label_values	2/2 (100.00%)
loki	labels	4/4 (100.00%)
loki	metrics	2/2 (100.00%)
loki	otel	1/1 (100.00%)
loki	query_range	1/1 (100.00%)
loki	series	1/1 (100.00%)
vl	detected_fields	4/4 (100.00%)
vl	field_values	3/3 (100.00%)
vl	index_stats	1/1 (100.00%)
vl	stream_translation	1/1 (100.00%)
vl	synthetic_labels	1/1 (100.00%)
vl	volume_range	1/1 (100.00%)

Showing 18/18 rows.

Synthetic handler microbenchmarks

Synthetic handler elapsed time and allocations; not process CPU, RSS or physical disk I/O. Repeated-sample medians.

Signal	Base	PR	Delta
QueryRange cache-hit elapsed	2703 ns/op	2671 ns/op	-1.2% (stable)
QueryRange cache-hit allocated bytes	1256 B/op	1256 B/op	0.0% (stable)
QueryRange cache-hit allocations	28 allocs/op	28 allocs/op	0.0% (stable)
QueryRange cache-bypass elapsed	3139 ns/op	3204 ns/op	+2.1% (stable)
QueryRange cache-bypass allocated bytes	1309 B/op	1309 B/op	0.0% (stable)
QueryRange cache-bypass allocations	28 allocs/op	28 allocs/op	0.0% (stable)
Labels cache-hit elapsed	695.8 ns/op	730.2 ns/op	+4.9% (stable)
Labels cache-hit allocated bytes	48 B/op	48 B/op	0.0% (stable)
Labels cache-hit allocations	3 allocs/op	3 allocs/op	0.0% (stable)
Labels cache-bypass elapsed	1183 ns/op	1205 ns/op	+1.9% (stable)
Labels cache-bypass allocated bytes	397 B/op	397 B/op	0.0% (stable)
Labels cache-bypass allocations	14 allocs/op	14 allocs/op	0.0% (stable)

Labels follow the existing gate percentage, absolute and low-baseline guards, including pooled-path allowances.

Required checks decide merge safety. This report summarizes selected evidence; rendering starts no tests or heavy workloads.

[szibis]

Expanded the report using existing CI output only; read-only scoring corrected in 947f5c8.

The headline now flags specific outcome/coverage/contract regressions; collapsed tables retain named cases, package coverage and broader API/Grafana execution scopes alongside the existing A/B, visual and resource/cache evidence. Small smoke scores no longer claim a base-product runtime comparison, because both collector invocations can share the candidate Docker image.

Corrected current declared inventory at 947f5c8 (runtime proxy code unchanged by this scope/report update):

Declared read inventory	Base	PR
Implemented read endpoints	24/27	24/27
Read endpoints linked to differential tests	18/27	18/27
Declared proven read endpoints	11/27	11/27
Open compatibility gaps	67	67

Write-only routes are excluded. Both push endpoints were already outside the previous 33-endpoint inventory. Five deprecated read aliases are explicitly unsupported and excluded; /api/prom/rules* remains in scope. Deletion management (1 routed, partial endpoint) is separate from the 27 read endpoints; POST query transports remain reads. Two existing rule-group routes are correctly recognized by the extractor. These are scope/classification corrections, not new runtime behavior. index/shards stays a counted optional planner-only gap, ranked after ordinary and last-priority work. These are checked-in registry declarations, not measured runtime compatibility. Separately, the existing Loki pinned job passed 15 parent tests + 64 subtests, with no observed failures/skips. The scopes are intentionally not pooled.

Validation of the new report code: 409 tests pass, pinned actionlint/Zizmor pass, signed commit verified, pinned Gitleaks scans 76 available local commits with no leaks. This includes retained failures, partial-run safeguards, 12,000 named outcomes, stale revisions, separate Playwright invocations and malformed artifact fixtures. Fresh remote CI is pending for 947f5c8.

Exact unit outcome comparisons populate from the newly retained existing quality runs. Cross-workflow automatic refresh activates when the publisher reaches the default branch; it adds no measurement workload.
Independent review found a malformed JSON root could crash the new optional-deletion inventory reader. Reproduced with both files, fixed to return unavailable, and added regression coverage; the reviewer rechecked the fix with no remaining findings.