diff --git a/CHANGELOG.md b/CHANGELOG.md index bbf296fea..8453ce8be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +- Include form-body selectors and time bounds in POST read-cache identities. + Requests with different form parameters no longer share a URL-only memo or + metadata response. Matching GET/POST parameters can reuse metadata answers; + normalization preserves body precedence and the original request form. + Affected metadata cache namespaces are versioned so earlier ambiguous entries + cannot be reused from memory, disk or peers after upgrade. + +- Serve `GET/POST /loki/api/v1/label` directly through the existing labels + handler, matching Loki's singular labels alias. Both paths share tenant/auth + admission, request semantics and labels metadata-cache answers; the alias no longer + redirects to an invalid label-values URL. + ### Security - Restrict a Gitleaks false-positive exception to one historical browser-proof diff --git a/README.md b/README.md index 3b59618b5..0f8fd4240 100644 --- a/README.md +++ b/README.md @@ -94,14 +94,14 @@ Loki profile shown: `loki-defaults`, Loki with its own defaults (align_queries_w | | Now | |---|---| -| Loki endpoints the proxy routes | **24 of 27** (89%) | -| Endpoints with tests that compare the proxy against Loki | **18 of 27** (67%) | -| Endpoints whose registry state is `proven` | **11 of 27** (41%) | -| Routed endpoints answered natively by VictoriaLogs | 5 of 24 | +| Loki endpoints the proxy routes | **25 of 27** (93%) | +| Endpoints with tests that compare the proxy against Loki | **19 of 27** (70%) | +| Endpoints whose registry state is `proven` | **12 of 27** (44%) | +| Routed endpoints answered natively by VictoriaLogs | 6 of 25 | | Behaviour and feature items | 6 proven, 19 partial, 3 gaps | | Known differences from Loki (ranked) | **87**: 67 open, 12 documented, 6 owner-kept, 2 upstream | -Read routes not implemented: `/loki/api/v1/cache/generation_numbers`, `/loki/api/v1/label`, `/loki/api/v1/index/shards` (optional planner gap, low priority). +Read routes not implemented: `/loki/api/v1/cache/generation_numbers`, `/loki/api/v1/index/shards` (optional planner gap, low priority). These are registry evidence counts, not a full runtime compatibility percentage. Optional deletion management is excluded from the read totals; deprecated aliases and write-only APIs are outside this score. diff --git a/bench/read-api/README.md b/bench/read-api/README.md new file mode 100644 index 000000000..bccd219de --- /dev/null +++ b/bench/read-api/README.md @@ -0,0 +1,103 @@ +# Read API alias and POST cache proof + +This proof compares a base proxy, the candidate proxy and Loki against one +dual-written fixture. It checks a real filtered labels response, GET/POST form +equivalence, direct singular-alias responses, isolation from a second empty +selector and proxy upstream select counters. +It does not measure general query performance or fleet behavior. + +`compose.json` is an isolated four-service subset of the existing compatibility +stack. Loki 3.7.9 and VictoriaLogs 1.52.0 have separate named volumes from the +playground. Each proxy has its own disk cache; peers are disabled. Primary L1 is +256 MiB with 10,000 entries, the additional GET compatibility cache is 10%, and +the logical disk cache budget is 512 MiB per proxy. `GOMEMLIMIT=2GiB` is a Go +runtime target, not a cgroup limit or measured RSS. Loki has a 12 GiB container +limit and VictoriaLogs an 8 GiB limit with a 3 GiB internal memory allowance. + +## Reproduce + +Run from the repository root with Docker, Go and Python 3 available. Use the +base/candidate revisions in the retained result. Build the base image from a +clean worktree at the base revision, then build the candidate from its clean +worktree. The commands below assume the current checkout is the candidate. + +```sh +git worktree add --detach /tmp/read-api-base +docker build --build-arg REVISION= -t loki-vl-proxy:read-api-base /tmp/read-api-base +docker build --build-arg REVISION= -t loki-vl-proxy:read-api-local . +docker compose -p read-api-proof -f bench/read-api/compose.json up -d --no-build +``` + +Wait until Loki `/ready` on port 34101 and both proxy `/ready` endpoints on +34100/34102 return 200. On a **new proof stack**, ingest once: + +```sh +LOKI_URL=http://127.0.0.1:34101 PROXY_URL=http://127.0.0.1:34100 VL_URL=http://127.0.0.1:34428 \ + go test -tags=e2e ./test/e2e-compat -run '^TestSetup_IngestLogs$' -count=1 -v +``` + +Do not ingest the fixture repeatedly into retained volumes: the backends have +different duplicate-ingestion behavior. Then run the CI-wired core regression +and the read-only comparison: + +```sh +LOKI_URL=http://127.0.0.1:34101 PROXY_URL=http://127.0.0.1:34100 VL_URL=http://127.0.0.1:34428 \ + go test -tags=e2e ./test/e2e-compat -run '^TestCompat_LabelsSingularAlias$' -count=1 -v +python3 bench/read-api/labels_alias_proof.py \ + --base-sha --pr-sha --out /tmp/read-api-result.json +``` + +The harness uses one fixed one-hour window and `{app="e2e-test"}`. Canonical GET +precedes its alias request; POST follows GET. The stack can already be warm from +the endpoint test, so this is not a controlled cold-cache measurement. Responses must be +nonempty and the candidate must match Loki. The retained counter rows, rather +than the script exit status alone, establish whether alias reuse avoided a +backend request. Absent counters are `null`, not zero. Counter resets abort the +proof. Do not run unrelated traffic against these proof proxies during capture. +The next POST uses a missing-app selector before its GET control. It must not +reuse the preceding app's labels. Loki omits `data` for this empty response; +the proxy returns `data: []`. Both mean no labels; the raw shapes are retained. + +## Historical retained comparison + +[Measured JSON](results/2026-10-31-labels-alias.json) records base `28a5a2f` +and candidate `c320190`, the exact committed runtime sources used for the images. +This capture predates the merge of #947. The branch now includes that S2 change +after a rebase, so these rows are not measurements of the rebased head. The +original measured identities remain unchanged; they must not be relabelled as +the current main and candidate. + +| Signal | Historical main runtime | Historical candidate | Loki reference | +|---|---|---|---| +| Singular alias, GET and POST | 307 redirect | 200, same filtered labels as plural | 200 | +| Filtered canonical POST immediately after GET, VL selects | 4 | 0 | Not collected | +| Missing-app POST after nonempty app POST | Wrong previous labels | Empty, matches its GET control | Empty | +| Missing-app POST, VL selects | 0, wrong cache reuse | 3, required new answer | Not collected | +| Alias requests after canonical priming, VL selects | 0, redirect only | 0, correct labels | Not collected | + +These are one sequence's process-counter deltas. They include the cost of +rejecting unsafe reuse: the candidate reads VL for the genuinely new selector. +CPU, RSS, physical disk traffic, per-tier occupancy/evictions, p50/p95 and fleet +scaling are not measured. Budgets and node count are unchanged; no capacity or +general performance improvement is claimed. + +## Interpretation and limits + +- Primary labels metadata entries are shared by the two routes and equivalent + GET/POST forms. The optional outer GET compatibility cache is path-specific. +- Unit tests cover differing selectors/windows, body precedence, preserved raw + forms, malformed partial forms, tenant/auth isolation and migration away from + ambiguous old namespaces. HTTP migration tests seed old RAM, SSD and outer + compatibility entries. The same versioned keys address peers and stale reads; + this proof does not run a live peer-loss or mixed-version fleet experiment. +- Namespace fencing causes initial misses after an upgrade. Old records remain + subject to existing expiry and eviction; this change does not implement a + deletion-generation protocol or change configured cache capacity. +- Upstream select counters are logical proxy-to-VL requests. They are not SSD + reads, IOPS, CPU, RSS, backend-server resource use or per-tier hit ratios. +- Shifted windows, dashboard refresh, disk restart, pressure, peer loss, p50/p95, + throughput, allocation cost and process resource deltas are not measured here. + Existing admission, completeness and freshness rules continue to apply. + +This is a manually reproducible retained proof. It adds no CI measurement job; +the Go endpoint regression runs in the existing `TestCompat_.*` core shard. diff --git a/bench/read-api/compose.json b/bench/read-api/compose.json new file mode 100644 index 000000000..01ec351bc --- /dev/null +++ b/bench/read-api/compose.json @@ -0,0 +1,243 @@ +{ + "name": "read-api-proof", + "services": { + "loki": { + "command": [ + "-config.file=/etc/loki/local-config.yaml", + "-compactor.retention-enabled=true", + "-compactor.working-directory=/tmp/loki/compactor", + "-compactor.delete-request-store=filesystem", + "-compactor.compaction-interval=10m", + "-compactor.retention-delete-delay=1h", + "-store.retention=72h", + "-validation.reject-old-samples.max-age=72h" + ], + "entrypoint": null, + "environment": { + "GOGC": "80", + "GOMEMLIMIT": "10GiB" + }, + "healthcheck": { + "disable": true + }, + "image": "grafana/loki:3.7.9", + "mem_limit": "12884901888", + "ports": [ + { + "mode": "ingress", + "target": 3100, + "published": "34101", + "protocol": "tcp", + "host_ip": "127.0.0.1" + } + ], + "restart": "unless-stopped", + "user": "0:0", + "volumes": [ + { + "type": "bind", + "source": "../../test/e2e-compat/loki-local-config.yaml", + "target": "/etc/loki/local-config.yaml", + "read_only": true, + "bind": {} + }, + { + "type": "volume", + "source": "e2e-loki-data", + "target": "/tmp/loki", + "volume": {} + } + ] + }, + "victorialogs": { + "command": [ + "-storageDataPath=/vlogs", + "-retentionPeriod=3d", + "-httpListenAddr=:9428", + "-search.maxConcurrentRequests=16", + "-search.maxQueueDuration=60s", + "-search.maxQueryDuration=60s", + "-blockcache.missesBeforeCaching=1", + "-internStringCacheExpireDuration=15m", + "-memory.allowedBytes=3GiB", + "-inmemoryDataFlushInterval=1s" + ], + "entrypoint": null, + "healthcheck": { + "test": [ + "CMD", + "/victoria-logs-prod", + "-version" + ], + "timeout": "2s", + "interval": "3s", + "retries": 10, + "start_period": "5s" + }, + "image": "victoriametrics/victoria-logs:v1.52.0", + "mem_limit": "8589934592", + "ports": [ + { + "mode": "ingress", + "target": 9428, + "published": "34428", + "protocol": "tcp", + "host_ip": "127.0.0.1" + } + ], + "restart": "unless-stopped", + "volumes": [ + { + "type": "volume", + "source": "vl-data", + "target": "/vlogs", + "volume": {} + } + ] + }, + "loki-vl-proxy": { + "command": [ + "-listen=:3100", + "-backend=http://victorialogs:9428", + "-max-stats-query-series=1000000", + "-tenant-default-limits={\"max_entries_limit_per_query\":0,\"max_query_length\":\"721h\",\"query_timeout\":\"5m\",\"retention_period\":\"3d\"}", + "-backend-timeout=5m", + "-log-level=info", + "-cache-ttl=5s", + "-label-values-indexed-cache=true", + "-label-values-hot-limit=200", + "-label-values-index-max-entries=50000", + "-label-values-index-persist-path=/cache/label-values-index.json", + "-label-values-index-persist-interval=5s", + "-label-values-index-startup-stale-threshold=30s", + "-label-values-index-startup-peer-warm-timeout=2s", + "-disk-cache-path=/cache/proxy-l2.bolt", + "-disk-cache-max-bytes=536870912", + "-disk-cache-min-ttl=1s", + "-drilldown-scan-timeout=3s", + "-rate-limit-per-second=0", + "-rate-limit-burst=0", + "-max-concurrent=0", + "-metadata-default-lookback=1h", + "-label-style=underscores", + "-metadata-field-mode=translated", + "-emit-structured-metadata=true", + "-server.register-instrumentation=true" + ], + "depends_on": { + "victorialogs": { + "condition": "service_healthy", + "required": true + } + }, + "entrypoint": null, + "environment": { + "GOMEMLIMIT": "2GiB" + }, + "healthcheck": { + "test": [ + "CMD", + "/usr/local/bin/healthcheck" + ], + "timeout": "2s", + "interval": "3s", + "retries": 10, + "start_period": "5s" + }, + "image": "loki-vl-proxy:read-api-local", + "ports": [ + { + "mode": "ingress", + "target": 3100, + "published": "34100", + "protocol": "tcp", + "host_ip": "127.0.0.1" + } + ], + "user": "root", + "volumes": [ + { + "type": "volume", + "source": "e2e-proxy-cache", + "target": "/cache", + "volume": {} + } + ] + }, + "loki-vl-proxy-base": { + "command": [ + "-listen=:3100", + "-backend=http://victorialogs:9428", + "-max-stats-query-series=1000000", + "-tenant-default-limits={\"max_entries_limit_per_query\":0,\"max_query_length\":\"721h\",\"query_timeout\":\"5m\",\"retention_period\":\"3d\"}", + "-backend-timeout=5m", + "-log-level=info", + "-cache-ttl=5s", + "-label-values-indexed-cache=true", + "-label-values-hot-limit=200", + "-label-values-index-max-entries=50000", + "-label-values-index-persist-path=/cache/label-values-index.json", + "-label-values-index-persist-interval=5s", + "-label-values-index-startup-stale-threshold=30s", + "-label-values-index-startup-peer-warm-timeout=2s", + "-disk-cache-path=/cache/proxy-l2.bolt", + "-disk-cache-max-bytes=536870912", + "-disk-cache-min-ttl=1s", + "-drilldown-scan-timeout=3s", + "-rate-limit-per-second=0", + "-rate-limit-burst=0", + "-max-concurrent=0", + "-metadata-default-lookback=1h", + "-label-style=underscores", + "-metadata-field-mode=translated", + "-emit-structured-metadata=true", + "-server.register-instrumentation=true" + ], + "depends_on": { + "victorialogs": { + "condition": "service_healthy", + "required": true + } + }, + "entrypoint": null, + "environment": { + "GOMEMLIMIT": "2GiB" + }, + "healthcheck": { + "test": [ + "CMD", + "/usr/local/bin/healthcheck" + ], + "timeout": "2s", + "interval": "3s", + "retries": 10, + "start_period": "5s" + }, + "image": "loki-vl-proxy:read-api-base", + "ports": [ + { + "mode": "ingress", + "target": 3100, + "published": "34102", + "protocol": "tcp", + "host_ip": "127.0.0.1" + } + ], + "user": "root", + "volumes": [ + { + "type": "volume", + "source": "e2e-proxy-base-cache", + "target": "/cache", + "volume": {} + } + ] + } + }, + "volumes": { + "vl-data": {}, + "e2e-loki-data": {}, + "e2e-proxy-base-cache": {}, + "e2e-proxy-cache": {} + } +} diff --git a/bench/read-api/labels_alias_proof.py b/bench/read-api/labels_alias_proof.py new file mode 100644 index 000000000..aa61e3e8b --- /dev/null +++ b/bench/read-api/labels_alias_proof.py @@ -0,0 +1,133 @@ +#!/usr/bin/env python3 +"""Read-only alias proof against an already seeded base/PR/Loki stack.""" +import argparse +import datetime +import json +from pathlib import Path +import re +import time +import urllib.error +import urllib.parse +import urllib.request + + +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, *args): + return None + + +def request(base, path, method="GET", params=None): + data = urllib.parse.urlencode(params or {}).encode() + target = base.rstrip("/") + path + headers = {} + if method == "GET": + target += "?" + data.decode() if data else "" + data = None + else: + headers["Content-Type"] = "application/x-www-form-urlencoded" + req = urllib.request.Request(target, data=data, headers=headers, method=method) + try: + response = urllib.request.build_opener(NoRedirect).open(req, timeout=30) + except urllib.error.HTTPError as error: + response = error + with response: + body = response.read(2 * 1024 * 1024 + 1) + if len(body) > 2 * 1024 * 1024: + raise ValueError("proof response exceeds bound") + return response.code, body.decode() + + +def backend_reads(base): + code, metrics = request(base, "/metrics") + if code != 200: + raise ValueError("proxy metrics unavailable") + values = [] + for line in metrics.splitlines(): + if (line.startswith("loki_vl_proxy_requests_total{") and 'system="vl"' in line + and 'direction="upstream"' in line and 'route="/select/logsql/' in line): + values.append(float(line.rsplit(" ", 1)[1])) + # These series are created lazily. Absence cannot supply a cold delta; + # after the canonical request its exported counters can prove alias reuse. + return sum(values) if values else None + + +def delta(before, after): + if before is None or after is None: + return None + if after < before: + raise ValueError("upstream counter reset during proof") + return after - before + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--base", default="http://127.0.0.1:34102") + parser.add_argument("--pr", default="http://127.0.0.1:34100") + parser.add_argument("--loki", default="http://127.0.0.1:34101") + parser.add_argument("--base-sha", required=True) + parser.add_argument("--pr-sha", required=True) + parser.add_argument("--out", type=Path, required=True) + args = parser.parse_args() + for sha in (args.base_sha, args.pr_sha): + if not re.fullmatch(r"[a-f0-9]{40}", sha): + parser.error("proof revisions must be full commit SHAs") + end = time.time_ns() + params = {"start": str(end - 3600 * 10**9), "end": str(end), "query": '{app="e2e-test"}'} + rows = [] + references = {} + for name, base in (("base", args.base), ("pr", args.pr), ("loki", args.loki)): + get_reference = None + for method in ("GET", "POST"): + before = backend_reads(base) if name != "loki" else None + plural_code, plural_body = request(base, "/loki/api/v1/labels", method, params) + after_plural = backend_reads(base) if name != "loki" else None + alias_code, alias_body = request(base, "/loki/api/v1/label", method, params) + after_alias = backend_reads(base) if name != "loki" else None + plural = json.loads(plural_body) + if plural_code != 200 or plural.get("status") != "success" or "app" not in plural.get("data", []): + raise ValueError(f"{name} {method}: canonical fixture labels unavailable") + if method == "GET": + get_reference = plural + references[name] = plural + canonical_matches_get = plural == get_reference + same = json.loads(alias_body) == plural if alias_code == 200 else False + if name in {"pr", "loki"} and (not same or not canonical_matches_get): + raise ValueError(f"{name} {method}: alias differs from canonical labels") + rows.append({"case": "singular_alias", "source": name, "method": method, "plural_status": plural_code, + "alias_status": alias_code, "alias_equals_plural": same, + "canonical_matches_get": canonical_matches_get, + "labels": plural["data"], + "plural_upstream_selects": delta(before, after_plural), + "alias_upstream_selects": delta(after_plural, after_alias)}) + missing = dict(params, query='{app="read-api-proof-missing-fixture"}') + before = backend_reads(base) if name != "loki" else None + post_code, post_body = request(base, "/loki/api/v1/labels", "POST", missing) + after_post = backend_reads(base) if name != "loki" else None + get_code, get_body = request(base, "/loki/api/v1/labels", "GET", missing) + post, get = json.loads(post_body), json.loads(get_body) + if post_code != 200 or get_code != 200 or get.get("status") != "success" or get.get("data", []) != []: + raise ValueError(f"{name}: empty-selector fixture check failed") + same = post == get + if name in {"pr", "loki"} and not same: + raise ValueError(f"{name}: POST reused another selector's labels") + rows.append({"case": "post_selector_isolation", "source": name, + "query": missing["query"], "post_status": post_code, "get_status": get_code, + "post_matches_get": same, "post_labels": post.get("data"), "get_labels": get.get("data"), + "post_upstream_selects": delta(before, after_post)}) + if references["pr"] != references["loki"]: + raise ValueError("candidate canonical labels differ from Loki") + result = {"base": args.base_sha, "candidate": args.pr_sha, + "generated_at": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "scope": "dedicated compose, same seeded Loki/VL data; peers disabled; GET then POST; no redirect following", + "params": params, "rows": rows, + "filtered_nonempty_response_matches_loki": True, + "limits": ["Request counters are proxy upstream selects, not physical storage operations.", + "No CPU, RSS, SSD IOPS, throughput or fleet-scaling comparison measured.", + "The optional outer GET compatibility cache remains path-specific."]} + args.out.parent.mkdir(parents=True, exist_ok=True) + args.out.write_text(json.dumps(result, indent=2) + "\n") + print(json.dumps(rows)) + + +if __name__ == "__main__": + main() diff --git a/bench/read-api/results/2026-10-31-labels-alias.json b/bench/read-api/results/2026-10-31-labels-alias.json new file mode 100644 index 000000000..43fc2e641 --- /dev/null +++ b/bench/read-api/results/2026-10-31-labels-alias.json @@ -0,0 +1,185 @@ +{ + "base": "28a5a2f6994240d85d254b5daa23dbc04ecfd891", + "candidate": "c320190c3b2cda0137b36518491e406bbc8835d3", + "generated_at": "2026-10-31T21:42:27.113601+00:00", + "scope": "dedicated compose, same seeded Loki/VL data; peers disabled; GET then POST; no redirect following", + "params": { + "start": "1793479346772386000", + "end": "1793482946772386000", + "query": "{app=\"e2e-test\"}" + }, + "rows": [ + { + "case": "singular_alias", + "source": "base", + "method": "GET", + "plural_status": 200, + "alias_status": 307, + "alias_equals_plural": false, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": 4.0, + "alias_upstream_selects": 0.0 + }, + { + "case": "singular_alias", + "source": "base", + "method": "POST", + "plural_status": 200, + "alias_status": 307, + "alias_equals_plural": false, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": 4.0, + "alias_upstream_selects": 0.0 + }, + { + "case": "post_selector_isolation", + "source": "base", + "query": "{app=\"read-api-proof-missing-fixture\"}", + "post_status": 200, + "get_status": 200, + "post_matches_get": false, + "post_labels": [ + "app", + "env", + "level", + "service_name" + ], + "get_labels": [], + "post_upstream_selects": 0.0 + }, + { + "case": "singular_alias", + "source": "pr", + "method": "GET", + "plural_status": 200, + "alias_status": 200, + "alias_equals_plural": true, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": 4.0, + "alias_upstream_selects": 0.0 + }, + { + "case": "singular_alias", + "source": "pr", + "method": "POST", + "plural_status": 200, + "alias_status": 200, + "alias_equals_plural": true, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": 0.0, + "alias_upstream_selects": 0.0 + }, + { + "case": "post_selector_isolation", + "source": "pr", + "query": "{app=\"read-api-proof-missing-fixture\"}", + "post_status": 200, + "get_status": 200, + "post_matches_get": true, + "post_labels": [], + "get_labels": [], + "post_upstream_selects": 3.0 + }, + { + "case": "singular_alias", + "source": "loki", + "method": "GET", + "plural_status": 200, + "alias_status": 200, + "alias_equals_plural": true, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": null, + "alias_upstream_selects": null + }, + { + "case": "singular_alias", + "source": "loki", + "method": "POST", + "plural_status": 200, + "alias_status": 200, + "alias_equals_plural": true, + "canonical_matches_get": true, + "labels": [ + "app", + "env", + "level", + "service_name" + ], + "plural_upstream_selects": null, + "alias_upstream_selects": null + }, + { + "case": "post_selector_isolation", + "source": "loki", + "query": "{app=\"read-api-proof-missing-fixture\"}", + "post_status": 200, + "get_status": 200, + "post_matches_get": true, + "post_labels": null, + "get_labels": null, + "post_upstream_selects": null + } + ], + "filtered_nonempty_response_matches_loki": true, + "limits": [ + "Request counters are proxy upstream selects, not physical storage operations.", + "No CPU, RSS, SSD IOPS, throughput or fleet-scaling comparison measured.", + "The optional outer GET compatibility cache remains path-specific." + ], + "provenance": { + "images": { + "candidate": { + "tag": "loki-vl-proxy:read-api-local", + "image_id": "sha256:c332b56001ec1fafa7a09f59500bb4a3ab20d5e85f52bec972d79f9f5269ab5b" + }, + "base": { + "tag": "loki-vl-proxy:read-api-base", + "image_id": "sha256:3b4b58e06e89fa585ebe6fef9ffff5f30c3c47bc5c0b24eb7fc0bd59d0ba821c" + }, + "loki": { + "tag": "grafana/loki:3.7.9", + "image_id": "sha256:0db216b630ae1f348a1584c67f3cead8b4c1fc839169a39501db664744158e28" + }, + "victorialogs": { + "tag": "victoriametrics/victoria-logs:v1.52.0", + "image_id": "sha256:47b820890d64c4575a2a0a46415dcd8a4fd59a0f1fcd6a377693d7aea639442e" + } + }, + "candidate_revision_label": "c320190c3b2cda0137b36518491e406bbc8835d3", + "fixture": "TestSetup_IngestLogs, ingested once; retained dedicated volumes; no UI generator", + "compose": "bench/read-api/compose.json", + "compose_sha256": "841b2df3f8c8f1db6b8ca1a79f52d912d08f1a716106b68490ab546983edc470", + "prior_priming": "Earlier endpoint proof runs; not a controlled cold-cache capture", + "topology": "One base and one candidate proxy, separate cache volumes, same Loki/VL fixture, peers disabled" + } +} diff --git a/conformance/registry/baseline.json b/conformance/registry/baseline.json index 99a6576d9..3654e34e9 100644 --- a/conformance/registry/baseline.json +++ b/conformance/registry/baseline.json @@ -1,10 +1,10 @@ { "endpoints": 28, - "proven": 11, + "proven": 12, "scores": { - "coverage": 0.6429, - "implementation": 0.8929, - "native_share_of_implemented": 0.24 + "coverage": 0.6786, + "implementation": 0.9286, + "native_share_of_implemented": 0.2692 }, "states": { "api_prom_label": "not-applicable", @@ -35,7 +35,7 @@ "loki_api_v1_index_stats": "proven", "loki_api_v1_index_volume": "proven", "loki_api_v1_index_volume_range": "proven", - "loki_api_v1_label": "gap", + "loki_api_v1_label": "proven", "loki_api_v1_label_name_values": "partial", "loki_api_v1_labels": "proven", "loki_api_v1_patterns": "proven", diff --git a/conformance/registry/generated/matrix-link.json b/conformance/registry/generated/matrix-link.json index 1271b1773..cd563f169 100644 --- a/conformance/registry/generated/matrix-link.json +++ b/conformance/registry/generated/matrix-link.json @@ -4,9 +4,9 @@ "live_cases_hold": 0, "live_recorded": "2026-09-22T09:56:52+00:00", "scores": { - "coverage": 0.6667, - "implementation": 0.8889, - "native_share_of_implemented": 0.2083 + "coverage": 0.7037, + "implementation": 0.9259, + "native_share_of_implemented": 0.24 } }, "tracks": [ diff --git a/conformance/registry/generated/proxy/coverage.json b/conformance/registry/generated/proxy/coverage.json index 580fec274..2e64e5247 100644 --- a/conformance/registry/generated/proxy/coverage.json +++ b/conformance/registry/generated/proxy/coverage.json @@ -686,7 +686,9 @@ }, { "differential_tests": { - "e2e": [], + "e2e": [ + "test/e2e-compat/labels_alias_test.go" + ], "ui": [], "unit": [] }, @@ -696,11 +698,20 @@ "POST" ], "path": "/loki/api/v1/label", - "proxy_route": null, + "proxy_route": { + "handler": "handleLabels", + "pattern": "/loki/api/v1/label", + "route": "labels" + }, "tests": { - "e2e": [], + "e2e": [ + "test/e2e-compat/labels_alias_test.go" + ], "ui": [], - "unit": [] + "unit": [ + "internal/proxy/labels_alias_test.go", + "internal/proxy/labels_post_form_cache_test.go" + ] } }, { @@ -807,6 +818,7 @@ "test/e2e-compat/features_test.go", "test/e2e-compat/heavy_query_protection_test.go", "test/e2e-compat/label_freshness_e2e_test.go", + "test/e2e-compat/labels_alias_test.go", "test/e2e-compat/labels_full_range_test.go", "test/e2e-compat/labels_zero_width_test.go", "test/e2e-compat/line_fields_parity_test.go", @@ -843,6 +855,7 @@ "test/e2e-compat/heavy_query_protection_test.go", "test/e2e-compat/label_cache_e2e_test.go", "test/e2e-compat/label_freshness_e2e_test.go", + "test/e2e-compat/labels_alias_test.go", "test/e2e-compat/labels_full_range_test.go", "test/e2e-compat/labels_zero_width_test.go", "test/e2e-compat/line_fields_parity_test.go", @@ -867,6 +880,7 @@ "internal/middleware/compress_test.go", "internal/middleware/middleware_test.go", "internal/proxy/backend_heavy_test.go", + "internal/proxy/cache_keys_post_form_test.go", "internal/proxy/cache_locality_test.go", "internal/proxy/circuit_breaker_internal_cancel_test.go", "internal/proxy/compat_cache_freshness_test.go", @@ -885,9 +899,11 @@ "internal/proxy/label_metadata_freshness_test.go", "internal/proxy/label_negative_cache_test.go", "internal/proxy/label_surface_test.go", + "internal/proxy/labels_alias_test.go", "internal/proxy/labels_fields_scale_bench_test.go", "internal/proxy/labels_inclusive_end_test.go", "internal/proxy/labels_perf_test.go", + "internal/proxy/labels_post_form_cache_test.go", "internal/proxy/lifecycle_leak_test.go", "internal/proxy/loki_profile_test.go", "internal/proxy/memleak_test.go", diff --git a/conformance/registry/generated/proxy/implementation.json b/conformance/registry/generated/proxy/implementation.json index 2e94480f8..b8a8f35c9 100644 --- a/conformance/registry/generated/proxy/implementation.json +++ b/conformance/registry/generated/proxy/implementation.json @@ -229,8 +229,22 @@ ] }, { + "execution": "native_vl", + "functions_reached": 121, + "handler": "handleLabels", "id": "loki_api_v1_label", - "implemented": false + "implemented": true, + "proxy_side_work": {}, + "route": "labels", + "vl_endpoints": [ + "/select/logsql/field_names" + ], + "vl_pipes": [ + "len" + ], + "where": [ + "internal/proxy/label_handlers.go:24 handleLabels" + ] }, { "execution": "native_vl", @@ -334,7 +348,7 @@ "unpack_json" ], "where": [ - "internal/proxy/proxy.go:2720 handleQuery" + "internal/proxy/proxy.go:2721 handleQuery" ] }, { @@ -367,7 +381,7 @@ "unpack_json" ], "where": [ - "internal/proxy/proxy.go:2300 handleQueryRange" + "internal/proxy/proxy.go:2301 handleQueryRange" ] }, { diff --git a/conformance/registry/generated/summary.json b/conformance/registry/generated/summary.json index adc30084a..4e2408bc7 100644 --- a/conformance/registry/generated/summary.json +++ b/conformance/registry/generated/summary.json @@ -2,11 +2,11 @@ "endpoints": 28, "execution": { "hybrid": 4, - "native_vl": 6, - "not_implemented": 3, + "native_vl": 7, + "not_implemented": 2, "proxy_side": 15 }, - "implemented": 25, + "implemented": 26, "loki_version": "v3.7.7", "optional_management": { "endpoints": 1, @@ -32,27 +32,27 @@ }, "with_differential_tests": 0 }, - "proven": 11, + "proven": 12, "read_api": { "endpoints": 27, "execution": { "hybrid": 4, - "native_vl": 5, - "not_implemented": 3, + "native_vl": 6, + "not_implemented": 2, "proxy_side": 15 }, - "implemented": 24, - "proven": 11, + "implemented": 25, + "proven": 12, "scope": "loki-read-api", "scores": { - "coverage": 0.6667, - "implementation": 0.8889, - "native_share_of_implemented": 0.2083 + "coverage": 0.7037, + "implementation": 0.9259, + "native_share_of_implemented": 0.24 }, "state": { - "gap": 3, + "gap": 2, "partial": 13, - "proven": 11, + "proven": 12, "unknown": 0, "waived": 0 }, @@ -61,26 +61,22 @@ "optional": false, "path": "/loki/api/v1/cache/generation_numbers" }, - { - "optional": false, - "path": "/loki/api/v1/label" - }, { "optional": true, "path": "/loki/api/v1/index/shards" } ], - "with_differential_tests": 18 + "with_differential_tests": 19 }, "scores": { - "coverage": 0.6429, - "implementation": 0.8929, - "native_share_of_implemented": 0.24 + "coverage": 0.6786, + "implementation": 0.9286, + "native_share_of_implemented": 0.2692 }, "state": { - "gap": 3, + "gap": 2, "partial": 14, - "proven": 11, + "proven": 12, "unknown": 0, "waived": 0 }, @@ -90,5 +86,5 @@ "stats_functions": 14, "version_gated": 2 }, - "with_differential_tests": 18 + "with_differential_tests": 19 } diff --git a/conformance/registry/generated/wiring.json b/conformance/registry/generated/wiring.json index e36c318de..fd6a21223 100644 --- a/conformance/registry/generated/wiring.json +++ b/conformance/registry/generated/wiring.json @@ -211,6 +211,11 @@ "test/e2e-compat/drilldown_volume_bytes_test.go", "test/e2e-compat/label_matcher_regexp_anchoring_test.go" ], + "loki_api_v1_label": [ + "internal/proxy/labels_alias_test.go", + "internal/proxy/labels_post_form_cache_test.go", + "test/e2e-compat/labels_alias_test.go" + ], "loki_api_v1_label_name_values": [ "internal/proxy/label_surface_test.go", "internal/proxy/metadata_edge_refresh_test.go", @@ -221,9 +226,12 @@ "loki_api_v1_labels": [ "internal/proxy/backend_heavy_test.go", "internal/proxy/label_surface_test.go", + "internal/proxy/labels_alias_test.go", + "internal/proxy/labels_post_form_cache_test.go", "internal/proxy/metadata_edge_refresh_test.go", "internal/proxy/metadata_inventory_test.go", "internal/proxy/metadata_scan_admission_test.go", + "test/e2e-compat/labels_alias_test.go", "test/e2e-compat/line_fields_parity_test.go", "test/e2e-compat/metadata_scan_admission_test.go" ], @@ -1471,6 +1479,10 @@ "operator-configurable-limits", "semantics/label-inventory-exact-and-incremental" ], + "internal/proxy/labels_alias_test.go": [ + "loki_api_v1_label", + "loki_api_v1_labels" + ], "internal/proxy/labels_cache_warm_test.go": [ "quality/label-cache-warm", "tenant-label-isolation", @@ -1479,6 +1491,10 @@ "internal/proxy/labels_inclusive_end_test.go": [ "semantics/labels-end-inclusive-like-loki" ], + "internal/proxy/labels_post_form_cache_test.go": [ + "loki_api_v1_label", + "loki_api_v1_labels" + ], "internal/proxy/lifecycle_leak_test.go": [ "shutdown-releases-background-work" ], @@ -2224,6 +2240,10 @@ "semantics/label-replace-new-label", "semantics/label-replace-range-binary-marker" ], + "test/e2e-compat/labels_alias_test.go": [ + "loki_api_v1_label", + "loki_api_v1_labels" + ], "test/e2e-compat/labels_zero_width_test.go": [ "semantics/labels-end-inclusive-like-loki" ], @@ -2484,7 +2504,7 @@ "profiles/structured-metadata-keys-per-profile" ] }, - "tests_with_markers": 249, + "tests_with_markers": 252, "unknown_ids": {}, "unwired_items": [ "api_prom_label", @@ -2506,7 +2526,6 @@ "loki_api_v1_format_query", "loki_api_v1_index_shards", "loki_api_v1_index_volume_range", - "loki_api_v1_label", "loki_api_v1_patterns", "loki_api_v1_push", "loki_api_v1_status_buildinfo", @@ -2576,5 +2595,5 @@ "versions/capabilities-floor-vs-matrix", "versions/rejected-stats-query-wording-vl-1.47" ], - "wired_items": 198 + "wired_items": 199 } diff --git a/conformance/registry/loki/endpoints/loki_api_v1_label.yaml b/conformance/registry/loki/endpoints/loki_api_v1_label.yaml index 32247e4a2..7e4d8fcaf 100644 --- a/conformance/registry/loki/endpoints/loki_api_v1_label.yaml +++ b/conformance/registry/loki/endpoints/loki_api_v1_label.yaml @@ -5,43 +5,48 @@ path: "/loki/api/v1/label" methods: [GET, POST] generated: loki_version: v3.7.7 - proxy_route: null - proxy_handler: null + proxy_route: "labels" + proxy_handler: "handleLabels" tests: - unit: 44 + unit: 2 unit_vs_loki: 0 - e2e: 17 - e2e_vs_loki: 11 - ui: 1 + e2e: 1 + e2e_vs_loki: 1 + ui: 0 ui_vs_loki: 0 implementation: - implemented: false - where: null - execution: not_implemented - vl_endpoints: [] - vl_pipes: [] + implemented: true + where: "internal/proxy/label_handlers.go:24 handleLabels" + execution: native_vl + vl_endpoints: [/select/logsql/field_names] + vl_pipes: [len] proxy_side_work: [] + description: | - TODO: what this endpoint returns and the Loki behaviour a client depends on. + Singular Loki label-names alias of /loki/api/v1/labels. GET and form-encoded POST + use the same handler, tenant/auth admission and metadata cache identity. + The optional outer GET response cache remains path-specific. context: consumers: + - explore + - drilldown + - datasource - api notes: | - TODO: how each consumer calls it, and how their parameter combinations differ. + Served directly without the prior subtree redirect to /loki/api/v1/label/. + Label-values paths below /loki/api/v1/label/{name}/values keep their own route. examples: - consumer: api request: | - TODO: a real request, with the exact parameters a client sends. + GET /loki/api/v1/label?start=1790871720000000000&end=1790875320000000000 expectation: | - TODO: Loki's response shape and the values that must match. + Same status, JSON shape and labels as /loki/api/v1/labels for the same scope. victorialogs: - # Confirmed by the differential runner at runtime; the generated block above - # is a static approximation. Say what VictoriaLogs answers natively, what the - # proxy has to compute, and why VictoriaLogs cannot do that part. native: [] - transformed: [] + transformed: + - stream_field_names vl_cannot: | - TODO: the VictoriaLogs limitation that forces proxy-side work, if any. -edge_cases: [] # case ids under conformance/registry/cases/ + The proxy applies the existing Loki label translation and synthetic labels. +edge_cases: [] diff --git a/conformance/registry/state/loki/loki_api_v1_label.yaml b/conformance/registry/state/loki/loki_api_v1_label.yaml index 24142e1dd..dbadf6be3 100644 --- a/conformance/registry/state/loki/loki_api_v1_label.yaml +++ b/conformance/registry/state/loki/loki_api_v1_label.yaml @@ -1,6 +1,6 @@ id: loki_api_v1_label -state: gap +state: proven reason: | - Seeded from the repository: no proxy route, 0 test files compare it against Loki. + Uses the same route middleware, labels handler and labels metadata-cache identity as /loki/api/v1/labels. TestLabelsAliasSharesCanonicalHandlerAndMetadataCache covers GET/POST and cold/warm metadata reuse in both directions; TestCompat_LabelsSingularAlias compares the singular and plural routes against real Loki with nonempty ingested data. The optional outer GET response cache remains path-specific. owner_decision: null waiver: null diff --git a/conformance/reports/compatibility-matrix.md b/conformance/reports/compatibility-matrix.md index 62d1ec4d0..13d6d0ab3 100644 --- a/conformance/reports/compatibility-matrix.md +++ b/conformance/reports/compatibility-matrix.md @@ -23,7 +23,7 @@ Generated by `conformance/scripts/matrix_link.py`. Versions come from `test/e2e- ## Registry state behind the matrix -- Read endpoints with a test comparing the proxy against Loki: **67%** -- Read endpoints the proxy routes: **89%** -- Implemented endpoints answered natively by VictoriaLogs: **21%** +- Read endpoints with a test comparing the proxy against Loki: **70%** +- Read endpoints the proxy routes: **93%** +- Implemented endpoints answered natively by VictoriaLogs: **24%** - Live case proof, last run 2026-09-22T09:56:52+00:00: **0 hold**, **0 gaps** diff --git a/conformance/reports/coverage-map.md b/conformance/reports/coverage-map.md index a80337882..df708a529 100644 --- a/conformance/reports/coverage-map.md +++ b/conformance/reports/coverage-map.md @@ -15,10 +15,10 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Score | Value | Meaning | |---|---|---| -| Implementation | 24/27 | endpoints the proxy routes | -| Differential evidence | 18/27 | endpoints with tests comparing the proxy against Loki | -| Proven | 11/27 | endpoints whose registry state is `proven` | -| VictoriaLogs native share | 5/24 | implemented endpoints answered natively | +| Implementation | 25/27 | endpoints the proxy routes | +| Differential evidence | 19/27 | endpoints with tests comparing the proxy against Loki | +| Proven | 12/27 | endpoints whose registry state is `proven` | +| VictoriaLogs native share | 6/25 | implemented endpoints answered natively | Counts exclude optional deletion management as well as out-of-scope writes and deprecated aliases. A route or a linked test is not a full compatibility claim. @@ -38,11 +38,12 @@ A route or a linked test is not a full compatibility claim. | `/loki/api/v1/index/stats` | GET, POST | proven | VictoriaLogs native | explore, datasource, api | 20 | 7 | internal/proxy/label_handlers.go:546 | | `/loki/api/v1/index/volume` | GET, POST | proven | proxy-side | drilldown, api | 29 | 10 | internal/proxy/volume.go:192 | | `/loki/api/v1/index/volume_range` | GET, POST | proven | proxy-side | drilldown, api | 21 | 4 | internal/proxy/volume.go:200 | +| `/loki/api/v1/label` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 3 | 1 | internal/proxy/label_handlers.go:24 | | `/loki/api/v1/label/{name}/values` | GET, POST | partial | VictoriaLogs native | explore, drilldown, datasource, api | 50 | 15 | internal/proxy/label_handlers.go:124 | -| `/loki/api/v1/labels` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 80 | 17 | internal/proxy/label_handlers.go:24 | +| `/loki/api/v1/labels` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 84 | 18 | internal/proxy/label_handlers.go:24 | | `/loki/api/v1/patterns` | GET, POST | proven | hybrid | drilldown, api | 23 | 3 | internal/proxy/patterns.go:108 | -| `/loki/api/v1/query` | GET, POST | proven | hybrid | explore, drilldown, api | 104 | 40 | internal/proxy/proxy.go:2720 | -| `/loki/api/v1/query_range` | GET, POST | proven | hybrid | explore, drilldown, api | 217 | 64 | internal/proxy/proxy.go:2300 | +| `/loki/api/v1/query` | GET, POST | proven | hybrid | explore, drilldown, api | 104 | 40 | internal/proxy/proxy.go:2721 | +| `/loki/api/v1/query_range` | GET, POST | proven | hybrid | explore, drilldown, api | 217 | 64 | internal/proxy/proxy.go:2301 | | `/loki/api/v1/rules` | GET | partial | proxy-side | api | 10 | 2 | internal/proxy/alerting.go:95 | | `/loki/api/v1/rules/{namespace}` | GET | partial | proxy-side | api | 4 | 0 | internal/proxy/alerting.go:95 | | `/loki/api/v1/rules/{namespace}/{groupName}` | GET | partial | proxy-side | api | 4 | 0 | internal/proxy/alerting.go:95 | @@ -53,7 +54,6 @@ A route or a linked test is not a full compatibility claim. | `/prometheus/api/v1/rules` | GET | partial | proxy-side | api | 5 | 1 | internal/proxy/alerting.go:95 | | `/loki/api/v1/cache/generation_numbers` | GET | gap | not implemented | api | 0 | 0 | — | | `/loki/api/v1/index/shards` | GET, POST | gap (optional) | not implemented | api | 0 | 0 | — | -| `/loki/api/v1/label` | GET, POST | gap | not implemented | api | 0 | 0 | — | Optional API gaps remain included in the totals and scores: diff --git a/conformance/reports/gaps.md b/conformance/reports/gaps.md index f676a800a..75389494c 100644 --- a/conformance/reports/gaps.md +++ b/conformance/reports/gaps.md @@ -20,7 +20,6 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | 6 | `/loki/api/v1/detected_labels` | drilldown, api | proxy_side | 21 | 7 | proxy-side: justify in the registry or push down to VictoriaLogs | | 5 | `/loki/api/v1/status/buildinfo` | datasource, api | proxy_side | 8 | 4 | no test wired to the registry; proxy-side: justify in the registry or push down to VictoriaLogs | | 4 | `/loki/api/v1/patterns` | drilldown, api | hybrid | 23 | 3 | no test wired to the registry | -| 4 | `/loki/api/v1/label` | api | not_implemented | 0 | 0 | not implemented; no test compares it against Loki; no test wired to the registry | | 4 | `/loki/api/v1/cache/generation_numbers` | api | not_implemented | 0 | 0 | not implemented; no test compares it against Loki; no test wired to the registry | | 3 | `/loki/api/v1/drilldown-limits` | api | proxy_side | 8 | 2 | proxy-side: justify in the registry or push down to VictoriaLogs | | 6 | `/prometheus/api/v1/alerts` | api | proxy_side | 5 | 0 | no test compares it against Loki; proxy-side: justify in the registry or push down to VictoriaLogs | diff --git a/conformance/reports/roadmap.md b/conformance/reports/roadmap.md index d3e458324..dca1316ed 100644 --- a/conformance/reports/roadmap.md +++ b/conformance/reports/roadmap.md @@ -6,7 +6,7 @@ The proxy serves Loki's read API; write endpoints such as push are not supported ## Where we are -- Read endpoints with a test comparing the proxy against Loki: **67%** +- Read endpoints with a test comparing the proxy against Loki: **70%** - Behaviour and feature items: **6 proven**, 19 partial, 3 gaps, 0 waived diff --git a/docs/KNOWN_ISSUES.md b/docs/KNOWN_ISSUES.md index cf355d70b..c0889b889 100644 --- a/docs/KNOWN_ISSUES.md +++ b/docs/KNOWN_ISSUES.md @@ -22,9 +22,9 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Read API signal | Registry evidence | |---|---| -| Routed endpoints | 24/27 | -| Endpoints with Loki differential tests | 18/27 | -| Endpoints declared `proven` | 11/27 | +| Routed endpoints | 25/27 | +| Endpoints with Loki differential tests | 19/27 | +| Endpoints declared `proven` | 12/27 | Routing, linked tests and declared proof are different signals; none means every LogQL shape or every backend version is compatible. See the @@ -35,7 +35,6 @@ Read routes without an implementation: | Route | Current scope | |---|---| | `/loki/api/v1/cache/generation_numbers` | Not implemented | -| `/loki/api/v1/label` | Not implemented | | `/loki/api/v1/index/shards` | Optional planner gap; low priority, counted in scores | `index/shards` is not required by the pinned Grafana/Drilldown request paths @@ -61,7 +60,6 @@ routing these reads does not implement rule writes or prove full ruler parity. | Area | What to expect | |---|---| -| POST metadata cache identity | The current helper keys metadata reads from URL parameters, omitting form-body selectors and bounds. Different POST forms at the same URL can reuse the previous selector's labels. Until [#949](https://github.com/ReliablyObserve/loki-vl-proxy/pull/949) lands, use GET with selector and range in the URL for affected metadata reads. Routing a POST does not prove its cache identity is correct. | | Label vs field surfaces | With `-label-style=underscores` and `-metadata-field-mode=hybrid`, label APIs remain Loki-safe (`service_name`) while field-oriented APIs can expose both `service.name` and `service_name`. This is expected compatibility behavior, not duplicate data corruption. | | Grafana dotted-field builder UX | Grafana builder paths can still tokenize dotted field names awkwardly even when the generated query executes correctly. For click-to-filter flows, underscore aliases are the safer UI path. | | Parsed-only field freshness | `detected_fields` and `detected_field/{name}/values` prefer native VictoriaLogs metadata when possible, but parsed-only or very new fields can still fall back to bounded sampling. That means freshness can differ from indexed metadata. | @@ -147,6 +145,7 @@ caps the worst case. These are not current open issues in this codebase: +- POST metadata form identities — selectors and bounds in the effective form are included in the cache key with body precedence, without mutating the request form. Upgraded binaries fence earlier ambiguous metadata entries across memory, disk, peers and the outer GET cache. Equivalent forms share metadata entries; the outer GET cache remains path-specific. The [retained main/PR/Loki proof](https://github.com/ReliablyObserve/loki-vl-proxy/pull/949) covers selector isolation and exact reuse, not distributed deletion invalidation. - read-path query and label fanout across multiple tenants - Grafana Logs Drilldown contract coverage as a tracked compatibility product - Loki-compatible `/loki/api/v1/patterns` support with persistence and peer warm diff --git a/docs/api-reference.md b/docs/api-reference.md index 8b870b85c..26ec5f21e 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -15,9 +15,9 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Read API signal | Registry evidence | |---|---| -| Routed endpoints | 24/27 | -| Endpoints with Loki differential tests | 18/27 | -| Endpoints declared `proven` | 11/27 | +| Routed endpoints | 25/27 | +| Endpoints with Loki differential tests | 19/27 | +| Endpoints declared `proven` | 12/27 | Routing, linked tests and declared proof are different signals; none means every LogQL shape or every backend version is compatible. See the @@ -28,7 +28,6 @@ Read routes without an implementation: | Route | Current scope | |---|---| | `/loki/api/v1/cache/generation_numbers` | Not implemented | -| `/loki/api/v1/label` | Not implemented | | `/loki/api/v1/index/shards` | Optional planner gap; low priority, counted in scores | `index/shards` is not required by the pinned Grafana/Drilldown request paths @@ -54,6 +53,7 @@ Conditional and unsupported APIs are listed separately below. | `GET/POST /loki/api/v1/query_range` (metrics) | Routed | `/select/logsql/stats_query_range` | 5m (2) | | `GET/POST /loki/api/v1/query` | Routed | `/select/logsql/query` or `stats_query` | 5m (2) | | `GET/POST /loki/api/v1/labels` | Routed | `/select/logsql/stream_field_names` with fallback to `/select/logsql/field_names` | 5m (3) | +| `GET/POST /loki/api/v1/label` | Routed, singular labels alias | Same labels handler and metadata cache as `/labels`; outer GET cache remains path-specific | 5m (3) | | `GET/POST /loki/api/v1/label/{name}/values` | Routed | `field_names` (label alias resolution) → `stream_field_values` when the backend supports stream metadata endpoints, else `field_values` (also used when `stream_field_values` returns a 4xx) | 5m (3) | | `GET/POST /loki/api/v1/series` | Routed | `/select/logsql/streams` | 30s | | `GET/POST /loki/api/v1/index/stats` | Routed | `/select/logsql/query` (`stats count(), count_uniq_hash(_stream_id)`) | 10s | diff --git a/docs/architecture.md b/docs/architecture.md index eb16ce7e9..c8de71edc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -297,6 +297,19 @@ flowchart LR Only **1** request reaches VictoriaLogs. All clients get the same response. Coalescing and cache keys include a scope fingerprint derived from the resolved tenant routing, backend identity, field mappings and forwarded identity, so requests from different tenants or authorization scopes never share a backend response. +Metadata read keys include the effective POST form parameters, with body values +taking precedence over URL values as they do in the handlers. Normalization uses +a copy of the form; POST bypasses the URL-only key memo. Equivalent GET and POST +requests can reuse metadata entries. The singular `/loki/api/v1/label` route uses +the labels handler and metadata identity of `/loki/api/v1/labels`; the outer +GET compatibility cache still distinguishes their paths. + +The affected label, detected-metadata, index-stats and volume namespaces fence +entries created before form parameters were included. Upgrading can therefore +cause initial metadata misses across memory, disk and peers. Old records become +unreachable and follow existing expiry and eviction policies. This namespace +change is an upgrade fence, not a distributed deletion-invalidation protocol. + ## Query And Metadata Flow ```mermaid diff --git a/docs/compatibility-coverage.md b/docs/compatibility-coverage.md index a80337882..df708a529 100644 --- a/docs/compatibility-coverage.md +++ b/docs/compatibility-coverage.md @@ -15,10 +15,10 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Score | Value | Meaning | |---|---|---| -| Implementation | 24/27 | endpoints the proxy routes | -| Differential evidence | 18/27 | endpoints with tests comparing the proxy against Loki | -| Proven | 11/27 | endpoints whose registry state is `proven` | -| VictoriaLogs native share | 5/24 | implemented endpoints answered natively | +| Implementation | 25/27 | endpoints the proxy routes | +| Differential evidence | 19/27 | endpoints with tests comparing the proxy against Loki | +| Proven | 12/27 | endpoints whose registry state is `proven` | +| VictoriaLogs native share | 6/25 | implemented endpoints answered natively | Counts exclude optional deletion management as well as out-of-scope writes and deprecated aliases. A route or a linked test is not a full compatibility claim. @@ -38,11 +38,12 @@ A route or a linked test is not a full compatibility claim. | `/loki/api/v1/index/stats` | GET, POST | proven | VictoriaLogs native | explore, datasource, api | 20 | 7 | internal/proxy/label_handlers.go:546 | | `/loki/api/v1/index/volume` | GET, POST | proven | proxy-side | drilldown, api | 29 | 10 | internal/proxy/volume.go:192 | | `/loki/api/v1/index/volume_range` | GET, POST | proven | proxy-side | drilldown, api | 21 | 4 | internal/proxy/volume.go:200 | +| `/loki/api/v1/label` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 3 | 1 | internal/proxy/label_handlers.go:24 | | `/loki/api/v1/label/{name}/values` | GET, POST | partial | VictoriaLogs native | explore, drilldown, datasource, api | 50 | 15 | internal/proxy/label_handlers.go:124 | -| `/loki/api/v1/labels` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 80 | 17 | internal/proxy/label_handlers.go:24 | +| `/loki/api/v1/labels` | GET, POST | proven | VictoriaLogs native | explore, drilldown, datasource, api | 84 | 18 | internal/proxy/label_handlers.go:24 | | `/loki/api/v1/patterns` | GET, POST | proven | hybrid | drilldown, api | 23 | 3 | internal/proxy/patterns.go:108 | -| `/loki/api/v1/query` | GET, POST | proven | hybrid | explore, drilldown, api | 104 | 40 | internal/proxy/proxy.go:2720 | -| `/loki/api/v1/query_range` | GET, POST | proven | hybrid | explore, drilldown, api | 217 | 64 | internal/proxy/proxy.go:2300 | +| `/loki/api/v1/query` | GET, POST | proven | hybrid | explore, drilldown, api | 104 | 40 | internal/proxy/proxy.go:2721 | +| `/loki/api/v1/query_range` | GET, POST | proven | hybrid | explore, drilldown, api | 217 | 64 | internal/proxy/proxy.go:2301 | | `/loki/api/v1/rules` | GET | partial | proxy-side | api | 10 | 2 | internal/proxy/alerting.go:95 | | `/loki/api/v1/rules/{namespace}` | GET | partial | proxy-side | api | 4 | 0 | internal/proxy/alerting.go:95 | | `/loki/api/v1/rules/{namespace}/{groupName}` | GET | partial | proxy-side | api | 4 | 0 | internal/proxy/alerting.go:95 | @@ -53,7 +54,6 @@ A route or a linked test is not a full compatibility claim. | `/prometheus/api/v1/rules` | GET | partial | proxy-side | api | 5 | 1 | internal/proxy/alerting.go:95 | | `/loki/api/v1/cache/generation_numbers` | GET | gap | not implemented | api | 0 | 0 | — | | `/loki/api/v1/index/shards` | GET, POST | gap (optional) | not implemented | api | 0 | 0 | — | -| `/loki/api/v1/label` | GET, POST | gap | not implemented | api | 0 | 0 | — | Optional API gaps remain included in the totals and scores: diff --git a/docs/compatibility-loki.md b/docs/compatibility-loki.md index b2a712659..047066a5c 100644 --- a/docs/compatibility-loki.md +++ b/docs/compatibility-loki.md @@ -17,9 +17,9 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Read API signal | Registry evidence | |---|---| -| Routed endpoints | 24/27 | -| Endpoints with Loki differential tests | 18/27 | -| Endpoints declared `proven` | 11/27 | +| Routed endpoints | 25/27 | +| Endpoints with Loki differential tests | 19/27 | +| Endpoints declared `proven` | 12/27 | Routing, linked tests and declared proof are different signals; none means every LogQL shape or every backend version is compatible. See the @@ -30,7 +30,6 @@ Read routes without an implementation: | Route | Current scope | |---|---| | `/loki/api/v1/cache/generation_numbers` | Not implemented | -| `/loki/api/v1/label` | Not implemented | | `/loki/api/v1/index/shards` | Optional planner gap; low priority, counted in scores | `index/shards` is not required by the pinned Grafana/Drilldown request paths diff --git a/docs/real-window-compatibility-gaps.md b/docs/real-window-compatibility-gaps.md index c824256d4..a26a53b44 100644 --- a/docs/real-window-compatibility-gaps.md +++ b/docs/real-window-compatibility-gaps.md @@ -58,14 +58,14 @@ Loki profile shown: `loki-defaults`, Loki with its own defaults (align_queries_w | | Now | |---|---| -| Loki endpoints the proxy routes | **24 of 27** (89%) | -| Endpoints with tests that compare the proxy against Loki | **18 of 27** (67%) | -| Endpoints whose registry state is `proven` | **11 of 27** (41%) | -| Routed endpoints answered natively by VictoriaLogs | 5 of 24 | +| Loki endpoints the proxy routes | **25 of 27** (93%) | +| Endpoints with tests that compare the proxy against Loki | **19 of 27** (70%) | +| Endpoints whose registry state is `proven` | **12 of 27** (44%) | +| Routed endpoints answered natively by VictoriaLogs | 6 of 25 | | Behaviour and feature items | 6 proven, 19 partial, 3 gaps | | Known differences from Loki (ranked) | **87**: 67 open, 12 documented, 6 owner-kept, 2 upstream | -Read routes not implemented: `/loki/api/v1/cache/generation_numbers`, `/loki/api/v1/label`, `/loki/api/v1/index/shards` (optional planner gap, low priority). +Read routes not implemented: `/loki/api/v1/cache/generation_numbers`, `/loki/api/v1/index/shards` (optional planner gap, low priority). These are registry evidence counts, not a full runtime compatibility percentage. Optional deletion management is excluded from the read totals; deprecated aliases and write-only APIs are outside this score. diff --git a/docs/roadmap.md b/docs/roadmap.md index ef73e565e..decd4101d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -15,9 +15,9 @@ The proxy serves Loki's read API; write endpoints such as push are not supported | Read API signal | Registry evidence | |---|---| -| Routed endpoints | 24/27 | -| Endpoints with Loki differential tests | 18/27 | -| Endpoints declared `proven` | 11/27 | +| Routed endpoints | 25/27 | +| Endpoints with Loki differential tests | 19/27 | +| Endpoints declared `proven` | 12/27 | Routing, linked tests and declared proof are different signals; none means every LogQL shape or every backend version is compatible. See the @@ -28,7 +28,6 @@ Read routes without an implementation: | Route | Current scope | |---|---| | `/loki/api/v1/cache/generation_numbers` | Not implemented | -| `/loki/api/v1/label` | Not implemented | | `/loki/api/v1/index/shards` | Optional planner gap; low priority, counted in scores | `index/shards` is not required by the pinned Grafana/Drilldown request paths diff --git a/internal/proxy/cache_keys.go b/internal/proxy/cache_keys.go index d094b57a3..c3e939148 100644 --- a/internal/proxy/cache_keys.go +++ b/internal/proxy/cache_keys.go @@ -15,15 +15,21 @@ import ( // /label/{name}/values and the label-name inventory. The same key addresses the // memory, disk (L2) and peer (L3) tiers and the stale-on-error lookup, so // bumping it makes entries written by older binaries unreachable everywhere. -// "@full-range-v2": older binaries cached answers covering only the most recent +// "@full-range-form-v3": older binaries cached answers covering only the most recent // minutes (names) or hours (values) of the requested range. '@' cannot appear // in a Loki label name or in an encoded query string, so versioned keys never -// collide with unversioned ones. -const labelMetadataCacheKeyVersion = "@full-range-v2" +// collide with unversioned ones. The form version also fences entries written +// by POST requests whose body parameters were absent from the key. +const labelMetadataCacheKeyVersion = "@full-range-form-v3" // volumeCacheKeyVersion: older binaries cached volumes as line counts stamped -// at bucket starts; volumes are now bytes stamped like Loki. -const volumeCacheKeyVersion = "@bytes-v1" +// at bucket starts; volumes are now bytes stamped like Loki. The form version +// fences entries written under URL-only POST identities. +const volumeCacheKeyVersion = "@bytes-form-v2" + +// indexStatsCacheKeyVersion fences URL-only POST identities, which could omit +// a selector or window and collide with a different request's response. +const indexStatsCacheKeyVersion = "@form-v1" // logLineCacheKeyVersion versions the cache keys of log query responses and // query_range windows, which the disk (L2) and peer (L3) tiers keep for up to @@ -33,10 +39,11 @@ const volumeCacheKeyVersion = "@bytes-v1" const logLineCacheKeyVersion = "line-v3" // detectedMetadataCacheKeyVersion versions the cache keys of /detected_fields, -// /detected_field/{name}/values and /detected_labels. "@detected-level-v1": +// /detected_field/{name}/values and /detected_labels. "@detected-level-form-v2": // older binaries listed detected_level with parsers and the stored level -// values, and hid a detected_level stream label. -const detectedMetadataCacheKeyVersion = "@detected-level-v1" +// values, and hid a detected_level stream label. The form version also fences +// entries written under URL-only POST identities. +const detectedMetadataCacheKeyVersion = "@detected-level-form-v2" // readCacheKeyVersion returns the key version segment for endpoint, if any. func readCacheKeyVersion(endpoint string) string { @@ -45,6 +52,8 @@ func readCacheKeyVersion(endpoint string) string { return labelMetadataCacheKeyVersion case "volume", "volume_range": return volumeCacheKeyVersion + case "index_stats": + return indexStatsCacheKeyVersion case "detected_fields", "detected_field_values", "detected_labels": return detectedMetadataCacheKeyVersion default: @@ -109,7 +118,8 @@ func (p *Proxy) canonicalReadCacheKey(endpoint, orgID string, r *http.Request, e } func buildCanonicalReadCacheMemoKey(endpoint, orgID string, r *http.Request, extraParts []string) (canonicalReadCacheMemoKey, bool) { - if r == nil || len(extraParts) > 1 { + // POST identity includes form-body parameters absent from this URL-only memo. + if r == nil || r.Method == http.MethodPost || len(extraParts) > 1 { return canonicalReadCacheMemoKey{}, false } key := canonicalReadCacheMemoKey{ @@ -125,6 +135,15 @@ func buildCanonicalReadCacheMemoKey(endpoint, orgID string, r *http.Request, ext func computeCanonicalReadCacheKey(endpoint, orgID string, r *http.Request, extraParts ...string) string { params := r.URL.Query() + if r.Method == http.MethodPost { + // Match FormValue's body-before-URL precedence, including partial forms + // on parse errors. Normalize a copy so handlers still see the raw form. + _ = r.ParseForm() + params = make(url.Values, len(r.Form)) + for name, values := range r.Form { + params[name] = append([]string(nil), values...) + } + } normalizeReadCacheParams(endpoint, params) switch endpoint { case "detected_fields", "detected_field_values", "detected_labels": diff --git a/internal/proxy/cache_keys_post_form_test.go b/internal/proxy/cache_keys_post_form_test.go new file mode 100644 index 000000000..e89f06ee1 --- /dev/null +++ b/internal/proxy/cache_keys_post_form_test.go @@ -0,0 +1,131 @@ +package proxy + +import ( + "net/http" + "net/http/httptest" + "net/url" + "reflect" + "strings" + "testing" + "time" + + "github.com/ReliablyObserve/Loki-VL-proxy/internal/cache" +) + +func postReadCacheRequest(path string, params url.Values) *http.Request { + r := httptest.NewRequest(http.MethodPost, path, strings.NewReader(params.Encode())) + r.Header.Set("Content-Type", "application/x-www-form-urlencoded") + return r +} + +func TestCanonicalReadCacheKeyPOSTFormMatchesGET(t *testing.T) { + for _, endpoint := range []string{"labels", "label_values", "index_stats", "volume", "volume_range", "detected_fields", "detected_field_values", "detected_labels"} { + t.Run(endpoint, func(t *testing.T) { + params := url.Values{"query": {`{app="api"}`}, "from": {"1790871720000000000"}, "to": {"1790875320000000000"}, "q": {"svc"}, "line_limit": {"1000"}, "step": {"60"}, "match[]": {`{app="api"}`, `{env="prod"}`}} + p := &Proxy{} + get := httptest.NewRequest(http.MethodGet, "/?"+params.Encode(), nil) + post := postReadCacheRequest("/", params) + if got, want := p.canonicalReadCacheKey(endpoint, "tenant-a", post), p.canonicalReadCacheKey(endpoint, "tenant-a", get); got != want { + t.Fatalf("POST form and GET params differ: got %q want %q", got, want) + } + }) + } +} + +func TestCanonicalReadCacheKeyPOSTFormPrecedenceAndImmutable(t *testing.T) { + body := url.Values{"query": {`{app="body"}`, `{app="second"}`}, "from": {"1790871720000000000"}, "to": {"1790875320000000000"}, "q": {" svc "}, "match[]": {`{env="prod"}`, `{env="staging"}`}} + r := postReadCacheRequest("/?query=%7Bapp%3D%22url%22%7D&from=1&to=2&match%5B%5D=%7Benv%3D%22url%22%7D", body) + if err := r.ParseForm(); err != nil { + t.Fatal(err) + } + before, postBefore := r.Form.Encode(), r.PostForm.Encode() + p := &Proxy{} + got := p.canonicalReadCacheKey("labels", "tenant-a", r) + expected := url.Values{"query": {`{app="body"}`}, "start": {"1790871720000000000"}, "end": {"1790875320000000000"}, "search": {"svc"}, "match[]": {`{env="prod"}`, `{env="staging"}`, `{env="url"}`}} + want := p.canonicalReadCacheKey("labels", "tenant-a", httptest.NewRequest(http.MethodGet, "/?"+expected.Encode(), nil)) + if got != want { + t.Errorf("POST must use body precedence and retain multiple values: got %q want %q", got, want) + } + if r.Form.Encode() != before || r.PostForm.Encode() != postBefore { + t.Fatalf("cache key normalization mutated request form: Form=%v PostForm=%v", r.Form, r.PostForm) + } + if !reflect.DeepEqual(r.Form["query"], []string{`{app="body"}`, `{app="second"}`, `{app="url"}`}) { + t.Fatalf("request query values changed: %v", r.Form["query"]) + } +} + +func TestCanonicalReadCacheKeyPOSTFormSeparatesSelectorsAndWindows(t *testing.T) { + p := &Proxy{} + cases := []url.Values{ + {"query": {`{app="a"}`}, "start": {"1790871720000000000"}, "end": {"1790875320000000000"}}, + {"query": {`{app="b"}`}, "start": {"1790871720000000000"}, "end": {"1790875320000000000"}}, + {"query": {`{app="a"}`}, "start": {"1790785320000000000"}, "end": {"1790875320000000000"}}, + {"query": {`{app="a"}`}, "start": {"1790871720000000000"}, "end": {"1790961720000000000"}}, + } + seen := map[string]bool{} + for _, params := range cases { + key := p.canonicalReadCacheKey("labels", "", postReadCacheRequest("/loki/api/v1/labels", params)) + if seen[key] { + t.Errorf("POST form identity collided: %v -> %q", params, key) + } + seen[key] = true + if again := p.canonicalReadCacheKey("labels", "", postReadCacheRequest("/loki/api/v1/labels", params)); again != key { + t.Errorf("repeat identity changed: %q != %q", again, key) + } + } +} + +func TestCanonicalReadCacheKeyPOSTPartialFormPreservesEffectiveParameters(t *testing.T) { + r := httptest.NewRequest(http.MethodPost, "/?query=%7Bapp%3D%22url%22%7D", strings.NewReader("query=%7Bapp%3D%22body%22%7D&invalid=%zz&start=1790871720000000000&end=1790875320000000000")) + r.Header.Set("Content-Type", "application/x-www-form-urlencoded") + if err := r.ParseForm(); err == nil { + t.Fatal("fixture must contain a malformed form value") + } + before := r.Form.Encode() + p := &Proxy{} + got := p.canonicalReadCacheKey("labels", "tenant-a", r) + params := url.Values{"query": {`{app="body"}`}, "start": {"1790871720000000000"}, "end": {"1790875320000000000"}} + want := p.canonicalReadCacheKey("labels", "tenant-a", httptest.NewRequest(http.MethodGet, "/?"+params.Encode(), nil)) + if got != want || r.Form.Encode() != before { + t.Fatalf("partial form identity or original form changed: got=%q want=%q form=%v", got, want, r.Form) + } +} + +// previousPOSTReadCacheKey reconstructs the namespace used by binaries whose +// URL-only POST keys could store a filtered answer under a wildcard identity. +func previousPOSTReadCacheKey(endpoint, key string) string { + previous := map[string]string{ + "labels": "@full-range-v2", "label_values": "@full-range-v2", "label_inventory": "@full-range-v2", + "detected_fields": "@detected-level-v1", "detected_field_values": "@detected-level-v1", "detected_labels": "@detected-level-v1", + "volume": "@bytes-v1", "volume_range": "@bytes-v1", "index_stats": "", + } + if version := readCacheKeyVersion(endpoint); version != "" { + replacement := ":" + if previous[endpoint] != "" { + replacement += previous[endpoint] + ":" + } + return strings.Replace(key, ":"+version+":", replacement, 1) + } + return key +} + +func TestCanonicalReadCacheKeyPOSTMigrationRejectsPriorNamespace(t *testing.T) { + for _, endpoint := range []string{"labels", "label_values", "label_inventory", "index_stats", "volume", "volume_range", "detected_fields", "detected_field_values", "detected_labels"} { + t.Run(endpoint, func(t *testing.T) { + c := cache.New(time.Hour, 10) + t.Cleanup(c.Close) + p := &Proxy{cache: c} + req := httptest.NewRequest(http.MethodGet, "/?query=%2A", nil) + key := p.canonicalReadCacheKey(endpoint, "tenant-a", req) + previous := previousPOSTReadCacheKey(endpoint, key) + c.SetWithTTL(previous, []byte(`{"status":"success","data":["poisoned-selector"]}`), time.Hour) + if body, _, _, hit := p.endpointReadCacheEntry(endpoint, key); hit { + t.Fatalf("current GET reused a prior POST namespace: endpoint=%s key=%q body=%s", endpoint, key, body) + } + c.SetWithTTL(key, []byte("current-answer"), time.Hour) + if body, _, _, hit := p.endpointReadCacheEntry(endpoint, key); !hit || string(body) != "current-answer" { + t.Fatalf("current namespace did not retain its own entry: hit=%v body=%s", hit, body) + } + }) + } +} diff --git a/internal/proxy/labels_alias_test.go b/internal/proxy/labels_alias_test.go new file mode 100644 index 000000000..0e65b1556 --- /dev/null +++ b/internal/proxy/labels_alias_test.go @@ -0,0 +1,79 @@ +package proxy + +import ( + "context" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" +) + +func TestLabelsAliasPreservesTenantAdmission(t *testing.T) { + backend, calls := fieldNamesServer(t, []string{"app"}) + p, err := New(Config{BackendURL: backend.URL, AuthEnabled: true, RequireTenantHeader: true, LogLevel: "error"}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = p.Shutdown(context.Background()) }) + mux := http.NewServeMux() + p.RegisterRoutes(mux) + var canonical int + for _, path := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + w := httptest.NewRecorder() + mux.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil)) + if canonical == 0 { + canonical = w.Code + } + if w.Code < 400 || w.Code != canonical { + t.Fatalf("%s bypassed tenant admission: status=%d canonical=%d", path, w.Code, canonical) + } + } + if got := calls.Load(); got != 0 { + t.Fatalf("rejected request reached backend: %d calls", got) + } +} + +// conformance: loki_api_v1_label, loki_api_v1_labels +func TestLabelsAliasSharesCanonicalHandlerAndMetadataCache(t *testing.T) { + for _, first := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + t.Run(first, func(t *testing.T) { + backend, calls := fieldNamesServer(t, []string{"app", "env"}) + _, mux := newBehaviorProxy(t, backend.URL, 0) + params := url.Values{"start": {"1790871720000000000"}, "end": {"1790875320000000000"}} + get := func(method, path string) *httptest.ResponseRecorder { + t.Helper() + req := httptest.NewRequest(method, path+"?"+params.Encode(), nil) + if method == http.MethodPost { + req = httptest.NewRequest(method, path, strings.NewReader(params.Encode())) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + } + w := httptest.NewRecorder() + mux.ServeHTTP(w, req) + if w.Code != http.StatusOK { + t.Fatalf("%s %s: status=%d body=%s", method, path, w.Code, w.Body.String()) + } + if w.Header().Get("Location") != "" { + t.Fatalf("alias must serve directly, got Location=%q", w.Header().Get("Location")) + } + return w + } + initial := get(http.MethodGet, first) + baseline := calls.Load() + if baseline == 0 { + t.Fatal("cold request did not reach backend") + } + for _, method := range []string{http.MethodGet, http.MethodPost} { + for _, path := range []string{"/loki/api/v1/label", "/loki/api/v1/labels"} { + got := get(method, path) + if got.Body.String() != initial.Body.String() { + t.Fatalf("%s %s response differs: %s vs %s", method, path, got.Body.String(), initial.Body.String()) + } + if got := calls.Load(); got != baseline { + t.Fatalf("alias duplicated backend read: calls=%d want=%d", got, baseline) + } + } + } + }) + } +} diff --git a/internal/proxy/labels_post_form_cache_test.go b/internal/proxy/labels_post_form_cache_test.go new file mode 100644 index 000000000..ffb7b594a --- /dev/null +++ b/internal/proxy/labels_post_form_cache_test.go @@ -0,0 +1,154 @@ +package proxy + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/ReliablyObserve/Loki-VL-proxy/internal/cache" +) + +// conformance: loki_api_v1_label, loki_api_v1_labels +func TestLabelsPOSTFormCacheSeparatesSelectors(t *testing.T) { + for _, path := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + t.Run(path, func(t *testing.T) { + var calls atomic.Int64 + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/health" { + w.WriteHeader(http.StatusOK) + return + } + calls.Add(1) + switch r.URL.Path { + case "/select/logsql/stream_field_names", "/select/logsql/field_names": + query := r.URL.Query().Get("query") + switch { + case strings.Contains(query, "alpha"): + writeVLFieldNames(w, []fieldHit{{"app", 1}, {"alpha_label", 1}}) + case strings.Contains(query, "beta"): + writeVLFieldNames(w, []fieldHit{{"app", 1}, {"beta_label", 1}}) + default: + t.Errorf("unexpected backend selector %q", query) + writeVLFieldNames(w, nil) + } + default: + writeVLFieldValues(w, nil) + } + })) + t.Cleanup(backend.Close) + _, mux := newBehaviorProxy(t, backend.URL, 0) + request := func(method, route, selector string) string { + t.Helper() + params := url.Values{"query": {selector}, "start": {"1790871720000000000"}, "end": {"1790875320000000000"}} + req := postReadCacheRequest(route, params) + if method == http.MethodGet { + req = httptest.NewRequest(method, route+"?"+params.Encode(), nil) + } + w := httptest.NewRecorder() + mux.ServeHTTP(w, req) + if w.Code != http.StatusOK { + t.Fatalf("%s %s returned %d: %s", method, route, w.Code, w.Body.String()) + } + var resp struct{ Data []string } + if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil { + t.Fatal(err) + } + want, other := "alpha_label", "beta_label" + if strings.Contains(selector, "beta") { + want, other = other, want + } + if !contains(resp.Data, want) || contains(resp.Data, other) { + t.Errorf("%s %s selector %s returned labels %v", method, route, selector, resp.Data) + } + return w.Body.String() + } + alpha := request(http.MethodPost, path, `{app="alpha"}`) + alphaCalls := calls.Load() + beta := request(http.MethodPost, path, `{app="beta"}`) + if calls.Load() <= alphaCalls { + t.Error("different POST selector reused the first response") + } + baseline := calls.Load() + for _, method := range []string{http.MethodPost, http.MethodGet} { + for _, route := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + if got := request(method, route, `{app="alpha"}`); got != alpha { + t.Errorf("repeat alpha payload differs: %s", got) + } + if got := request(method, route, `{app="beta"}`); got != beta { + t.Errorf("repeat beta payload differs: %s", got) + } + } + } + if got := calls.Load(); got != baseline { + t.Errorf("equivalent forms/aliases did not reuse response cache: got %d backend calls want %d", got, baseline) + } + }) + } +} + +// conformance: loki_api_v1_label, loki_api_v1_labels +func TestLabelsPOSTFormMigrationIgnoresPriorWildcardAnswer(t *testing.T) { + for _, path := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + for _, tier := range []string{"memory", "disk", "compatibility"} { + t.Run(path+"/"+tier, func(t *testing.T) { + backend, calls := fieldNamesServer(t, []string{"app", "fresh_label"}) + c := cache.New(time.Hour, 100) + t.Cleanup(c.Close) + compat := cache.New(time.Hour, 100) + t.Cleanup(compat.Close) + var disk *cache.DiskCache + if tier == "disk" { + var err error + disk, err = cache.NewDiskCache(cache.DiskCacheConfig{Path: t.TempDir() + "/cache.db"}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = disk.Close() }) + c.SetL2(disk) + } + p, err := New(Config{BackendURL: backend.URL, Cache: c, CompatCache: compat, LogLevel: "error"}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = p.Shutdown(context.Background()) }) + mux := http.NewServeMux() + p.RegisterRoutes(mux) + req := httptest.NewRequest(http.MethodGet, path, nil) + key := p.canonicalReadCacheKey("labels", "", req) + poisoned := lokiLabelsResponse([]string{"poisoned_selector"}) + previous := previousPOSTReadCacheKey("labels", key) + switch tier { + case "memory": + c.SetWithTTL(previous, poisoned, time.Hour) + case "disk": + disk.Set(previous, poisoned, time.Hour) + disk.Flush() + case "compatibility": + outer, ok := p.compatCacheKey("labels", req) + if !ok { + t.Fatal("GET labels is not compatibility-cacheable") + } + compat.SetWithTTL(previousPOSTReadCacheKey("labels", outer), poisoned, time.Hour) + } + rec := httptest.NewRecorder() + mux.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, path, nil)) + var response struct{ Data []string } + if err := json.Unmarshal(rec.Body.Bytes(), &response); err != nil { + t.Fatal(err) + } + if rec.Code != http.StatusOK || !contains(response.Data, "fresh_label") || contains(response.Data, "poisoned_selector") { + t.Fatalf("GET reused a wildcard answer poisoned by an older POST: status=%d body=%s", rec.Code, rec.Body.String()) + } + if calls.Load() == 0 { + t.Fatal("migration did not fetch a fresh backend answer") + } + }) + } + } +} diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go index fca33e7d6..ea24edd7f 100644 --- a/internal/proxy/proxy.go +++ b/internal/proxy/proxy.go @@ -1876,6 +1876,7 @@ func (p *Proxy) RegisterProxyRoutes(mux *http.ServeMux) { // Metadata endpoints — rate-limited but cached mux.Handle("/loki/api/v1/labels", p.routeHandler("labels", "/loki/api/v1/labels", p.handleLabels)) + mux.Handle("/loki/api/v1/label", p.routeHandler("labels", "/loki/api/v1/label", p.handleLabels)) mux.Handle("/loki/api/v1/label/", p.routeHandler("label_values", "/loki/api/v1/label/{name}/values", p.handleLabelValues)) mux.Handle("/loki/api/v1/detected_fields", p.routeHandler("detected_fields", "/loki/api/v1/detected_fields", p.handleDetectedFields)) mux.Handle("/loki/api/v1/detected_field/", p.routeHandler("detected_field_values", "/loki/api/v1/detected_field/{name}/values", p.handleDetectedFieldValues)) diff --git a/test/e2e-compat/labels_alias_test.go b/test/e2e-compat/labels_alias_test.go new file mode 100644 index 000000000..f1242c2b3 --- /dev/null +++ b/test/e2e-compat/labels_alias_test.go @@ -0,0 +1,101 @@ +//go:build e2e + +package e2e_compat + +import ( + "encoding/json" + "io" + "net/http" + "net/url" + "reflect" + "strconv" + "strings" + "testing" + "time" +) + +// conformance: loki_api_v1_label, loki_api_v1_labels +func TestCompat_LabelsSingularAlias(t *testing.T) { + now := time.Now() + params := url.Values{"start": {strconv.FormatInt(now.Add(-time.Hour).UnixNano(), 10)}, "end": {strconv.FormatInt(now.UnixNano(), 10)}, "query": {`{app="e2e-test"}`}} + client := &http.Client{Timeout: 30 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + var lokiReference map[string]any + for _, backend := range []struct{ name, base string }{{"Loki", lokiURL}, {"proxy", proxyURL}} { + name, base := backend.name, backend.base + t.Run(name, func(t *testing.T) { + var reference map[string]any + for _, path := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + for _, method := range []string{http.MethodGet, http.MethodPost} { + req, err := http.NewRequest(method, base+path+"?"+params.Encode(), nil) + if method == http.MethodPost { + req, err = http.NewRequest(method, base+path, strings.NewReader(params.Encode())) + if err == nil { + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + } + } + if err != nil { + t.Fatal(err) + } + resp, err := client.Do(req) + if err != nil { + t.Fatal(err) + } + body, err := io.ReadAll(resp.Body) + resp.Body.Close() + if err != nil { + t.Fatal(err) + } + if resp.StatusCode != http.StatusOK { + t.Fatalf("%s %s: %d %s", method, path, resp.StatusCode, body) + } + var result map[string]any + if err := json.Unmarshal(body, &result); err != nil { + t.Fatalf("%s %s: %v", method, path, err) + } + if result["status"] != "success" { + t.Fatalf("%s %s: %s", method, path, body) + } + if labels := extractStrings(result, "data"); !contains(labels, "app") { + t.Fatalf("%s %s did not include the ingested app label: %s", method, path, body) + } + if reference == nil { + reference = result + } else if !reflect.DeepEqual(reference, result) { + t.Fatalf("%s %s differs from canonical labels: got=%s reference=%v", method, path, body, reference) + } + if lokiReference == nil { + lokiReference = result + } else if !reflect.DeepEqual(lokiReference, result) { + t.Fatalf("%s %s differs from Loki: got=%s Loki=%v", method, path, body, lokiReference) + } + } + } + // A second form selector must not reuse the nonempty response above. + empty := url.Values{"start": {params.Get("start")}, "end": {params.Get("end")}, "query": {`{app="read-api-proof-missing-fixture"}`}} + for _, path := range []string{"/loki/api/v1/labels", "/loki/api/v1/label"} { + req, err := http.NewRequest(http.MethodPost, base+path, strings.NewReader(empty.Encode())) + if err != nil { + t.Fatal(err) + } + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + resp, err := client.Do(req) + if err != nil { + t.Fatal(err) + } + body, err := io.ReadAll(resp.Body) + resp.Body.Close() + if err != nil { + t.Fatal(err) + } + var result map[string]any + if err := json.Unmarshal(body, &result); err != nil { + t.Fatal(err) + } + // Loki omits data for an empty labels response; the proxy uses []. + if resp.StatusCode != http.StatusOK || result["status"] != "success" || len(extractStrings(result, "data")) != 0 { + t.Fatalf("POST %s reused a different selector's labels: status=%d body=%s", path, resp.StatusCode, body) + } + } + }) + } +}