diff --git a/AGENTS.md b/AGENTS.md index a647cf6..445f18e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -159,7 +159,7 @@ External knowledge sources are configured with active `*.inputs.toml` manifests - **B2 project fallback identity + historical re-resolution**: when the global registry (`~/.config/backscroll/projects.toml`) does not match a session's cwd, `projects.Identify()` derives a sanitized fallback id from the cwd basename (lowercase, `[a-z0-9-_]`; registry always wins when it matches). `rebuild` runs a re-resolution pass: `ReresolveProjects` decodes Claude's session-dir encoding (dashes-for-slashes; if the decoded path exists on disk its basename wins, else the last dash segment — the encoding is lossy and this heuristic is documented as ambiguous for dir names containing dashes) and relabels rows stuck at `project='unknown'`, returning distinct files resolved. Fallback labels are not revisited once set — followed by registry-aware re-resolution below. - **B2.1 registry-aware re-resolution**: `projects.Identify()` now carries a `FromRegistry` flag to distinguish registry matches from fallback labels. `rebuild` includes a second phase that loads `~/.config/backscroll/projects.toml` and re-resolves historical sessions labeled with fallback IDs. For each session path, it decodes the cwd, applies cross-host normalization, and calls `Identify` with the registry. If a registry entry matches (FromRegistry=true) and differs from the stored fallback, the rows are updated. Only registry matches count — fallback-only paths are not re-labeled (no churn). This allows a future registry entry to correct historical misattribution without touching already-correct sessions. Wired into rebuild via `ReresolveProjectsWithRegistry` (queries.go). - **B3 retroactive mining over stored text**: for sessions that expire from disk before Template/Correction/Sequence mining runs, `BackfillDerived()` recovers templates, correction signals, and lossy tool_events from stored text in search_items. Stale template re-mining (F2a) is integrated: BackfillDerived first discovers stale paths via `StaleTemplatePaths(CurrentNormalizationVersion)` and re-mines templates under v2 heuristics, updating existing templates to normalization_version=2. Template mining reuses `internal/templates/Miner` over tool-text rows. Correction detection filters input to prose only (role='user' AND content_type IN ('text','code')) to avoid tool_result false positives; lexicon, denial, and rephrase detectors run on prose; interrupt detector runs on all user messages. Lossy tool_events reverse-parse toolfmt input serialization ( ...; heuristic: first token has no '=', at least one subsequent has '=') to extract tool_name and command_head; outputs unattributable without tool_use_id so skipped; extraction_version=0 marks lossy rows (inputs only, unrecoverable once disk source expires). `rebuild` command now runs after mutation-class startup sync and then: (1) re-derives FTS from DB, (2) BackfillDerived (stale-path discovery + re-mining + expired-file discovery + batch mining + progress reporting), (3) re-resolves projects, and (4) applies registry-aware re-resolution. It performs no second sync. The recover path retains the remediation lock through post-install sync. -- **Early input validation**: CLI commands validate positional arguments and semantic flag values through Cobra `Args`, which runs before the root `PersistentPreRunE`; the root also checks required flags and flag groups before invoking the startup policy for the command class. Invalid invocations therefore fail before opening, creating, synchronizing, or recovering the database. `internal/services` owns pure query validation as the shared CLI/MCP boundary. Direct service calls validate before database access. Search accepts limit 0 for the historical internal default of 100, or an explicit limit from 1 through 300. Context accepts max_tokens from 64 through 16384 and uses 2000 by default. +- **Early input validation**: CLI commands validate positional arguments and semantic flag values through Cobra `Args`, which runs before the root `PersistentPreRunE`; the root also checks required flags and flag groups before invoking the startup policy for the command class. Invalid invocations therefore fail before opening, creating, synchronizing, or recovering the database. `internal/services` owns pure query validation, including opt-in relaxation syntax, as the shared CLI/MCP boundary. Direct service calls validate before database access. Search accepts limit 0 for the historical internal default of 100, or an explicit limit from 1 through 300. Context accepts max_tokens from 64 through 16384 and uses 2000 by default. - **Coverage gate**: CI enforces ≥85% aggregate statement coverage via `go test ./... -race -coverprofile`. Local check: `bash scripts/check-coverage.sh`. Tests that depend on local machine state (e.g. `~/.config/backscroll/projects.toml`) must use `t.Setenv("HOME", tempDir)` to stay reproducible on CI. Likewise, `InputsDir` branches requiring `BACKSCROLL_CONFIG_DIR` to be unset must set it to `""` via `t.Setenv`. To test the `Validate` orphan path, insert directly into `search_items` without a matching `indexed_files` row. - **Zero-result guidance**: when `search`/`list` return no rows, actionable suggestions (`--all-projects`, `--content-type tool`, `backscroll status`) are printed to STDERR — never STDOUT, so `--json` stays a clean empty payload. - **Search robot output contract**: robot mode on search emits `result_N_field=value` lines exactly once-wrapped (the robot path writes lines directly; passing pre-formatted lines through the picokit formatter double-wraps them as `result_N=result_N_field=...`). Search robot string values escape backslash as `\\`, carriage return as `\r`, and newline as `\n`. Minimal fields use bounded snippets; full fields retain indexed content. Positive `--max-tokens` budgets count the complete escaped payload with one Picokit estimate, including omission metadata; never sum independently rounded line/result estimates. Truncation removes whole trailing results, and emits no output when even the omission record cannot fit. diff --git a/cmd/backscroll/search_relaxation_output_test.go b/cmd/backscroll/search_relaxation_output_test.go index 6104320..f0de2a6 100644 --- a/cmd/backscroll/search_relaxation_output_test.go +++ b/cmd/backscroll/search_relaxation_output_test.go @@ -6,6 +6,8 @@ import ( "strings" "testing" + "github.com/pablontiv/backscroll/internal/models" + "github.com/pablontiv/backscroll/internal/services" "github.com/pablontiv/backscroll/internal/storage" picokitoutput "github.com/pablontiv/picokit/output" ) @@ -34,6 +36,27 @@ func TestSearchRelaxationBudgetKeepsProvenanceWithEachResult(t *testing.T) { } } +func TestSearchRelaxationValidationMatchesServiceBytes(t *testing.T) { + testEnv(t) + query := `"unclosed` + serviceErr := services.ValidateSearchRequest(services.SearchRequest{ + Query: query, + Options: models.SearchOptions{Limit: 20}, + Relax: true, + }) + if serviceErr == nil { + t.Fatal("service validation accepted invalid relaxation syntax") + } + + stdout, stderr, cliErr := runCmd("search", "--text", query, "--relax", "--robot") + if cliErr == nil || cliErr.Error() != serviceErr.Error() { + t.Fatalf("CLI error = %v, service error = %v", cliErr, serviceErr) + } + if stdout != "" || stderr != "" { + t.Fatalf("CLI output = (%q, %q), want empty output", stdout, stderr) + } +} + func TestSearchRelaxationInvalidSyntaxPrecedesStartup(t *testing.T) { testEnv(t) for _, query := range []string{`"unclosed`, "+", `""`, "+ word", "handshake adaptation ...", "violet && handshake", "+...", `"... &&"`} { diff --git a/internal/services/query_test.go b/internal/services/query_test.go index c177615..9a1d809 100644 --- a/internal/services/query_test.go +++ b/internal/services/query_test.go @@ -150,6 +150,10 @@ func TestValidateSearchRequestExactErrorsAndOrder(t *testing.T) { request: SearchRequest{Query: `"unclosed`, Options: models.SearchOptions{Limit: 20}, Relax: true}, want: "invalid --relax query: quoted phrases must be closed and whitespace-delimited", }, + { + name: "relaxation syntax stays opt-in", + request: SearchRequest{Query: `"unclosed`, Options: models.SearchOptions{Limit: 20}}, + }, } for _, test := range tests { t.Run(test.name, func(t *testing.T) { @@ -158,6 +162,19 @@ func TestValidateSearchRequestExactErrorsAndOrder(t *testing.T) { } } +func TestQueryServiceRejectsInvalidRelaxationBeforeDatabaseAccess(t *testing.T) { + service := QueryService{} + response, err := service.Search(context.Background(), SearchRequest{ + Query: `"unclosed`, + Options: models.SearchOptions{Limit: 20}, + Relax: true, + }) + assertExactValidationError(t, err, "invalid --relax query: quoted phrases must be closed and whitespace-delimited") + if len(response.Results) != 0 || response.Provenance.RelaxationStages != nil { + t.Fatalf("response = %+v, want empty response", response) + } +} + func TestValidateExplicitSearchScopeComposesBaseValidation(t *testing.T) { tests := []struct { name string