Implement FQL stdlib migration with support for safe legacy call replacements

README.md

@@ -149,9 +149,50 @@ return for item in 1..3 {
 }
 ```
 
-Only a structurally recognized final top-level `FOR` without an explicit
-terminal `return` is changed. Nested, assigned, expression-contained,
-function-contained, non-final, and already-returned loops remain untouched.
+For loop migration, only a structurally recognized final top-level `FOR` without
+an explicit terminal `return` is changed. Nested, assigned, expression-contained,
+function-contained, non-final, and already-returned loops retain their structure.
+
+The command also migrates safe unqualified legacy stdlib calls to canonical
+lowercase names. For example, `JSON_PARSE(body)` becomes
+`encoding::json_parse(body)`, `has(obj, key)` becomes `object::has_key(obj, key)`,
+and `abs(value)` becomes `math::abs(value)`. Supported mappings include:
+
+| Namespace | Legacy calls migrated automatically |
+| --- | --- |
+| `encoding::` | `json_parse`, `json_stringify`, `encode_uri_component` → `query_escape`, `decode_uri_component` → `query_unescape`, `to_base64` → `base64_encode`, `from_base64` → `base64_decode`, `escape_html` → `html_escape`, `unescape_html` → `html_unescape` |
+| `crypto::` | `md5`, `sha1`, `sha512`, `random_token` |
+| `path::` | `base`, `clean`, `dir`, `ext`, `is_abs`, `separate`, `match` |
+| `object::` | `values`, `has` → `has_key`, `zip`, `keep_keys`, `merge`, `merge_recursive` → `merge_deep`, and one-argument `keys(obj)` |
+| `datetime::` | `now`, `date` → `parse`, `date_dayofweek` → `day_of_week`, `date_dayofyear` → `day_of_year`, `date_leapyear` → `is_leap_year`; `date_year`, `date_month`, `date_day`, `date_hour`, `date_minute`, `date_second`, `date_millisecond`, `date_quarter`, `date_days_in_month`, `date_format`, `date_add`, `date_subtract` lose their `date_` prefix |
+| `math::` | `pi`, `abs`, `acos`, `asin`, `atan`, `atan2`, `ceil`, `cos`, `degrees`, `exp`, `exp2`, `floor`, `log`, `log2`, `log10`, `pow`, `radians`, `round`, `sin`, `sqrt`, `tan` |
+
+Only parsed call targets are replaced; arguments retain their meaning, and
+strings, comments, object keys, and variable names are not matched. Nested calls
+are migrated independently, and already-qualified targets are preserved. Object
+replacements use immutable operations, never `object::mut`.
+
+Manual follow-up includes the original path and line and explains why a call
+was preserved:
+
+- `join` is ambiguous between legacy path joining and modern global string joining.
+- `keys` with any arity other than one needs argument-aware review.
+- `date_compare` has component-range semantics that differ from `datetime::same`;
+  legacy `date_diff` integer/floating behavior differs from `datetime::diff`.
+- `average`, `sum`, `min`, `max`, `median`, `percentile`, `stddev_population`,
+  `stddev_sample`, `variance_population`, and `variance_sample` have permissive
+  legacy behavior that differs from strict canonical math.
+- A matching function declaration or function alias anywhere in the file may
+  change call resolution. These checks are deliberately conservative, including
+  case variants and declarations in nested scopes. A namespace alias blocks a
+  replacement only when it would redirect that canonical target.
+
+Other safe calls in the same file still migrate. Array migrations and
+`rand`/`range` are deferred and receive no new diagnostics in this pass.
+Rerunning migration produces no further edits; unresolved manual actions remain.
+If the formatter cannot preserve comments, the entire file is left unchanged
+and reported for manual follow-up.
+
 Changed FQL is canonically formatted; files needing only formatting remain
 byte-for-byte unchanged. FQL-only targets do not require a Go module or Go
 toolchain and do not change Go dependencies. A directory containing eligible Go
@@ -193,6 +234,14 @@ files.
 
 ### Checking FQL compatibility
 
+`migrate check` reports final collecting `FOR` compatibility and the same legacy
+stdlib findings as `migrate run`: encoding, crypto, path, immutable object,
+datetime, and scalar math replacements, plus calls requiring manual review.
+It uses the same conservative declaration and alias guards described above.
+Already-qualified calls are left alone; arrays and `rand`/`range` remain outside
+this pass. A clean check covers these supported rules, not all v1 application
+semantics.
+
 Check a standalone FQL file or recursively inspect a directory without
 modifying source files:
 
@@ -209,19 +258,24 @@ nested Go modules. They skip `.git`, `.hg`, `.svn`, `vendor`, and
 `node_modules`, and do not follow directory symlinks.
 
 Compatibility findings use editor-friendly locations and include a suggested
-manual fix:
+replacement or a manual-review explanation:
 
 ```text
 1_hackernews.fql:2:1: Final collecting FOR no longer becomes the script result in Ferret v2.
   help: Add `return` before this loop.
 
-Found 1 v1 compatibility issue in 1 of 12 FQL files.
+query.fql:1:8: Legacy stdlib call `has` should use `object::has_key`.
+  help: Preview automatic replacements with `ferret migrate run --print`.
+
+Found 2 v1 compatibility issues in 2 of 12 FQL files.
 ```
 
-The command exits nonzero when it finds a compatibility issue or cannot parse
-an FQL file. Malformed files are reported while the remaining files continue
-to be checked. Filesystem, cancellation, and internal failures stop the check
-immediately.
+The command exits nonzero for automatic replacement suggestions, manual-review
+findings, or FQL parse failures. Use `ferret migrate run --print path/to/source`
+to preview edits; the check does not verify whether the formatter can preserve
+the source during rewriting. Malformed files are reported while the remaining
+files continue to be checked. Filesystem, cancellation, and internal failures
+stop the check immediately.
 
 ## Module lifecycle
 

cmd/internal/migrate/check.go

@@ -16,6 +16,10 @@ func newCompatibilityCheckCommand(service Service) *cobra.Command {
 		Short: "Check FQL source for Ferret version compatibility",
 		Long: "Check a standalone FQL file or recursively inspect a directory for supported behavior changes " +
 			"between Ferret versions. The check does not require a Go module and never modifies source files.\n\n" +
+			"Checks final collecting FOR and legacy stdlib calls in encoding, crypto, path, object, datetime, and math. " +
+			"Findings suggest canonical replacements or manual review for ambiguous calls, semantic differences, " +
+			"and local function or use alias collisions. Arrays and rand/range are outside this pass. " +
+			"Preview automatic replacements with migrate run --print.\n\n" +
 			"Directory scans include testdata, hidden and underscore-prefixed directories, and nested Go modules. " +
 			"They skip .git, .hg, .svn, vendor, and node_modules and do not follow directory symlinks.\n\n" +
 			"Compatibility findings and malformed FQL make the command fail after all readable files are checked.",

cmd/internal/migrate/command_test.go

@@ -422,6 +422,11 @@ func TestMigrateCheckHelpDocumentsReadOnlyScanBoundaries(t *testing.T) {
 	for _, expected := range []string{
 		"migrate check [path]",
 		"does not require a Go module and never modifies source files",
+		"final collecting FOR and legacy stdlib calls in encoding, crypto, path, object, datetime, and math",
+		"manual review",
+		"local function or use alias collisions",
+		"Arrays and rand/range are outside this pass",
+		"migrate run --print",
 		"include testdata, hidden and underscore-prefixed directories, and nested Go modules",
 		"skip .git, .hg, .svn, vendor, and node_modules",
 		"currently only v1",

cmd/internal/migrate/run.go

@@ -26,6 +26,10 @@ func newRunCommand(store *config.Store, service Service) *cobra.Command {
 			"directories, and nested Go modules. The selected directory itself is scanned regardless of its name. " +
 			"Directory symlinks are not followed.\n\n" +
 			"FQL migration returns and canonically formats a structurally recognized final top-level FOR. " +
+			"It also migrates safe legacy stdlib calls to encoding, crypto, path, object, datetime, and math namespaces. " +
+			"Ambiguous join, non-unary keys, date_compare, date_diff, aggregate math, and calls affected by " +
+			"local function or use alias collisions are left for manual follow-up. " +
+			"Arrays and rand/range are outside this pass. Already-qualified calls are preserved. " +
 			"Malformed FQL is left unchanged and reported for manual follow-up.\n\n" +
 			"The command performs only documented mechanical import, dependency, and source changes. " +
 			"It does not convert application logic or arbitrary APIs to native Ferret v2 equivalents.",

cmd/internal/migrate/stdlib_test.go

@@ -0,0 +1,146 @@
+package migrate
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/MontFerret/cli/v2/internal/migration"
+)
+
+func TestMigrateRunStdlibPreviewAndApply(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "query.fql")
+	before := "return [abs(-2), average(xs)]"
+	want := "return [math::abs(-2), average(xs)]"
+	if err := os.WriteFile(path, []byte(before), 0o600); err != nil {
+		t.Fatal(err)
+	}
+
+	service := migration.New(nil)
+	for _, mode := range []string{"--dry-run", "--print", "apply"} {
+		args := []string{"run", path}
+		if mode != "apply" {
+			args = append(args, mode)
+		}
+
+		stdout, stderr, err := executeMigrateCommand(t, service, args...)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if !strings.Contains(stderr, "query.fql:1: average(...)") || !strings.Contains(stderr, "heterogeneous") {
+			t.Fatalf("missing manual warning in mode %s: %s", mode, stderr)
+		}
+
+		if mode == "--print" {
+			if !strings.HasPrefix(stdout, "--- a/query.fql\n+++ b/query.fql\n") ||
+				!strings.Contains(stdout, "+"+want) || strings.Contains(stdout, "Manual follow-up") ||
+				strings.Contains(stdout, "Scanned") || !strings.Contains(stderr, "Scanned 1 FQL file") {
+				t.Fatalf("print mixed diff and diagnostics: stdout=%q stderr=%q", stdout, stderr)
+			}
+		} else if !strings.Contains(stdout, "query.fql") || strings.Contains(stdout, "average(...)") {
+			t.Fatalf("unexpected human output: %s", stdout)
+		}
+
+		contents, err := os.ReadFile(path)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		expected := before
+		if mode == "apply" {
+			expected = want
+		}
+
+		if string(contents) != expected {
+			t.Fatalf("mode %s contents = %q, want %q", mode, contents, expected)
+		}
+	}
+
+	stdout, stderr, err := executeMigrateCommand(t, service, "run", "--print", path)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if stdout != "" || !strings.Contains(stderr, "average(...)") || !strings.Contains(stderr, "No safe automatic changes available") {
+		t.Fatalf("manual-only second run: stdout=%q stderr=%q", stdout, stderr)
+	}
+}
+
+func TestMigrateRunStdlibHelp(t *testing.T) {
+	stdout, _, err := executeMigrateCommand(t, new(fakeMigrationService), "run", "--help")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	for _, text := range []string{"safe legacy stdlib", "encoding, crypto, path, object, datetime, and math", "local function or use alias", "Arrays and rand/range"} {
+		if !strings.Contains(stdout, text) {
+			t.Fatalf("help does not contain %q: %s", text, stdout)
+		}
+	}
+}
+
+func TestMigrateCheckStdlibReportsFindingsWithoutWriting(t *testing.T) {
+	tests := []struct {
+		name, input, diagnostic string
+	}{
+		{
+			name:  "has replacement",
+			input: `return has({ foo: "bar" }, "baz")`,
+			diagnostic: ":1:8: Legacy stdlib call `has` should use `object::has_key`.\n" +
+				"  help: Preview automatic replacements with `ferret migrate run --print`.\n\n",
+		},
+		{
+			name:  "manual review only",
+			input: "return average(xs)",
+			diagnostic: ":1:8: Stdlib call `average` needs manual review.\n" +
+				"  help: legacy aggregate behavior is permissive for heterogeneous collections; " +
+				"the strict math API requires a separate semantic migration\n\n",
+		},
+		{
+			name:  "canonical source",
+			input: `return object::has_key({ foo: "bar" }, "baz")`,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			path := filepath.Join(t.TempDir(), "query.fql")
+			if err := os.WriteFile(path, []byte(test.input), 0o600); err != nil {
+				t.Fatal(err)
+			}
+
+			workingDirectory, err := os.Getwd()
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			displayPath, err := filepath.Rel(workingDirectory, path)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			for range 2 {
+				stdout, stderr, err := executeMigrateCommand(t, migration.New(nil), "check", path)
+				if test.diagnostic == "" {
+					if err != nil || stderr != "" || stdout != "✓ No v1 compatibility issues found in 1 FQL file.\n" {
+						t.Fatalf("unexpected clean check: stdout=%q stderr=%q err=%v", stdout, stderr, err)
+					}
+				} else if err == nil || err.Error() != "Found 1 v1 compatibility issue in 1 of 1 FQL file." ||
+					stdout != "" || stderr != filepath.ToSlash(displayPath)+test.diagnostic {
+					t.Fatalf("unexpected check finding: stdout=%q stderr=%q err=%v", stdout, stderr, err)
+				}
+
+				contents, err := os.ReadFile(path)
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if string(contents) != test.input {
+					t.Fatalf("check changed source: %q", contents)
+				}
+			}
+		})
+	}
+}

internal/migration/compatibility.go

@@ -59,7 +59,7 @@ func checkFQLCompatibility(ctx context.Context, options CompatibilityOptions) (*
 		}
 
 		src := source.New(displayPath, string(data))
-		loop, err := finalTopLevelFQLFor(src)
+		program, err := parseFQLSource(src)
 		if err != nil {
 			detail, line, column := fqlDiagnosticDetails(src, err)
 			result.Diagnostics = append(result.Diagnostics, CompatibilityDiagnostic{
@@ -74,6 +74,14 @@ func checkFQLCompatibility(ctx context.Context, options CompatibilityOptions) (*
 			continue
 		}
 
+		stdlib, err := checkFQLStdlib(src, program)
+		if err != nil {
+			return nil, fmt.Errorf("locate compatibility issue in %s: %w", displayPath, err)
+		}
+
+		result.Diagnostics = append(result.Diagnostics, stdlib...)
+
+		loop := finalFQLFor(program)
 		if loop == nil {
 			continue
 		}

internal/migration/compatibility_stdlib.go

@@ -0,0 +1,46 @@
+package migration
+
+import (
+	"fmt"
+
+	"github.com/MontFerret/ferret/v2/pkg/parser/fql"
+	"github.com/MontFerret/ferret/v2/pkg/source"
+)
+
+func checkFQLStdlib(src source.Source, program *fql.ProgramContext) ([]CompatibilityDiagnostic, error) {
+	var diagnostics []CompatibilityDiagnostic
+	for _, finding := range analyzeFQLStdlib(program) {
+		name := finding.name
+		spelling := name.GetText()
+
+		span, ok := fqlByteSpan(src.Content(), source.Span{
+			Start: name.GetStart().GetStart(),
+			End:   name.GetStop().GetStop() + 1,
+		})
+		if !ok || span.End <= span.Start {
+			return nil, fmt.Errorf("locate stdlib call %s in Ferret source", spelling)
+		}
+
+		position := src.PositionAt(span)
+		if position.Line == 0 || position.Column == 0 {
+			return nil, fmt.Errorf("resolve stdlib call %s location", spelling)
+		}
+
+		diagnostic := CompatibilityDiagnostic{
+			Path:    src.Name(),
+			Message: fmt.Sprintf("Legacy stdlib call `%s` should use `%s`.", spelling, finding.target),
+			Help:    "Preview automatic replacements with `ferret migrate run --print`.",
+			Line:    position.Line,
+			Column:  position.Column,
+			Kind:    CompatibilityDiagnosticIssue,
+		}
+		if finding.reason != "" {
+			diagnostic.Message = fmt.Sprintf("Stdlib call `%s` needs manual review.", spelling)
+			diagnostic.Help = finding.reason
+		}
+
+		diagnostics = append(diagnostics, diagnostic)
+	}
+
+	return diagnostics, nil
+}