From bc350ddc04acd8874073508d8771d8a086acc43b Mon Sep 17 00:00:00 2001 From: Anton Date: Sat, 19 Sep 2026 18:57:43 +0200 Subject: [PATCH] feat(lang): add a `while` special form MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rayfall had no way to stop iterating before the end of a sequence. Every iteration primitive — map, pmap, fold, fold-left/right, scan, scan-left/right, prior — consumes its whole input, and there was no loop construct, so a "repeat until done" loop had to be written as a fold over a fixed range whose full length was paid on every call however early the work finished. `return` did not help: it exits the lambda, not the iteration, so the fold kept calling the lambda for every remaining element. Recursion was not an alternative either, since there is no TCE and the stack tops out around 1-2k frames. (while cond body...) evaluates cond, and while it is truthy evaluates each body expression in order, then tests again. It always returns null — a statement form run for effect, and a never-taken loop has no last value to report. Zero body expressions is legal, so a condition with side effects can be the whole loop; that is the shape a drain wants, where there is no sequence to iterate over and the range was only ever scaffolding. Implemented on both evaluators, which must agree: - ray_while_fn (eval.c), registered beside `if` and `do`. - an sf_while case in the bytecode compiler emitting JMPF over a backward JMP — the first backward branch the compiler produces. The VM's op_jmp already checked for a pending interrupt when the displacement is negative (dormant until now), so a runaway loop is Ctrl-C-able for free. Compiling the body inline, rather than letting it fall through to the generic special-form path, is what keeps `return` working inside a loop: it reaches the sf_return case and unwinds the lambda instead of degrading to the tree walker's identity `return`. Neither path pushes a scope around the body. `let` binds only in the top frame (env_bind_local), so a per-iteration frame would discard loop-carried `let` state on the tree-walking path while the compiled path — whose `let` writes a function-level slot — kept it, and the two evaluators would disagree on the same source. A caller wanting a fresh frame per pass writes (while cond (do ...)), which composes. patch_jump and emit_jump_back now share write_jump_offset; an out-of-range displacement still sets c->error, dropping the lambda to the interpreter. Measured on the drain shape from the issue (4-step drain under a 4096 safety bound, release build, driver baseline subtracted): fold-left over (til 4096) 1420 us/batch, the nested 64x64 workaround 45.8 us/batch, `while` 3.8 us/batch — 374x over the original and 12x over the workaround. Closes #588 --- docs/docs/language/control-flow.md | 51 +++++++++++ docs/docs/reference/all-functions.md | 3 +- src/lang/compile.c | 60 ++++++++++++- src/lang/env.c | 2 +- src/lang/eval.c | 42 +++++++++ src/lang/eval.h | 1 + test/rfl/lang/while.rfl | 129 +++++++++++++++++++++++++++ 7 files changed, 283 insertions(+), 5 deletions(-) create mode 100644 test/rfl/lang/while.rfl diff --git a/docs/docs/language/control-flow.md b/docs/docs/language/control-flow.md index 23628d5ec..8cf4d7c69 100644 --- a/docs/docs/language/control-flow.md +++ b/docs/docs/language/control-flow.md @@ -30,6 +30,57 @@ Without an else branch, `if` returns `0`: 30 ``` +## Iteration: while + +`while` evaluates `cond`, and while it is truthy evaluates each body expression +in order, then tests again. It always returns null — it is a statement form, +run for effect. + +```text +‣ (set n 5) +‣ (set total 0) +‣ (while (> n 0) (set total (+ total n)) (set n (- n 1))) +‣ total +15 +``` + +It is the only iteration form that can stop early. `map`, `fold`, `scan` and +`prior` all consume their whole input, so a "repeat until done" loop written as +a fold over a fixed range pays that range's full length on every call, however +early the work finishes. `while` stops when the condition says stop, allocates +no range, and does not recurse — so it is not bounded by the stack depth a +recursive loop would hit. + +The body may be omitted, in which case a condition with side effects is the +whole loop. That is the natural shape when there is no sequence to iterate over +in the first place: + +```text +‣ (while (drain-one-batch)) +``` + +Unlike `do`, `while` pushes no scope of its own. A `let` in the body binds in +the enclosing frame and therefore survives the iteration, which is what makes a +`let` usable as a loop variable inside a lambda: + +```lisp +((fn [n] + (let i 0) + (let acc 0) + (while (< i n) (let acc (+ acc i)) (let i (+ i 1))) + acc) 4) ; => 6 +``` + +When a fresh binding per pass is wanted instead, wrap the body in `do`, which +does push a scope: + +```lisp +(while (< i 3) (do (let tmp (* i i)) (use tmp)) (set i (+ i 1))) +``` + +A loop whose condition never goes false runs until interrupted; Ctrl-C breaks +out of one at the REPL. + ## Variable Binding: set and let `set` creates a global binding. `let` creates a local binding scoped to the enclosing `do`: diff --git a/docs/docs/reference/all-functions.md b/docs/docs/reference/all-functions.md index 85975352c..fff747dea 100644 --- a/docs/docs/reference/all-functions.md +++ b/docs/docs/reference/all-functions.md @@ -12,7 +12,7 @@ |---|---|---| | [Arithmetic](#arithmetic) (24) | [Comparison](#comparison) (7) | [Logic](#logic) (3) | | [Aggregation](#aggregation) (25) | [Higher-Order](#higher-order) (13) | [Collection](#collection) (41) | -| [Sorting & Ordering](#sorting) (10) | [Control Flow & Special Forms](#control) (11) | [Table Operations](#table-ops) (20) | +| [Sorting & Ordering](#sorting) (10) | [Control Flow & Special Forms](#control) (12) | [Table Operations](#table-ops) (20) | | [Query](#query) (4) | [Joins](#joins) (7) | [Pivot](#pivot) (1) | | [String](#string-ops) (11) | [Temporal](#temporal) (3) | [Type & Introspection](#type-ops) (5) | | [I/O & Output](#io) (12) | [System & Utility](#system) (15) | [Serialization](#serialization) (2) | @@ -359,6 +359,7 @@ Special forms receive their arguments unevaluated. These are the core language p | `let` | binary | special | Bind value to local variable (lexical scope) | `(let y (+ x 1))` | | `if` | variadic | special | Conditional: (if cond then else) | `(if (> x 0) "pos" "neg")` | | `do` | variadic | special | Sequential execution, returns last value | `(do (set x 1) (set y 2) (+ x y))` | +| `while` | variadic | special | Iterate while cond is truthy; returns null | `(while (> n 0) (set n (- n 1)))` | | `fn` | variadic | special | Create lambda function | `(fn [x y] (+ x y))` | | `try` | binary | special | Error handling: (try expr handler-fn-or-fallback-value) | `(try (/ 1 0) (fn [e] 0))` | | `raise` | unary | — | Throw an error with message | `(raise "bad input")` | diff --git a/src/lang/compile.c b/src/lang/compile.c index 5384ff31f..a20ad8390 100644 --- a/src/lang/compile.c +++ b/src/lang/compile.c @@ -183,16 +183,39 @@ static int32_t emit_jump(compiler_t *c, uint8_t opcode) { return patch_pos; } -static void patch_jump(compiler_t *c, int32_t pos) { - int32_t raw = c->code_len - pos - 2; +/* Write a jump displacement into the 2-byte operand at `pos`, which is + * relative to the instruction's end (pos + 2). Out of int16_t range sets + * c->error, aborting bytecode emission so the lambda falls back to the + * tree-walking interpreter — the same graceful degradation as the other + * compile-time bailouts. */ +static void write_jump_offset(compiler_t *c, int32_t pos, int32_t target) { + int32_t raw = target - pos - 2; if (raw > 32767 || raw < -32768) { c->error = true; return; } int16_t offset = (int16_t)raw; c->code[pos] = (uint8_t)((uint16_t)offset >> 8); c->code[pos + 1] = (uint8_t)(offset & 0xFF); } +/* Resolve a forward jump emitted earlier, now that its target is here. */ +static void patch_jump(compiler_t *c, int32_t pos) { + write_jump_offset(c, pos, c->code_len); +} + +/* Emit an unconditional jump BACKWARD to an already-emitted address — the + * mirror of patch_jump, where the target is known up front and the + * displacement comes out negative. The VM's op_jmp checks for a pending + * interrupt whenever the offset is negative, so a runaway loop built from + * this stays Ctrl-C-able. */ +static void emit_jump_back(compiler_t *c, int32_t target) { + emit(c, OP_JMP); + int32_t pos = c->code_len; + emit(c, 0); + emit(c, 0); + write_jump_offset(c, pos, target); +} + /* Cached sym IDs for special forms */ -static _Thread_local int64_t sf_set = -1, sf_let = -1, sf_if = -1, sf_do = -1, sf_fn = -1, sf_self = -1, sf_try = -1, sf_return = -1, sf_null = -1; +static _Thread_local int64_t sf_set = -1, sf_let = -1, sf_if = -1, sf_do = -1, sf_while = -1, sf_fn = -1, sf_self = -1, sf_try = -1, sf_return = -1, sf_null = -1; static _Thread_local int64_t sf_eval = -1, sf_resolve = -1; static void init_sf_syms(void) { @@ -201,6 +224,7 @@ static void init_sf_syms(void) { sf_let = ray_sym_intern("let", 3); sf_if = ray_sym_intern("if", 2); sf_do = ray_sym_intern("do", 2); + sf_while= ray_sym_intern("while", 5); sf_fn = ray_sym_intern("fn", 2); sf_self = ray_sym_intern("self", 4); sf_try = ray_sym_intern("try", 3); @@ -369,6 +393,36 @@ static void compile_list(compiler_t *c, ray_t *ast) { return; } + /* (while cond body...) — the only backward branch the compiler + * emits. Body values are discarded (OP_POP each), and the form + * yields null however many times it iterated, matching + * ray_while_fn. No scope is pushed: `let` in the body writes this + * frame's local slot, so loop-carried state survives the pass + * exactly as it does on the tree-walking path. + * + * Compiling the body inline — rather than letting this fall through + * to the generic special-form path below — is what keeps `return` + * working inside a loop: it reaches the sf_return case and unwinds + * the lambda, instead of degrading to the tree walker's identity + * `return` (issue 588). */ + if (sym_id == sf_while && n >= 2) { + int32_t top = c->code_len; + compile_expr(c, elems[1]); + /* Truthiness belongs to the materialized value, not to the + * non-NULL lazy handle containing it — same rule as sf_if. */ + emit(c, OP_FORCE); + int32_t jmpf_pos = emit_jump(c, OP_JMPF); + for (int64_t i = 2; i < n; i++) { + compile_expr(c, elems[i]); + emit(c, OP_POP); + } + emit_jump_back(c, top); + patch_jump(c, jmpf_pos); + int32_t idx = add_constant(c, RAY_NULL_OBJ); + emit_const(c, idx); + return; + } + /* (fn [params] body...) — nested lambda via dynamic eval */ if (sym_id == sf_fn && n >= 3) { if (ast_refs_locals(c, ast)) { diff --git a/src/lang/env.c b/src/lang/env.c index a4356f630..886bbd47a 100644 --- a/src/lang/env.c +++ b/src/lang/env.c @@ -839,7 +839,7 @@ int32_t ray_env_list_user(int64_t* sym_ids, ray_t** vals, int32_t max_entries) { /* ---- Prefix lookup ---- */ static const char* s_keywords[] = { - "def", "do", "false", "fn", "if", "let", "set", "true", NULL + "def", "do", "false", "fn", "if", "let", "set", "true", "while", NULL }; /* Compare helper for qsort on const char* */ diff --git a/src/lang/eval.c b/src/lang/eval.c index d72841d44..defce74d0 100644 --- a/src/lang/eval.c +++ b/src/lang/eval.c @@ -1926,6 +1926,47 @@ ray_t* ray_do_fn(ray_t** args, int64_t n) { return result; } +/* (while cond body...) — iterate while cond is truthy. Receives + * unevaluated args. Always returns null: it is a statement form run for + * effect, and a never-taken loop has no last value to report. + * + * Zero body expressions is legal — a condition with side effects is then + * the whole loop, which is the shape a "repeat until done" drain wants + * (issue 588): there is no sequence to iterate, so folding over a range + * was only ever scaffolding. + * + * Unlike ray_do_fn this pushes NO scope around the body. `let` binds in + * the top frame only (env_bind_local), so a per-iteration frame would + * discard loop-carried `let` state here while the compiled path — whose + * `let` writes a function-level bytecode slot — kept it, and the two + * evaluators would disagree on the same source. A caller wanting a fresh + * frame per pass writes (while cond (do ...)), which composes. + * + * No interrupt check is needed: the condition goes through ray_eval on + * every pass, whose entry guard raises `cancel` when a Ctrl-C has landed. + * The compiled form is emitted by the bytecode compiler — see compile.c. */ +ray_t* ray_while_fn(ray_t** args, int64_t n) { + if (n < 1) return ray_error("domain", "while: expected at least 1 arg (cond), got %lld", (long long)n); + for (;;) { + ray_t* cond = ray_eval(args[0]); + if (RAY_IS_ERR(cond)) return cond; + /* Materialize lazy handles before testing truthiness — the + * truthiness belongs to the value, not to the non-NULL handle + * that happens to contain it (same rule as ray_cond_fn). */ + if (ray_is_lazy(cond)) + cond = ray_lazy_materialize(cond); + if (RAY_IS_ERR(cond)) return cond; + int truthy = is_truthy(cond); + ray_release(cond); + if (!truthy) return RAY_NULL_OBJ; + for (int64_t i = 1; i < n; i++) { + ray_t* val = ray_eval(args[i]); + if (RAY_IS_ERR(val)) return val; + ray_release(val); + } + } +} + /* ══════════════════════════════════════════ * Lambda functions * ══════════════════════════════════════════ */ @@ -3149,6 +3190,7 @@ static void ray_register_builtins(void) { register_binary("let", RAY_FN_SPECIAL_FORM, ray_let_fn); register_vary("if", RAY_FN_SPECIAL_FORM, ray_cond_fn); register_vary("do", RAY_FN_SPECIAL_FORM, ray_do_fn); + register_vary("while", RAY_FN_SPECIAL_FORM, ray_while_fn); register_vary("fn", RAY_FN_SPECIAL_FORM, ray_fn); /* Aggregation builtins */ diff --git a/src/lang/eval.h b/src/lang/eval.h index d52e67b3e..0630700cf 100644 --- a/src/lang/eval.h +++ b/src/lang/eval.h @@ -363,6 +363,7 @@ ray_t* ray_set_fn(ray_t* name_obj, ray_t* val_expr); ray_t* ray_let_fn(ray_t* name_obj, ray_t* val_expr); ray_t* ray_cond_fn(ray_t** args, int64_t n); ray_t* ray_do_fn(ray_t** args, int64_t n); +ray_t* ray_while_fn(ray_t** args, int64_t n); ray_t* ray_fn(ray_t** args, int64_t n); ray_t* ray_raise_fn(ray_t* val); ray_t* ray_try_fn(ray_t* expr, ray_t* handler_expr); diff --git a/test/rfl/lang/while.rfl b/test/rfl/lang/while.rfl new file mode 100644 index 000000000..8b1f24cef --- /dev/null +++ b/test/rfl/lang/while.rfl @@ -0,0 +1,129 @@ +;; (while cond body...) — the language's only early-terminating loop. +;; +;; Motivation (issue 588): every iteration primitive is exhaustive, so a +;; "repeat until done" loop had to be a fold over a fixed range whose full +;; length was paid on every call. `while` stops when the condition says so. +;; +;; Two implementations must agree: the tree walker (ray_while_fn in eval.c) +;; and the bytecode compiler (the sf_while case in compile.c, which emits a +;; backward OP_JMP). Every behavioural assertion below is therefore made +;; twice where it can be — once at top level (tree walk) and once inside a +;; lambda (compiled). The `let`-as-loop-variable cases are the ones that +;; would catch the two paths drifting apart: `let` binds in the top scope +;; frame (env.c env_bind_local), so a per-iteration scope push here would +;; make loop-carried `let` state work compiled and hang interpreted. + +;; ── basic iteration ────────────────────────────────────────────────── +(set i 0) +(set n 0) +(while (< i 5) (set n (+ n 1)) (set i (+ i 1))) +i -- 5 +n -- 5 + +;; a statement form: always null, whatever the body evaluated to +(set i 0) +(nil? (while (< i 2) (set i (+ i 1)) 'discarded)) -- true + +;; zero-trip loop: body never runs, result is still null +(set x 99) +(nil? (while false (set x 1))) -- true +x -- 99 + +;; a false-from-the-start condition that is an expression, not a literal +(set g 0) +(while (> g 0) (set g (+ g 1))) +g -- 0 + +;; ── body-less form ─────────────────────────────────────────────────── +;; The condition alone drives the loop — there is no sequence to fold over, +;; which is the shape the issue's drain loop actually wanted. +(set k 0) +(while (do (set k (+ k 1)) (< k 4))) +k -- 4 + +;; ── arity ──────────────────────────────────────────────────────────── +(while) !- domain + +;; ── termination is condition-driven, not bounded by a range ────────── +;; The step runs exactly as many times as there is work. This is precisely +;; what the fold-left workaround could not do: it called the lambda for all +;; DRAIN_STEPS elements however early the drain finished. +(set work 3) +(set calls 0) +(set step (fn [] (set calls (+ calls 1)) (set work (- work 1)) (> work 0))) +(set more true) +(while more (set more (step))) +calls -- 3 +work -- 0 + +;; ── locals: `let` as loop-carried state ────────────────────────────── +;; Compiled path: `let` writes a function-level slot, so it survives the +;; iteration. If the interpreter ever pushes a per-iteration scope, the +;; same source hangs instead — which is why this is asserted both ways. +((fn [n] (let i 0) (let acc 0) (while (< i n) (let acc (+ acc i)) (let i (+ i 1))) acc) 4) -- 6 + +;; the loop condition reads a lambda PARAM (the special_form_locals hazard) +((fn [n] (let c 0) (while (< c n) (let c (+ c 1))) c) 3) -- 3 + +;; tree-walk path, same arithmetic, via globals +(set i 0) +(set acc 0) +(while (< i 4) (set acc (+ acc i)) (set i (+ i 1))) +acc -- 6 + +;; a lambda-local loop must not leak its names to the global env +((fn [] (let scratch 7) (while false (let scratch 8)) scratch)) -- 7 +scratch !- name + +;; ── `return` from inside a body exits the enclosing lambda ─────────── +;; The compiled `while` must route its body through the normal sf_return +;; path. If `while` fell back to a dynamic tree-walk eval, `return` would +;; degrade to identity and the loop would run all ten iterations. +(set hits 0) +((fn [] (let i 0) (while (< i 10) (set hits (+ hits 1)) (if (> i 2) (return 42)) (let i (+ i 1))) 7)) -- 42 +hits -- 4 + +;; without the early return the same loop runs to completion +(set hits 0) +((fn [] (let i 0) (while (< i 10) (set hits (+ hits 1)) (let i (+ i 1))) 7)) -- 7 +hits -- 10 + +;; ── errors ─────────────────────────────────────────────────────────── +;; An error raised in the body propagates out and stops the loop — note the +;; loop would otherwise never terminate, so `c` pins that it stopped on the +;; FIRST pass. `raise` reports as a bare `domain` error and carries its +;; payload on __VM->raise_val (eval.c ray_raise_fn), so the payload is +;; asserted through `try`, which is the only way to observe it. +(set c 0) +(== (try (while true (set c (+ c 1)) (raise 'boom)) (fn [e] e)) 'boom) -- true +c -- 1 + +;; an error raised in the condition propagates before the body ever runs +(set c 0) +(== (try (while (raise 'bad-cond) (set c (+ c 1))) (fn [e] e)) 'bad-cond) -- true +c -- 0 + +;; same, unwinding out of a compiled lambda +(== (try ((fn [] (while true (raise 'inner)))) (fn [e] e)) 'inner) -- true + +;; a plain (non-raise) error in the body also terminates the loop +(set c 0) +(while true (set c (+ c 1)) (fold-right +)) !- domain +c -- 1 + +;; ── nesting ────────────────────────────────────────────────────────── +(set total 0) +(set i 0) +(while (< i 3) (set j 0) (while (< j 2) (set total (+ total 1)) (set j (+ j 1))) (set i (+ i 1))) +total -- 6 + +;; nested, compiled, with let-locals on both levels +((fn [] (let i 0) (let t 0) (while (< i 3) (let j 0) (while (< j 2) (let t (+ t 1)) (let j (+ j 1))) (let i (+ i 1))) t)) -- 6 + +;; ── explicit per-iteration isolation still composes ────────────────── +;; `while` does not push a scope; `do` does. Wrapping the body in `do` +;; gives a fresh frame per pass when that is what you want. +(set out (list)) +(set i 0) +(while (< i 3) (do (let sq (* i i)) (set out (concat out sq))) (set i (+ i 1))) +out -- (list 0 1 4)