Repository navigation
Recursively interpret Core.eval expressions - #783
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #783 +/- ##
==========================================
+ Coverage 90.83% 90.86% +0.02%
==========================================
Files 12 12
Lines 3535 3579 +44
==========================================
+ Hits 3211 3252 +41
- Misses 324 327 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
aviatesk
force-pushed
the
avi/module-interpretation
branch
from
October 3, 2026 10:45
152f442 to
5e5d4cd
Compare
aviatesk
force-pushed
the
avi/recurse-eval
branch
from
October 3, 2026 10:51
a1db24b to
0018f56
Compare
aviatesk
force-pushed
the
avi/recurse-eval
branch
3 times, most recently
from
October 3, 2026 11:45
c834403 to
24d6182
Compare
aviatesk
added a commit
that referenced
this pull request
Oct 3, 2026
`:s` on a statement of a top-level driver frame, as created by `Frame(mod, ex)` for a `:toplevel` or `:module` expression, entered the callee of the surface call expression directly. Since the surface expression is not lowered, this failed when the call's arguments contained calls (e.g. `f(g(1))` threw "unknown call f introduced by ccall lowering g"), and it skipped the rest of the statement, such as the assignment of `x = f(1)`. Such frames come only from `Frame(mod, ex)` called at the root, which debugging sessions rarely do, but #783 interprets the code passed to `Core.eval` in such frames, so `:s` inside `@eval`ed or `include`d code takes this path too. The two changes are independent of each other, but stepping into evaluated code with `:s` needs both. Now `:s` enters the lowered frame of the statement like a callee, before any of the statement's side effects, and stops at its first call. As when stepping into a method, it steps through the statements before that call: global declarations, which are builtin calls on some Julia versions (`Core.set_binding_type!` for typed globals on 1.10-1.11, and `Core.declare_global` on 1.13+), `:latestworld`, and global loads. The handling of method frames is factored out together into `maybe_step_through_prelude!`. `:si` stops at the entry of the lowered frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aviatesk
added a commit
that referenced
this pull request
Oct 3, 2026
`:s` on a statement of a top-level driver frame, as created by `Frame(mod, ex)` for a `:toplevel` or `:module` expression, entered the callee of the surface call expression directly. Since the surface expression is not lowered, this failed when the call's arguments contained calls (e.g. `f(g(1))` threw "unknown call f introduced by ccall lowering g"), and it skipped the rest of the statement, such as the assignment of `x = f(1)`. Such frames come only from `Frame(mod, ex)` called at the root, which debugging sessions rarely do, but #783 interprets the code passed to `Core.eval` in such frames, so `:s` inside `@eval`ed or `include`d code takes this path too. The two changes are independent of each other, but stepping into evaluated code with `:s` needs both. Now `:s` enters the lowered frame of the statement like a callee, before any of the statement's side effects, and stops at its first call. As when stepping into a method, it steps through the statements before that call: global declarations, which are builtin calls on some Julia versions (`Core.set_binding_type!` for typed globals on 1.10-1.11, and `Core.declare_global` on 1.13+), `:latestworld`, and global loads. The handling of method frames is factored out together into `maybe_step_through_prelude!`. `:si` stops at the entry of the lowered frame. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
aviatesk
force-pushed
the
avi/recurse-eval
branch
from
October 3, 2026 12:33
24d6182 to
d8c7b96
Compare
With `RecursiveInterpreter`, the code passed to `Core.eval`, also through `Base.eval`, `@eval`, and `include`, is now interpreted in top-level frames linked to the calling frame, instead of being evaluated natively. `FrameCode` replaces the runtime call in the method of `Core.eval` with a call to `eval_in_frame`, which `evaluate_eval!` handles. Like the compilation of `llvmcall`s, this is done regardless of `optimize`, so that debugger commands can step into the evaluation also with `optimize=false`. Breakpoints, stepping, and exception handling extend into the evaluated code: the calling frame's handlers are passed down (`caller_will_catch_err`), and interpreted dynamic scopes reach both the evaluation and the lowering of the code. Evaluation starts in the latest world while the calling method keeps its own, and `:module` expressions go through the native module evaluation of `Frame`. `NonRecursiveInterpreter`, or adding the method of `Core.eval` to `compiled_methods`, keeps native evaluation. Top-level frames now follow native world semantics: on Julia 1.12 and later they advance their world only at `:latestworld`, so a nested eval does not by itself advance the enclosing thunk's world. Thunks start in the latest world, which internally created ones take when they are created, and which `Frame(mod, ex)` and `Frame(mod, src::CodeInfo)` now use by default. Selective evaluation must still advance the world at the `:latestworld` statements it skips, which LoweredCodeUtils does in a coordinated release. A top-level frame with a caller, e.g. from an eval in a method, runs in its own world even when a debugger resumes it with a method at the root; the task world is switched only when it differs, to keep stepping fast. Related fixes to top-level interpretation: - A statement that fails to lower throws the error of native evaluation. - The frame of a nested `:thunk` is linked to its caller, so breakpoints, errors, and its value propagate (including a `:method` returned by older lowering). - A `ReturnNode` value as a surface statement is not taken for the driver frame's own return. - `get_return` of a driver frame returns `nothing` if its last statement recorded no value, as when an `Interpreter` overload handles statements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aviatesk
force-pushed
the
avi/recurse-eval
branch
from
October 3, 2026 12:38
d8c7b96 to
b8925b8
Compare
Merged
aviatesk
referenced
this pull request
Oct 3, 2026
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 3, 2026
aviatesk
added a commit
to aviatesk/JETLS.jl
that referenced
this pull request
Oct 4, 2026
JuliaInterpreter 0.12 interprets the code passed to `Core.eval` (JuliaDebug/JuliaInterpreter.jl#783), and JET 0.12.3 builds on it, so full analysis of scripts, notebooks and test files now interprets `@eval`ed code like other top-level code: methods and structs defined by it are analyzed, `concretization_timeout` can stop it, and the stack traces of timeout and error reports extend into it. The new JET also reports malformed module expressions generated by macros as top-level errors instead of aborting the analysis. This commit updates JET to the JETLS branch rebased onto JET 0.12.3, and Revise and TestRunner to revisions that allow JuliaInterpreter 0.12. Revise-based package analysis and package source analysis run `Core.eval` natively and are unaffected. The documentation and schema descriptions of `concretization_timeout` no longer list `Core.eval` among the native calls that cannot be interrupted, and name the `__init__` functions of modules it evaluates instead. The concretization timeout test now expects the interpreted frames of the `@eval`ed `sleep` in script mode. New tests cover diagnostics in methods and structs defined by `@eval`, and malformed module expressions generated by macros. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aviatesk
added a commit
to aviatesk/JETLS.jl
that referenced
this pull request
Oct 4, 2026
JuliaInterpreter 0.12 interprets the code passed to `Core.eval` (JuliaDebug/JuliaInterpreter.jl#783), and JET 0.12.3 builds on it, so full analysis of scripts, notebooks and test files now interprets `@eval`ed code like other top-level code: methods and structs defined by it are analyzed, `concretization_timeout` can stop it, and the stack traces of timeout and error reports extend into it. The new JET also reports malformed module expressions generated by macros as top-level errors instead of aborting the analysis. This commit updates JET to the JETLS branch rebased onto JET 0.12.3, and Revise and TestRunner to revisions that allow JuliaInterpreter 0.12. Revise-based package analysis and package source analysis run `Core.eval` natively and are unaffected. The documentation and schema descriptions of `concretization_timeout` no longer list `Core.eval` among the native calls that cannot be interrupted, and name the `__init__` functions of modules it evaluates instead. The concretization timeout test now expects the interpreted frames of the `@eval`ed `sleep` in script mode. New tests cover diagnostics in methods and structs defined by `@eval`, and malformed module expressions generated by macros. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
RecursiveInterpreter, the code passed toCore.eval, also throughBase.eval,@eval, andinclude, is now interpreted in top-level frames linked to the calling frame, instead of being evaluated natively.FrameCodereplaces the runtime call in the method ofCore.evalwith a call toeval_in_frame, whichevaluate_eval!handles. Like the compilation ofllvmcalls, this is done regardless ofoptimize, so that debugger commands can step into the evaluation also withoptimize=false. Breakpoints, stepping, and exception handling extend into the evaluated code: the calling frame's handlers are passed down (caller_will_catch_err), and interpreted dynamic scopes reach both the evaluation and the lowering of the code. Evaluation starts in the latest world while the calling method keeps its own, and:moduleexpressions go through the native module evaluation ofFrame.NonRecursiveInterpreter, or adding the method ofCore.evaltocompiled_methods, keeps native evaluation.Top-level frames now follow native world semantics: on Julia 1.12 and later they advance their world only at
:latestworld, so a nested eval does not by itself advance the enclosing thunk's world. Thunks start in the latest world, which internally created ones take when they are created, and whichFrame(mod, ex)andFrame(mod, src::CodeInfo)now use by default. Selective evaluation must still advance the world at the:latestworldstatements it skips, which LoweredCodeUtils does in a coordinated release (xref: JuliaDebug/LoweredCodeUtils.jl#171). A top-level frame with a caller, e.g. from an eval in a method, runs in its own world even when a debugger resumes it with a method at the root; the task world is switched only when it differs, to keep stepping fast.Related fixes to top-level interpretation:
:thunkis linked to its caller, so breakpoints, errors, and its value propagate (including a:methodreturned by older lowering).ReturnNodevalue as a surface statement is not taken for the driver frame's own return.get_returnof a driver frame returnsnothingif its last statement recorded no value, as when anInterpreteroverload handles statements.Stepping with
:sinside the evaluated code goes through statements of top-level frames driven byFrame(mod, ex), which builds on the fix of:sfor them in #784.