diff --git a/README.md b/README.md index c4bb494..f5f9e7f 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,55 @@ -# libkore — the Kore application standard library - -`0.1.0-alpha.1` is the first **Rust foundation**, replacing the aspirational -scaffold on `alpha`. It provides a real filesystem API, common prelude, native -runtime macros, typed sleep, process facilities, automatically seeded random -values, hash collections and an opt-in driver substrate. -The `kore-ffi` package adds generated C bindings and a translated Zig module for -filesystem requests, clocks/sleep, random values, async task preparation, -joined thread/SPMD scopes and independent native create/join/detach, driven by -a Rust-founded host. It also exposes -opaque hash tables, C allocation functions and a Zig `std.mem.Allocator` -adapter over the application's installed heap. -It is `no_std`; all authority comes from the application's actual endowments. +# libkore — application libraries for Kore + +libkore `0.1.0-alpha.1` provides `no_std` libraries for programs running under +the [Kore capability microkernel](https://github.com/MicroPerceptron/kore). +It is an application facade, not a kernel package or a hosted operating-system +runtime. Filesystem, clocks, process operations and devices use capabilities +actually endowed to the process; selecting a Cargo feature grants none of them. +The libraries do not install a global allocator or start a second executor. + +| Package | Use | +| --- | --- | +| `kore-std` | Rust facade: filesystem, prelude, time, random, collections, process and optional driver APIs | +| `kore-entry` | Executable-only adapter for a returning Rust `fn main()` on Kore | +| `kore-ffi` | C headers and Zig module over a Rust-founded host, including filesystem, time, random, allocation and worker APIs; no standalone C/Zig CRT | +| `kore-lua` | Embeddable Lua 5.5 engine and bridge; interactive and opt-in restricted worker policies | +| `kore-wasi` | WASI 0.2 component host composed by an embedder with its Wasmtime engine and native owners | + +All five package manifests set `publish = false`; consume a reviewed Git +revision or a deliberately composed local checkout, not a registry version. +The WASI host supports the `wasi:cli/command` world and native-backed I/O, +filesystem, clocks and random imports; unsupported imports such as sockets +fail at instantiation. Its embedder configures the Wasmtime engine. +For this PR's source revision, name the `kore-std` dependency `kore` to avoid +confusion with Kore's distinct kernel package: ```toml [dependencies] -kore = { package = "kore-std", git = "https://github.com/MicroPerceptron/libkore", branch = "alpha" } +kore = { package = "kore-std", git = "https://github.com/MicroPerceptron/libkore", rev = "06cddb199eee59739841887bca843d8ce493de7b" } +``` + +The pinned revision is a source dependency, not an ABI stability promise. Choose +a newer reviewed commit when updating. See [local integration](docs/integration.md) +before combining this Git source with packages from a local Kore checkout: +Cargo treats otherwise identical path and Git dependencies as different owners. + +## First use + +The collection facade can be exercised in a host test. A Kore process needs +its founded HEAP supply before an allocating collection can do the same: + +```rust +# use kore_std as kore; +use kore::collections::HashMap; +let mut values = HashMap::new(); +values.insert("ready", 1); +assert_eq!(values["ready"], 1); ``` -That branch is the integration destination; the alpha implementation is available -there after its PR merges. Pin a reviewed commit for reproducible applications. -The package name avoids collision with Kore's kernel package, also named `kore`. +Native service calls require an attached client, an executor and the relevant +endowed face; they are not host emulations. For example, given an owned DagFS +view and an attached filesystem client, this helper writes its complete input +starting at offset zero: ```rust # use kore_std as kore; @@ -35,9 +65,10 @@ async fn store(fs: &mut Fs<'_, '_>, file: &View, bytes: &[u8]) ``` Writes do not implicitly truncate, sync, or atomically replace a file. Use the -native `Fs::resize` and sync operations explicitly. Ordinary operations build on -DagFS; graph/revision/grant facilities remain under `kore::fs::dag`. See -[the filesystem contract](https://github.com/MicroPerceptron/libkore/blob/alpha/docs/filesystem.md) for partial writes and cleanup. +native `Fs::resize` and sync operations explicitly. An error reports the +acknowledged prefix, not a safe replay offset. Ordinary operations build on +DagFS; graph/revision/grant facilities remain under `kore::fs::dag`. See the +[filesystem contract](docs/filesystem.md) for partial writes and cleanup. | Feature | Effect | | --- | --- | @@ -62,27 +93,41 @@ slots. `kore::exit_codes!` names those slots without accepting arbitrary integer native exit/join preserves the full signed 32-bit status. See [native exit outcomes](docs/runtime.md#native-exit-outcomes). -See [C and Zig allocation](docs/allocators.md), [hash collections](docs/collections.md), [time and concurrency](docs/runtime.md) -and [random values](docs/random.md) for API examples, admission and failure contracts. +See [C and Zig allocation](docs/allocators.md), [hash collections](docs/collections.md), +[time and concurrency](docs/runtime.md), [random values](docs/random.md), +[Lua modes](docs/lua.md), and the [WASI host](crates/wasi/src/lib.rs) for API examples, +admission and failure contracts. ## Build and integration -The toolchain, dependency locks and native source revision are pinned. A normal -clone needs no sibling checkout, private cross-repository token or submodule setup: +The pinned Rust toolchain is `nightly-2026-07-13`. Kore executables use the +architecture-specific JSON user targets in `targets/` (x86_64, aarch64 and +riscv64gc), with `-Z build-std=core,alloc,compiler_builtins` and +`-Z build-std-features=compiler-builtins-mem`. A normal clone needs no sibling +checkout or private cross-repository token. Start with host-facing checks: ```sh cargo fetch --locked +cargo test --locked -p kore-std +cargo test --locked -p kore-lua +``` + +For the full source/target matrix, install **official Zig 0.16.0 release** and +`cbindgen 0.29.4` using [language integration](docs/languages.md), then run: + +```sh make fmt test check docs owners cross make miri bindings-check ``` -For the full workspace checks, install the pinned C/Zig generation tools as -described in [language integration](https://github.com/MicroPerceptron/libkore/blob/alpha/docs/languages.md). Ordinary Rust dependency builds do not need them. +The host tests exercise Rust logic and the linked foreign/Lua harnesses; they +do not prove founded service behavior or physical-device operation. Ordinary +Rust dependency builds do not need the C/Zig generation tools. The roughly 5 MiB `native/` export contains 14 required packages, including optional and test dependencies, with unchanged source paths and manifests. It contains no kernel or boot images. `make native-check` detects source drift. For work beside -a local Kore tree, follow [local integration](https://github.com/MicroPerceptron/libkore/blob/alpha/docs/integration.md). +a local Kore tree, follow [local integration](docs/integration.md). ## Alpha boundary @@ -91,27 +136,29 @@ a local Kore tree, follow [local integration](https://github.com/MicroPerceptron | Rust | Working foundation with native ownership types | | C | Generated filesystem, time, random, task and joined worker APIs | | Zig | Target-specific C translation and generated module namespaces | -| Lua | Lua 5.5 on a worker thread, with `io`/`os` over the task's own filesystem | +| Lua | Embeddable Lua 5.5: interactive `io`/`os` over the task's filesystem, or an opt-in restricted read-only worker | +| WASI | Embedder-composed WASI 0.2 component host; no standalone runner | | Python | Planned handwritten ownership/scheduling adapter | Filesystem support remains a requirement for **every initial language surface**. A clocks-only foreign binding would not satisfy that requirement. The C/Zig owner uses scoped generational handles and bounded copied buffers; callbacks return to the native executor for each request. The `kore-lua` package carries the pinned Lua 5.5 engine, its freestanding host and the `kore.*` -standard library a script sees; `io` and `os` are written over the task's own -filesystem rather than vendored, because both of Lua's are libc over POSIX and -this target has neither. `os.execute` and `io.popen` are absent, and arrive -through `Extension` like `kore.accel`: both launch a process, which is a service -face the embedding program holds. See +standard library a script sees. Interactive `io` and `os` use the task's own +filesystem; `os.execute` and `io.popen` are absent. The restricted worker +opens only pure libraries and read-only `kore.fs` queries, with local quotas; +its embedder must provide a read-only filesystem face and enforce Stop/deadlines +by terminating the whole task. See [Lua modes](docs/lua.md) and [`crates/lua/KORE-VENDOR.md`](crates/lua/KORE-VENDOR.md) for the provenance, the selected libraries and every place the manual leaves a detail to the system. -Standalone foreign entry/CRT, allocator adapters and the bounded Lua driver -experiment interface remain follow-on work. See [the language contract](https://github.com/MicroPerceptron/libkore/blob/alpha/docs/languages.md) for ownership, cancellation and validation boundaries. +Standalone foreign entry/CRT and a bounded Lua driver experiment interface +remain follow-on work. See [the language contract](docs/languages.md) for +ownership, cancellation and validation boundaries. The previous sketches and address/collection experiments remain in Git history at `8a6381c`; their parallel runtime, allocator placeholder and empty language -headers are no longer part of the active library. See [provenance](https://github.com/MicroPerceptron/libkore/blob/alpha/docs/provenance.md) -for upstream dependencies, licensing records and validation boundaries. +headers are no longer part of the active library. See [provenance](docs/provenance.md) +for upstream dependencies and validation boundaries. Pthread-shaped C/Zig lifecycle calls use a process-lifetime native controller; @@ -134,3 +181,13 @@ within their process. `kore::crash` exposes bounded evidence collection, text rendering, and explicit parent-side persistence. Reports carry a bounded task name and opaque 128-bit public identity; IDs convey no authority. See [crash reports](docs/crash.md) for lifecycle ordering and validation limits. + +## Contributing and licenses + +Read [AGENTS.md](AGENTS.md) for ownership and verification rules before changing +code, and [integration](docs/integration.md) for the single native owner graph. +The repository retains its [MPL-2.0 license](./LICENSE); the exported native +sources retain their [upstream notice](native/LICENSE), and adapted Kore files +retain the [source notice](docs/KORE-LICENSE). The Lua vendor record is +[here](crates/lua/KORE-VENDOR.md). See [provenance](docs/provenance.md) for +the precise source and validation boundaries. diff --git a/crates/lua/build.rs b/crates/lua/build.rs index 9208316..14a054a 100644 --- a/crates/lua/build.rs +++ b/crates/lua/build.rs @@ -4,6 +4,7 @@ use std::{ fs, path::{Path, PathBuf}, process::{Command, ExitStatus}, + time::{Duration, Instant}, }; const SUPPORTED_ZIG_MAJOR: u64 = 0; @@ -40,6 +41,32 @@ fn run(mut command: Command, what: &str) -> ExitStatus { status } +fn run_supervised(binary: &Path, name: &str, argument: Option<&str>, timeout: Duration) { + let mut command = Command::new(binary); + if let Some(argument) = argument { + command.arg(argument); + } + let mut child = command + .spawn() + .unwrap_or_else(|error| panic!("could not start supervised Lua {name}: {error}")); + let deadline = Instant::now() + timeout; + loop { + match child + .try_wait() + .expect("could not observe supervised Lua test") + { + Some(status) if status.success() => return, + Some(status) => panic!("Lua {name} failed with {status}"), + None if Instant::now() < deadline => std::thread::sleep(Duration::from_millis(10)), + None => { + let _ = child.kill(); + let _ = child.wait(); + panic!("Lua {name} exceeded its supervised deadline"); + } + } + } +} + fn zig_command(zig: &OsStr, out: &Path) -> Command { let mut command = Command::new(zig); command @@ -86,9 +113,18 @@ fn host_smoke(out: &Path, host: &str) -> bool { .args(["src/kore_lua.c", "tests/host_smoke.c", "-lm", "-o"]) .arg(&host_smoke); run(smoke_cc, "native Lua protected-call smoke compilation"); - run( - Command::new(&host_smoke), - "native Lua protected-call smoke execution", + run_supervised(&host_smoke, "host smoke", None, Duration::from_secs(30)); + run_supervised( + &host_smoke, + "quota-adversary", + Some("quota-adversary"), + Duration::from_secs(2), + ); + run_supervised( + &host_smoke, + "teardown-adversary", + Some("teardown-adversary"), + Duration::from_secs(2), ); // Compile the C engine a second time without its libc numeric oracle, then @@ -152,9 +188,11 @@ fn host_smoke(out: &Path, host: &str) -> bool { .arg(format!("link-arg={}", target_smoke_object.display())) .args(["-C", "link-arg=-lm"]); run(shim_rustc, "native Lua target-shim smoke compilation"); - run( - Command::new(&target_shim_smoke), - "native Lua target-shim smoke execution", + run_supervised( + &target_shim_smoke, + "target-shim smoke", + None, + Duration::from_secs(30), ); true diff --git a/crates/lua/src/bridge.rs b/crates/lua/src/bridge.rs index 713bc9e..fb0e246 100644 --- a/crates/lua/src/bridge.rs +++ b/crates/lua/src/bridge.rs @@ -73,7 +73,7 @@ use kore::runtime::{ use kore_late::{LoadError, Loader, Search}; use crate::vm::{EvalResult, ReplResult, Vm}; -use crate::{Answer, Extension, OP_EMBEDDER_BASE}; +use crate::{Answer, Extension, OP_EMBEDDER_BASE, Policy}; /// The bound on a source module (`.lua`). Sized like a generous /// script, not like a library: the whole buffer is staged in the task's @@ -181,6 +181,8 @@ use core::ffi::c_void; #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Outcome { Ok, + /// Lua source failed to compile; stderr includes the chunk and line. + CompileFailed, Failed, /// The chunk ran `os.exit`. The sentinel it raised unwound every /// `` handler on the way out, so this is a completed evaluation @@ -292,7 +294,7 @@ unsafe fn payload<'a>() -> &'a mut Payload { /// The worker's entry: build the VM, then serve evaluations until `QUIT`. fn vm_thread() { - let Some(mut vm) = Vm::new(E::LIBRARIES) else { + let Some(mut vm) = Vm::new_with_policy(E::LIBRARIES, E::POLICY) else { // SAFETY: the worker owns the cell while the word reads EVAL_DONE // and thread 0 has not yet posted (it posts only after spawn). unsafe { payload().outcome = Outcome::Startup }; @@ -342,6 +344,7 @@ fn run_eval(vm: &mut Vm) { KIND_REPL => match vm.eval_repl(script) { ReplResult::Complete(true) => Outcome::Ok, ReplResult::Complete(false) => Outcome::Failed, + ReplResult::CompileFailed => Outcome::CompileFailed, ReplResult::Exited => Outcome::Exited, ReplResult::Incomplete => Outcome::Incomplete, ReplResult::ResourceRefused => Outcome::ResourceRefused, @@ -354,7 +357,9 @@ fn run_eval(vm: &mut Vm) { unsafe { core::slice::from_raw_parts(cell.chunk_name, cell.chunk_name_len) }; match vm.eval(script, name) { EvalResult::Ok => Outcome::Ok, + EvalResult::CompileFailed => Outcome::CompileFailed, EvalResult::Failed => Outcome::Failed, + EvalResult::BudgetExceeded => Outcome::ResourceRefused, EvalResult::Exited => Outcome::Exited, } } @@ -908,21 +913,21 @@ async fn serve_call(services: &mut Services<'_, '_>, extension: &m // SAFETY: CALL_POSTED. let op = unsafe { payload() }.op; if op == OP_REQUIRE { - serve_require(services).await; + serve_require(services, E::POLICY).await; } else { - serve_value(services, extension, op).await; + serve_value(services, extension, op, E::POLICY).await; } } /// Answer the `require` the cell holds: read the source, or link the /// native module and look its opener up. -async fn serve_require(services: &mut Services<'_, '_>) { +async fn serve_require(services: &mut Services<'_, '_>, policy: Policy) { let (name, len) = { // SAFETY: CALL_POSTED. let cell = unsafe { payload() }; (cell.name, cell.name_len) }; - let reply = resolve(services, &name[..len.min(FS_NAME_BYTES)]).await; + let reply = resolve(services, &name[..len.min(FS_NAME_BYTES)], policy).await; // SAFETY: CALL_POSTED. let cell = unsafe { payload() }; match reply { @@ -949,7 +954,22 @@ async fn serve_require(services: &mut Services<'_, '_>) { /// Answer every value-returning op. The outcome is written back as a /// status, up to two words, and a text; nothing here panics on a refusal. -async fn serve_value(services: &mut Services<'_, '_>, extension: &mut E, op: u32) { +async fn serve_value( + services: &mut Services<'_, '_>, + extension: &mut E, + op: u32, + policy: Policy, +) { + if policy.is_restricted() + && !matches!( + op, + OP_FS_READ_FILE | OP_FS_STAT | OP_FS_LIST | OP_FS_ENTRIES | OP_FS_RIGHTS + ) + { + return finish_answer(Answer::Unavailable(alloc::string::String::from( + "host operation unavailable to read-only worker", + ))); + } let args = { // SAFETY: CALL_POSTED. unsafe { payload() }.args @@ -976,7 +996,7 @@ async fn serve_value(services: &mut Services<'_, '_>, extension: & OP_FS_READ_FILE | OP_FS_SAVE | OP_FS_ENTRIES | OP_FS_RIGHTS ) { - let answer = serve_fs(services, op, args, input).await; + let answer = serve_fs(services, op, args, input, policy).await; return finish_answer(answer); } let answer = match op { @@ -1028,6 +1048,7 @@ async fn serve_fs( op: u32, args: [u64; 3], input: &[u8], + policy: Policy, ) -> Answer { // Split the borrow: `files` and `fs` are distinct fields, and every // verb below needs both at once. @@ -1050,7 +1071,9 @@ async fn serve_fs( OP_FS_OPEN => crate::files::open(fs, files, input, args[0], args[1]).await, OP_FS_CLOSE => crate::files::close(fs, files, args[0]).await, OP_FS_READ => crate::files::read(fs, files, args[0], args[1], args[2]).await, - OP_FS_READ_FILE => crate::files::read_file(fs, input).await, + OP_FS_READ_FILE => { + crate::files::read_file_with_limit(fs, input, policy.fs_result_bytes()).await + } OP_FS_SAVE => crate::files::save(fs, files, args[0], input).await, OP_FS_WRITE => crate::files::write(fs, files, args[0], args[1], input).await, OP_FS_SYNC => crate::files::sync(fs, files, args[0]).await, @@ -1063,8 +1086,10 @@ async fn serve_fs( Some(cut) => crate::files::rename(fs, &input[..cut], &input[cut + 1..]).await, None => Answer::Refused(alloc::string::String::from("invalid-name")), }, - OP_FS_LIST => crate::files::list(fs, input).await, - OP_FS_ENTRIES => crate::files::entries(fs, input).await, + OP_FS_LIST => crate::files::list_with_limit(fs, input, policy.fs_result_bytes()).await, + OP_FS_ENTRIES => { + crate::files::entries_with_limit(fs, input, policy.fs_result_bytes()).await + } OP_FS_RIGHTS => crate::files::rights(fs, input).await, _ => Answer::Unavailable(alloc::string::String::from("unknown filesystem call")), } @@ -1086,16 +1111,17 @@ enum Reply { Error(&'static CStr), } -async fn resolve(services: &mut Services<'_, '_>, stem: &[u8]) -> Reply { +async fn resolve(services: &mut Services<'_, '_>, stem: &[u8], policy: Policy) -> Reply { let search = Search { prefixes: SEARCH }; let Some(fs) = services.fs().await else { return Reply::Error(c"filesystem face is unavailable"); }; let mut buf = Vec::new(); - if buf.try_reserve_exact(MAX_SOURCE_MODULE_BYTES).is_err() { + let source_limit = MAX_SOURCE_MODULE_BYTES.min(policy.fs_result_bytes()); + if buf.try_reserve_exact(source_limit).is_err() { return Reply::Error(c"source buffer allocation refused"); } - buf.resize(MAX_SOURCE_MODULE_BYTES, 0); + buf.resize(source_limit, 0); match Loader::read_source(fs, search, stem, &mut buf).await { Ok(len) => { buf.truncate(len); @@ -1105,6 +1131,9 @@ async fn resolve(services: &mut Services<'_, '_>, stem: &[u8]) -> Reply { Err(error) => return Reply::Error(describe(error)), } drop(buf); + if policy.is_restricted() { + return Reply::Error(c"source-only worker cannot load native modules"); + } if services.loader.is_none() { match Loader::new(services.rt) { Ok(loader) => services.loader = Some(loader), diff --git a/crates/lua/src/files.rs b/crates/lua/src/files.rs index 885d8a4..6ca0b4d 100644 --- a/crates/lua/src/files.rs +++ b/crates/lua/src/files.rs @@ -596,11 +596,17 @@ pub async fn read( /// Bounded by `MAX_FILE_BYTES` (16 MiB), refused as `range` rather than /// grown without a ceiling. pub async fn read_file(fs: &mut Fs<'_, '_>, path: &[u8]) -> Answer { + read_file_with_limit(fs, path, MAX_FILE_BYTES).await +} + +/// Read a whole file with the caller's additional result ceiling. +pub async fn read_file_with_limit(fs: &mut Fs<'_, '_>, path: &[u8], limit: usize) -> Answer { + let limit = limit.min(MAX_FILE_BYTES); let mut bytes: Vec = Vec::new(); let mut overflow = false; let read = fs .read_file(path, |chunk| { - if bytes.len() + chunk.len() > MAX_FILE_BYTES { + if chunk.len() > limit.saturating_sub(bytes.len()) { overflow = true; return Err(Refusal::Fs(FsError::Range)); } @@ -873,12 +879,18 @@ pub async fn rename(fs: &mut Fs<'_, '_>, from: &[u8], to: &[u8]) -> Answer { /// identity would be wrong in a way nothing would tell it. The stable /// identity is [`stat`]'s, where it is named as such. pub async fn entries(fs: &mut Fs<'_, '_>, path: &[u8]) -> Answer { + entries_with_limit(fs, path, MAX_READ_BYTES).await +} + +/// Enumerate directory edges with the caller's additional result ceiling. +pub async fn entries_with_limit(fs: &mut Fs<'_, '_>, path: &[u8], limit: usize) -> Answer { + let limit = limit.min(MAX_READ_BYTES); let mut records: Vec = Vec::new(); let mut overflow = false; let listed = fs .list_directory(path, |entry| { let name = entry.name.as_bytes(); - if records.len() + name.len() + 3 > MAX_READ_BYTES { + if name.len().saturating_add(3) > limit.saturating_sub(records.len()) { overflow = true; return Err(Refusal::Fs(FsError::Range)); } @@ -928,13 +940,19 @@ pub async fn rights(fs: &mut Fs<'_, '_>, path: &[u8]) -> Answer { /// kind of thing a script iterates once, and paging it would put a second /// piece of mutable state in the handle table for no gain at these sizes. pub async fn list(fs: &mut Fs<'_, '_>, path: &[u8]) -> Answer { + list_with_limit(fs, path, MAX_READ_BYTES).await +} + +/// Enumerate directory names with the caller's additional result ceiling. +pub async fn list_with_limit(fs: &mut Fs<'_, '_>, path: &[u8], limit: usize) -> Answer { + let limit = limit.min(MAX_READ_BYTES); let mut names: Vec = Vec::new(); let mut overflow = false; let listed = fs .list_directory(path, |entry| { // Stop at the cap rather than growing without bound; the // caller is told by `Range` that the listing was not complete. - if names.len() + entry.name.len() + 1 > MAX_READ_BYTES { + if entry.name.len().saturating_add(1) > limit.saturating_sub(names.len()) { overflow = true; return Err(Refusal::Fs(FsError::Range)); } diff --git a/crates/lua/src/kore_lua.c b/crates/lua/src/kore_lua.c index 6131331..2848cbd 100644 --- a/crates/lua/src/kore_lua.c +++ b/crates/lua/src/kore_lua.c @@ -2228,6 +2228,18 @@ static const luaL_Reg kore_lua_fs_lib[] = { {NULL, NULL} }; +/* Only verbs whose bridge arm cannot mutate the store. The worker receives + a READ-only face as a second, independent authority boundary. */ +static const luaL_Reg kore_lua_fs_read_only_lib[] = { + {"read_file", kore_lua_fs_read_file}, + {"stat", kore_lua_fs_stat}, + {"entries", kore_lua_fs_entries}, + {"rights", kore_lua_fs_rights}, + {"exists", kore_lua_fs_exists}, + {"list", kore_lua_fs_list}, + {NULL, NULL} +}; + /* * ----------------------------------------------------------------- io * @@ -3081,27 +3093,32 @@ static int luaopen_kore(lua_State *L) { const kore_lua_library *libraries = (const kore_lua_library *)lua_touserdata(L, lua_upvalueindex(1)); size_t count = (size_t)lua_tointeger(L, lua_upvalueindex(2)); + int restricted = lua_toboolean(L, lua_upvalueindex(3)); lua_newtable(L); /* the kore table */ lua_newtable(L); /* the caps array, in registration order */ lua_Integer listed = 0; - luaL_newlib(L, kore_lua_time_lib); - kore_lua_module(L, "time", &listed); - - luaL_newlib(L, kore_lua_random_lib); - kore_lua_module(L, "random", &listed); + if (!restricted) { + luaL_newlib(L, kore_lua_time_lib); + kore_lua_module(L, "time", &listed); + luaL_newlib(L, kore_lua_random_lib); + kore_lua_module(L, "random", &listed); + } - luaL_newlib(L, kore_lua_fs_lib); + if (restricted) luaL_newlib(L, kore_lua_fs_read_only_lib); + else luaL_newlib(L, kore_lua_fs_lib); kore_lua_module(L, "fs", &listed); - luaL_newlib(L, kore_lua_proc_lib); - lua_newtable(L); - lua_pushcfunction(L, kore_lua_proc_index); - lua_setfield(L, -2, "__index"); - lua_setmetatable(L, -2); - kore_lua_module(L, "proc", &listed); + if (!restricted) { + luaL_newlib(L, kore_lua_proc_lib); + lua_newtable(L); + lua_pushcfunction(L, kore_lua_proc_index); + lua_setfield(L, -2, "__index"); + lua_setmetatable(L, -2); + kore_lua_module(L, "proc", &listed); + } - for (size_t i = 0; libraries != NULL && i < count; i++) { + for (size_t i = 0; !restricted && libraries != NULL && i < count; i++) { const kore_lua_library *library = &libraries[i]; if (library->name == NULL || library->opener == NULL) continue; lua_pushcfunction(L, library->opener); @@ -3245,6 +3262,7 @@ static int luaopen_kore_os(lua_State *L) { } static int kore_lua_open_libraries(lua_State *L) { + int restricted = lua_toboolean(L, 3); luaL_requiref(L, LUA_GNAME, luaopen_base, 1); /* Omit inert stdio/warning APIs and the unverified binary-chunk loader. */ static const char *const omitted_base[] = { @@ -3253,11 +3271,28 @@ static int kore_lua_open_libraries(lua_State *L) { for (const char *const *name = omitted_base; *name != NULL; name++) { lua_pushnil(L); lua_setfield(L, -2, *name); } + if (restricted) { + /* Lua runs xpcall's message handler while the count hook is disabled, + and __gc finalizers run with hooks disabled too. There is no safe + in-process instruction ceiling for either callback. A restricted VM + has no way to catch the count-hook error or install a finalizer. Keep + the unrestricted REPL's base library intact. */ + static const char *const unsafe_callbacks[] = { + "pcall", "xpcall", "setmetatable", "getmetatable", "rawset", + "collectgarbage", NULL + }; + for (const char *const *name = unsafe_callbacks; *name != NULL; name++) { + lua_pushnil(L); lua_setfield(L, -2, *name); + } + } /* The one loader the host offers: over the store, through the executor. */ lua_pushcfunction(L, kore_lua_require); lua_setfield(L, -2, "require"); lua_pop(L, 1); - luaL_requiref(L, LUA_COLIBNAME, luaopen_coroutine, 1); lua_pop(L, 1); + /* Coroutines do not inherit the main state's instruction hook. */ + if (!restricted) { + luaL_requiref(L, LUA_COLIBNAME, luaopen_coroutine, 1); lua_pop(L, 1); + } luaL_requiref(L, LUA_TABLIBNAME, luaopen_table, 1); lua_pop(L, 1); luaL_requiref(L, LUA_STRLIBNAME, luaopen_string, 1); /* No partial printf and no producer for unsupported binary chunks. */ @@ -3267,9 +3302,11 @@ static int kore_lua_open_libraries(lua_State *L) { luaL_requiref(L, LUA_MATHLIBNAME, luaopen_math, 1); lua_pop(L, 1); luaL_requiref(L, LUA_UTF8LIBNAME, luaopen_utf8, 1); lua_pop(L, 1); /* The file object's metatable, before anything that can create one. */ - kore_lua_file_metatable(L); - luaL_requiref(L, LUA_IOLIBNAME, luaopen_kore_io, 1); lua_pop(L, 1); - luaL_requiref(L, LUA_OSLIBNAME, luaopen_kore_os, 1); lua_pop(L, 1); + if (!restricted) { + kore_lua_file_metatable(L); + luaL_requiref(L, LUA_IOLIBNAME, luaopen_kore_io, 1); lua_pop(L, 1); + luaL_requiref(L, LUA_OSLIBNAME, luaopen_kore_os, 1); lua_pop(L, 1); + } /* kore's own standard library, provisioned from this process's faces. One table rather than a scatter of globals: `require "kore"` answers it, and an embedder's own libraries sit beside them. Registered by @@ -3278,7 +3315,8 @@ static int kore_lua_open_libraries(lua_State *L) { requiref does once the opener has run. */ lua_pushlightuserdata(L, lua_touserdata(L, 1)); lua_pushvalue(L, 2); - lua_pushcclosure(L, luaopen_kore, 2); + lua_pushboolean(L, restricted); + lua_pushcclosure(L, luaopen_kore, 3); lua_call(L, 0, 1); luaL_getsubtable(L, LUA_REGISTRYINDEX, LUA_LOADED_TABLE); lua_pushvalue(L, -2); @@ -3288,20 +3326,52 @@ static int kore_lua_open_libraries(lua_State *L) { return 0; } -lua_State *kore_lua_new(lua_Alloc alloc, void *ud, unsigned seed, - const kore_lua_library *libraries, size_t count) { +lua_State *kore_lua_new_with_policy(lua_Alloc alloc, void *ud, unsigned seed, + const kore_lua_library *libraries, + size_t count, int restricted) { lua_State *L = lua_newstate(alloc, ud, seed); if (L == NULL) return NULL; lua_pushcfunction(L, kore_lua_open_libraries); lua_pushlightuserdata(L, (void *)(uintptr_t)libraries); lua_pushinteger(L, (lua_Integer)count); - if (lua_pcall(L, 2, 0, 0) != LUA_OK) { + lua_pushboolean(L, restricted); + if (lua_pcall(L, 3, 0, 0) != LUA_OK) { lua_close(L); return NULL; } return L; } +lua_State *kore_lua_new(lua_Alloc alloc, void *ud, unsigned seed, + const kore_lua_library *libraries, size_t count) { + return kore_lua_new_with_policy(alloc, ud, seed, libraries, count, 0); +} + +/* The counter lives in the Rust VM, not a process-global. Restricted VMs + omit coroutines so every instruction runs under this one state hook. */ +typedef struct { + unsigned long long remaining; + int exhausted; +} kore_lua_instruction_budget; + +static void kore_lua_instruction_hook(lua_State *L, lua_Debug *ar) { + (void)ar; + kore_lua_instruction_budget *budget = + *(kore_lua_instruction_budget **)lua_getextraspace(L); + if (budget == NULL || budget->remaining == 0) { + if (budget != NULL) budget->exhausted = 1; + luaL_error(L, "kore:instruction-limit"); + } + --budget->remaining; +} + +void kore_lua_set_instruction_budget(lua_State *L, + kore_lua_instruction_budget *budget) { + *(kore_lua_instruction_budget **)lua_getextraspace(L) = budget; + lua_sethook(L, budget != NULL ? kore_lua_instruction_hook : NULL, + LUA_MASKCOUNT, 1); +} + static int kore_lua_install_print_inner(lua_State *L) { const kore_lua_writer *config = (const kore_lua_writer *)lua_touserdata(L, 1); if (config == NULL || config->append == NULL) @@ -3338,6 +3408,25 @@ int kore_lua_call(lua_State *L, int arguments, int results) { return lua_pcall(L, arguments, results, 0); } +/* The restricted tool reports a Lua stack while the failing call's frames + still exist. The message handler itself runs inside lua_pcall's recovery + frame, so even a quota or allocation failure cannot longjmp through Rust. */ +static int kore_lua_traceback(lua_State *L) { + const char *message = lua_type(L, 1) == LUA_TSTRING + ? lua_tostring(L, 1) : "Lua error"; + luaL_traceback(L, L, message, 1); + return 1; +} + +int kore_lua_call_traceback(lua_State *L, int arguments, int results) { + int function = lua_gettop(L) - arguments; + lua_pushcfunction(L, kore_lua_traceback); + lua_insert(L, function); + int status = lua_pcall(L, arguments, results, function); + lua_remove(L, function); + return status; +} + static int kore_lua_print_results_inner(lua_State *L) { int results = lua_gettop(L); lua_getglobal(L, "print"); @@ -3358,6 +3447,14 @@ int kore_lua_print_results(lua_State *L) { return lua_pcall(L, results, 0, 0); } +int kore_lua_print_results_traceback(lua_State *L) { + int results = lua_gettop(L); + if (results == 0) return LUA_OK; + lua_pushcfunction(L, kore_lua_print_results_inner); + lua_insert(L, 1); + return kore_lua_call_traceback(L, results, 0); +} + int kore_lua_top(lua_State *L) { return lua_gettop(L); } void kore_lua_set_top(lua_State *L, int top) { lua_settop(L, top); } int kore_lua_type(lua_State *L, int index) { return lua_type(L, index); } diff --git a/crates/lua/src/lib.rs b/crates/lua/src/lib.rs index db8c279..54387ba 100644 --- a/crates/lua/src/lib.rs +++ b/crates/lua/src/lib.rs @@ -137,6 +137,8 @@ pub mod bridge; pub mod files; #[cfg(target_os = "none")] mod numeric; +#[cfg(any(target_os = "none", test))] +mod output; #[cfg(target_os = "none")] mod shim; #[cfg(target_os = "none")] @@ -209,6 +211,65 @@ pub enum Answer { /// without this crate learning that the library exists. pub const OP_EMBEDDER_BASE: u32 = 64; +/// Opt-in limits for a one-shot, read-only Lua worker. The default keeps the +/// existing interactive REPL surface and limits unchanged. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Policy { + restricted: bool, + heap_bytes: usize, + instructions: u64, + output_bytes: usize, + fs_result_bytes: usize, +} + +impl Policy { + pub const INTERACTIVE: Self = Self { + restricted: false, + heap_bytes: usize::MAX, + instructions: u64::MAX, + output_bytes: usize::MAX, + fs_result_bytes: usize::MAX, + }; + + /// Exposes pure Lua libraries and read-only `kore.fs` queries. `require` + /// can load bounded source modules, never native images. The embedder + /// must also provide a read-only filesystem face to the worker task. + pub const fn read_only_worker( + heap_bytes: usize, + instructions: u64, + output_bytes: usize, + fs_result_bytes: usize, + ) -> Self { + Self { + restricted: true, + heap_bytes, + instructions, + output_bytes, + fs_result_bytes, + } + } + + pub const fn is_restricted(self) -> bool { + self.restricted + } + + pub const fn heap_bytes(self) -> usize { + self.heap_bytes + } + + pub const fn instructions(self) -> u64 { + self.instructions + } + + pub const fn output_bytes(self) -> usize { + self.output_bytes + } + + pub const fn fs_result_bytes(self) -> usize { + self.fs_result_bytes + } +} + /// Libraries and ops the embedding program adds to the VM. /// /// The split is deliberate: this crate owns the engine, the transport and @@ -232,6 +293,10 @@ pub trait Extension { /// Registered beneath `kore` at VM construction, in this order. const LIBRARIES: &'static [Library] = &[]; + /// A one-shot worker may opt into restricted libraries and resource + /// budgets. Existing embedders inherit the interactive policy. + const POLICY: Policy = Policy::INTERACTIVE; + /// Serve one op at or above [`OP_EMBEDDER_BASE`], on the executor /// thread, with the cell's three argument words. fn serve(&mut self, op: u32, args: [u64; 3]) -> impl Future; diff --git a/crates/lua/src/output.rs b/crates/lua/src/output.rs new file mode 100644 index 0000000..0a2e3ad --- /dev/null +++ b/crates/lua/src/output.rs @@ -0,0 +1,114 @@ +//! One aggregate stdout/stderr budget for Lua print and error paths. + +use alloc::vec::Vec; + +#[derive(Default)] +pub(crate) struct Capture { + pub stdout: Vec, + pub stderr: Vec, + pub limit: usize, + pub exhausted: bool, +} + +impl Capture { + fn remaining(&self) -> usize { + self.limit + .saturating_sub(self.stdout.len().saturating_add(self.stderr.len())) + } + + pub fn append(&mut self, bytes: &[u8], stderr: bool) -> bool { + if bytes.len() > self.remaining() { + self.exhausted = true; + return false; + } + let output = if stderr { + &mut self.stderr + } else { + &mut self.stdout + }; + if output.try_reserve(bytes.len()).is_err() { + self.exhausted = true; + return false; + } + output.extend_from_slice(bytes); + true + } + + /// Errors have priority within the same aggregate limit as `print`. + /// Reclaim the tail of stdout if it filled the budget before a failure; + /// otherwise a near-limit print could erase the only diagnosis. Borrow + /// the Lua string and never allocate proportionally to its length. + pub fn append_error(&mut self, error: &[u8]) { + let body = error.strip_suffix(b"\n").unwrap_or(error); + let wanted = body.len().saturating_add(1).min(self.limit); + let reclaim = wanted + .saturating_sub(self.remaining()) + .min(self.stdout.len()); + self.stdout.truncate(self.stdout.len() - reclaim); + let remaining = self.remaining(); + if remaining == 0 { + return; + } + let copied = body.len().min(remaining - 1); + if self.stderr.try_reserve(copied + 1).is_ok() { + self.stderr.extend_from_slice(&body[..copied]); + self.stderr.push(b'\n'); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn long_error_is_truncated_at_policy_limit() { + let mut capture = Capture { + limit: 32, + ..Capture::default() + }; + capture.append_error(&[b'x'; 4096]); + assert_eq!(capture.stderr.len(), 32); + assert_eq!(capture.stderr[31], b'\n'); + assert_eq!(capture.remaining(), 0); + } + + #[test] + fn print_quota_error_reclaims_stdout_within_shared_budget() { + let mut capture = Capture { + limit: 8, + ..Capture::default() + }; + assert!(capture.append(b"hello", false)); + assert!(!capture.append(b"more", false)); + assert!(capture.exhausted); + capture.append_error(b"quota exceeded: a deliberately long error"); + assert!(capture.stdout.is_empty()); + assert_eq!(capture.stderr, b"quota e\n"); + assert_eq!(capture.stdout.len() + capture.stderr.len(), 8); + assert_eq!(capture.stderr[7], b'\n'); + assert_eq!(capture.remaining(), 0); + } + + #[test] + fn nearly_full_stdout_preserves_later_error() { + let mut capture = Capture { + limit: 2048, + ..Capture::default() + }; + assert!(capture.append(&[b'x'; 2047], false)); + capture.append_error(b"=qwen-tool:1: sentinel\nstack traceback:"); + assert!(capture.stderr.windows(8).any(|part| part == b"sentinel")); + assert!(capture.stderr.ends_with(b"\n")); + assert!(capture.stdout.len() + capture.stderr.len() <= 2048); + } + + #[test] + fn zero_budget_has_no_error_escape() { + let mut capture = Capture::default(); + capture.append_error(b"long error"); + assert!(capture.stderr.is_empty()); + assert!(!capture.append(b"x", false)); + assert!(capture.exhausted); + } +} diff --git a/crates/lua/src/shim.rs b/crates/lua/src/shim.rs index aad4cdc..5853d14 100644 --- a/crates/lua/src/shim.rs +++ b/crates/lua/src/shim.rs @@ -8,6 +8,56 @@ use core::{ const LUA_ALIGNMENT: usize = 16; +/// Per-VM live Lua allocation accounting. The callback's `ud` points to +/// this value for the lifetime of the state; no process-wide allocator is +/// installed and host-side bridge buffers remain separately bounded. +pub struct Budget { + pub limit: usize, + pub used: usize, + pub exhausted: bool, +} + +impl Budget { + pub const fn new(limit: usize) -> Self { + Self { + limit, + used: 0, + exhausted: false, + } + } +} + +pub unsafe extern "C" fn allocate_bounded( + ud: *mut c_void, + ptr: *mut c_void, + old_size: usize, + new_size: usize, +) -> *mut c_void { + let Some(budget) = (unsafe { ud.cast::().as_mut() }) else { + return ptr::null_mut(); + }; + let old = if ptr.is_null() { 0 } else { old_size }; + let Some(next) = budget + .used + .checked_sub(old) + .and_then(|used| used.checked_add(new_size)) + else { + budget.exhausted = true; + return ptr::null_mut(); + }; + if next > budget.limit { + budget.exhausted = true; + return ptr::null_mut(); + } + let result = unsafe { allocate(ptr::null_mut(), ptr, old_size, new_size) }; + if new_size == 0 || !result.is_null() { + budget.used = next; + } else { + budget.exhausted = true; + } + result +} + /// Lua's per-state allocator, backed by this task's capability-grown heap. pub unsafe extern "C" fn allocate( _ud: *mut c_void, diff --git a/crates/lua/src/vm.rs b/crates/lua/src/vm.rs index b2ec5d8..2cf8407 100644 --- a/crates/lua/src/vm.rs +++ b/crates/lua/src/vm.rs @@ -8,8 +8,9 @@ use core::{ }; use kore::abi::process::PROCESS_OUTPUT_BYTES; -use crate::Library; +use crate::output::Capture; use crate::shim; +use crate::{Library, Policy}; const LUA_OK: c_int = 0; const LUA_MULTRET: c_int = -1; @@ -41,6 +42,15 @@ unsafe extern "C" { libraries: *const Library, library_count: usize, ) -> *mut LuaState; + fn kore_lua_new_with_policy( + alloc: LuaAlloc, + ud: *mut c_void, + seed: u32, + libraries: *const Library, + library_count: usize, + restricted: c_int, + ) -> *mut LuaState; + fn kore_lua_set_instruction_budget(state: *mut LuaState, budget: *mut InstructionBudget); fn kore_lua_install_print(state: *mut LuaState, append: LuaAppend, ud: *mut c_void) -> c_int; fn kore_lua_load( state: *mut LuaState, @@ -49,17 +59,19 @@ unsafe extern "C" { name: *const c_char, ) -> c_int; fn kore_lua_call(state: *mut LuaState, arguments: c_int, results: c_int) -> c_int; + fn kore_lua_call_traceback(state: *mut LuaState, arguments: c_int, results: c_int) -> c_int; fn kore_lua_print_results(state: *mut LuaState) -> c_int; + fn kore_lua_print_results_traceback(state: *mut LuaState) -> c_int; fn kore_lua_top(state: *mut LuaState) -> c_int; fn kore_lua_set_top(state: *mut LuaState, top: c_int); fn kore_lua_error_string(state: *mut LuaState, len: *mut usize) -> *const c_char; fn kore_lua_close(state: *mut LuaState); } -#[derive(Default)] -struct Capture { - stdout: Vec, - stderr: Vec, +#[repr(C)] +struct InstructionBudget { + remaining: u64, + exhausted: c_int, } unsafe extern "C" fn append( @@ -73,29 +85,12 @@ unsafe extern "C" fn append( return 0; } let bytes = unsafe { slice::from_raw_parts(bytes.cast::(), len) }; - let used = unsafe { - (*capture) - .stdout - .len() - .saturating_add((*capture).stderr.len()) - }; - if bytes.len() > PROCESS_OUTPUT_BYTES.saturating_sub(used) { - return 0; - } - let output = if stderr == 0 { - unsafe { &mut (*capture).stdout } - } else { - unsafe { &mut (*capture).stderr } - }; - if output.try_reserve(bytes.len()).is_err() { - return 0; - } - output.extend_from_slice(bytes); - 1 + i32::from(unsafe { &mut *capture }.append(bytes, stderr != 0)) } pub enum ReplResult { Complete(bool), + CompileFailed, /// The chunk ran `os.exit`. Exited, Incomplete, @@ -105,7 +100,9 @@ pub enum ReplResult { /// How one chunk ended. pub enum EvalResult { Ok, + CompileFailed, Failed, + BudgetExceeded, /// The chunk ran `os.exit`: the sentinel unwound every `` on the /// way out and the recorded code is the answer, not the raise. Exited, @@ -114,25 +111,61 @@ pub enum EvalResult { pub struct Vm { state: NonNull, capture: Box, + budget: Option>, + instructions: Option>, + policy: Policy, } impl Vm { /// Build a VM, registering `libraries` beneath the `kore` table /// after this crate's own modules. - pub fn new(libraries: &'static [Library]) -> Option { - let mut capture = Box::try_new(Capture::default()).ok()?; + pub fn new_with_policy(libraries: &'static [Library], policy: Policy) -> Option { + let mut capture = Box::try_new(Capture { + stdout: Vec::new(), + stderr: Vec::new(), + limit: PROCESS_OUTPUT_BYTES.min(policy.output_bytes()), + exhausted: false, + }) + .ok()?; + let mut budget = if policy.is_restricted() { + Some(Box::try_new(shim::Budget::new(policy.heap_bytes())).ok()?) + } else { + None + }; + let instructions = if policy.is_restricted() { + Some( + Box::try_new(InstructionBudget { + remaining: policy.instructions(), + exhausted: 0, + }) + .ok()?, + ) + } else { + None + }; // Seed from the CPU, not from a constant: this word decides string // hashing and table iteration order, and a fixed one made every // boot's ordering identical. `math.random`'s own seed comes from // the same source, through `luai_makeseed` (`kore_lua.c`). let state = NonNull::new(unsafe { - kore_lua_new( - shim::allocate, - ptr::null_mut(), - crate::bridge::seed(), - libraries.as_ptr(), - libraries.len(), - ) + if let Some(ref mut budget) = budget { + kore_lua_new_with_policy( + shim::allocate_bounded, + (&mut **budget as *mut shim::Budget).cast(), + crate::bridge::seed(), + libraries.as_ptr(), + libraries.len(), + 1, + ) + } else { + kore_lua_new( + shim::allocate, + ptr::null_mut(), + crate::bridge::seed(), + libraries.as_ptr(), + libraries.len(), + ) + } })?; let status = unsafe { kore_lua_install_print( @@ -147,7 +180,13 @@ impl Vm { unsafe { kore_lua_close(state.as_ptr()) }; return None; } - Some(Self { state, capture }) + Some(Self { + state, + capture, + budget, + instructions, + policy, + }) } fn clear_stack(&mut self) { @@ -192,30 +231,36 @@ impl Vm { fn publish_error(&mut self) { let (error, len) = self.error_parts(); - let used = self - .capture - .stdout - .len() - .saturating_add(self.capture.stderr.len()); - let remaining = PROCESS_OUTPUT_BYTES.saturating_sub(used); - if remaining != 0 { - // Always reserve the final byte for one deterministic line - // terminator. An attacker-controlled Lua error therefore cannot - // bypass the aggregate ProcessIo output bound or trigger a large - // temporary allocation while it is being reported. - let error = unsafe { slice::from_raw_parts(error, len) }; - let body = error.strip_suffix(b"\n").unwrap_or(error); - let copied = body.len().min(remaining - 1); - if self.capture.stderr.try_reserve(copied + 1).is_ok() { - self.capture.stderr.extend_from_slice(&body[..copied]); - self.capture.stderr.push(b'\n'); - } - } + // The Lua string stays live until clear_stack below. + self.capture + .append_error(unsafe { slice::from_raw_parts(error, len) }); self.clear_stack(); } + fn budget_hit(&self) -> bool { + self.instructions + .as_ref() + .is_some_and(|budget| budget.exhausted != 0) + || self.capture.exhausted + || self.budget.as_ref().is_some_and(|budget| budget.exhausted) + } + fn execute_loaded(&mut self, print_results: bool) -> EvalResult { - if unsafe { kore_lua_call(self.state.as_ptr(), 0, LUA_MULTRET) } != LUA_OK { + self.capture.exhausted = false; + if let Some(ref mut budget) = self.budget { + budget.exhausted = false; + } + if let Some(ref mut remaining) = self.instructions { + remaining.remaining = self.policy.instructions(); + remaining.exhausted = 0; + unsafe { kore_lua_set_instruction_budget(self.state.as_ptr(), &mut **remaining) }; + } + let call = if self.policy.is_restricted() { + unsafe { kore_lua_call_traceback(self.state.as_ptr(), 0, LUA_MULTRET) } + } else { + unsafe { kore_lua_call(self.state.as_ptr(), 0, LUA_MULTRET) } + }; + if call != LUA_OK { // `os.exit`'s unwind reaches here as an error, because that is // how it ran the `` handlers on its way out. It is not // one, so it is not reported as one. @@ -223,19 +268,33 @@ impl Vm { self.clear_stack(); return EvalResult::Exited; } + let budget_hit = self.budget_hit(); self.publish_error(); - return EvalResult::Failed; + return if budget_hit { + EvalResult::BudgetExceeded + } else { + EvalResult::Failed + }; } - if print_results - && unsafe { kore_lua_top(self.state.as_ptr()) } > 0 - && unsafe { kore_lua_print_results(self.state.as_ptr()) } != LUA_OK - { + let printed = !print_results + || unsafe { kore_lua_top(self.state.as_ptr()) } == 0 + || if self.policy.is_restricted() { + (unsafe { kore_lua_print_results_traceback(self.state.as_ptr()) }) == LUA_OK + } else { + (unsafe { kore_lua_print_results(self.state.as_ptr()) }) == LUA_OK + }; + if !printed { if self.error_is_exit() { self.clear_stack(); return EvalResult::Exited; } + let budget_hit = self.budget_hit(); self.publish_error(); - return EvalResult::Failed; + return if budget_hit { + EvalResult::BudgetExceeded + } else { + EvalResult::Failed + }; } self.clear_stack(); EvalResult::Ok @@ -245,7 +304,7 @@ impl Vm { self.clear_stack(); if self.load(bytes, name) != LUA_OK { self.publish_error(); - EvalResult::Failed + EvalResult::CompileFailed } else { self.execute_loaded(false) } @@ -254,7 +313,9 @@ impl Vm { fn repl_result(result: EvalResult) -> ReplResult { match result { EvalResult::Ok => ReplResult::Complete(true), + EvalResult::CompileFailed => ReplResult::CompileFailed, EvalResult::Failed => ReplResult::Complete(false), + EvalResult::BudgetExceeded => ReplResult::ResourceRefused, EvalResult::Exited => ReplResult::Exited, } } @@ -288,7 +349,7 @@ impl Vm { ReplResult::Incomplete } else { self.publish_error(); - ReplResult::Complete(false) + ReplResult::CompileFailed } } diff --git a/crates/lua/tests/host_smoke.c b/crates/lua/tests/host_smoke.c index fcf57c8..98b810e 100644 --- a/crates/lua/tests/host_smoke.c +++ b/crates/lua/tests/host_smoke.c @@ -18,11 +18,20 @@ typedef struct { const char *name; lua_CFunction opener; } kore_lua_library; extern lua_State *kore_lua_new(lua_Alloc, void *, unsigned, const kore_lua_library *, size_t); +extern lua_State *kore_lua_new_with_policy(lua_Alloc, void *, unsigned, + const kore_lua_library *, size_t, + int); +typedef struct { unsigned long long remaining; int exhausted; } + kore_lua_instruction_budget; +extern void kore_lua_set_instruction_budget(lua_State *, + kore_lua_instruction_budget *); typedef int (*kore_lua_append)(void *, const char *, size_t, int); extern int kore_lua_install_print(lua_State *, kore_lua_append, void *); extern int kore_lua_load(lua_State *, const char *, size_t, const char *); extern int kore_lua_call(lua_State *, int, int); +extern int kore_lua_call_traceback(lua_State *, int, int); extern int kore_lua_print_results(lua_State *); +extern int kore_lua_print_results_traceback(lua_State *); extern void kore_lua_set_top(lua_State *, int); extern void kore_lua_close(lua_State *); /* The sizes the read path turns on, as src/kore_lua.c compiled them. */ @@ -90,6 +99,110 @@ static int run(lua_State *L, const char *source, int print_results) { return !print_results || kore_lua_print_results(L) == LUA_OK; } +static int restricted_smoke(void) { + lua_State *L = kore_lua_new_with_policy(host_alloc, NULL, 0x4b4f5245u, + NULL, 0, 1); + if (L == NULL) return 1; + const char *surface = + "assert(io==nil and os==nil and coroutine==nil)\n" + "assert(pcall==nil and xpcall==nil)\n" + "assert(setmetatable==nil and getmetatable==nil)\n" + "assert(rawset==nil and collectgarbage==nil)\n" + "assert(kore.time==nil and kore.random==nil and kore.proc==nil)\n" + "assert(kore.fs and kore.fs.read_file and kore.fs.stat)\n" + "assert(kore.fs.list and kore.fs.entries and kore.fs.rights)\n" + "assert(kore.fs.open==nil and kore.fs.write_file==nil)\n" + "assert(kore.fs.remove==nil and kore.fs.rename==nil)\n" + "assert(kore.fs.mkdir==nil and kore.fs.resize==nil)\n" + "assert(load==nil and loadfile==nil and dofile==nil)\n"; + if (!run(L, surface, 0)) { kore_lua_close(L); return 2; } + kore_lua_instruction_budget budget = {128, 0}; + kore_lua_set_instruction_budget(L, &budget); + if (kore_lua_load(L, "while true do end", 17, "=quota") != LUA_OK) { + kore_lua_close(L); return 3; + } + if (kore_lua_call(L, 0, LUA_MULTRET) == LUA_OK) { + kore_lua_close(L); return 4; + } + if (!budget.exhausted) { kore_lua_close(L); return 5; } + size_t len = 0; + const char *error = lua_tolstring(L, -1, &len); + if (error == NULL || strstr(error, "kore:instruction-limit") == NULL) { + kore_lua_close(L); return 6; + } + kore_lua_set_top(L, 0); + budget.remaining = 128; + budget.exhausted = 0; + if (!run(L, "assert(6*7==42)", 0)) { + kore_lua_close(L); return 7; + } + kore_lua_set_top(L, 0); + budget.remaining = 128; + budget.exhausted = 0; + static const char print_spin[] = + "print=function() while true do end end; return 42"; + if (kore_lua_load(L, print_spin, sizeof print_spin - 1, + "=print-quota") != LUA_OK || + kore_lua_call_traceback(L, 0, LUA_MULTRET) != LUA_OK || + kore_lua_print_results_traceback(L) == LUA_OK || + !budget.exhausted) { + kore_lua_close(L); return 8; + } + kore_lua_close(L); + return 0; +} + +/* Run under build.rs's hard child-process deadline. If a removed base entry + accidentally comes back, the hostile Lua closure can spin in a callback + whose debug hook is disabled; the build cannot be allowed to hang. */ +static int restricted_adversary(void) { + lua_State *L = kore_lua_new_with_policy(host_alloc, NULL, 0x4b4f5245u, + NULL, 0, 1); + if (L == NULL) return 1; + kore_lua_instruction_budget budget = {128, 0}; + kore_lua_set_instruction_budget(L, &budget); + static const char *const hostile[] = { + "xpcall(function() while true do end end, function() while true do end end)", + "pcall(function() while true do end end)", + "setmetatable({}, {__gc=function() while true do end end}); collectgarbage()", + NULL + }; + for (const char *const *source = hostile; *source != NULL; source++) { + if (kore_lua_load(L, *source, strlen(*source), "=adversary") != LUA_OK) { + kore_lua_close(L); return 2; + } + if (kore_lua_call(L, 0, LUA_MULTRET) == LUA_OK || budget.exhausted) { + kore_lua_close(L); return 3; + } + kore_lua_set_top(L, 0); + } + if (kore_lua_load(L, "while true do end", 17, "=quota") != LUA_OK) { + kore_lua_close(L); return 4; + } + if (kore_lua_call(L, 0, LUA_MULTRET) == LUA_OK || !budget.exhausted) { + kore_lua_close(L); return 5; + } + kore_lua_close(L); + return 0; +} + +static int restricted_teardown_adversary(void) { + lua_State *L = kore_lua_new_with_policy(host_alloc, NULL, 0x4b4f5245u, + NULL, 0, 1); + if (L == NULL) return 1; + /* The chunk itself succeeds. If metatable installation returns to this + restricted surface, close runs a finalizer with hooks disabled and the + supervising build process terminates this child at its deadline. */ + const char *source = + "local t={}\n" + "if setmetatable then setmetatable(t,{__gc=function() while true do end end}) end\n" + "_G.keep=t\n"; + int good = run(L, source, 0); + if (!good) { kore_lua_close(L); return 2; } + kore_lua_close(L); + return 0; +} + static int protected_setup_smoke(void) { struct fault_alloc baseline = {0, 0}; lua_State *L = kore_lua_new(fault_allocator, &baseline, 0x4b4f5245u, NULL, 0); @@ -1098,7 +1211,11 @@ static int streaming_smoke(void) { #if defined(KORE_LUA_USE_TARGET_NUMERIC) int kore_lua_host_smoke(void) { #else -int main(void) { +int main(int argc, char **argv) { + if (argc == 2 && strcmp(argv[1], "quota-adversary") == 0) + return restricted_adversary(); + if (argc == 2 && strcmp(argv[1], "teardown-adversary") == 0) + return restricted_teardown_adversary(); #endif static const char selftest[] = "local co=coroutine.create(function() coroutine.yield(6) end)\n" @@ -1272,6 +1389,8 @@ int main(void) { "print(6*7)\n"; int setup_status = protected_setup_smoke(); if (setup_status != 0) return 10 + setup_status; + int restricted_status = restricted_smoke(); + if (restricted_status != 0) return 120 + restricted_status; /* A store with nothing in it: the selftest below asserts what an absent path answers, and the pass after it fills one in. */ host_fs_reset(""); @@ -1309,6 +1428,19 @@ int main(void) { if (kore_lua_load(L, "error('expected')", 17, "=error") != LUA_OK) return 4; if (kore_lua_call(L, 0, LUA_MULTRET) == LUA_OK) return 5; kore_lua_set_top(L, 0); + static const char traced_script[] = + "local function broken() error('tool failure') end\n" + "broken()\n"; + if (kore_lua_load(L, traced_script, sizeof traced_script - 1, + "=qwen-tool") != LUA_OK) return 14; + if (kore_lua_call_traceback(L, 0, LUA_MULTRET) == LUA_OK) return 15; + { + const char *why = lua_tostring(L, -1); + if (why == NULL || strstr(why, "qwen-tool:1: tool failure") == NULL || + strstr(why, "stack traceback:") == NULL || + strstr(why, "qwen-tool:2") == NULL) return 16; + } + kore_lua_set_top(L, 0); if (!run(L, "return 6*7", 1)) return 6; if (capture.len != 6 || memcmp(capture.bytes + 3, "42\n", 3) != 0) return 7; kore_lua_set_top(L, 0); @@ -1318,6 +1450,20 @@ int main(void) { "return 6*7", 1)) return 8; + kore_lua_set_top(L, 0); + static const char print_error[] = + "print=function() error('print result sentinel') end; return 42"; + if (kore_lua_load(L, print_error, sizeof print_error - 1, + "=print-result") != LUA_OK || + kore_lua_call(L, 0, LUA_MULTRET) != LUA_OK || + kore_lua_print_results_traceback(L) == LUA_OK) + return 17; + { + const char *why = lua_tostring(L, -1); + if (why == NULL || strstr(why, "print result sentinel") == NULL || + strstr(why, "stack traceback:") == NULL || + strstr(why, "print-result:1") == NULL) return 18; + } kore_lua_close(L); return 0; } diff --git a/docs/entry.md b/docs/entry.md index 7256454..9333644 100644 --- a/docs/entry.md +++ b/docs/entry.md @@ -12,7 +12,7 @@ edition-specific prelude: ```toml [dependencies] -std = { package = "kore-entry", git = "https://github.com/MicroPerceptron/libkore", branch = "alpha" } +std = { package = "kore-entry", git = "https://github.com/MicroPerceptron/libkore", rev = "06cddb199eee59739841887bca843d8ce493de7b" } ``` ```rust @@ -23,6 +23,10 @@ fn main() -> Result { } ``` +This is a source-only Git dependency (`publish = false`); choose a newer +reviewed commit when updating, and use the matching Kore runtime described +above. + There is no `no_main`, handwritten `_start`, entry attribute, or application language-item feature. `fn main() {}` also works. The alias does not claim full compatibility with upstream Rust std: Kore's implemented APIs and authority diff --git a/docs/lua.md b/docs/lua.md index 0030ac0..ae98dfe 100644 --- a/docs/lua.md +++ b/docs/lua.md @@ -1,5 +1,89 @@ # Lua on Kore +`kore-lua` is an embeddable Lua 5.5 library, not a hosted `lua` executable. +Its VM, bridge and service calls run in a founded Kore process +(`target_os = "none"`); host builds test the pure Rust logic and linked C +engine harness. The embedding program owns the process, service faces and +executor. + +## Choose a mode + +| Mode | Selection | Script surface | Lifecycle | +| --- | --- | --- | --- | +| Interactive REPL | `Extension::POLICY` default (`Policy::INTERACTIVE`) | `io`, `os`, coroutine and the task's `kore.*` libraries | Embedder may evaluate successive chunks | +| Read-only worker | Opt in with `Policy::read_only_worker(...)` | Pure Lua libraries and read-only `kore.fs` queries | Embedder launches a delegated worker task, then terminates and reaps it on Stop/deadline | + +For example, an embedder can select a restricted policy without registering +additional service operations: + +```rust +use alloc::string::String; +use kore_lua::{Answer, Extension, Policy}; + +struct ReadOnly; +impl Extension for ReadOnly { + const POLICY: Policy = Policy::read_only_worker( + 8 * 1024 * 1024, // live Lua heap bytes per VM + 1_000_000, // instructions per evaluation + 2_048, // combined captured output bytes + 4_096, // bytes in one filesystem result + ); + + async fn serve(&mut self, _op: u32, _args: [u64; 3]) -> Answer { + Answer::Unavailable(String::from("no extension operation delegated")) + } +} +``` + +On Kore, `Host::start::(&transport)` starts the VM worker. The +embedder then awaits `host.eval(&mut services, &mut extension, script, +b"=worker\0", false)`, which returns `(Outcome, stdout, stderr)` on a live +bridge. For a first script, `print(6 * 7)` uses no service capability and +prints `42`; with `repl = false`, a bare `return 6 * 7` does not automatically +print the returned value. `repl = true` selects the interactive +expression-then-statement grammar. Host tests do not establish that a founded +worker has run on hardware. + +## Read-only worker policy + +An embedding program can set `Extension::POLICY` to +`Policy::read_only_worker(heap_bytes, instructions, output_bytes, +fs_result_bytes)`. The default policy remains the existing interactive REPL. +The restricted policy opens pure Lua base, table, string, math and UTF-8 +libraries and a small `kore.fs` table (`read_file`, `stat`, `exists`, `list`, +`entries`, `rights`). It omits `io`, `os`, coroutine, time, random, process and +embedder libraries. Its `require` loads bounded source modules only; the +bridge rejects native modules and every non-read host operation even if a +script reaches an internal C function. `load`, `loadfile`, `dofile` and Lua +bytecode remain unavailable. + +The restricted base library also omits `pcall`, `xpcall`, `setmetatable`, +`getmetatable`, `rawset`, and `collectgarbage`. Lua disables instruction hooks +while running an `xpcall` error handler or a `__gc` finalizer; those callbacks +could loop after the counter expired. With no protected-call entry or way to +install a script finalizer, a quota error escapes the one-shot chunk to its +host. The ordinary interactive REPL retains these Lua functions. + +The Lua allocator accounts live bytes per VM, and the instruction hook bounds +executed Lua instructions on that restricted surface. The instruction allowance +resets for each evaluation; it is not a cumulative session budget. The bridge +bounds the *combined* captured stdout, stderr, and error text, as well as each +filesystem result. A source module is limited to the smaller of 64 KiB and +`fs_result_bytes`. The embedding task must hold only a delegated read-only +filesystem face: this policy does not mint or attenuate that capability. A wall-time +deadline and Stop must terminate and reap the *whole task*. `Host::eval` +borrows the script across the worker-thread execution and cannot be safely +cancelled by dropping its future while the VM may still read that pointer. + +`Host::eval` distinguishes successful evaluation, compile failure, runtime +failure, REPL continuation (`Incomplete`), REPL allocation refusal +(`ResourceRefused`), VM startup failure and `os.exit` (`Exited`) in `Outcome`. +Compile diagnostics include the chunk and line; restricted runtime failures +use a protected traceback. The returned stdout and stderr stay separate and +share the capture bound. A broken bridge is a `Fatal` error, not a Lua result. + +## Interactive Lua surface + Lua 5.5, freestanding, over the task's own filesystem face. A script sees two surfaces and they are one implementation: **`kore.*` is the store's vocabulary spelled for Lua, and `io`/`os` is the same code with the capabilities