Add built-in filesystem tools and complete SDK size optimizations

[vigoo]

Summary

Adds Rust and MoonBit implementations of the built-in read-file, write-file, and edit-file tools, alongside WASM size optimizations across the Rust, MoonBit, Scala, TypeScript, and Effect SDKs.

The filesystem components shrink from 1,219,887 B to 394,326 B for Rust (67.7%) and 1,070,119 B to 672,478 B for MoonBit (37.2%), while retaining the existing five-interface export contract.

The main changes operate at four distinct layers:

• Compile-time specialization: generate concrete descriptors, dispatch, and clients instead of interpreting schemas at runtime.
• Dead-code elimination: stop unused capabilities and reflection from retaining their implementation stacks.
• Direct wire encoding: avoid the owned schema model on ordinary generated invocation paths.
• Canonical-ABI deduplication: share necessary lifting, lowering, and cleanup code in wit-bindgen rather than emitting copies per export.

Filesystem-tool scope

• Adds Rust and MoonBit read-file, write-file, and edit-file components.
• Provisions all six suffixed built-in tool releases idempotently, allowing both implementations to be evaluated.
• Adds real-worker coverage for sandbox filesystem behavior, guest invocation, snapshot recovery, and registry provisioning.

How the size reductions work

1. Specialize at generation time instead of interpreting at runtime

Generated code already knows the command tree, argument types, result types, and error declarations. Previously, ordinary generated paths still used general-purpose builders and dispatch machinery to rediscover that information at startup or on each invocation.

The generators now emit:

• Final wire descriptors for schemas, command trees, metadata, and errors.
• Direct command-path matches and concrete argument bindings.
• Typed clients with precomputed command paths, input layouts, output shapes, and error decoders.

For example, an illustrative files/read(path) command changes from:

Before: find command in tree → inspect input schema → project "path" → call handler
After:  generated path match → decode the known "path" field        → call handler

Arity, shape, command identity, and protocol checks remain. What disappears is the generic machinery for discovering how to perform them.

Where registration still requires descriptor construction or validation, the result is cached instead of reconstructed and revalidated on every call. MoonBit and Scala code generation also resolve capability hooks and binding information during compilation rather than retaining broader runtime discovery paths.

2. Make unused capabilities and reflection removable

An unused feature is not free if an export, registration function, or package import keeps its runtime reachable.

Rust and MoonBit now use small canonical-ABI entrypoints for absent capabilities. The required exports remain present, but their full typed dispatchers are linked only when an implementation exists. Rust registration constructors install those dispatchers for concrete implementations rather than rooting them from common SDK code.

For example, a component with no tool implementation still supplies the required tool exports, but does not need the typed tool-dispatch stack behind them. The minimal exports still obey allocation, post-return, and async task-return ownership rules.

TypeScript and Effect address the same retention problem through capability-specific bundle entrypoints, separate reflection entrypoints, and narrow side-effect declarations. Generated Rust clients no longer query agent types merely to construct a typed client.

Explicit dynamic and reflected APIs remain available. Ordinary typed clients no longer pull them in accidentally.

3. Encode directly at the wire boundary

Specialized dispatch alone is insufficient if every argument and result still passes through a generic owned representation.

Generated invocation codecs now read and write the WIT wire arena directly, bypassing the intermediate SchemaGraph, SchemaValue, and TypedSchemaValue model.

For an illustrative record value:

Before: { path, contents } → owned schema/value model → WIT wire arena
After:  { path, contents } → generated encoder        → WIT wire arena

The same principle extends beyond basic invocation arguments:

• Tool composition and middleware projection use wire descriptors and typed invocation boundaries, preserving subtree forwarding, aliases, and custom errors.
• Generated guests perform focused structural checks; rich graph validation remains available to host and dynamic APIs.
• Snapshots and restore context use narrow typed JSON/principal codecs, and built-in string errors are encoded directly into their wire representation.

These changes remove otherwise independent retention paths into generic model, validation, and JSON-value machinery. They do not remove JSON where the protocol requires it or where application code uses it.

Resource ownership remains part of the wire boundary: encoding performs a resource-only preflight before transferring handles, decoding tracks ownership in the arena, and failure/cancellation paths clean up resources still owned by the guest. This covers resource-bearing values such as streams, secrets, and quota tokens.

4. Share live canonical-ABI code in wit-bindgen

Even after unused SDK code is eliminated, active exports still need canonical-ABI lifting, lowering, allocation, resource transfer, and cleanup.

wit-bindgen previously emitted similar nested walkers separately for exports using the same resource-bearing shapes. The generator now outlines reusable helpers, including async lowering and post-return reclamation.

Before: export A → its own walker for shape T
        export B → another walker for shape T

After:  exports A and B → shared walker for shape T

This is distinct from dead-code elimination: the walkers are needed, but their implementations no longer need to be duplicated. Live tag dispatch, nested allocation/copy, resource transfer, and reclamation remain.

Additional runtime improvement: MoonBit provider stdout now writes in bounded 64 KiB windows, preserving partial writes, stream errors, and finish/drop behavior while reducing boundary-crossing overhead.

Size results

Rust release artifacts now consistently strip symbol/debug residue, and size-analysis fixtures record reproducible build settings across SDKs.

Filesystem components

Both implementations retain the same five exported interfaces.

Implementation	Initial stripped component	Final committed component	Reduction
Rust	1,219,887 B	394,326 B	67.7%
MoonBit	1,070,119 B	672,478 B	37.2%

The final Rust component is 41.4% smaller than the final MoonBit component.

Isolated effect of shared ABI walkers

Controlled before/after measurements, in stripped bytes:

Fixture	Before outlining	After outlining
Typed-tool floor, full export contract	166,453 B	150,652 B
Tool-only control	135,973 B	120,164 B
Rust filesystem	491,124 B	394,326 B
MoonBit filesystem core module	647,648 B	607,644 B

The tool-only fixture is a measurement control, not a change to the shipped export contract. The MoonBit row measures a core module, not the complete component in the preceding table.

The ≤170 KB typed-tool-floor target was met. The aspirational ≤350 KB Rust filesystem target was missed by 44,326 B.

Twiggy attribution identifies live tag dispatch, nested allocation/copy, resource transfer, and reclamation in the remaining shared helpers—not duplicated export walkers. Table-driven canonical-ABI interpretation or a different representation/contract are possible directions for further investigation, not changes implemented here.

Capability-free SDK overhead

The corrected empty Rust fixture uses export_golem_component!() without registering an implementation:

• Capability-free SDK component: 69,890 B stripped.
• Pure wit-bindgen control with the same export contract: 69,730 B.
• Measured difference: 160 B.

The previous “empty” fixture registered a durable agent and therefore also measured registry, snapshot, principal, and JSON behavior.

Worker performance

Optimized-host measurements for 64 KiB files, with 100 samples after 10 warmups:

Operation	Rust median (p95), ms	MoonBit median (p95), ms
Write	4.854 (5.172)	5.520 (6.492)
Read	5.102 (5.994)	5.488 (7.210)
Edit	6.516 (8.125)	6.816 (8.260)

These include worker dispatch, component execution, and sandbox filesystem overhead. The linked benchmark executable was invoked directly, so compilation is excluded.

These timings predate the final size follow-ups. Final artifacts were revalidated with real-worker filesystem tests, but the 100-sample latency matrix was not rerun. This is a latency comparison between the measured implementations, not a before/after speedup measurement for the optimization work.

Validation and limitations

Behavior, ABI, and retention

• Rust and MoonBit filesystem WASMs validate, retain the expected five-interface export contract, and introduce no new imports.
• Real-worker filesystem behavior, guest invocation, snapshot recovery, and registry provisioning/idempotency checks pass.
• Raw absent-interface ABI and resource-ownership checks pass across all capability combinations.
• Size-analysis tests and component/export-contract checks pass.

CI retention checks inspect every embedded core module in the checked fixtures using wasm-tools, demangling, and Twiggy evidence. Empty and ordinary typed fixtures exclude forbidden generic model/validation families and reflection imports. A positive reflection fixture retains the expected host import and symbols, checking that the negative tests can actually detect those dependencies.

SDK checks

• Rust: macro, SDK, and filesystem tests; formatting; scoped Clippy with -D warnings pass.
• MoonBit: SDK, codegen, direct-wire, reflection, and ownership tests, plus deterministic regeneration, pass.
• Scala: macro and codegen suites pass (131 + 170 tests).
• TypeScript: 1,010 tests pass, 20 skipped; lint has no errors; bundle checks confirm ordinary paths omit generic models.
• Effect: 986 tests, typecheck, lint, and formatting pass.

Remaining validation gaps

• Scala: the broader test-agent link remains unverified because this checkout lacks the generated golem/wasm/agent_guest.wasm prerequisite.
• MoonBit: the white-box filesystem runner lacks the WASI 0.3 future-drop-readable intrinsic. Committed components validate and the real-worker filesystem tests pass.

Dependencies

• Base prerequisite: Simplify SDKs to a single agent world #3923, already merged.
• Generator fork: golemcloud/wit-bindgen#1, pinned at d1d16370eff379655f68df661891b7a2116c7557.
• No other source repositories contain unreleased changes required by this PR.