Skip to content

Harden the generated FFM runtime and copy value types verbatim - #32

Merged
kdroidFilter merged 2 commits into
mainfrom
fix/ffm-runtime-hardening
Oct 5, 2026
Merged

kdroidFilter merged 2 commits into
mainfrom
fix/ffm-runtime-hardening

Conversation

@kdroidFilter

Copy link
Copy Markdown
Collaborator

Summary

Enums and data classes are now generated on the JVM as verbatim copies of their native declarations. While verifying this end to end, torture tests and seeded monkey tests uncovered a series of runtime bugs in the generated FFM layer, all fixed here.

Value types

  • JVM enum class / data class keep default values, body members, entry bodies, companion objects, KDoc and annotations (e.g. a custom equals/hashCode on a data class with a ByteArray field was previously dropped).
  • Imports of native-only packages (kotlinx.cinterop, platform.*, kotlin.native) and of module packages are removed.

Runtime bugs fixed

Bug Symptom before Fix
Exception thrown in a JVM callback JVM terminated (uncaught exception in upcall) Upcalls catch, native caller unwinds, original exception rethrown
Global native error state Errors lost or raised on the wrong thread Thread-local error
Collections > 4096 elements / 8 KB strings IndexOutOfBoundsException or crash (returns, sets, maps, suspend, DC fields) Oversized result parked natively, read back with a grown buffer
String / ByteArray > 8 KB Native function called twice (side effects duplicated) Read the parked result instead
Data class String field > 8 KB Silently truncated Marker + StableRef read back
Flow with slow collector 19 935 / 20 000 elements dropped Credit-based backpressure
Suspend cancelled before native start close() spins forever CoroutineStart.ATOMIC, completion always signalled
Cancel racing completion Use-after-free of the job StableRef Job handle owned and serialized by the JVM
Call after close(), concurrent close(), closed object as argument Use-after-free Per-object lease: IllegalStateException, deferred disposal
Cleaner action captured the proxy Unclosed objects never reclaimed Action only references the lease
Per-call upcall stubs in long-lived arenas Unbounded memory growth → OutOfMemoryError Static coroutine stubs + per-call callback arenas
Abstract/sealed hierarchies Native objects never disposed Generic StableRef disposal
List<Short> / List<Byte>, non-String map keys Corrupted values / ClassCastException Consistent element encoding
NUL in String arguments Silently truncated IllegalArgumentException (returns decoded by length)

Behavior changes

  • Callbacks are valid for the duration of the call only (like JNI). Native code must not store a JVM callback; event streams should be exposed as Flow<T> (as systeminfo already does). README updated.
  • Calls on a closed object throw IllegalStateException.
  • String arguments containing \u0000 throw IllegalArgumentException.

Tests

  • ValueTypeParityTest, ValueTypeTortureTest: value types across FFM, overflow parity, class-init races in fresh classloaders, re-entrant callbacks.
  • RuntimeTortureTest, BufferTortureTest: every buffer past its limit, error isolation across threads, lifecycle races, cleaner reclamation, flows and cancellations.
  • MonkeyTest: seeded model-based monkey (each native result checked against a JVM model) and concurrent chaos monkey (8 threads, random closes, throwing callbacks, cancellations, GC). Replay with MONKEY_SEED=<seed>, widen with MONKEY_SEEDS=<n> (add --rerun).
  • systeminfo: screen capture test accepts the PNG produced on macOS.

Results on macOS arm64: calculator 1785 tests green, monkeys green on 500 seeds each; benchmark and systeminfo green; plugin unit tests green.

The macOS implementation encodes the capture as PNG while Linux and
Windows produce BMP; the test only accepted the BMP header.
Value types
- Enums and data classes are emitted on the JVM as verbatim copies of the
  native declarations (defaults, body members, companions, KDoc), with
  native-only imports dropped.

Runtime fixes found by torture and seeded monkey tests
- Exceptions thrown by JVM callbacks no longer kill the VM: they unwind the
  native caller and are rethrown with their original type.
- Native error state is thread-local (errors leaked across threads).
- Collections, strings, byte arrays and data class string fields of any size:
  oversized results are parked natively and read back instead of overflowing
  fixed buffers or calling the function twice.
- Flows use credit-based backpressure (slow collectors lost elements).
- Suspend calls always signal completion (ATOMIC start); job handles are
  owned by the JVM to avoid use-after-free on cancellation.
- Object lifecycle uses a lease: calls after close() throw, close() is safe
  concurrently and from callbacks, arguments stay alive during calls, and the
  Cleaner no longer captures the proxy (unclosed objects leaked).
- Coroutine completion uses static upcall stubs dispatched by call id and
  callback stubs are released per call (unbounded memory growth).
- List<Short>/List<Byte> and non-String map keys are encoded consistently.
- Strings containing NUL are rejected instead of silently truncated.

Callbacks are now valid for the duration of the call only; event streams
should be exposed as Flow.
@kdroidFilter
kdroidFilter merged commit a4ae354 into main Oct 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant