From 6d79d524b00c6dc54682a56a4d5059d45de05d21 Mon Sep 17 00:00:00 2001 From: gcdepaula Date: Tue, 22 Sep 2026 18:11:04 -0300 Subject: [PATCH 01/11] build: bump Cartesi Machine to v0.21.0 and vendor cartesi-tools Pin the emulator, the paired ctsi-2 kernel and guest tools 0.18.0, and rebuild the canonical images. v0.21 stores are not loadable by v0.20 and vice versa, so every image and watchdog checkpoint is regenerated. Vendor the cartesi-tools-rs crates (libcmt-sys, trolley, testsi, types) under cartesi-tools/ instead of pinning a git rev: testsi moves to the v0.21 Rust bindings, trolley follows the libcmt 0.18 renames, and libcmt-sys builds the vendored libcmt 0.18 sources without downloading kernel headers. CI installs the pinned emulator for canonical-test. The watchdog's Lua host loop does not run on v0.21 (it feeds inputs without a revert root hash); its rewrite follows in this branch. --- .../install-cartesi-machine/action.yml | 51 ++ .../actions/setup-guest-toolchain/action.yml | 28 +- .github/workflows/ci.yml | 10 + AGENTS.md | 5 +- Cargo.lock | 76 +-- Cargo.toml | 16 +- cartesi-tools/README.md | 50 ++ cartesi-tools/libcmt-sys/Cargo.toml | 12 + cartesi-tools/libcmt-sys/build.rs | 101 +++ cartesi-tools/libcmt-sys/libcmt/AUTHORS | 7 + cartesi-tools/libcmt-sys/libcmt/LICENSE | 202 ++++++ cartesi-tools/libcmt-sys/libcmt/Makefile | 254 ++++++++ .../libcmt-sys/libcmt/include/libcmt/abi.h | 578 ++++++++++++++++++ .../libcmt-sys/libcmt/include/libcmt/buf.h | 82 +++ .../libcmt-sys/libcmt/include/libcmt/io.h | 168 +++++ .../libcmt-sys/libcmt/include/libcmt/ioctl.h | 67 ++ .../libcmt-sys/libcmt/include/libcmt/keccak.h | 131 ++++ .../libcmt-sys/libcmt/include/libcmt/merkle.h | 118 ++++ .../libcmt-sys/libcmt/include/libcmt/rollup.h | 260 ++++++++ .../libcmt-sys/libcmt/include/libcmt/util.h | 34 ++ cartesi-tools/libcmt-sys/libcmt/src/abi.c | 315 ++++++++++ cartesi-tools/libcmt-sys/libcmt/src/buf.c | 121 ++++ cartesi-tools/libcmt-sys/libcmt/src/io-mock.c | 319 ++++++++++ cartesi-tools/libcmt-sys/libcmt/src/io.c | 163 +++++ cartesi-tools/libcmt-sys/libcmt/src/keccak.c | 156 +++++ cartesi-tools/libcmt-sys/libcmt/src/merkle.c | 202 ++++++ cartesi-tools/libcmt-sys/libcmt/src/rollup.c | 455 ++++++++++++++ cartesi-tools/libcmt-sys/libcmt/src/util.c | 52 ++ cartesi-tools/libcmt-sys/src/lib.rs | 5 + cartesi-tools/testsi/Cargo.toml | 23 + cartesi-tools/testsi/src/lib.rs | 7 + cartesi-tools/testsi/src/machine.rs | 374 ++++++++++++ cartesi-tools/testsi/src/test_runner.rs | 47 ++ cartesi-tools/testsi/src/types.rs | 155 +++++ cartesi-tools/testsi/testsi-macros/Cargo.toml | 14 + cartesi-tools/testsi/testsi-macros/src/lib.rs | 114 ++++ cartesi-tools/trolley/Cargo.toml | 11 + cartesi-tools/trolley/src/cmt.rs | 212 +++++++ cartesi-tools/trolley/src/lib.rs | 81 +++ cartesi-tools/types/Cargo.toml | 11 + cartesi-tools/types/src/lib.rs | 236 +++++++ examples/canonical-app/Dockerfile | 8 +- examples/canonical-app/justfile | 5 +- tests/e2e/src/watchdog_compare.rs | 2 +- toolchain-pins.env | 10 +- 45 files changed, 5235 insertions(+), 113 deletions(-) create mode 100644 .github/actions/install-cartesi-machine/action.yml create mode 100644 cartesi-tools/README.md create mode 100644 cartesi-tools/libcmt-sys/Cargo.toml create mode 100644 cartesi-tools/libcmt-sys/build.rs create mode 100644 cartesi-tools/libcmt-sys/libcmt/AUTHORS create mode 100644 cartesi-tools/libcmt-sys/libcmt/LICENSE create mode 100644 cartesi-tools/libcmt-sys/libcmt/Makefile create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/abi.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/buf.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/io.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/ioctl.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/keccak.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/merkle.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/rollup.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/include/libcmt/util.h create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/abi.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/buf.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/io-mock.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/io.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/keccak.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/merkle.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/rollup.c create mode 100644 cartesi-tools/libcmt-sys/libcmt/src/util.c create mode 100644 cartesi-tools/libcmt-sys/src/lib.rs create mode 100644 cartesi-tools/testsi/Cargo.toml create mode 100644 cartesi-tools/testsi/src/lib.rs create mode 100644 cartesi-tools/testsi/src/machine.rs create mode 100644 cartesi-tools/testsi/src/test_runner.rs create mode 100644 cartesi-tools/testsi/src/types.rs create mode 100644 cartesi-tools/testsi/testsi-macros/Cargo.toml create mode 100644 cartesi-tools/testsi/testsi-macros/src/lib.rs create mode 100644 cartesi-tools/trolley/Cargo.toml create mode 100644 cartesi-tools/trolley/src/cmt.rs create mode 100644 cartesi-tools/trolley/src/lib.rs create mode 100644 cartesi-tools/types/Cargo.toml create mode 100644 cartesi-tools/types/src/lib.rs diff --git a/.github/actions/install-cartesi-machine/action.yml b/.github/actions/install-cartesi-machine/action.yml new file mode 100644 index 00000000..b09c4617 --- /dev/null +++ b/.github/actions/install-cartesi-machine/action.yml @@ -0,0 +1,51 @@ +name: "Install cartesi-machine" +description: "Install the pinned machine-emulator .deb and export LIBCARTESI_PATH / INCLUDECARTESI_PATH for the Rust bindings." + +inputs: + version: + description: "cartesi-machine release tag" + required: true + sha256-amd64: + description: "SHA-256 for the amd64 .deb" + required: true + sha256-arm64: + description: "SHA-256 for the arm64 .deb" + required: true + +runs: + using: composite + steps: + - name: Install cartesi-machine + shell: bash + run: | + set -euo pipefail + ARCH="$(dpkg --print-architecture)" + VERSION="${{ inputs.version }}" + + case "${ARCH}" in + amd64) + DEB_SHA256="${{ inputs.sha256-amd64 }}" + ;; + arm64) + DEB_SHA256="${{ inputs.sha256-arm64 }}" + ;; + *) + echo "unsupported arch for machine-emulator: ${ARCH}" >&2 + exit 1 + ;; + esac + + wget -O /tmp/machine-emulator.deb "https://github.com/cartesi/machine-emulator/releases/download/${VERSION}/machine-emulator_${ARCH}.deb" + echo "${DEB_SHA256} /tmp/machine-emulator.deb" | sha256sum --check + sudo apt-get install -y /tmp/machine-emulator.deb + rm -f /tmp/machine-emulator.deb + cartesi-machine --version + + # testsi's cartesi-machine bindings (cartesi/dave) link this prebuilt + # emulator; they build only with LIBCARTESI_PATH set (external_cartesi). + # The static archive references libslirp, so callers must also install + # libslirp-dev. + test -f /usr/lib/libcartesi.a + test -f /usr/include/cartesi-machine/cm.h + echo "LIBCARTESI_PATH=/usr/lib" >> "${GITHUB_ENV}" + echo "INCLUDECARTESI_PATH=/usr/include/cartesi-machine" >> "${GITHUB_ENV}" diff --git a/.github/actions/setup-guest-toolchain/action.yml b/.github/actions/setup-guest-toolchain/action.yml index 972153dc..3b95829d 100644 --- a/.github/actions/setup-guest-toolchain/action.yml +++ b/.github/actions/setup-guest-toolchain/action.yml @@ -80,29 +80,11 @@ runs: sudo apt-get install -y /tmp/xgenext2fs.deb - name: Install cartesi-machine - shell: bash - run: | - set -euo pipefail - ARCH="$(dpkg --print-architecture)" - VERSION="${{ inputs.cartesi-machine-version }}" - - case "${ARCH}" in - amd64) - DEB_SHA256="${{ inputs.cartesi-machine-sha256-amd64 }}" - ;; - arm64) - DEB_SHA256="${{ inputs.cartesi-machine-sha256-arm64 }}" - ;; - *) - echo "unsupported arch for machine-emulator: ${ARCH}" >&2 - exit 1 - ;; - esac - - wget -O /tmp/machine-emulator.deb "https://github.com/cartesi/machine-emulator/releases/download/${VERSION}/machine-emulator_${ARCH}.deb" - echo "${DEB_SHA256} /tmp/machine-emulator.deb" | sha256sum --check - sudo apt-get install -y /tmp/machine-emulator.deb - cartesi-machine --version + uses: ./.github/actions/install-cartesi-machine + with: + version: ${{ inputs.cartesi-machine-version }} + sha256-amd64: ${{ inputs.cartesi-machine-sha256-amd64 }} + sha256-arm64: ${{ inputs.cartesi-machine-sha256-arm64 }} - name: Set up QEMU uses: docker/setup-qemu-action@v4 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7ae4ff6b..88ebca0e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -30,6 +30,16 @@ jobs: liblua5.4-dev \ libslirp-dev + # canonical-test links the emulator through testsi's cartesi-machine + # bindings, so the workspace build needs the pinned CM .deb and the + # LIBCARTESI_PATH / INCLUDECARTESI_PATH this action exports. + - name: Install cartesi-machine + uses: ./.github/actions/install-cartesi-machine + with: + version: ${{ env.CARTESI_MACHINE_VERSION }} + sha256-amd64: ${{ env.CARTESI_MACHINE_SHA256_AMD64 }} + sha256-arm64: ${{ env.CARTESI_MACHINE_SHA256_ARM64 }} + - name: Install Rust toolchain uses: dtolnay/rust-toolchain@stable with: diff --git a/AGENTS.md b/AGENTS.md index 500dcdd6..e8e3ef72 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -161,6 +161,7 @@ signing, and SSZ batch encoding. - `examples/c-wallet-sequencer/` — binary composing the C-engine host with the reference wallet engine. - `examples/canonical-app/` — on-chain scheduler reference implementation. - `examples/canonical-test/` — e2e test harness for the canonical app. +- `cartesi-tools/` — vendored guest and guest-test tooling (`libcmt-sys`, `trolley`, `testsi`, `types`); provenance in its README. - `sdk/rust-client/` — Rust client library for the sequencer API. - `tests/{benchmarks,e2e,harness}/` — test infrastructure. @@ -370,7 +371,9 @@ Prefer black-box tests around `POST /tx` and commit outcomes for integration. Some `sequencer` tests use Anvil (Foundry). They run by default and fail with a clear message if `anvil` is not on PATH. Use the configured Nix/direnv environment -or install Foundry. `canonical-test` additionally needs libslirp. +or install Foundry. `canonical-test` additionally needs the Cartesi Machine +library named by `LIBCARTESI_PATH`/`INCLUDECARTESI_PATH` (the devshell exports +both) and libslirp. ## Shell and Commands diff --git a/Cargo.lock b/Cargo.lock index bfd831a5..fd68f875 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -763,12 +763,6 @@ dependencies = [ "types", ] -[[package]] -name = "ar" -version = "0.9.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d67af77d68a931ecd5cbd8a3b5987d63a1d1d1278f7f6a60ae33db485cdebb69" - [[package]] name = "ark-ff" version = "0.3.0" @@ -1458,7 +1452,7 @@ dependencies = [ [[package]] name = "cartesi-machine" version = "2.0.0" -source = "git+https://github.com/cartesi/dave?branch=feature%2Fbump-emulator-2-coutinho#317a88a32a1b4a6002d561f37bb833843df55cba" +source = "git+https://github.com/cartesi/dave?rev=79874c0a1b40678465daf533acc976c88301c6d4#79874c0a1b40678465daf533acc976c88301c6d4" dependencies = [ "base64", "cartesi-machine-sys", @@ -1472,13 +1466,11 @@ dependencies = [ [[package]] name = "cartesi-machine-sys" version = "2.0.0" -source = "git+https://github.com/cartesi/dave?branch=feature%2Fbump-emulator-2-coutinho#317a88a32a1b4a6002d561f37bb833843df55cba" +source = "git+https://github.com/cartesi/dave?rev=79874c0a1b40678465daf533acc976c88301c6d4#79874c0a1b40678465daf533acc976c88301c6d4" dependencies = [ "bindgen", - "cfg-if", - "hex-literal 1.1.0", "link-cplusplus", - "sha1", + "sha2", ] [[package]] @@ -2484,18 +2476,6 @@ dependencies = [ "arrayvec", ] -[[package]] -name = "hex-literal" -version = "0.4.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6fe2267d4ed49bc07b63801559be28c718ea06c4738b7a03c94df7386d2cde46" - -[[package]] -name = "hex-literal" -version = "1.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e712f64ec3850b98572bffac52e2c6f282b29fe6c5fa6d42334b30be438d95c1" - [[package]] name = "hmac" version = "0.12.1" @@ -2593,7 +2573,6 @@ dependencies = [ "tokio", "tokio-rustls", "tower-service", - "webpki-roots", ] [[package]] @@ -2984,16 +2963,8 @@ checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" [[package]] name = "libcmt-sys" version = "0.1.0" -source = "git+https://github.com/GCdePaula/cartesi-tools-rs?rev=ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22#ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" dependencies = [ - "ar", "bindgen", - "bytes", - "hex-literal 0.4.1", - "reqwest 0.12.28", - "sha2", - "tar", - "xz2", ] [[package]] @@ -3085,17 +3056,6 @@ version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" -[[package]] -name = "lzma-sys" -version = "0.1.20" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5fda04ab3764e6cde78b9974eec4f779acaba7c4e84b36eca3cf77c581b85d27" -dependencies = [ - "cc", - "libc", - "pkg-config", -] - [[package]] name = "macro-string" version = "0.2.0" @@ -3763,28 +3723,22 @@ checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147" dependencies = [ "base64", "bytes", - "futures-channel", "futures-core", "futures-util", "http", "http-body", "http-body-util", "hyper", - "hyper-rustls", "hyper-util", "js-sys", "log", "percent-encoding", "pin-project-lite", - "quinn", - "rustls", - "rustls-pki-types", "serde", "serde_json", "serde_urlencoded", "sync_wrapper", "tokio", - "tokio-rustls", "tokio-util", "tower", "tower-http", @@ -3794,7 +3748,6 @@ dependencies = [ "wasm-bindgen-futures", "wasm-streams", "web-sys", - "webpki-roots", ] [[package]] @@ -4036,7 +3989,6 @@ checksum = "0283386ce02abc0151e1761d08802dfe86c173b0b494af5cbc086574e453da06" dependencies = [ "aws-lc-rs", "once_cell", - "ring", "rustls-pki-types", "rustls-webpki", "subtle", @@ -4734,7 +4686,6 @@ dependencies = [ [[package]] name = "testsi" version = "0.1.0" -source = "git+https://github.com/GCdePaula/cartesi-tools-rs?rev=ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22#ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" dependencies = [ "cartesi-machine", "inventory", @@ -4747,7 +4698,6 @@ dependencies = [ [[package]] name = "testsi-macros" version = "0.1.0" -source = "git+https://github.com/GCdePaula/cartesi-tools-rs?rev=ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22#ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" dependencies = [ "proc-macro2", "quote", @@ -5136,7 +5086,6 @@ dependencies = [ [[package]] name = "trolley" version = "0.1.0" -source = "git+https://github.com/GCdePaula/cartesi-tools-rs?rev=ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22#ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" dependencies = [ "libcmt-sys", "types", @@ -5190,7 +5139,6 @@ checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" [[package]] name = "types" version = "0.1.0" -source = "git+https://github.com/GCdePaula/cartesi-tools-rs?rev=ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22#ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" dependencies = [ "alloy-primitives", "alloy-sol-types", @@ -5448,15 +5396,6 @@ dependencies = [ "rustls-pki-types", ] -[[package]] -name = "webpki-roots" -version = "1.0.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7dcd9d09a39985f5344844e66b0c530a33843579125f23e21e9f0f220850f22a" -dependencies = [ - "rustls-pki-types", -] - [[package]] name = "winapi-util" version = "0.1.11" @@ -5656,15 +5595,6 @@ dependencies = [ "rustix", ] -[[package]] -name = "xz2" -version = "0.1.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "388c44dc09d76f1536602ead6d325eb532f5c122f17782bd57fb47baeeb767e2" -dependencies = [ - "lzma-sys", -] - [[package]] name = "yoke" version = "0.8.3" diff --git a/Cargo.toml b/Cargo.toml index 4846cc62..1522ae5f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -15,6 +15,11 @@ members = [ "tests/benchmarks", "tests/harness", "tests/e2e", + "cartesi-tools/libcmt-sys", + "cartesi-tools/trolley", + "cartesi-tools/testsi", + "cartesi-tools/testsi/testsi-macros", + "cartesi-tools/types", ] default-members = ["sequencer", "examples/wallet-sequencer"] @@ -33,12 +38,11 @@ authors = [ ] [workspace.dependencies] -# ── Git-pinned Cartesi tooling ─────────────────────────────── -libcmt-sys = { version = "0.1", git = "https://github.com/GCdePaula/cartesi-tools-rs", rev = "ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" } -trolley = { version = "0.1", git = "https://github.com/GCdePaula/cartesi-tools-rs", rev = "ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" } -testsi = { version = "0.1", git = "https://github.com/GCdePaula/cartesi-tools-rs", rev = "ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" } -types = { version = "0.1", git = "https://github.com/GCdePaula/cartesi-tools-rs", rev = "ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22" } -cartesi-machine = { version = "2", git = "https://github.com/cartesi/dave", rev = "317a88a32a1b4a6002d561f37bb833843df55cba", features = ["download_uarch"] } +# ── Vendored Cartesi tooling (see cartesi-tools/README.md) ── +libcmt-sys = { version = "0.1", path = "cartesi-tools/libcmt-sys" } +trolley = { version = "0.1", path = "cartesi-tools/trolley" } +testsi = { version = "0.1", path = "cartesi-tools/testsi" } +types = { version = "0.1", path = "cartesi-tools/types" } # ── Contracts (exact pin — the single place the version lives) ── cartesi-rollups-contracts = "=3.0.0-alpha.6" diff --git a/cartesi-tools/README.md b/cartesi-tools/README.md new file mode 100644 index 00000000..3a4b4540 --- /dev/null +++ b/cartesi-tools/README.md @@ -0,0 +1,50 @@ +# cartesi-tools + +Rust tooling for building and testing the canonical Cartesi Machine guest +(`examples/canonical-app`, `examples/canonical-test`). Vendored in-tree; these +crates are workspace members and are not published. + +## Provenance + +- **Rust crates:** imported from + [GCdePaula/cartesi-tools-rs](https://github.com/GCdePaula/cartesi-tools-rs) + at rev `ed14b98ecfe9796dc3ca7c9b96bfdbf0ef9baf22` + (`guest/libcmt-sys`, `guest/trolley`, `host/testsi`, `types`; its examples + were not imported). Since then they have been updated here for Cartesi + Machine v0.21.0 and machine-guest-tools v0.18.0. Apache-2.0 licensed. +- **libcmt C sources** (`libcmt-sys/libcmt/`): the `sys-utils/libcmt` + `Makefile`, `include/` and `src/`, plus the top-level `LICENSE` (Apache-2.0) + and `AUTHORS`, from + [cartesi/machine-guest-tools](https://github.com/cartesi/machine-guest-tools) + tag `v0.18.0` (commit `5222250c69371f7cbe96e6082699a8ca22b5969c`), taken + from `https://github.com/cartesi/machine-guest-tools/archive/refs/tags/v0.18.0.tar.gz` + (sha256 `a16ee31a7abb0522ae70497721bf4856665297e1dd26d875a43fadbe48814bb6`). + The files are unmodified. libcmt must match the guest-tools version pinned + in `examples/canonical-app/Dockerfile`, which pairs with the emulator + release in `toolchain-pins.env`. + +## Crates + +- **`libcmt-sys`**: raw bindgen bindings to libcmt. On `riscv64` targets + `build.rs` builds the real library with `riscv64-unknown-linux-musl-gcc` (from + the pinned `cross` image). On the host it builds libcmt's mock I/O backend, so + the guest crates still compile and lint there. +- **`trolley`**: a safe guest-side rollup API over libcmt: the input loop, + vouchers, notices, reports and GIO. +- **`testsi`** (with its proc-macro crate `testsi-macros`): a host-side test + harness that loads a stored machine image and feeds it advance-state inputs. + It uses the Cartesi Machine v0.21 Rust bindings from + [cartesi/dave](https://github.com/cartesi/dave), pinned by rev in + `testsi/Cargo.toml`. These link the prebuilt emulator named by + `LIBCARTESI_PATH` / `INCLUDECARTESI_PATH`: the dev flake exports both, and so + does CI when it installs the emulator `.deb`. +- **`types`**: the Cartesi Rollups ABI types (`EvmAdvance` input, vouchers, + notices, portal deposit encodings), shared by the guest and host crates. + +## Updating libcmt + +1. Download the new machine-guest-tools tag tarball and record its sha256. +2. Replace `libcmt-sys/libcmt/{Makefile,include,src,LICENSE,AUTHORS}` with the + new tag's files. +3. Update the version and hashes above. +4. Bump the guest-tools pin in `examples/canonical-app/Dockerfile` to match. diff --git a/cartesi-tools/libcmt-sys/Cargo.toml b/cartesi-tools/libcmt-sys/Cargo.toml new file mode 100644 index 00000000..4b1e41bf --- /dev/null +++ b/cartesi-tools/libcmt-sys/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "libcmt-sys" +version = "0.1.0" +license = "Apache-2.0" +edition = "2021" +description = "Raw bindings to the vendored libcmt (machine-guest-tools v0.18.0) guest library" +publish = false + +[dependencies] + +[build-dependencies] +bindgen = "0.72" diff --git a/cartesi-tools/libcmt-sys/build.rs b/cartesi-tools/libcmt-sys/build.rs new file mode 100644 index 00000000..5ae9ab06 --- /dev/null +++ b/cartesi-tools/libcmt-sys/build.rs @@ -0,0 +1,101 @@ +use std::env; +use std::path::{Path, PathBuf}; +use std::process::Command; + +// Vendored libcmt sources (machine-guest-tools v0.18.0, see ../README.md). +// libcmt 0.18 bundles its cmio ioctl definitions (`libcmt/ioctl.h`), so the +// riscv64 build no longer needs the Cartesi Linux kernel headers. +const LIBCMT_PATH: &str = "libcmt"; + +fn main() { + let target = env::var("TARGET").unwrap(); + let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); + let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap()); + let is_riscv_target = target.starts_with("riscv64"); + + // 1. Prepare Source (Sandbox) + let original_src = manifest_dir.join(LIBCMT_PATH); + let build_dir = out_dir.join("libcmt_build"); + + // Always clean and recopy to ensure no stale artifacts + if build_dir.exists() { + std::fs::remove_dir_all(&build_dir).unwrap(); + } + copy_dir_recursive(&original_src, &build_dir).expect("Failed to copy source"); + + let headers_path = build_dir.join("include/libcmt"); + + // 2. Build with Make + let lib_output_dir = if is_riscv_target { + let status = Command::new("make") + .current_dir(&build_dir) + .env("TOOLCHAIN_PREFIX", "riscv64-unknown-linux-musl-") + .status() + .expect("Failed to execute make"); + + if !status.success() { + panic!("libcmt Makefile failed"); + } + + build_dir.join("build/riscv64") + } else { + println!("Skipping real build on host; building mock"); + + // Mock build for Host / LSP + let status = Command::new("make") + .current_dir(&build_dir) + .arg("mock") + .status() + .expect("Failed to execute make"); + + if !status.success() { + panic!("libcmt Makefile mock failed"); + } + + build_dir.join("build/mock") + }; + + // 3. Generate Bindings + let mut bindgen = bindgen::Builder::default() + .header(headers_path.join("rollup.h").to_str().unwrap()) + .use_core() + .ctypes_prefix("core::ffi"); + if is_riscv_target { + bindgen = bindgen.clang_arg("--target=riscv64-unknown-linux-musl"); + } + let bindings = bindgen.generate().expect("Unable to generate bindings"); + + bindings + .write_to_file(out_dir.join("bindings.rs")) + .expect("Couldn't write bindings!"); + + // 4. Link + // Sanity check to save hours of debugging + if !lib_output_dir.join("libcmt.a").exists() { + panic!( + "Build succeeded but libcmt.a not found at expected path: {}", + lib_output_dir.display() + ); + } + + println!( + "cargo:rustc-link-search=native={}", + lib_output_dir.display() + ); + println!("cargo:rustc-link-lib=static=cmt"); + println!("cargo:rerun-if-changed={}", LIBCMT_PATH); + println!("cargo:rerun-if-changed=build.rs"); +} + +fn copy_dir_recursive(src: &Path, dst: &Path) -> std::io::Result<()> { + std::fs::create_dir_all(dst)?; + for entry in std::fs::read_dir(src)? { + let entry = entry?; + if entry.file_type()?.is_dir() { + copy_dir_recursive(&entry.path(), &dst.join(entry.file_name()))?; + } else { + std::fs::copy(entry.path(), dst.join(entry.file_name()))?; + } + } + Ok(()) +} diff --git a/cartesi-tools/libcmt-sys/libcmt/AUTHORS b/cartesi-tools/libcmt-sys/libcmt/AUTHORS new file mode 100644 index 00000000..2880febe --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/AUTHORS @@ -0,0 +1,7 @@ +Diego Nehab +Eduardo Barthel +Gabriel de Quadros Ligneul +Marcelo Politzer +Marko Atanasievski +Marcos Pernambuco Motta +Victor Fusco diff --git a/cartesi-tools/libcmt-sys/libcmt/LICENSE b/cartesi-tools/libcmt-sys/libcmt/LICENSE new file mode 100644 index 00000000..d6456956 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/cartesi-tools/libcmt-sys/libcmt/Makefile b/cartesi-tools/libcmt-sys/libcmt/Makefile new file mode 100644 index 00000000..e642201a --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/Makefile @@ -0,0 +1,254 @@ +# Copyright Cartesi and individual authors (see AUTHORS) +# +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +PREFIX ?= /usr +CC := $(TOOLCHAIN_PREFIX)gcc +AR := $(TOOLCHAIN_PREFIX)ar +CFLAGS += -Wvla -O2 -g -Wall -Wextra -Iinclude \ + -fno-strict-aliasing -fno-strict-overflow -fPIC +LIBCMT_CFLAGS += -ftrivial-auto-var-init=zero -Wstrict-aliasing=3 + +all: libcmt +host: mock tools +#------------------------------------------------------------------------------- +examples_SRC := \ + doc/examples/abi_encode_000.c \ + doc/examples/abi_encode_001.c \ + doc/examples/abi_encode_002.c \ + doc/examples/abi_decode_000.c \ + doc/examples/abi_decode_001.c \ + doc/examples/abi_decode_002.c \ + doc/examples/io.c \ + doc/examples/rollup.c + +examples_OBJDIR := build/examples +examples_OBJ := $(patsubst %.c,$(examples_OBJDIR)/%.o,$(examples_SRC)) + +$(examples_OBJ): $(examples_OBJDIR)/%.o: %.c + @mkdir -p $(@D) + $(CC) $(CFLAGS) -MT $@ -MMD -MP -MF $(@:.o=.d) -c -o $@ $< + +# no need to link, just ensure examples build correctly +examples: $(examples_OBJ) + +#------------------------------------------------------------------------------- +libcmt_SRC := \ + src/buf.c \ + src/abi.c \ + src/keccak.c \ + src/merkle.c \ + src/rollup.c \ + src/util.c \ + src/io.c + +libcmt_OBJDIR := build/riscv64 +libcmt_OBJ := $(patsubst %.c,$(libcmt_OBJDIR)/%.o,$(libcmt_SRC)) +libcmt_LIB := $(libcmt_OBJDIR)/libcmt.a +libcmt_SO := $(libcmt_OBJDIR)/libcmt.so + +$(libcmt_OBJ): $(libcmt_OBJDIR)/%.o: %.c + @mkdir -p $(@D) + $(CC) $(CFLAGS) $(LIBCMT_CFLAGS) -MT $@ -MMD -MP -MF $(@:.o=.d) -c -o $@ $< + +$(libcmt_LIB): $(libcmt_OBJ) + $(AR) rcs $@ $^ + +$(libcmt_SO): $(libcmt_OBJ) + $(CC) -shared -o $@ $^ + +libcmt: $(libcmt_LIB) $(libcmt_SO) +install-run: $(libcmt_SO) + mkdir -p $(DESTDIR)$(PREFIX)/lib + cp -f $(libcmt_SO) $(DESTDIR)$(PREFIX)/lib + +install-dev: $(libcmt_LIB) build/ffi.h + mkdir -p $(DESTDIR)$(PREFIX)/lib + cp -f $(libcmt_LIB) $(DESTDIR)$(PREFIX)/lib + mkdir -p $(DESTDIR)$(PREFIX)/include/libcmt/ + cp -f include/libcmt/*.h $(DESTDIR)$(PREFIX)/include/libcmt/ + cp -f build/ffi.h $(DESTDIR)$(PREFIX)/include/libcmt/ + mkdir -p $(DESTDIR)$(PREFIX)/lib/pkgconfig + sed -e 's|@PREFIX@|$(PREFIX)|g' \ + tools/libcmt.pc.in > $(DESTDIR)$(PREFIX)/lib/pkgconfig/libcmt.pc + +install: install-run install-dev + +#------------------------------------------------------------------------------- +mock_SRC := \ + src/abi.c \ + src/buf.c \ + src/keccak.c \ + src/merkle.c \ + src/rollup.c \ + src/util.c \ + src/io-mock.c + +mock_OBJDIR := build/mock +mock_OBJ := $(patsubst %.c,$(mock_OBJDIR)/%.o,$(mock_SRC)) +mock_LIB := $(mock_OBJDIR)/libcmt.a +mock_SO := $(mock_OBJDIR)/libcmt.so + +$(mock_OBJ): $(mock_OBJDIR)/%.o: %.c + @mkdir -p $(@D) + $(CC) $(CFLAGS) -MT $@ -MMD -MP -MF $(@:.o=.d) -c -o $@ $< + +$(mock_LIB): $(mock_OBJ) + $(AR) rcs $@ $^ + +$(mock_SO): $(mock_OBJ) + $(CC) -shared -o $@ $^ + +mock: $(mock_LIB) $(mock_SO) + +install-mock: $(mock_LIB) $(mock_SO) build/ffi.h + mkdir -p $(DESTDIR)$(PREFIX)/lib + cp -f $(mock_LIB) $(mock_SO) $(DESTDIR)$(PREFIX)/lib + mkdir -p $(DESTDIR)$(PREFIX)/include/libcmt/ + cp -f include/libcmt/*.h $(DESTDIR)$(PREFIX)/include/libcmt/ + cp -f build/ffi.h $(DESTDIR)$(PREFIX)/include/libcmt/ + mkdir -p $(DESTDIR)$(PREFIX)/lib/pkgconfig + sed -e 's|@ARG_PREFIX@|$(PREFIX)|g' tools/libcmt.pc.in > $(DESTDIR)$(PREFIX)/lib/pkgconfig/libcmt.pc + +#------------------------------------------------------------------------------- +unittests_BINS := \ + $(mock_OBJDIR)/abi-multi \ + $(mock_OBJDIR)/abi-single \ + $(mock_OBJDIR)/buf \ + $(mock_OBJDIR)/gio \ + $(mock_OBJDIR)/keccak \ + $(mock_OBJDIR)/merkle \ + $(mock_OBJDIR)/progress \ + $(mock_OBJDIR)/rollup + +$(mock_OBJDIR)/abi-multi: tests/abi-multi.c $(mock_LIB) + $(CC) $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/abi-single: tests/abi-single.c $(mock_LIB) + $(CC) $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/buf: tests/buf.c $(mock_LIB) + $(CC) $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/keccak: tests/keccak.c $(mock_LIB) + $(CC) $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/merkle: tests/merkle.c $(mock_LIB) + $(CC) $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/gio: tests/gio.c tests/data.h $(mock_LIB) + $(CC) -Itests $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/rollup: tests/rollup.c tests/data.h $(mock_LIB) + $(CC) -Itests $(CFLAGS) -o $@ $^ + +$(mock_OBJDIR)/progress: tests/progress.c $(mock_LIB) + $(CC) -Itests $(CFLAGS) -o $@ $^ + +test: $(unittests_BINS) + $(foreach test,$(unittests_BINS),$(test) &&) true + +tests/data.h: tests/create-data.sh + $< > $@ + +#------------------------------------------------------------------------------- +tools_OBJDIR := build/tools +tools_BINS := \ + $(tools_OBJDIR)/funsel + +$(tools_OBJDIR)/funsel: tools/funsel.c $(mock_LIB) + @mkdir -p $(@D) + $(CC) $(CFLAGS) -o $@ $^ + +tools: $(tools_BINS) + +HDRS := $(patsubst %,include/libcmt/%, buf.h abi.h keccak.h merkle.h io.h rollup.h) +build/ffi.h: $(HDRS) + cat $^ | sh tools/prepare-ffi.sh > $@ +#------------------------------------------------------------------------------- +LINTER_IGNORE_SOURCES=src/io.c +LINTER_IGNORE_HEADERS= +LINTER_SOURCES=$(filter-out $(LINTER_IGNORE_SOURCES),$(strip $(wildcard src/*.c) $(wildcard tests/*.c) $(wildcard tools/*.c))) +LINTER_HEADERS=$(filter-out $(LINTER_IGNORE_HEADERS),$(strip $(wildcard src/*.h))) + +CLANG_TIDY=clang-tidy +CLANG_TIDY_TARGETS=$(patsubst %.c,%.clang-tidy,$(LINTER_SOURCES)) + +CLANG_FORMAT=clang-format +CLANG_FORMAT_FILES:=$(wildcard src/*.c) $(wildcard src/*.h) $(wildcard tests/*.c) $(wildcard tools/*.c) +CLANG_FORMAT_IGNORE_FILES:= +CLANG_FORMAT_FILES:=$(strip $(CLANG_FORMAT_FILES)) +CLANG_FORMAT_FILES:=$(filter-out $(CLANG_FORMAT_IGNORE_FILES),$(strip $(CLANG_FORMAT_FILES))) + +EMPTY:= +SPACE:=$(EMPTY) $(EMPTY) +CLANG_TIDY_HEADER_FILTER=$(CURDIR)/($(subst $(SPACE),|,$(LINTER_HEADERS))) + +%.clang-tidy: %.c + @$(CLANG_TIDY) --header-filter='$(CLANG_TIDY_HEADER_FILTER)' $< -- $(CFLAGS) 2>/dev/null + @$(CC) $(CFLAGS) $< -MM -MT $@ -MF $@.d > /dev/null 2>&1 + @touch $@ + +clangd-config: + @echo "$(CFLAGS)" | sed -e $$'s/ \{1,\}/\\\n/g' | grep -v "MMD" > compile_flags.txt + +format: + @$(CLANG_FORMAT) -i $(CLANG_FORMAT_FILES) + +check-format: + @$(CLANG_FORMAT) -Werror --dry-run $(CLANG_FORMAT_FILES) + +lint: $(CLANG_TIDY_TARGETS) + +#------------------------------------------------------------------------------- + +help: + @echo "Targets: (default: '*')" + @echo "* all - Build libcmt and host targets" + @echo " host - Build mock and tools targets" + @echo " libcmt - Build the library, tools and examples; to run on the cartesi-machine." + @echo " (requires the cartesi Linux headers to build)" + @echo " mock - Build a mocked version of the library, tools and examples; to run on the host system." + @echo " tools - Build tools on top of the mocked library to run on the host system." + @echo " test - Build and run tests on top of the mocked library on the host system." + @echo " doc - Build the documentation and API references as html." + @echo " clean - remove the binaries and objects." + @echo " install - Install the library and C headers; on the host system." + @echo " Use DESTDIR and PREFIX to customize the installation." + @echo " install-mock - Install the mocked version of the library and C headers; on the host system." + @echo " Use DESTDIR and PREFIX to customize the installation." + +doc/theme: + git clone git@github.com:jothepro/doxygen-awesome-css.git $@ + git -C doc/theme checkout 8cea9a073ecd50a5b2c0958a3df100292d6c7374 + +doc: doc/theme examples + doxygen doc/Doxyfile + +clean: + @rm -rf build + @rm -rf src/*.clang-tidy src/*.d + @rm -rf tests/*.clang-tidy tests/*.d + @rm -rf tools/*.clang-tidy tools/*.d + @rm -rf *.bin + +distclean: clean + @rm -rf doc/html doc/theme + @rm -rf compile_flags.txt + +OBJ := $(mock_OBJ) $(libcmt_OBJ) $(examples_OBJ) $(tools_OBJ) + +.PHONY: install install-run install-dev +-include $(OBJ:%.o=%.d) diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/abi.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/abi.h new file mode 100644 index 00000000..6f3958e7 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/abi.h @@ -0,0 +1,578 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_abi abi + * + * This is a C library to encode and decode Ethereum Virtual Machine (EVM) + * calldata. This format is used to interacts with contracts in the Ethereum + * ecosystem. + * + * We will cover the basic concepts required to use the API we provide, but for + * a complete reference consult the solidity specification, it can be found + * here: https://docs.soliditylang.org/en/latest/abi-spec.html + * + * ## Function Selector {#funsel} + * + * 4 bytes that identify the function name and parameter types. This is used to + * distinguish between different formats. To compute it, take the four first + * bytes of the `keccak` digest of the solidity function declaration. It should + * respect a canonical format and such as having no the variable names, check + * docs for details. For reference, for a hypotetical "FunctionName" function, + * it should look something like this: + * `keccak("FunctionName(type1,type2,...,typeN)");` + * + * ## Data Sections {#sections} + * + * After the function selector, we can start encoding the function parameters. + * One by one, left to right in two sections. `Static` for fixed sized values. + * Think ints, bools, addresses, offsets, etc. Then `Dynamic` for variable + * sized data such as the contents of @b bytes. Values that need a dynamic + * section will also have an entry in the static section. + * + * ### Static Section {#static-section} + * + * [uint](@ref cmt_abi_put_uint), [bool](@ref cmt_abi_put_bool) and + * [address](@ref cmt_abi_put_address) values are encoded directly in the + * static section. In addition to those, @b bytes gets an entry in both + * sections. The static part is done with [this](@ref cmt_abi_put_bytes_s) call. + * + * ### Dynamic Section {#dynamic-section} + * + * The Dynamic section encodes the contents of variable sized types. Every entry + * in this section requires a corresponding entry in the static section as well. + * + * So types with variable size are encoded in both sections. + * + * - `static` section gets some kind of reference / offset to the dynamic section. + * - `dynamic` section gets the actual contents + * + * In more concrete terms, the @b bytes type is encoded first with a call to @ref + * cmt_abi_put_bytes_s for its `static` section part and then with a call to + * @ref cmt_abi_put_bytes_d for its `dynamic` section part. + * + * ## Encoder + * + * Lets look at some code starting with a simple case. A function that encodes + * the function selector and a single @b address value into the buffer: + * + * @includelineno "examples/abi_encode_000.c" + * + * For @b bytes, we need both sections. static and dynamic. + * + * @includelineno "examples/abi_encode_001.c" + * + * For multiple values in the dynamic section, do them in order. + * + * @includelineno "examples/abi_encode_002.c" + * + * ## Decoder + * + * Lets look at code that decodes the examples above. We'll _check_ instead of + * _put_ for funsel. And _get_ instead _put_ for most of everything else. + * + * @includelineno "examples/abi_decode_000.c" + * + * Retrieving @b bytes is a bit different since the API doesn't do dynamic + * memory allocation. @b data points inside the @p rd buffer itself, into its + * dynamic section. This makes the API very lightweight and fast but requires + * care in its usage. If @p rd gets free'd or reused while there is still a + * reference to @p data, we'll get memory corruption. If in doublt create a + * copy of @p data and use it instead. + * + * @includelineno "examples/abi_decode_001.c" + * + * ## Complete + * + * Lets look at some code on how to tie everything together. We'll build a @b + * echo of sorts that decodes the contents of @p rd and re-encodes it into @p + * wr. We'll use previous examples as a starting point to implement encode_echo + * and decode_echo. With the entrypoint being @p f. We'll use @ref + * cmt_abi_peek_funsel to switch on the message function selector, only one + * valid case in this example. Decode and if we succeed, encode it back. If the + * caller needs the encoded size, we can compute it by storing @p wr at the + * start and compute the difference. + * + * @includelineno "examples/abi_multi.c" + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_ABI_H +#define CMT_ABI_H +#include "buf.h" +#include + +enum { + CMT_ABI_U256_LENGTH = 32, /**< length of a evm word in bytes */ + CMT_ABI_ADDRESS_LENGTH = 20, /**< length of a evm address in bytes */ +}; + +/** Compile time equivalent to @ref cmt_abi_funsel + * @note don't port. use @ref cmt_abi_funsel instead */ +#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ +#define CMT_ABI_FUNSEL(A, B, C, D) \ + (((uint32_t) (D) << 000) | ((uint32_t) (C) << 010) | ((uint32_t) (B) << 020) | ((uint32_t) (A) << 030)) +#else +#define CMT_ABI_FUNSEL(A, B, C, D) \ + (((uint32_t) (A) << 000) | ((uint32_t) (B) << 010) | ((uint32_t) (C) << 020) | ((uint32_t) (D) << 030)) +#endif + +/** EVM address */ +typedef struct cmt_abi_address { + uint8_t data[CMT_ABI_ADDRESS_LENGTH]; +} cmt_abi_address_t; + +/** EVM u256 in big endian format */ +typedef struct cmt_abi_u256 { + uint8_t data[CMT_ABI_U256_LENGTH]; +} cmt_abi_u256_t; + +typedef struct cmt_abi_bytes { + size_t length; + void *data; +} cmt_abi_bytes_t; + +/** Create a function selector from an array of bytes + * @param [in] funsel function selector bytes + * @return + * - function selector converted to big endian (as expected by EVM) */ +uint32_t cmt_abi_funsel(uint8_t a, uint8_t b, uint8_t c, uint8_t d); + +/** Create a frame for the dynamic section. Read the EVM ABI for the details + * @param [in] me reader or writer buffer + * @param [out] frame start of the parameters frame + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | + */ +int cmt_abi_mark_frame(const cmt_buf_t *me, cmt_buf_t *frame); + +// put section --------------------------------------------------------------- + +/** Encode a function selector into the buffer @p me + * + * @param [in,out] me a initialized buffer working as iterator + * @param [in] funsel function selector + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | + * + * @note A function selector can be compute it with: @ref cmt_keccak_funsel. + * It is always represented in big endian. */ +int cmt_abi_put_funsel(cmt_buf_t *me, uint32_t funsel); + +/** Encode a native endianness unsigned integer of up to 32bytes of data into + * the buffer + * + * @param [in,out] me a initialized buffer working as iterator + * @param [in] n size of @p data in bytes + * @param [in] data pointer to a integer + * + * @return + * | | | + * |-------:|------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * | -EDOM| integer not representable in @p 32 bytes | + * + * + * @code + * ... + * cmt_buf_t it = ...; + * uint64_t x = UINT64_C(0xdeadbeef); + * cmt_abi_put_uint(&it, sizeof x, &x); + * ... + * @endcode */ +int cmt_abi_put_uint(cmt_buf_t *me, size_t data_length, const void *data); + +/** Encode a big endian unsigned integer of up to 32bytes of data into the + * buffer + * + * @param [in,out] me a initialized buffer working as iterator + * @param [in] length size of @p data in bytes + * @param [in] data pointer to a integer + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * | -EDOM| integer not representable in @p data_length bytes | + * + * @code + * ... + * cmt_buf_t it = ...; + * uint8_t small[] = { + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, + * }; + * cmt_abi_put_uint(&it, sizeof small, &small); + * ... + * uint8_t big[] = { + * 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + * }; + * cmt_abi_put_uint(&it, sizeof big, &big); + * @endcode + * @note This function takes care of endianness conversions */ +int cmt_abi_put_uint_be(cmt_buf_t *me, size_t data_length, const void *data); + +/** Encode a @ref cmt_abi_u256_t into the buffer + * + * @param [in,out] me a initialized buffer working as iterator + * @param [in] data pointer to a @ref cmt_abi_u256_t + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_put_uint256(cmt_buf_t *me, const cmt_abi_u256_t *value); + +/** Encode a bool into the buffer + * + * @param [in,out] me a initialized buffer working as iterator + * @param [in] value boolean value + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * + * @code + * ... + * cmt_buf_t it = ...; + * cmt_abi_put_bool(&it, true); + * ... + * @endcode + * @note This function takes care of endianness conversions */ +int cmt_abi_put_bool(cmt_buf_t *me, bool value); + +/** Encode @p address (exactly @ref CMT_ABI_ADDRESS_LENGTH bytes) into the buffer + * + * @param [in,out] me initialized buffer + * @param [in] address a value of type @ref cmt_abi_address_t + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_put_address(cmt_buf_t *me, const cmt_abi_address_t *address); + +/** Encode the static part of @b bytes into the message, + * used in conjunction with @ref cmt_abi_put_bytes_d + * + * @param [in,out] me initialized buffer + * @param [out] offset initialize for @ref cmt_abi_put_bytes_d + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_put_bytes_s(cmt_buf_t *me, cmt_buf_t *offset); + +/** Encode the dynamic part of @b bytes into the message, + * used in conjunction with @ref cmt_abi_put_bytes_d + * + * @param [in,out] me initialized buffer + * @param [in] offset initialized from @ref cmt_abi_put_bytes_h + * @param [in] n size of @b data + * @param [in] data array of bytes + * @param [in] start starting point for offset calculation (first byte after funsel) + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +//int cmt_abi_put_bytes_d(cmt_buf_t *me, cmt_buf_t *offset, size_t n, const void *data, const void *start); +int cmt_abi_put_bytes_d(cmt_buf_t *me, cmt_buf_t *offset, const cmt_buf_t *frame, const cmt_abi_bytes_t *payload); + +/** Reserve @b n bytes of data from the buffer into @b res to be filled by the + * caller + * + * @param [in,out] me initialized buffer + * @param [in] n amount of bytes to reserve + * @param [out] res slice of bytes extracted from @p me + * @param [in] start starting point for offset calculation (first byte after funsel) + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * + * @note @p me must outlive @p res. + * Create a duplicate otherwise */ +int cmt_abi_reserve_bytes_d(cmt_buf_t *me, cmt_buf_t *of, size_t n, cmt_buf_t *out, const void *start); + +// get section --------------------------------------------------------------- + +/** Read the funsel without consuming it from the buffer @p me, 0 is returned + * if there are less than 4 bytes in the buffer. + * + * @return + * | | | + * |-------:|-------------------------| + * | != 0| function selector value | + * | 0| failure | + * + * @code + * ... + * if (cmt_buf_length(it) < 4) + * return EXIT_FAILURE; + * switch (cmt_abi_peek_funsel(it) { + * case CMT_ABI_FUNSEL(...): // known type, try to parse it + * case CMT_ABI_FUNSEL(...): // known type, try to parse it + * default: + * return EXIT_FAILURE; + * } + * @endcode */ +uint32_t cmt_abi_peek_funsel(cmt_buf_t *me); + +/** Consume funsel from the buffer @p me and ensure it matches @p expected_funsel + * + * @param [in,out] me initialized buffer + * @param [in] expected expected function selector + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * |-EBADMSG| funsel mismatch | */ +int cmt_abi_check_funsel(cmt_buf_t *me, uint32_t expected); + +/** Decode a @ref cmt_abi_u256_t from the buffer + * + * @param [in,out] me initialized buffer + * @param [out] data value of type @ref cmt_abi_u256_t + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_get_uint256(cmt_buf_t *me, cmt_abi_u256_t *value); + +/** Decode a unsigned integer of up to 32bytes, in native endianness, from the buffer + * + * @param [in,out] me initialized buffer + * @param [in] n size of @p data in bytes + * @param [out] data pointer to a integer + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * | -EDOM| integer not representable in @p data_length bytes | */ +int cmt_abi_get_uint(cmt_buf_t *me, size_t n, void *data); + +/** Decode @p length big-endian bytes, up to 32, from the buffer into @p data + * + * @param [in,out] me initialized buffer + * @param [in] length size of @p data in bytes + * @param [out] data pointer to a integer + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * | -EDOM| integer not representable in @p data_length bytes | */ +int cmt_abi_get_uint_be(cmt_buf_t *me, size_t n, void *data); + +/** Consume and decode a bool from the buffer + * + * @param [in,out] me a initialized buffer working as iterator + * @param [out] value boolean value + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * + * @code + * ... + * cmt_buf_t it = ...; + * bool value; + * int rc = cmt_abi_put_bool(&it, &value); + * assert(rc == 0); + * ... + * @endcode + * @note This function takes care of endianness conversions */ +int cmt_abi_get_bool(cmt_buf_t *me, bool *value); + +/** Consume and decode @b address from the buffer + * + * @param [in,out] me initialized buffer + * @param [out] address value of type @ref cmt_abi_address_t + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_get_address(cmt_buf_t *me, cmt_abi_address_t *value); + +/** Consume and decode the offset @p of + * + * @param [in,out] me initialized buffer + * @param [out] of offset to @p bytes data, for use in conjunction with @ref cmt_abi_get_bytes_d + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | */ +int cmt_abi_get_bytes_s(cmt_buf_t *me, cmt_buf_t of[1]); + +/** Decode @b bytes from the buffer by taking a pointer to its contents. + * + * @param [in] start initialized buffer (from the start after funsel) + * @param [out] of offset to @p bytes data + * @param [out] n amount of data available in @b bytes + * @param [out] data memory range with contents + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * + * @note @p of can be initialized by calling @ref cmt_abi_get_bytes_s */ +int cmt_abi_get_bytes_d(const cmt_buf_t *start, cmt_buf_t of[1], size_t *n, void **data); + +/** Decode @b bytes from the buffer by taking a pointer to its contents. + * + * @param [in] start initialized buffer (from the start after funsel) + * @param [out] of offset to @p bytes data + * @param [out] n amount of data available in @b bytes + * @param [out] bytes memory range with contents + * + * @return + * | | | + * |-------:|------------------------| + * | 0| success | + * |-ENOBUFS| no space left in @p me | + * + * @note @p of can be initialized by calling @ref cmt_abi_get_bytes_s */ +int cmt_abi_peek_bytes_d(const cmt_buf_t *start, cmt_buf_t of[1], cmt_buf_t *bytes); + +// raw codec section -------------------------------------------------------- + +/** Encode @p n bytes of @p data into @p out (up to 32). + * + * @param [in] n size of @p data in bytes + * @param [in] data integer value to encode into @p out + * @param [out] out encoded result + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | */ +int cmt_abi_encode_uint(size_t n, const void *data, uint8_t out[CMT_ABI_U256_LENGTH]); + +/** Encode @p n bytes of @p data into @p out (up to 32) in reverse order. + * + * @param [in] n size of @p data in bytes + * @param [in] data integer value to encode into @p out + * @param [out] out encoded result + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | + * + * @note use @ref cmt_abi_encode_uint instead */ +int cmt_abi_encode_uint_nr(size_t n, const uint8_t *data, uint8_t out[CMT_ABI_U256_LENGTH]); + +/** Encode @p n bytes of @p data into @p out (up to 32) in normal order. + * + * @param [in] n size of @p data in bytes + * @param [in] data integer value to encode into @p out + * @param [out] out encoded result + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | + * + * @note use @ref cmt_abi_encode_uint instead */ +int cmt_abi_encode_uint_nn(size_t n, const uint8_t *data, uint8_t out[CMT_ABI_U256_LENGTH]); + +/** Decode @p n bytes of @p data into @p out (up to 32). + * + * @param [in] data integer value to decode into @p out + * @param [in] n size of @p data in bytes + * @param [out] out decoded output + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | */ +int cmt_abi_decode_uint(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out); + +/** Decode @p n bytes of @p data into @p out (up to 32) in reverse order. + * + * @param [in] data integer value to decode into @p out + * @param [in] n size of @p data in bytes + * @param [out] out decoded output + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | + * + * @note if in doubt, use @ref cmt_abi_decode_uint */ +int cmt_abi_decode_uint_nr(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out); + +/** Decode @p n bytes of @p data into @p out (up to 32) in normal order. + * + * @param [in] data integer value to decode into @p out + * @param [in] n size of @p data in bytes + * @param [out] out decoded output + * + * @return + * | | | + * |-------:|---------------------------------------------------| + * | 0| success | + * | -EDOM| integer not representable in @p data_length bytes | + * + * @note if in doubt, use @ref cmt_abi_decode_uint */ +int cmt_abi_decode_uint_nn(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out); + +#endif /* CMT_ABI_H */ +/** @} */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/buf.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/buf.h new file mode 100644 index 00000000..d8259376 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/buf.h @@ -0,0 +1,82 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_buf buf + * slice of contiguous memory + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_BUF_H +#define CMT_BUF_H +#include +#include +#include + +/** A slice of contiguous memory from @b begin to @b end, as an open range. + * Size can be taken with: `end - begin`. + * + * `begin == end` indicate an empty buffer */ +typedef struct { + uint8_t *begin; /**< begin of memory region */ + uint8_t *end; /**< end of memory region */ +} cmt_buf_t; + +/** Initialize @p me buffer backed by @p data, @p length bytes in size + * + * @param [out] me a uninitialized instance + * @param [in] length size in bytes of @b data + * @param [in] data the backing memory to be used. + * + * @note @p data memory must outlive @p me. + * user must copy the contents otherwise */ +void cmt_buf_init(cmt_buf_t *me, size_t length, void *data); + +/** Split a buffer in two, @b lhs with @b lhs_length bytes and @b rhs with the rest + * + * @param [in,out] me initialized buffer + * @param [in] lhs_length bytes in @b lhs + * @param [out] lhs left hand side + * @param [out] rhs right hand side + * + * @return + * - 0 success + * - negative value on error. -ENOBUFS when length(me) < lhs_length. */ +int cmt_buf_split(const cmt_buf_t *me, size_t lhs_length, cmt_buf_t *lhs, cmt_buf_t *rhs); + +/** Length in bytes of @p me + * + * @param [in] me initialized buffer + * + * @return + * - size in bytes */ +size_t cmt_buf_length(const cmt_buf_t *me); + +/** Print the contents of @b me buffer to stdout + * + * @param [in] begin start of memory region + * @param [in] end end of memory region + * @param [in] bytes_per_line bytes per line (must be a power of 2). */ +void cmt_buf_xxd(void *begin, void *end, int bytes_per_line); + +/** Take the substring @p x from @p xs start to the first @p , (comma). + * @param [out] x substring + * @param [in,out] xs string (iterator) + * + * @note @p x points inside @p xs, make a copy if it outlives @p xs. */ +bool cmt_buf_split_by_comma(cmt_buf_t *x, cmt_buf_t *xs); + +#endif /* CMT_BUF_H */ +/** @} */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/io.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/io.h new file mode 100644 index 00000000..99f718e0 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/io.h @@ -0,0 +1,168 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_io_driver Cartesi Machine Input/Output Driver + * Low level abstraction of the Cartesi Machine kernel driver. + * + * Interaction is as follows: + * - Open the kernel device and mmap the shared memory ranges (tx and rx) with: + * @ref cmt_io_init. + * - Retrieve the memory regions from the library with: @ref cmt_io_get_tx and + * @ref cmt_io_get_rx. + * - Do requests (next input, emit output ...) by yielding the control back to + * the emulator with: @ref cmt_io_yield. + * + * Lets look at some code: + * + * @include examples/io.c + * + * There are two implementations provided: `ioctl` that interacts with the + * cartesi-machine linux driver and `mock` that reads and writes the host + * filesystem, used for testing. + * + * @section Environment variables + * + * this module exposes two environment variables: @p CMT_DEBUG and @p CMT_INPUTS. + * + * @p CMT_DEBUG prints runtime information during application execution. + * + * ``` + * CMT_DEBUG=yes ./application + * ``` + * + * @p CMT_INPUTS feeds inputs to the mock (ignored for ioctl). + * + * ``` + * CMT_INPUTS=0:advance.bin,1:inspect.bin ./application + * ``` + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_IO_H +#define CMT_IO_H +#include "buf.h" +#include + +/** Device */ +enum { + HTIF_DEVICE_YIELD = 2, /**< Yield device */ +}; + +/** Commands */ +enum { + HTIF_YIELD_CMD_AUTOMATIC = 0, /**< Automatic yield */ + HTIF_YIELD_CMD_MANUAL = 1, /**< Manual yield */ +}; + +/** Automatic reasons */ +enum { + HTIF_YIELD_AUTOMATIC_REASON_PROGRESS = 1, /**< Progress */ + HTIF_YIELD_AUTOMATIC_REASON_TX_OUTPUT = 2, /**< emit an output */ + HTIF_YIELD_AUTOMATIC_REASON_TX_REPORT = 4, /**< emit a report */ +}; + +/** Manual reasons */ +enum { + HTIF_YIELD_MANUAL_REASON_RX_ACCEPTED = 1, /**< Accept and load next input */ + HTIF_YIELD_MANUAL_REASON_RX_REJECTED = 2, /**< Reject and revert */ + HTIF_YIELD_MANUAL_REASON_TX_EXCEPTION = 4, /**< emit a exception and halt execution */ +}; + +/** Reply reason when requesting @ref HTIF_YIELD_MANUAL_REASON_RX_ACCEPTED or HTIF_YIELD_MANUAL_REASON_RX_REJECTED */ +enum { + HTIF_YIELD_REASON_ADVANCE_STATE = 0, /**< Input is advance */ + HTIF_YIELD_REASON_INSPECT_STATE = 1, /**< Input is inspect */ +}; + +typedef struct { + cmt_buf_t tx[1]; + cmt_buf_t rx[1]; + int fd; +} cmt_io_driver_ioctl_t; + +typedef struct { + cmt_buf_t tx[1]; + cmt_buf_t rx[1]; + cmt_buf_t inputs_left; + + int input_type; + char input_filename[128]; + char input_fileext[16]; + + int input_seq; + int output_seq; + int report_seq; + int exception_seq; + int gio_seq; +} cmt_io_driver_mock_t; + +/** Implementation specific cmio state. */ +typedef union cmt_io_driver { + cmt_io_driver_ioctl_t ioctl; + cmt_io_driver_mock_t mock; +} cmt_io_driver_t; + +/** yield struct cmt_io_yield */ +typedef struct cmt_io_yield { + uint8_t dev; + uint8_t cmd; + uint16_t reason; + uint32_t data; +} cmt_io_yield_t; + +/** Open the io device and initialize the driver. Release its resources with @ref cmt_io_fini. + * + * @param [in] me A uninitialized @ref cmt_io_driver state + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_io_init(cmt_io_driver_t *me); + +/** Release the driver resources and close the io device. + * + * @param [in] me A successfully initialized state by @ref cmt_io_init + * @note usage of @p me after this call is a BUG and will cause undefined behaviour */ +void cmt_io_fini(cmt_io_driver_t *me); + +/** Retrieve the transmit buffer @p tx + * + * @param [in] me A successfully initialized state by @ref cmt_io_init + * @return + * - writable memory region as defined in the setup structure (check @ref cmt_buf_t) + * @note memory is valid until @ref cmt_io_fini is called. */ +cmt_buf_t cmt_io_get_tx(cmt_io_driver_t *me); + +/** Retrieve the receive buffer @p rx + * + * @param [in] me A successfully initialized state by @ref cmt_io_init + * @return + * - readable memory region as defined in the setup structure (check @ref cmt_buf_t) + * @note memory is valid until @ref cmt_io_fini is called. */ +cmt_buf_t cmt_io_get_rx(cmt_io_driver_t *me); + +/** Perform the yield encoded in @p rr. + * + * @param [in] me A successfully initialized state by @ref cmt_io_init + * @param [in,out] rr Request and Reply + * @return + * - 0 on success + * - negative errno code on error */ +int cmt_io_yield(cmt_io_driver_t *me, cmt_io_yield_t *rr); + +#endif /* CMT_IO_H */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/ioctl.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/ioctl.h new file mode 100644 index 00000000..a13f09b6 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/ioctl.h @@ -0,0 +1,67 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_ioctl Cartesi Machine cmio ioctl interface + * ABI definitions of the Cartesi Machine `cmio` kernel driver. + * + * The structures and command numbers below are copied verbatim from the Linux + * kernel UAPI header `` and must be kept in sync with it. + * Vendoring them here allows libcmt to be compiled (and cross-compiled) without + * requiring the Cartesi kernel headers package to be installed in the sysroot. + * + * The ioctl command numbers are precomputed and assume the @b asm-generic + * `ioctl` encoding. + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_IOCTL_H +#define CMT_IOCTL_H + +#include + +/** A `cmio` shared memory region as reported by the kernel driver */ +struct cmt_ioctl_cmio_buffer { + uint64_t data; /**< physical address of the memory region */ + uint64_t length; /**< length of the memory region in bytes */ +}; + +/** Layout of the tx and rx buffers returned by @ref CMT_IOCTL_CMIO_SETUP */ +struct cmt_ioctl_cmio_setup { + struct cmt_ioctl_cmio_buffer tx; /**< transmit buffer (guest to emulator) */ + struct cmt_ioctl_cmio_buffer rx; /**< receive buffer (emulator to guest) */ +}; + +/** Return a @ref cmt_ioctl_cmio_setup structure filled with tx and rx buffer + * details. Use these values to mmap them into the user-space. + * + * @note Equivalent to the kernel's `_IOR(0xd3, 0, struct cmio_setup)`. + * + * @return + * - 0 on success. + * - -1 on error and errno is set. */ +#define CMT_IOCTL_CMIO_SETUP 0x8020d300UL + +/** Yield the machine execution and transfer control back to the emulator. + * + * @note Equivalent to the kernel's `_IOWR(0xd3, 1, uint64_t)`. + * + * @return + * - 0 on success. + * - -1 on error and errno is set. */ +#define CMT_IOCTL_CMIO_YIELD 0xc008d301UL + +#endif /* CMT_IOCTL_H */ +/** @} */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/keccak.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/keccak.h new file mode 100644 index 00000000..96ec0812 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/keccak.h @@ -0,0 +1,131 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_keccak keccak + * Keccak 256 digest + * + * Data can be inserted in pieces: + * @code + * ... + * uint8_t h[CMT_KECCAK_LENGTH]; + * cmt_keccak_t st[1]; + * + * cmt_keccak_init(st); + * cmt_keccak_update(st, 1, "h"); + * cmt_keccak_update(st, 1, "e"); + * cmt_keccak_update(st, 1, "l"); + * cmt_keccak_update(st, 1, "l"); + * cmt_keccak_update(st, 1, "o"); + * cmt_keccak_final(st, h); + * ... + * @endcode + * + * all at once: + * @code + * ... + * const char data[] = "hello"; + * uint8_t h[CMT_KECCAK_LENGTH]; + * cmt_keccak_data(h, sizeof(data)-1, data); + * ... + * @endcode + * + * or with a specialized call to generate @ref funsel data: + * @code + * ... + * uint32_t funsel = cmt_keccak_funsel("FunctionCall(address,bytes)"); + * ... + * @endcode + * + * Code based on https://github.com/mjosaarinen/tiny_sha3 with the sha3 to + * keccak change as indicated by this comment: + * https://github.com/mjosaarinen/tiny_sha3#updated-07-aug-15 + * + * @ingroup libcmtcmt + * @{ */ +#ifndef CMT_KECCAK_H +#define CMT_KECCAK_H +#include +#include + +enum { + CMT_KECCAK_LENGTH = 32, /**< Bytes in the hash digest */ +}; + +/** Opaque internal keccak state */ +typedef union cmt_keccak_state { + uint8_t b[200]; + uint64_t q[25]; +} cmt_keccak_state_t; + +/** Opaque Keccak state, used to do hash computations, initialize with: + * - @ref cmt_keccak_init + * - @ref CMT_KECCAK_INIT + * - @ref CMT_KECCAK_DECL */ +typedef struct cmt_keccak { + cmt_keccak_state_t st; + int pt, rsiz; +} cmt_keccak_t; + +/** Initialize a @ref cmt_keccak_t hasher state. + * + * @param [out] state uninitialized @ref cmt_keccak_t */ +void cmt_keccak_init(cmt_keccak_t *state); + +/** Hash @b n bytes of @b data + * + * @param [in,out] state initialize the hasher state + * @param [in] length bytes in @b data to process + * @param [in] data data to hash */ +void cmt_keccak_update(cmt_keccak_t *state, size_t n, const void *data); + +/** Finalize the hash calculation from @b state and store it in @b md + * + * @param [in] state initialize the hasher state (with all data already added to it) + * @param [out] md 32bytes to store the computed hash */ +void cmt_keccak_final(cmt_keccak_t *state, void *md); + +/** Hash all @b n bytes of @b data at once + * + * @param [in] length bytes in @b data to process + * @param [in] data data to hash + * @param [out] md 32bytes to store the computed hash + * @return pointer to @b md + * + * Equivalent to: + * @code + * cmt_keccak_t st = CMT_KECCAK_INIT(&st); + * cmt_keccak_update(&st, n, data); + * cmt_keccak_final(&st, md); + * return md; + * @endcode */ +uint8_t *cmt_keccak_data(size_t length, const void *data, uint8_t md[CMT_KECCAK_LENGTH]); + +/** Compute the function selector from the solidity declaration @p decl + * + * @param [in] decl solidity call declaration, without variable names + * @param [out] funsel function selector as described by @ref funsel + * @return A @p funsel value as if defined by @ref CMT_ABI_FUNSEL + * + * Example usage: + * @code + * ... + * uint32_t funsel = cmt_keccak_funsel("FunctionCall(address,bytes)"); + * ... + * @endcode + */ +uint32_t cmt_keccak_funsel(const char *decl); +#endif /* CMT_KECCAK_H */ +/** $@} */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/merkle.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/merkle.h new file mode 100644 index 00000000..08537c7a --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/merkle.h @@ -0,0 +1,118 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_merkle merkle + * merkle tree of keccak256 hashes + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_MERKLE_H +#define CMT_MERKLE_H +#include "keccak.h" + +enum { + CMT_MERKLE_TREE_HEIGHT = 63, /**< merkle tree height */ +}; + +/** Opaque Merkle tree state. + * initialize with: @ref cmt_merkle_init */ +typedef struct { + uint64_t leaf_count; /**< number of leaves in tree */ + uint8_t state[CMT_MERKLE_TREE_HEIGHT][CMT_KECCAK_LENGTH]; /**< hashes of complete subtrees */ +} cmt_merkle_t; + +/** Initialize a @ref cmt_merkle_t tree state. + * + * @param [in] me uninitialized state */ +void cmt_merkle_init(cmt_merkle_t *me); + +/** Resets a @ref cmt_merkle_t to pristine conditions. + * + * @param [in] me initialized state */ +void cmt_merkle_reset(cmt_merkle_t *me); + +/** Finalize a @ref cmt_merkle_t tree state. + * + * @param [in] me initialized state + * + * @note use of @p me after this call is undefined behavior. */ +void cmt_merkle_fini(cmt_merkle_t *me); + +/** Load the a @ref cmt_merkle_t tree from a @p file handle. + * + * @param [in] me either a initialized or uninitialized state + * @param [in] filepath which file to save the merkle state + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_merkle_load(cmt_merkle_t *me, const char *filepath); + +/** Save the a @ref cmt_merkle_t tree to a @p file handle. + * + * @param [in] me either a initialized or uninitialized state + * @param [in] filepath which file to save the merkle state + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_merkle_save(cmt_merkle_t *me, const char *filepath); + +/** Return number of leaves already in tree + * + * @param [in,out] me initialized state + * @return + * - leaf count */ +uint64_t cmt_merkle_get_leaf_count(cmt_merkle_t *me); + +/** Append a leaf node + * + * @param [in,out] me initialized state + * @param [in] hash value of the new leaf + * + * @return + * | | | + * |---------:|---------------------------------| + * | 0| success | + * | -ENOBUFS | indicates that the tree is full | + * | < 0| failure with a -errno value | */ +int cmt_merkle_push_back(cmt_merkle_t *me, const uint8_t hash[CMT_KECCAK_LENGTH]); + +/** Compute the keccak-256 hash of @p data and append it as a leaf node + * + * @param [in,out] me initialized state + * @param [in] length size of @p data in bytes + * @param [in] data array of bytes + * + * @return + * | | | + * |---------:|---------------------------------| + * | 0| success | + * | -ENOBUFS | indicates that the tree is full | + * | < 0| failure with a -errno value | */ +int cmt_merkle_push_back_data(cmt_merkle_t *me, size_t length, const void *data); + +/** Compute the root hash of the merkle tree + * + * @param [in] me initialized state + * @param [out] root root hash of the merkle tree */ +void cmt_merkle_get_root_hash(cmt_merkle_t *me, uint8_t root[CMT_KECCAK_LENGTH]); + +#endif /* CMT_MERKLE_H */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/rollup.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/rollup.h new file mode 100644 index 00000000..f8dcfc44 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/rollup.h @@ -0,0 +1,260 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/** @file + * @defgroup libcmt_rollup rollup + * Rollup abstraction layer + * + * Takes care of @ref libcmt_io_driver interactions, @ref libcmt_abi + * encoding/decoding and @ref libcmt_merkle tree handling. + * + * Mocked version has support for simulating I/O via environment variables: + * @p CMT_INPUTS="0:input.bin,..." and verbose output with @p CMT_DEBUG=yes. + * + * Lets look at some code: + * + * @include doc/examples/rollup.c + * + * @ingroup libcmt + * @{ */ +#ifndef CMT_ROLLUP_H +#define CMT_ROLLUP_H +#include "abi.h" +#include "io.h" +#include "merkle.h" + +typedef struct cmt_rollup { + union cmt_io_driver io[1]; + uint32_t fromhost_data; + cmt_merkle_t merkle[1]; + + // cache merkle values and repeat them on finish when the tree doesn't change + uint8_t finish_outputs_merkle_root[CMT_KECCAK_LENGTH]; + uint64_t finish_leaf_count; +} cmt_rollup_t; + +/** Public struct with the advance state contents */ +typedef struct cmt_rollup_advance { + uint64_t chain_id; /**< network */ + cmt_abi_address_t app_contract; /**< application contract address */ + cmt_abi_address_t msg_sender; /**< input sender address */ + uint64_t block_number; /**< block number of this input */ + uint64_t block_timestamp; /**< block timestamp of this input UNIX epoch format) */ + cmt_abi_u256_t prev_randao; /**< The latest RANDAO mix of the post beacon state of the previous block */ + uint64_t index; /**< input index (in relation to all inputs ever sent to the DApp) */ + cmt_abi_bytes_t payload; /**< payload for this input */ +} cmt_rollup_advance_t; + +/** Public struct with the inspect state contents */ +typedef struct cmt_rollup_inspect { + cmt_abi_bytes_t payload; /**< payload for this input */ +} cmt_rollup_inspect_t; + +/** Public struct with the finish state contents */ +typedef struct cmt_rollup_finish_s { + bool accept_previous_request; + int next_request_type; + uint32_t next_request_payload_length; +} cmt_rollup_finish_t; + +/** Public struct with generic io request/response */ +typedef struct cmt_gio { + uint16_t domain; /**< domain for the gio request */ + uint32_t id_length; /**< length of id */ + void *id; /**< id for the request */ + uint16_t response_code; /**< response */ + uint32_t response_data_length; /**< length of response data */ + void *response_data; /**< response data */ +} cmt_gio_t; + +/** Initialize a @ref cmt_rollup_t state. + * + * @param [in] me uninitialized state + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_init(cmt_rollup_t *me); + +/** Finalize a @ref cmt_rollup_t state previously initialized with @ref + * cmt_rollup_init + * + * @param [in] me initialized state + * + * @note use of @p me after this call is undefined behavior. */ +void cmt_rollup_fini(cmt_rollup_t *me); + +/** Emit a voucher + * + * Equivalent to the `Voucher(address,uint256,bytes)` solidity call. + * + * @param [in,out] me initialized @ref cmt_rollup_t instance + * @param [in] address destination data + * @param [in] value value data + * @param [in] data message contents + * @param [out] index index of emitted voucher, if successful + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_emit_voucher(cmt_rollup_t *me, const cmt_abi_address_t *address, const cmt_abi_u256_t *value, const cmt_abi_bytes_t *data, uint64_t *index); + +/** Emit a delegate call voucher + * + * Equivalent to the `DelegateCallVoucher(address,bytes)` solidity call. + * + * @param [in,out] me initialized @ref cmt_rollup_t instance + * @param [in] address destination data + * @param [in] data message contents + * @param [out] index index of emitted voucher, if successful + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_emit_delegate_call_voucher(cmt_rollup_t *me, const cmt_abi_address_t *address, const cmt_abi_bytes_t *data, uint64_t *index); + +/** Emit a notice + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] data message contents + * @param [out] index index of emitted notice, if successful + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_emit_notice(cmt_rollup_t *me, const cmt_abi_bytes_t *payload, uint64_t *index); + +/** Emit a report + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] n sizeof @p data in bytes + * @param [in] data message contents + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_emit_report(cmt_rollup_t *me, const cmt_abi_bytes_t *payload); + +/** Emit a exception + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] data_length data length in bytes + * @param [in] data message contents + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_emit_exception(cmt_rollup_t *me, const cmt_abi_bytes_t *payload); + +/** Report progress + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] progress progress value to be set + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_progress(cmt_rollup_t *me, uint32_t progress); + +/** Read advance state + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [out] advance cmt_rollup_advance_t instance (may be uninitialized) + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_read_advance_state(cmt_rollup_t *me, cmt_rollup_advance_t *advance); + +/** Read inspect state + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [out] inspect cmt_rollup_inspect_t instance (may be uninitialized) + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_read_inspect_state(cmt_rollup_t *me, cmt_rollup_inspect_t *inspect); + +/** Finish processing of current advance or inspect. + * Waits for and returns the next advance or inspect query when available. + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in,out] finish initialized cmt_rollup_finish_t instance + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_finish(cmt_rollup_t *me, cmt_rollup_finish_t *finish); + +/** Performs a generic IO request + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in,out] req initialized cmt_gio_t structure + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_gio_request(cmt_rollup_t *me, cmt_gio_t *req); + +/** Retrieve the merkle tree and intermediate state from a file @p path + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] file path to file (parent directories must exist) + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_load_merkle(cmt_rollup_t *me, const char *path); + +/** Store the merkle tree and intermediate state to a file @p path + * + * @param [in,out] me initialized cmt_rollup_t instance + * @param [in] file path to file (parent directories must exist) + * + * @return + * | | | + * |--:|-----------------------------| + * | 0| success | + * |< 0| failure with a -errno value | */ +int cmt_rollup_save_merkle(cmt_rollup_t *me, const char *path); + +/** Resets the merkle tree to pristine conditions + * + * @param [in,out] me initialized cmt_rollup_t instance */ +void cmt_rollup_reset_merkle(cmt_rollup_t *me); + +#endif /* CMT_ROLLUP_H */ diff --git a/cartesi-tools/libcmt-sys/libcmt/include/libcmt/util.h b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/util.h new file mode 100644 index 00000000..87d8c0ed --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/include/libcmt/util.h @@ -0,0 +1,34 @@ +#ifndef CMT_UTIL_H +#define CMT_UTIL_H +#include + +/** + */ +bool cmt_util_debug_enabled(void); + +/** Read whole file `name` contents into `data` and set `length`. + * @param name[in] - file path + * @param max[in] - size of `data` in bytes + * @param data[out] - file contents + * @param length[out] - actual size in `bytes` written to `data` + * + * @return + * | | | + * |-----|--------------------| + * | 0 |success | + * | < 0 |negative errno value| */ +int cmt_util_read_whole_file(const char *name, size_t max, void *data, size_t *length); + +/** Write the contents of `data` into file `name`. + * @param name[in] - file path + * @param length[in] - size of `data` in bytes + * @param data[out] - file contents + * + * @return + * | | | + * |-----|--------------------| + * | 0 |success | + * | < 0 |negative errno value| */ +int cmt_util_write_whole_file(const char *name, size_t length, const void *data); + +#endif /* CMT_UTIL_H */ diff --git a/cartesi-tools/libcmt-sys/libcmt/src/abi.c b/cartesi-tools/libcmt-sys/libcmt/src/abi.c new file mode 100644 index 00000000..bfa1d971 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/abi.c @@ -0,0 +1,315 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/abi.h" + +#include +#include + +static uintptr_t align_forward(uintptr_t p, size_t a) { + return (p + (a - 1)) & ~(a - 1); +} + +uint32_t cmt_abi_funsel(uint8_t a, uint8_t b, uint8_t c, uint8_t d) { + return CMT_ABI_FUNSEL(a, b, c, d); +} + +int cmt_abi_mark_frame(const cmt_buf_t *me, cmt_buf_t *frame) { + if (!me || !frame) { + return -EINVAL; + } + *frame = *me; + return 0; +} + +int cmt_abi_put_funsel(cmt_buf_t *me, uint32_t funsel) { + cmt_buf_t x[1]; + int rc = cmt_buf_split(me, sizeof(funsel), x, me); + if (rc) { + return rc; + } + // NOLINTNEXTLINE(clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + memcpy(x->begin, &funsel, sizeof(funsel)); + return 0; +} + +int cmt_abi_encode_uint_nr(size_t n, const uint8_t *data, uint8_t out[CMT_ABI_U256_LENGTH]) { + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + for (size_t i = 0; i < n; ++i) { + out[CMT_ABI_U256_LENGTH - 1 - i] = data[i]; + } + for (size_t i = n; i < CMT_ABI_U256_LENGTH; ++i) { + out[CMT_ABI_U256_LENGTH - 1 - i] = 0; + } + return 0; +} + +int cmt_abi_encode_uint_nn(size_t n, const uint8_t *data, uint8_t out[CMT_ABI_U256_LENGTH]) { + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + for (size_t i = 0; i < CMT_ABI_U256_LENGTH - n; ++i) { + out[i] = 0; + } + for (size_t i = CMT_ABI_U256_LENGTH - n; i < CMT_ABI_U256_LENGTH; ++i) { + out[i] = data[i - CMT_ABI_U256_LENGTH + n]; + } + return 0; +} + +int cmt_abi_encode_uint(size_t n, const void *data, uint8_t out[CMT_ABI_U256_LENGTH]) { +#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ + return cmt_abi_encode_uint_nn(n, data, out); +#else + return cmt_abi_encode_uint_nr(n, data, out); +#endif +} + +int cmt_abi_decode_uint_nr(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out) { + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + for (size_t i = 0; i < CMT_ABI_U256_LENGTH - n; ++i) { + if (data[i]) { + return -EDOM; + } + } + for (size_t i = CMT_ABI_U256_LENGTH - n; i < CMT_ABI_U256_LENGTH; ++i) { + out[CMT_ABI_U256_LENGTH - 1 - i] = data[i]; + } + return 0; +} + +int cmt_abi_decode_uint_nn(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out) { + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + for (size_t i = 0; i < CMT_ABI_U256_LENGTH - n; ++i) { + if (data[i]) { + return -EDOM; + } + } + for (size_t i = CMT_ABI_U256_LENGTH - n; i < CMT_ABI_U256_LENGTH; ++i) { + out[i - CMT_ABI_U256_LENGTH + n] = data[i]; + } + return 0; +} + +int cmt_abi_decode_uint(const uint8_t data[CMT_ABI_U256_LENGTH], size_t n, uint8_t *out) { +#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ + return cmt_abi_decode_uint_nn(data, n, out); +#else + return cmt_abi_decode_uint_nr(data, n, out); +#endif +} + +int cmt_abi_put_uint(cmt_buf_t *me, size_t data_length, const void *data) { + cmt_buf_t x[1]; + if (data_length > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + if (cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me)) { + return -ENOBUFS; + } + return cmt_abi_encode_uint(data_length, data, x->begin); +} + +int cmt_abi_put_uint_be(cmt_buf_t *me, size_t data_length, const void *data) { + cmt_buf_t x[1]; + if (data_length > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + if (cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me)) { + return -ENOBUFS; + } + return cmt_abi_encode_uint_nn(data_length, data, x->begin); +} +int cmt_abi_put_uint256(cmt_buf_t *me, const cmt_abi_u256_t *value) { + cmt_buf_t x[1]; + if (cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me)) { + return -ENOBUFS; + } + return cmt_abi_encode_uint_nn(sizeof(*value), value->data, x->begin); +} + +int cmt_abi_put_bool(cmt_buf_t *me, bool value) { + uint8_t boolean = !!value; + return cmt_abi_put_uint(me, sizeof(boolean), &boolean); +} + +int cmt_abi_put_address(cmt_buf_t *me, const cmt_abi_address_t *address) { + cmt_buf_t x[1]; + if (cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me)) { + return -ENOBUFS; + } + return cmt_abi_encode_uint_nn(sizeof(*address), address->data, x->begin); +} + +int cmt_abi_put_bytes_s(cmt_buf_t *me, cmt_buf_t *offset) { + return cmt_buf_split(me, CMT_ABI_U256_LENGTH, offset, me); +} + +int cmt_abi_reserve_bytes_d(cmt_buf_t *me, cmt_buf_t *of, size_t n, cmt_buf_t *out, const void *start) { + int rc = 0; + cmt_buf_t tmp[1]; + cmt_buf_t sz[1]; + size_t n32 = align_forward(n, CMT_ABI_U256_LENGTH); + + rc = cmt_buf_split(me, CMT_ABI_U256_LENGTH, sz, tmp); + if (rc) { + return rc; + } + rc = cmt_buf_split(tmp, n32, out, tmp); + if (rc) { + return rc; + } + + size_t offset = sz->begin - (uint8_t *) start; + rc = cmt_abi_encode_uint(sizeof(offset), &offset, of->begin); + if (rc) { + return rc; + } + rc = cmt_abi_encode_uint(sizeof(n), &n, sz->begin); + if (rc) { + return rc; + } + + *me = *tmp; // commit the buffer changes + // NOLINTNEXTLINE(clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + memset(out->begin + n, 0, n32 - n); // zero out the padding + return 0; +} + +int cmt_abi_put_bytes_d(cmt_buf_t *me, cmt_buf_t *offset, const cmt_buf_t *frame, const cmt_abi_bytes_t *payload) { + cmt_buf_t res[1]; + int rc = cmt_abi_reserve_bytes_d(me, offset, payload->length, res, frame->begin); + if (rc) { + return rc; + } + // NOLINTNEXTLINE(clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + memcpy(res->begin, payload->data, payload->length); + return 0; +} + +uint32_t cmt_abi_peek_funsel(cmt_buf_t *me) { + if (cmt_buf_length(me) < 4) { + return 0; + } + return CMT_ABI_FUNSEL(me->begin[0], me->begin[1], me->begin[2], me->begin[3]); +} + +int cmt_abi_check_funsel(cmt_buf_t *me, uint32_t expected) { + if (cmt_buf_length(me) < 4) { + return -ENOBUFS; + } + + if (cmt_abi_peek_funsel(me) != expected) { + return -EBADMSG; + } + + me->begin += 4; + return 0; +} + +int cmt_abi_get_uint(cmt_buf_t *me, size_t n, void *data) { + cmt_buf_t x[1]; + + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + int rc = cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me); + if (rc) { + return rc; + } + + return cmt_abi_decode_uint(x->begin, n, data); +} + +int cmt_abi_get_uint_be(cmt_buf_t *me, size_t n, void *data) { + cmt_buf_t x[1]; + + if (n > CMT_ABI_U256_LENGTH) { + return -EDOM; + } + int rc = cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me); + if (rc) { + return rc; + } + + return cmt_abi_decode_uint_nn(x->begin, n, data); +} + +int cmt_abi_get_uint256(cmt_buf_t *me, cmt_abi_u256_t *value) { + cmt_buf_t x[1]; + if (cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me)) { + return -ENOBUFS; + } + return cmt_abi_decode_uint_nn(x->begin, sizeof(*value), value->data); +} + +int cmt_abi_get_bool(cmt_buf_t *me, bool *value) { + bool boolean = 0; + int rc = cmt_abi_get_uint(me, sizeof(boolean), &boolean); + if (rc) { + return rc; + } + *value = boolean; + return 0; +} + +int cmt_abi_get_address(cmt_buf_t *me, cmt_abi_address_t *address) { + cmt_buf_t x[1]; + + int rc = cmt_buf_split(me, CMT_ABI_U256_LENGTH, x, me); + if (rc) { + return rc; + } + return cmt_abi_decode_uint_nn(x->begin, sizeof(*address), address->data); +} + +int cmt_abi_get_bytes_s(cmt_buf_t *me, cmt_buf_t of[1]) { + return cmt_buf_split(me, CMT_ABI_U256_LENGTH, of, me); +} + +int cmt_abi_peek_bytes_d(const cmt_buf_t *start, cmt_buf_t of[1], cmt_buf_t *bytes) { + int rc = 0; + uint64_t offset = 0; + uint64_t size = 0; + rc = cmt_abi_get_uint(of, sizeof(offset), &offset); + if (rc) { + return rc; + } + + /* from the beginning, after funsel */ + cmt_buf_t it[1] = {{start->begin + offset, start->end}}; + rc = cmt_abi_get_uint(it, sizeof(size), &size); + if (rc) { + return rc; + } + return cmt_buf_split(it, size, bytes, it); +} + +int cmt_abi_get_bytes_d(const cmt_buf_t *start, cmt_buf_t of[1], size_t *n, void **data) { + cmt_buf_t bytes[1]; + int rc = cmt_abi_peek_bytes_d(start, of, bytes); + if (rc) { + return rc; + } + *n = cmt_buf_length(bytes); + *data = bytes->begin; + return 0; +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/buf.c b/cartesi-tools/libcmt-sys/libcmt/src/buf.c new file mode 100644 index 00000000..e2cbf5ae --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/buf.c @@ -0,0 +1,121 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/buf.h" + +#include +#include +#include + +static inline int is_pow2(int l) { + return !(l & (l - 1)); +} + +void cmt_buf_init(cmt_buf_t *me, size_t length, void *data) { + if (!me) { + return; + } + me->begin = (uint8_t *) data; + me->end = (uint8_t *) data + length; +} + +int cmt_buf_split(const cmt_buf_t *me, size_t lhs_length, cmt_buf_t *lhs, cmt_buf_t *rhs) { + if (!me) { + return -EINVAL; + } + if (!lhs) { + return -EINVAL; + } + if (!rhs) { + return -EINVAL; + } + + uint8_t *begin = me->begin; + uint8_t *split = me->begin + lhs_length; + uint8_t *end = me->end; + + if (split < begin || end < split) { + return -ENOBUFS; + } + + lhs->begin = begin; + lhs->end = split; + rhs->begin = split; + rhs->end = end; + + return 0; +} + +static int comma(const uint8_t *s, const uint8_t *end) { + return s < end && *s == ','; +} + +static bool cmt_buf_split_by(cmt_buf_t *x, cmt_buf_t *xs, int (*nxt)(const uint8_t *s, const uint8_t *end)) { + if (xs->begin == xs->end) { + return false; + } + x->begin = xs->begin; + for (; *xs->begin && xs->begin < xs->end && !nxt(xs->begin, xs->end); ++xs->begin) { + ; + } + x->end = xs->begin; + xs->begin = xs->begin + nxt(xs->begin, xs->end); + return *x->begin; +} + +bool cmt_buf_split_by_comma(cmt_buf_t *x, cmt_buf_t *xs) { + return cmt_buf_split_by(x, xs, comma); +} + +size_t cmt_buf_length(const cmt_buf_t *me) { + if (!me) { + return 0; + } + return me->end - me->begin; +} + +static void xxd(const uint8_t *p, const uint8_t *q, size_t mask) { + if (q < p) { + return; + } + + for (size_t i = 0U, n = q - p; i < n; ++i) { + bool is_line_start = (i & mask) == 0; + bool is_line_end = (i & mask) == mask || (i + 1 == n); + char separator = is_line_end ? '\n' : ' '; + + if (is_line_start) { + printf("%p %4zu: ", (void *) (p + i), i); + } + printf("%02x%c", p[i], separator); + } +} + +void cmt_buf_xxd(void *begin, void *end, int bytes_per_line) { + if (!begin) { + return; + } + if (!end) { + return; + } + if (end <= begin) { + return; + } + if (!is_pow2(bytes_per_line)) { + return; + } + + xxd(begin, end, bytes_per_line - 1); +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/io-mock.c b/cartesi-tools/libcmt-sys/libcmt/src/io-mock.c new file mode 100644 index 00000000..f19c14e9 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/io-mock.c @@ -0,0 +1,319 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/io.h" +#include "libcmt/util.h" + +#include +#include +#include +#include + +/** track the number of open "devices". Mimic the kernel driver behavior by limiting it to 1 */ +static int open_count = 0; + +int cmt_io_init(cmt_io_driver_t *_me) { + if (!_me) { + return -EINVAL; + } + if (open_count) { + return -EBUSY; + } + + open_count++; + cmt_io_driver_mock_t *me = &_me->mock; + + size_t tx_length = 2U << 20; // 2MB + size_t rx_length = 2U << 20; // 2MB + cmt_buf_init(me->tx, tx_length, malloc(tx_length)); + cmt_buf_init(me->rx, rx_length, malloc(rx_length)); + + if (!me->tx->begin || !me->rx->begin) { + free(me->tx->begin); + free(me->rx->begin); + return -ENOMEM; + } + + char *inputs = getenv("CMT_INPUTS"); + if (inputs) { + cmt_buf_init(&me->inputs_left, strlen(inputs), inputs); + } else { + cmt_buf_init(&me->inputs_left, 0, ""); + } + + // in case the user writes something before loading any input + strcpy(me->input_filename, "none"); + strcpy(me->input_fileext, ".bin"); + + me->input_seq = 0; + me->output_seq = 0; + me->report_seq = 0; + me->exception_seq = 0; + me->gio_seq = 0; + + return 0; +} + +void cmt_io_fini(cmt_io_driver_t *_me) { + if (!_me) { + return; + } + + if (open_count == 0) { + return; + } + + open_count--; + cmt_io_driver_mock_t *me = &_me->mock; + + free(me->tx->begin); + free(me->rx->begin); + + memset(_me, 0, sizeof(*_me)); +} + +cmt_buf_t cmt_io_get_tx(cmt_io_driver_t *me) { + cmt_buf_t empty = {NULL, NULL}; + if (!me) { + return empty; + } + return *me->mock.tx; +} + +cmt_buf_t cmt_io_get_rx(cmt_io_driver_t *me) { + cmt_buf_t empty = {NULL, NULL}; + if (!me) { + return empty; + } + return *me->mock.rx; +} + +static int load_next_input(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + cmt_buf_t current_input; + char filepath[128] = {0}; + if (!cmt_buf_split_by_comma(¤t_input, &me->inputs_left)) { + return -EINVAL; + } + // NOLINTNEXTLINE(cert-err34-c,clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + if (sscanf((char *) current_input.begin, "%d:%127[^,]", &me->input_type, filepath) != 2) { + return -EINVAL; + } + + size_t file_length = 0; + int rc = cmt_util_read_whole_file(filepath, cmt_buf_length(me->rx), me->rx->begin, &file_length); + if (rc) { + if (cmt_util_debug_enabled()) { + (void) fprintf(stderr, "failed to load \"%s\". %s\n", filepath, strerror(-rc)); + } + return rc; + } + // NOLINTNEXTLINE(clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + if (sscanf(filepath, " %127[^.]%15s", me->input_filename, me->input_fileext) != 2) { + if (cmt_util_debug_enabled()) { + (void) fprintf(stderr, "failed to parse filename: \"%s\"\n", filepath); + } + return -EINVAL; + } + rr->reason = me->input_type; + rr->data = file_length; + + me->output_seq = 0; + me->report_seq = 0; + me->exception_seq = 0; + + if (cmt_util_debug_enabled()) { + (void) fprintf(stderr, "processing filename: \"%s\" (%lu), type: %d\n", filepath, file_length, me->input_type); + } + return 0; +} + +static int store_output(cmt_io_driver_mock_t *me, const char *filepath, struct cmt_io_yield *rr) { + if (rr->data > cmt_buf_length(me->rx)) { + return -ENOBUFS; + } + + int rc = cmt_util_write_whole_file(filepath, rr->data, me->tx->begin); + if (rc) { + (void) fprintf(stderr, "failed to store \"%s\". %s\n", filepath, strerror(-rc)); + return rc; + } + if (cmt_util_debug_enabled()) { + (void) fprintf(stderr, "wrote filename: \"%s\" (%u)\n", filepath, rr->data); + } + return 0; +} + +static int store_next_output(cmt_io_driver_mock_t *me, char *ns, int *seq, struct cmt_io_yield *rr) { + char filepath[128 + 32 + 8 + 16]; + // NOLINTNEXTLINE(cert-err33-c, clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + snprintf(filepath, sizeof filepath, "%s.%s%d%s", me->input_filename, ns, (*seq)++, me->input_fileext); + return store_output(me, filepath, rr); +} + +static int mock_progress(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + (void) me; + if (rr->cmd != HTIF_YIELD_CMD_AUTOMATIC) { + (void) fprintf(stderr, "Expected cmd to be AUTOMATIC\n"); + return -EINVAL; + } + (void) fprintf(stderr, "Progress: %6.2f\n", (double) rr->data / 10); + return 0; +} + +static int mock_rx_accepted(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + if (rr->cmd != HTIF_YIELD_CMD_MANUAL) { + (void) fprintf(stderr, "Expected cmd to be MANUAL\n"); + return -EINVAL; + } + if (me->input_seq++) { // skip the first + char filepath[128 + 32 + 8 + 16]; + // NOLINTNEXTLINE(cert-err33-c, clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling) + snprintf(filepath, sizeof filepath, "%s.outputs_merkle_root%s", me->input_filename, me->input_fileext); + int rc = store_output(me, filepath, rr); + if (rc) { + return rc; + } + } + if (load_next_input(me, rr)) { + return -ENODATA; + } + return 0; +} + +static int mock_rx_rejected(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + if (rr->cmd != HTIF_YIELD_CMD_MANUAL) { + (void) fprintf(stderr, "Expected cmd to be MANUAL\n"); + return -EINVAL; + } + (void) fprintf(stderr, "%s:%d no revert for the mock implementation\n", __FILE__, __LINE__); + if (load_next_input(me, rr)) { + return -ENOSYS; + } + return 0; +} + +static int mock_tx_output(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + if (rr->cmd != HTIF_YIELD_CMD_AUTOMATIC) { + (void) fprintf(stderr, "Expected cmd to be AUTOMATIC\n"); + return -EINVAL; + } + return store_next_output(me, "output-", &me->output_seq, rr); +} + +static int mock_tx_report(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + if (rr->cmd != HTIF_YIELD_CMD_AUTOMATIC) { + (void) fprintf(stderr, "Expected cmd to be AUTOMATIC\n"); + return -EINVAL; + } + return store_next_output(me, "report-", &me->report_seq, rr); +} + +static int mock_tx_exception(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + if (rr->cmd != HTIF_YIELD_CMD_MANUAL) { + (void) fprintf(stderr, "Expected cmd to be MANUAL\n"); + return -EINVAL; + } + return store_next_output(me, "exception-", &me->exception_seq, rr); +} + +static int mock_tx_gio(cmt_io_driver_mock_t *me, struct cmt_io_yield *rr) { + int rc = 0; + if (rr->cmd != HTIF_YIELD_CMD_MANUAL) { + (void) fprintf(stderr, "Expected cmd to be MANUAL\n"); + return -EINVAL; + } + + rc = store_next_output(me, "gio-", &me->gio_seq, rr); + if (rc) { + return rc; + } + + rc = load_next_input(me, rr); + if (rc) { + return rc; + } + return 0; +} + +/* These behaviours are defined by the cartesi-machine emulator */ +static int cmt_io_yield_inner(cmt_io_driver_t *_me, struct cmt_io_yield *rr) { + cmt_io_driver_mock_t *me = &_me->mock; + + if (rr->cmd == HTIF_YIELD_CMD_MANUAL) { + switch (rr->reason) { + case HTIF_YIELD_MANUAL_REASON_RX_ACCEPTED: + return mock_rx_accepted(me, rr); + case HTIF_YIELD_MANUAL_REASON_RX_REJECTED: + return mock_rx_rejected(me, rr); + case HTIF_YIELD_MANUAL_REASON_TX_EXCEPTION: + return mock_tx_exception(me, rr); + default: + return mock_tx_gio(me, rr); + } + } else if (rr->cmd == HTIF_YIELD_CMD_AUTOMATIC) { + switch (rr->reason) { + case HTIF_YIELD_AUTOMATIC_REASON_PROGRESS: + return mock_progress(me, rr); + case HTIF_YIELD_AUTOMATIC_REASON_TX_OUTPUT: + return mock_tx_output(me, rr); + case HTIF_YIELD_AUTOMATIC_REASON_TX_REPORT: + return mock_tx_report(me, rr); + default: + return -EINVAL; + } + } else { + return -EINVAL; + } + + return 0; +} + +/* emulate io.c:cmt_io_yield behavior (go and check it does if you change it) */ +int cmt_io_yield(cmt_io_driver_t *_me, struct cmt_io_yield *rr) { + if (!_me) { + return -EINVAL; + } + if (!rr) { + return -EINVAL; + } + + bool debug = cmt_util_debug_enabled(); + if (debug) { + (void) fprintf(stderr, + "tohost {\n" + "\t.dev = %d,\n" + "\t.cmd = %d,\n" + "\t.reason = %d,\n" + "\t.data = %d,\n" + "};\n", + rr->dev, rr->cmd, rr->reason, rr->data); + } + int rc = cmt_io_yield_inner(_me, rr); + if (rc) { + return rc; + } + + if (debug) { + (void) fprintf(stderr, + "fromhost {\n" + "\t.dev = %d,\n" + "\t.cmd = %d,\n" + "\t.reason = %d,\n" + "\t.data = %d,\n" + "};\n", + rr->dev, rr->cmd, rr->reason, rr->data); + } + return rc; +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/io.c b/cartesi-tools/libcmt-sys/libcmt/src/io.c new file mode 100644 index 00000000..1e9129c5 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/io.c @@ -0,0 +1,163 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/io.h" +#include "libcmt/ioctl.h" +#include "libcmt/util.h" + +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +int cmt_io_init(cmt_io_driver_t *_me) { + int rc = 0; + + if (!_me) { + return -EINVAL; + } + cmt_io_driver_ioctl_t *me = &_me->ioctl; + me->fd = open("/dev/cmio", O_RDWR); + + if (me->fd < 0) { + rc = -errno; + return rc; + } + + struct cmt_ioctl_cmio_setup setup; + if (ioctl(me->fd, CMT_IOCTL_CMIO_SETUP, &setup)) { + rc = -errno; + goto do_close; + } + + void *tx = mmap((void *) setup.tx.data, setup.tx.length, PROT_READ | PROT_WRITE, MAP_SHARED, me->fd, 0); + if (tx == MAP_FAILED) { + rc = -errno; + goto do_close; + } + + void *rx = mmap((void *) setup.rx.data, setup.rx.length, PROT_READ, MAP_SHARED, me->fd, 0); + if (rx == MAP_FAILED) { + rc = -errno; + goto do_unmap; + } + + cmt_buf_init(me->tx, setup.tx.length, tx); + cmt_buf_init(me->rx, setup.rx.length, rx); + return 0; + +do_unmap: + munmap(tx, setup.tx.length); +do_close: + close(me->fd); + return rc; +} + +void cmt_io_fini(cmt_io_driver_t *_me) { + if (!_me) { + return; + } + cmt_io_driver_ioctl_t *me = &_me->ioctl; + + munmap(me->tx->begin, cmt_buf_length(me->tx)); + munmap(me->rx->begin, cmt_buf_length(me->rx)); + close(me->fd); + + memset(me, 0, sizeof(*me)); + me->fd = -1; +} + +cmt_buf_t cmt_io_get_tx(cmt_io_driver_t *me) { + const cmt_buf_t empty = {NULL, NULL}; + if (!me) { + return empty; + } + return *me->ioctl.tx; +} + +cmt_buf_t cmt_io_get_rx(cmt_io_driver_t *me) { + const cmt_buf_t empty = {NULL, NULL}; + if (!me) { + return empty; + } + return *me->ioctl.rx; +} + +static uint64_t pack(struct cmt_io_yield *rr) { + // clang-format off + return ((uint64_t) rr->dev << 56) + | ((uint64_t) rr->cmd << 56 >> 8) + | ((uint64_t) rr->reason << 48 >> 16) + | ((uint64_t) rr->data << 32 >> 32); + // clang-format on +} + +static struct cmt_io_yield unpack(uint64_t x) { + // clang-format off + struct cmt_io_yield out = { + x >> 56, + x << 8 >> 56, + x << 16 >> 48, + x << 32 >> 32, + }; + // clang-format on + return out; +} + +/* io-mock.c:cmt_io_yield emulates this behavior (go and check it does if you change it) */ +int cmt_io_yield(cmt_io_driver_t *_me, struct cmt_io_yield *rr) { + if (!_me) { + return -EINVAL; + } + if (!rr) { + return -EINVAL; + } + cmt_io_driver_ioctl_t *me = &_me->ioctl; + + bool debug = cmt_util_debug_enabled(); + if (debug) { + (void) fprintf(stderr, + "tohost {\n" + "\t.dev = %d,\n" + "\t.cmd = %d,\n" + "\t.reason = %d,\n" + "\t.data = %d,\n" + "};\n", + rr->dev, rr->cmd, rr->reason, rr->data); + } + uint64_t req = pack(rr); + if (ioctl(me->fd, CMT_IOCTL_CMIO_YIELD, &req)) { + return -errno; + } + *rr = unpack(req); + + if (debug) { + (void) fprintf(stderr, + "fromhost {\n" + "\t.dev = %d,\n" + "\t.cmd = %d,\n" + "\t.reason = %d,\n" + "\t.data = %d,\n" + "};\n", + rr->dev, rr->cmd, rr->reason, rr->data); + } + return 0; +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/keccak.c b/cartesi-tools/libcmt-sys/libcmt/src/keccak.c new file mode 100644 index 00000000..6b82e4cb --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/keccak.c @@ -0,0 +1,156 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/keccak.h" +#include "libcmt/abi.h" + +#include + +// Helper macros for stringification +#define TO_STRING_HELPER(X) #X +#define TO_STRING(X) TO_STRING_HELPER(X) + +// Define loop unrolling depending on the compiler +#if defined(__clang__) +#define UNROLL_LOOP(n) _Pragma(TO_STRING(unroll(n))) +#elif defined(__GNUC__) && !defined(__clang__) +#define UNROLL_LOOP(n) _Pragma(TO_STRING(GCC unroll(n))) +#else +#define UNROLL_LOOP(n) +#endif + +#define ROTL64(x, y) (((x) << (y)) | ((x) >> (64 - (y)))) + +/** Initialize a keccak state + * @note don't port. use @ref cmt_keccak_init */ +// clang-format off +#define CMT_KECCAK_INIT(STATE) \ + { \ + .st.q = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }, \ + .pt = 0, .rsiz = 200 - 2 * CMT_KECCAK_LENGTH, \ + } + +// clang-format on + +static void keccakf(uint64_t st[25]) { + // clang-format off + const uint64_t keccakf_rndc[24] = { + 0x0000000000000001, 0x0000000000008082, 0x800000000000808a, 0x8000000080008000, 0x000000000000808b, + 0x0000000080000001, 0x8000000080008081, 0x8000000000008009, 0x000000000000008a, 0x0000000000000088, + 0x0000000080008009, 0x000000008000000a, 0x000000008000808b, 0x800000000000008b, 0x8000000000008089, + 0x8000000000008003, 0x8000000000008002, 0x8000000000000080, 0x000000000000800a, 0x800000008000000a, + 0x8000000080008081, 0x8000000000008080, 0x0000000080000001, 0x8000000080008008 + }; + const int keccakf_rotc[24] = { 1, 3, 6, 10, 15, 21, 28, 36, 45, 55, 2, 14, + 27, 41, 56, 8, 25, 43, 62, 18, 39, 61, 20, 44 }; + const int keccakf_piln[24] = { 10, 7, 11, 17, 18, 3, 5, 16, 8, 21, 24, 4, + 15, 23, 19, 13, 12, 2, 20, 14, 22, 9, 6, 1 }; + // clang-format on + +#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ + for (int i = 0; i < 25; i++) { + st[i] = __builtin_bswap64((uint64_t *) (st[i])); + } +#endif + + for (int r = 0; r < 24; r++) { + uint64_t t = 0; + uint64_t bc[5]; + + // Theta + UNROLL_LOOP(5) + for (int i = 0; i < 5; i++) { + bc[i] = st[i] ^ st[i + 5] ^ st[i + 10] ^ st[i + 15] ^ st[i + 20]; + } + + UNROLL_LOOP(5) + for (int i = 0; i < 5; i++) { + t = bc[(i + 4) % 5] ^ ROTL64(bc[(i + 1) % 5], 1); + for (int j = 0; j < 25; j += 5) { + st[j + i] ^= t; + } + } + + // Rho Pi + t = st[1]; + UNROLL_LOOP(24) + for (int i = 0; i < 24; i++) { + int j = keccakf_piln[i]; + bc[0] = st[j]; + st[j] = ROTL64(t, keccakf_rotc[i]); + t = bc[0]; + } + + // Chi + UNROLL_LOOP(25) + for (int j = 0; j < 25; j += 5) { + for (int i = 0; i < 5; i++) { + bc[i] = st[j + i]; + } + for (int i = 0; i < 5; i++) { + st[j + i] ^= (~bc[(i + 1) % 5]) & bc[(i + 2) % 5]; + } + } + + // Iota + st[0] ^= keccakf_rndc[r]; + } + +#if __BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__ + for (int i = 0; i < 25; i++) { + st[i] = __builtin_bswap64(st[i]); + } +#endif +} + +void cmt_keccak_init(cmt_keccak_t *state) { + *state = (cmt_keccak_t) CMT_KECCAK_INIT(state); +} + +void cmt_keccak_update(cmt_keccak_t *state, size_t n, const void *data) { + int j = state->pt; + for (size_t i = 0; i < n; i++) { + state->st.b[j++] ^= ((const uint8_t *) data)[i]; + if (j >= state->rsiz) { + keccakf(state->st.q); + j = 0; + } + } + state->pt = j; +} + +void cmt_keccak_final(cmt_keccak_t *state, void *md) { + state->st.b[state->pt] ^= 0x01; + state->st.b[state->rsiz - 1] ^= 0x80; + keccakf(state->st.q); + + for (int i = 0; i < CMT_KECCAK_LENGTH; i++) { + ((uint8_t *) md)[i] = state->st.b[i]; + } +} + +uint8_t *cmt_keccak_data(size_t length, const void *data, uint8_t md[CMT_KECCAK_LENGTH]) { + cmt_keccak_t c[1]; + cmt_keccak_init(c); + cmt_keccak_update(c, length, data); + cmt_keccak_final(c, md); + return md; +} + +uint32_t cmt_keccak_funsel(const char *decl) { + uint8_t md[32]; + cmt_keccak_data(strlen(decl), decl, md); + return CMT_ABI_FUNSEL(md[0], md[1], md[2], md[3]); +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/merkle.c b/cartesi-tools/libcmt-sys/libcmt/src/merkle.c new file mode 100644 index 00000000..b6dcd2ad --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/merkle.c @@ -0,0 +1,202 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/merkle.h" +#include "libcmt/util.h" + +#include +#include +#include +#include +#include + +// clang-format off +static const uint8_t pristine_hash[][CMT_KECCAK_LENGTH] = { + {0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, + {0xad,0x32,0x28,0xb6,0x76,0xf7,0xd3,0xcd,0x42,0x84,0xa5,0x44,0x3f,0x17,0xf1,0x96,0x2b,0x36,0xe4,0x91,0xb3,0x0a,0x40,0xb2,0x40,0x58,0x49,0xe5,0x97,0xba,0x5f,0xb5}, + {0xb4,0xc1,0x19,0x51,0x95,0x7c,0x6f,0x8f,0x64,0x2c,0x4a,0xf6,0x1c,0xd6,0xb2,0x46,0x40,0xfe,0xc6,0xdc,0x7f,0xc6,0x07,0xee,0x82,0x06,0xa9,0x9e,0x92,0x41,0x0d,0x30}, + {0x21,0xdd,0xb9,0xa3,0x56,0x81,0x5c,0x3f,0xac,0x10,0x26,0xb6,0xde,0xc5,0xdf,0x31,0x24,0xaf,0xba,0xdb,0x48,0x5c,0x9b,0xa5,0xa3,0xe3,0x39,0x8a,0x04,0xb7,0xba,0x85}, + {0xe5,0x87,0x69,0xb3,0x2a,0x1b,0xea,0xf1,0xea,0x27,0x37,0x5a,0x44,0x09,0x5a,0x0d,0x1f,0xb6,0x64,0xce,0x2d,0xd3,0x58,0xe7,0xfc,0xbf,0xb7,0x8c,0x26,0xa1,0x93,0x44}, + {0x0e,0xb0,0x1e,0xbf,0xc9,0xed,0x27,0x50,0x0c,0xd4,0xdf,0xc9,0x79,0x27,0x2d,0x1f,0x09,0x13,0xcc,0x9f,0x66,0x54,0x0d,0x7e,0x80,0x05,0x81,0x11,0x09,0xe1,0xcf,0x2d}, + {0x88,0x7c,0x22,0xbd,0x87,0x50,0xd3,0x40,0x16,0xac,0x3c,0x66,0xb5,0xff,0x10,0x2d,0xac,0xdd,0x73,0xf6,0xb0,0x14,0xe7,0x10,0xb5,0x1e,0x80,0x22,0xaf,0x9a,0x19,0x68}, + {0xff,0xd7,0x01,0x57,0xe4,0x80,0x63,0xfc,0x33,0xc9,0x7a,0x05,0x0f,0x7f,0x64,0x02,0x33,0xbf,0x64,0x6c,0xc9,0x8d,0x95,0x24,0xc6,0xb9,0x2b,0xcf,0x3a,0xb5,0x6f,0x83}, + {0x98,0x67,0xcc,0x5f,0x7f,0x19,0x6b,0x93,0xba,0xe1,0xe2,0x7e,0x63,0x20,0x74,0x24,0x45,0xd2,0x90,0xf2,0x26,0x38,0x27,0x49,0x8b,0x54,0xfe,0xc5,0x39,0xf7,0x56,0xaf}, + {0xce,0xfa,0xd4,0xe5,0x08,0xc0,0x98,0xb9,0xa7,0xe1,0xd8,0xfe,0xb1,0x99,0x55,0xfb,0x02,0xba,0x96,0x75,0x58,0x50,0x78,0x71,0x09,0x69,0xd3,0x44,0x0f,0x50,0x54,0xe0}, + {0xf9,0xdc,0x3e,0x7f,0xe0,0x16,0xe0,0x50,0xef,0xf2,0x60,0x33,0x4f,0x18,0xa5,0xd4,0xfe,0x39,0x1d,0x82,0x09,0x23,0x19,0xf5,0x96,0x4f,0x2e,0x2e,0xb7,0xc1,0xc3,0xa5}, + {0xf8,0xb1,0x3a,0x49,0xe2,0x82,0xf6,0x09,0xc3,0x17,0xa8,0x33,0xfb,0x8d,0x97,0x6d,0x11,0x51,0x7c,0x57,0x1d,0x12,0x21,0xa2,0x65,0xd2,0x5a,0xf7,0x78,0xec,0xf8,0x92}, + {0x34,0x90,0xc6,0xce,0xeb,0x45,0x0a,0xec,0xdc,0x82,0xe2,0x82,0x93,0x03,0x1d,0x10,0xc7,0xd7,0x3b,0xf8,0x5e,0x57,0xbf,0x04,0x1a,0x97,0x36,0x0a,0xa2,0xc5,0xd9,0x9c}, + {0xc1,0xdf,0x82,0xd9,0xc4,0xb8,0x74,0x13,0xea,0xe2,0xef,0x04,0x8f,0x94,0xb4,0xd3,0x55,0x4c,0xea,0x73,0xd9,0x2b,0x0f,0x7a,0xf9,0x6e,0x02,0x71,0xc6,0x91,0xe2,0xbb}, + {0x5c,0x67,0xad,0xd7,0xc6,0xca,0xf3,0x02,0x25,0x6a,0xde,0xdf,0x7a,0xb1,0x14,0xda,0x0a,0xcf,0xe8,0x70,0xd4,0x49,0xa3,0xa4,0x89,0xf7,0x81,0xd6,0x59,0xe8,0xbe,0xcc}, + {0xda,0x7b,0xce,0x9f,0x4e,0x86,0x18,0xb6,0xbd,0x2f,0x41,0x32,0xce,0x79,0x8c,0xdc,0x7a,0x60,0xe7,0xe1,0x46,0x0a,0x72,0x99,0xe3,0xc6,0x34,0x2a,0x57,0x96,0x26,0xd2}, + {0x27,0x33,0xe5,0x0f,0x52,0x6e,0xc2,0xfa,0x19,0xa2,0x2b,0x31,0xe8,0xed,0x50,0xf2,0x3c,0xd1,0xfd,0xf9,0x4c,0x91,0x54,0xed,0x3a,0x76,0x09,0xa2,0xf1,0xff,0x98,0x1f}, + {0xe1,0xd3,0xb5,0xc8,0x07,0xb2,0x81,0xe4,0x68,0x3c,0xc6,0xd6,0x31,0x5c,0xf9,0x5b,0x9a,0xde,0x86,0x41,0xde,0xfc,0xb3,0x23,0x72,0xf1,0xc1,0x26,0xe3,0x98,0xef,0x7a}, + {0x5a,0x2d,0xce,0x0a,0x8a,0x7f,0x68,0xbb,0x74,0x56,0x0f,0x8f,0x71,0x83,0x7c,0x2c,0x2e,0xbb,0xcb,0xf7,0xff,0xfb,0x42,0xae,0x18,0x96,0xf1,0x3f,0x7c,0x74,0x79,0xa0}, + {0xb4,0x6a,0x28,0xb6,0xf5,0x55,0x40,0xf8,0x94,0x44,0xf6,0x3d,0xe0,0x37,0x8e,0x3d,0x12,0x1b,0xe0,0x9e,0x06,0xcc,0x9d,0xed,0x1c,0x20,0xe6,0x58,0x76,0xd3,0x6a,0xa0}, + {0xc6,0x5e,0x96,0x45,0x64,0x47,0x86,0xb6,0x20,0xe2,0xdd,0x2a,0xd6,0x48,0xdd,0xfc,0xbf,0x4a,0x7e,0x5b,0x1a,0x3a,0x4e,0xcf,0xe7,0xf6,0x46,0x67,0xa3,0xf0,0xb7,0xe2}, + {0xf4,0x41,0x85,0x88,0xed,0x35,0xa2,0x45,0x8c,0xff,0xeb,0x39,0xb9,0x3d,0x26,0xf1,0x8d,0x2a,0xb1,0x3b,0xdc,0xe6,0xae,0xe5,0x8e,0x7b,0x99,0x35,0x9e,0xc2,0xdf,0xd9}, + {0x5a,0x9c,0x16,0xdc,0x00,0xd6,0xef,0x18,0xb7,0x93,0x3a,0x6f,0x8d,0xc6,0x5c,0xcb,0x55,0x66,0x71,0x38,0x77,0x6f,0x7d,0xea,0x10,0x10,0x70,0xdc,0x87,0x96,0xe3,0x77}, + {0x4d,0xf8,0x4f,0x40,0xae,0x0c,0x82,0x29,0xd0,0xd6,0x06,0x9e,0x5c,0x8f,0x39,0xa7,0xc2,0x99,0x67,0x7a,0x09,0xd3,0x67,0xfc,0x7b,0x05,0xe3,0xbc,0x38,0x0e,0xe6,0x52}, + {0xcd,0xc7,0x25,0x95,0xf7,0x4c,0x7b,0x10,0x43,0xd0,0xe1,0xff,0xba,0xb7,0x34,0x64,0x8c,0x83,0x8d,0xfb,0x05,0x27,0xd9,0x71,0xb6,0x02,0xbc,0x21,0x6c,0x96,0x19,0xef}, + {0x0a,0xbf,0x5a,0xc9,0x74,0xa1,0xed,0x57,0xf4,0x05,0x0a,0xa5,0x10,0xdd,0x9c,0x74,0xf5,0x08,0x27,0x7b,0x39,0xd7,0x97,0x3b,0xb2,0xdf,0xcc,0xc5,0xee,0xb0,0x61,0x8d}, + {0xb8,0xcd,0x74,0x04,0x6f,0xf3,0x37,0xf0,0xa7,0xbf,0x2c,0x8e,0x03,0xe1,0x0f,0x64,0x2c,0x18,0x86,0x79,0x8d,0x71,0x80,0x6a,0xb1,0xe8,0x88,0xd9,0xe5,0xee,0x87,0xd0}, + {0x83,0x8c,0x56,0x55,0xcb,0x21,0xc6,0xcb,0x83,0x31,0x3b,0x5a,0x63,0x11,0x75,0xdf,0xf4,0x96,0x37,0x72,0xcc,0xe9,0x10,0x81,0x88,0xb3,0x4a,0xc8,0x7c,0x81,0xc4,0x1e}, + {0x66,0x2e,0xe4,0xdd,0x2d,0xd7,0xb2,0xbc,0x70,0x79,0x61,0xb1,0xe6,0x46,0xc4,0x04,0x76,0x69,0xdc,0xb6,0x58,0x4f,0x0d,0x8d,0x77,0x0d,0xaf,0x5d,0x7e,0x7d,0xeb,0x2e}, + {0x38,0x8a,0xb2,0x0e,0x25,0x73,0xd1,0x71,0xa8,0x81,0x08,0xe7,0x9d,0x82,0x0e,0x98,0xf2,0x6c,0x0b,0x84,0xaa,0x8b,0x2f,0x4a,0xa4,0x96,0x8d,0xbb,0x81,0x8e,0xa3,0x22}, + {0x93,0x23,0x7c,0x50,0xba,0x75,0xee,0x48,0x5f,0x4c,0x22,0xad,0xf2,0xf7,0x41,0x40,0x0b,0xdf,0x8d,0x6a,0x9c,0xc7,0xdf,0x7e,0xca,0xe5,0x76,0x22,0x16,0x65,0xd7,0x35}, + {0x84,0x48,0x81,0x8b,0xb4,0xae,0x45,0x62,0x84,0x9e,0x94,0x9e,0x17,0xac,0x16,0xe0,0xbe,0x16,0x68,0x8e,0x15,0x6b,0x5c,0xf1,0x5e,0x09,0x8c,0x62,0x7c,0x00,0x56,0xa9}, + {0x27,0xae,0x5b,0xa0,0x8d,0x72,0x91,0xc9,0x6c,0x8c,0xbd,0xdc,0xc1,0x48,0xbf,0x48,0xa6,0xd6,0x8c,0x79,0x74,0xb9,0x43,0x56,0xf5,0x37,0x54,0xef,0x61,0x71,0xd7,0x57}, + {0xbf,0x55,0x8b,0xeb,0xd2,0xce,0xec,0x7f,0x3c,0x5d,0xce,0x04,0xa4,0x78,0x2f,0x88,0xc2,0xc6,0x03,0x6a,0xe7,0x8e,0xe2,0x06,0xd0,0xbc,0x52,0x89,0xd2,0x04,0x61,0xa2}, + {0xe2,0x19,0x08,0xc2,0x96,0x8c,0x06,0x99,0x04,0x0a,0x6f,0xd8,0x66,0xa5,0x77,0xa9,0x9a,0x9d,0x2e,0xc8,0x87,0x45,0xc8,0x15,0xfd,0x4a,0x47,0x2c,0x78,0x92,0x44,0xda}, + {0xae,0x82,0x4d,0x72,0xdd,0xc2,0x72,0xaa,0xb6,0x8a,0x8c,0x30,0x22,0xe3,0x6f,0x10,0x45,0x44,0x37,0xc1,0x88,0x6f,0x3f,0xf9,0x92,0x7b,0x64,0xf2,0x32,0xdf,0x41,0x4f}, + {0x27,0xe4,0x29,0xa4,0xbe,0xf3,0x08,0x3b,0xc3,0x1a,0x67,0x1d,0x04,0x6e,0xa5,0xc1,0xf5,0xb8,0xc3,0x09,0x4d,0x72,0x86,0x8d,0x9d,0xfd,0xc1,0x2c,0x73,0x34,0xac,0x5f}, + {0x74,0x3c,0xc5,0xc3,0x65,0xa9,0xa6,0xa1,0x5c,0x1f,0x24,0x0a,0xc2,0x58,0x80,0xc7,0xa9,0xd1,0xde,0x29,0x06,0x96,0xcb,0x76,0x60,0x74,0xa1,0xd8,0x3d,0x92,0x78,0x16}, + {0x4a,0xdc,0xf6,0x16,0xc3,0xbf,0xab,0xf6,0x39,0x99,0xa0,0x19,0x66,0xc9,0x98,0xb7,0xbb,0x57,0x27,0x74,0x03,0x5a,0x63,0xea,0xd4,0x9d,0xa7,0x3b,0x59,0x87,0xf3,0x47}, + {0x75,0x78,0x66,0x45,0xd0,0xc5,0xdd,0x7c,0x04,0xa2,0xf8,0xa7,0x5d,0xca,0xe0,0x85,0x21,0x36,0x52,0xf5,0xbc,0xe3,0xea,0x8b,0x9b,0x9b,0xed,0xd1,0xca,0xb3,0xc5,0xe9}, + {0xb8,0x8b,0x15,0x2c,0x9b,0x8a,0x7b,0x79,0x63,0x7d,0x35,0x91,0x18,0x48,0xb0,0xc4,0x1e,0x7c,0xc7,0xcc,0xa2,0xab,0x4f,0xe9,0xa1,0x5f,0x9c,0x38,0xbb,0x4b,0xb9,0x39}, + {0x0c,0x4e,0x2d,0x8c,0xe8,0x34,0xff,0xd7,0xa6,0xcd,0x85,0xd7,0x11,0x3d,0x45,0x21,0xab,0xb8,0x57,0x77,0x48,0x45,0xc4,0x29,0x1e,0x6f,0x6d,0x01,0x0d,0x97,0xe3,0x18}, + {0x5b,0xc7,0x99,0xd8,0x3e,0x3b,0xb3,0x15,0x01,0xb3,0xda,0x78,0x66,0x80,0xdf,0x30,0xfb,0xc1,0x8e,0xb4,0x1c,0xbc,0xe6,0x11,0xe8,0xc0,0xe9,0xc7,0x2f,0x69,0x57,0x1c}, + {0xa1,0x0d,0x3e,0xf8,0x57,0xd0,0x4d,0x9c,0x03,0xea,0xd7,0xc6,0x31,0x7d,0x79,0x7a,0x09,0x0f,0xa1,0x27,0x1a,0xd9,0xc7,0xad,0xdf,0xbc,0xb4,0x12,0xe9,0x64,0x3d,0x4f}, + {0xb3,0x3b,0x18,0x09,0xc4,0x26,0x23,0xf4,0x74,0x05,0x5f,0xa9,0x40,0x0a,0x20,0x27,0xa7,0xa8,0x85,0xc8,0xdf,0xa4,0xef,0xe2,0x06,0x66,0xb4,0xee,0x27,0xd7,0x52,0x9c}, + {0x13,0x4d,0x7f,0x28,0xd5,0x3f,0x17,0x5f,0x6b,0xf4,0xb6,0x2f,0xaa,0x21,0x10,0xd5,0xb7,0x6f,0x0f,0x77,0x0c,0x15,0xe6,0x28,0x18,0x1c,0x1f,0xcc,0x18,0xf9,0x70,0xa9}, + {0xc3,0x4d,0x24,0xb2,0xfc,0x8c,0x50,0xca,0x9c,0x07,0xa7,0x15,0x6e,0xf4,0xe5,0xff,0x4b,0xdf,0x00,0x2e,0xda,0x0b,0x11,0xc1,0xd3,0x59,0xd0,0xb5,0x9a,0x54,0x68,0x07}, + {0x04,0xdb,0xb9,0xdb,0x63,0x14,0x57,0x87,0x9b,0x27,0xe0,0xdf,0xdb,0xe5,0x01,0x58,0xfd,0x9c,0xf9,0xb4,0xcf,0x77,0x60,0x5c,0x4a,0xc4,0xc9,0x5b,0xd6,0x5f,0xc9,0xf6}, + {0xf9,0x29,0x5a,0x68,0x66,0x47,0xcb,0x99,0x90,0x90,0x81,0x9c,0xda,0x70,0x08,0x20,0xc2,0x82,0xc6,0x13,0xce,0xdc,0xd2,0x18,0x54,0x0b,0xbc,0x6f,0x37,0xb0,0x1c,0x65}, + {0x67,0xc4,0xa1,0xea,0x62,0x4f,0x09,0x2a,0x3a,0x5c,0xca,0x2d,0x6f,0x0f,0x0d,0xb2,0x31,0x97,0x2f,0xce,0x62,0x7f,0x0e,0xcc,0xa0,0xde,0xe6,0x0f,0x17,0x55,0x1c,0x5f}, + {0x8f,0xda,0xeb,0x5a,0xb5,0x60,0xb2,0xce,0xb7,0x81,0xcd,0xb3,0x39,0x36,0x1a,0x0f,0xbe,0xe1,0xb9,0xdf,0xfa,0xd5,0x91,0x15,0x13,0x8c,0x8d,0x6a,0x70,0xdd,0xa9,0xcc}, + {0xc1,0xbf,0x0b,0xbd,0xd7,0xfe,0xe1,0x57,0x64,0x84,0x5d,0xb8,0x75,0xf6,0x43,0x25,0x59,0xff,0x8d,0xbc,0x90,0x55,0x32,0x44,0x31,0xbc,0x34,0xe5,0xb9,0x3d,0x15,0xda}, + {0x30,0x73,0x17,0x84,0x9e,0xcc,0xd9,0x0c,0x0c,0x7b,0x98,0x87,0x0b,0x93,0x17,0xc1,0x5a,0x59,0x59,0xdc,0xfb,0x84,0xc7,0x6d,0xcc,0x90,0x8c,0x4f,0xe6,0xba,0x92,0x12}, + {0x63,0x39,0xbf,0x06,0xe4,0x58,0xf6,0x64,0x6d,0xf5,0xe8,0x3b,0xa7,0xc3,0xd3,0x5b,0xc2,0x63,0xb3,0x22,0x2c,0x8e,0x90,0x40,0x06,0x88,0x47,0x74,0x9c,0xa8,0xe8,0xf9}, + {0x50,0x45,0xe4,0x34,0x2a,0xeb,0x52,0x1e,0xb3,0xa5,0x58,0x7e,0xc2,0x68,0xed,0x3a,0xa6,0xfa,0xf3,0x2b,0x62,0xb0,0xbc,0x41,0xa9,0xd5,0x49,0x52,0x1f,0x40,0x6f,0xc3}, + {0x08,0x60,0x1d,0x83,0xcd,0xd3,0x4b,0x5f,0x7b,0x8d,0xf6,0x3e,0x7b,0x9a,0x16,0x51,0x9d,0x35,0x47,0x3d,0x0b,0x89,0xc3,0x17,0xbe,0xed,0x3d,0x3d,0x94,0x24,0xb2,0x53}, + {0x84,0xe3,0x5c,0x5d,0x92,0x17,0x13,0x76,0xca,0xe5,0xc8,0x63,0x00,0x82,0x2d,0x72,0x9c,0xd3,0xa8,0x47,0x95,0x83,0xbe,0xf0,0x95,0x27,0x02,0x7d,0xba,0x5f,0x11,0x26}, + {0x3c,0x5c,0xbb,0xeb,0x38,0x34,0xb7,0xa5,0xc1,0xcb,0xa9,0xaa,0x5f,0xee,0x0c,0x95,0xec,0x3f,0x17,0xa3,0x3e,0xc3,0xd8,0x04,0x7f,0xff,0x79,0x91,0x87,0xf5,0xae,0x20}, + {0x40,0xbb,0xe9,0x13,0xc2,0x26,0xc3,0x4c,0x9f,0xbe,0x43,0x89,0xdd,0x72,0x89,0x84,0x25,0x7a,0x81,0x68,0x92,0xb3,0xca,0xe3,0xe4,0x31,0x91,0xdd,0x29,0x1f,0x0e,0xb5}, + {0x14,0xaf,0x53,0x85,0xbc,0xbb,0x1e,0x47,0x38,0xbb,0xae,0x81,0x06,0x04,0x6e,0x6e,0x2f,0xca,0x42,0x87,0x5a,0xa5,0xc0,0x00,0xc5,0x82,0x58,0x77,0x42,0xbc,0xc7,0x48}, + {0x72,0xf2,0x96,0x56,0x80,0x3c,0x2f,0x4b,0xe1,0x77,0xb1,0xb8,0xdd,0x2a,0x51,0x37,0x89,0x2b,0x08,0x0b,0x02,0x21,0x00,0xfd,0xe4,0xe9,0x6d,0x93,0xef,0x8c,0x96,0xff}, + {0xd0,0x6f,0x27,0x06,0x1c,0x73,0x4d,0x78,0x25,0xb4,0x68,0x65,0xd0,0x0a,0xa9,0x00,0xe5,0xcc,0x3a,0x36,0x72,0x08,0x0e,0x52,0x71,0x71,0xe1,0x17,0x1a,0xa5,0x03,0x8a}, + {0x28,0x20,0x39,0x85,0xb5,0xf2,0xd8,0x77,0x09,0x17,0x16,0x78,0x16,0x97,0x39,0xf9,0x57,0xd2,0x74,0x5f,0x4b,0xfa,0x5c,0xc9,0x1e,0x2b,0x4b,0xd9,0xbf,0x48,0x3b,0x40}, + {0x0a,0x16,0x29,0x46,0xe5,0x61,0x58,0xba,0xc0,0x67,0x3e,0x6d,0xd3,0xbd,0xfd,0xc1,0xe4,0xa0,0xe7,0x74,0x4a,0x12,0x0f,0xdb,0x64,0x00,0x50,0xc8,0xd7,0xab,0xe1,0xc6}, + {0xe2,0xc3,0xed,0x40,0x52,0xee,0xb1,0xd6,0x05,0x14,0xb4,0xc3,0x8e,0xce,0x8d,0x73,0xa2,0x7f,0x37,0xfa,0x5b,0x36,0xdc,0xbf,0x33,0x8e,0x70,0xde,0x95,0x79,0x8c,0xaa} +}; +// clang-format on + +static void copy_hash(const uint8_t src[CMT_KECCAK_LENGTH], uint8_t out[CMT_KECCAK_LENGTH]) { + memcpy(out, src, CMT_KECCAK_LENGTH); +} + +static void concat_hash(const uint8_t lhs[CMT_KECCAK_LENGTH], const uint8_t rhs[CMT_KECCAK_LENGTH], + uint8_t out[CMT_KECCAK_LENGTH]) { + cmt_keccak_t c[1]; + cmt_keccak_init(c); + cmt_keccak_update(c, CMT_KECCAK_LENGTH, lhs); + cmt_keccak_update(c, CMT_KECCAK_LENGTH, rhs); + cmt_keccak_final(c, out); +} + +#if 0 // NOLINT +static void print_hash(uint8_t hash[CMT_KECCAK_LENGTH]) { + for (int i = 0; i < CMT_KECCAK_LENGTH; ++i) { + fprintf(stderr, "%02x", hash[i]); + } + fprintf(stderr, "\n"); +} +#endif + +void cmt_merkle_init(cmt_merkle_t *me) { + cmt_merkle_reset(me); +} + +void cmt_merkle_reset(cmt_merkle_t *me) { + me->leaf_count = 0; + memset(me->state, 0, sizeof(me->state)); +} + +void cmt_merkle_fini(cmt_merkle_t *me) { + // no-op + (void) me; +} + +int cmt_merkle_load(cmt_merkle_t *me, const char *filepath) { + if (!me) { + return -EINVAL; + } + size_t length = 0; + int rc = cmt_util_read_whole_file(filepath, sizeof *me, me, &length); + if (length != sizeof *me) { + return -EINVAL; + } + return rc; +} + +int cmt_merkle_save(cmt_merkle_t *me, const char *filepath) { + if (!me) { + return -EINVAL; + } + return cmt_util_write_whole_file(filepath, sizeof *me, me); +} + +uint64_t cmt_merkle_get_leaf_count(cmt_merkle_t *me) { + return me->leaf_count; +} + +int cmt_merkle_push_back(cmt_merkle_t *me, const uint8_t hash[CMT_KECCAK_LENGTH]) { + const uint64_t max_count = + (CMT_MERKLE_TREE_HEIGHT < 8 * sizeof(uint64_t)) ? (UINT64_C(1) << CMT_MERKLE_TREE_HEIGHT) : UINT64_MAX; + if (me->leaf_count == max_count) { + return -ENOBUFS; + } + uint8_t right[CMT_KECCAK_LENGTH]; + copy_hash(hash, right); + for (int i = 0; i < CMT_MERKLE_TREE_HEIGHT; ++i) { + uint64_t bit = ((uint64_t) 1) << i; + /* if we have a hash for a subtree of the current size in the state... */ + if (me->leaf_count & bit) { + /* ...concat it with current running right and replace running right with it. */ + concat_hash(me->state[i], right, right); + /* otherwise... */ + } else { + /* ...just copy the current subtree hash to the state and we are done. */ + copy_hash(right, me->state[i]); + break; + } + } + ++me->leaf_count; + return 0; +} + +void cmt_merkle_get_root_hash(cmt_merkle_t *me, uint8_t root[CMT_KECCAK_LENGTH]) { + copy_hash(pristine_hash[0], root); + for (int i = 0; i < CMT_MERKLE_TREE_HEIGHT; ++i) { + uint64_t bit = ((uint64_t) 1) << i; + /* if we have a hash for a subtree of the current size in the state... */ + if (me->leaf_count & bit) { + /* ...concat it with current root on the right and replace root */ + concat_hash(me->state[i], root, root); + /* otherwise... */ + } else { + /* ...concat root with pristine tree on the right and replace root */ + concat_hash(root, pristine_hash[i], root); + } + } +} + +int cmt_merkle_push_back_data(cmt_merkle_t *me, size_t length, const void *data) { + uint8_t hash[CMT_KECCAK_LENGTH]; + cmt_keccak_t c[1]; + cmt_keccak_init(c); + cmt_keccak_update(c, length, data); + cmt_keccak_final(c, hash); + return cmt_merkle_push_back(me, hash); +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/rollup.c b/cartesi-tools/libcmt-sys/libcmt/src/rollup.c new file mode 100644 index 00000000..fbd58e33 --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/rollup.c @@ -0,0 +1,455 @@ +/* Copyright Cartesi and individual authors (see AUTHORS) + * SPDX-License-Identifier: Apache-2.0 + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "libcmt/rollup.h" +#include "libcmt/abi.h" +#include "libcmt/merkle.h" +#include "libcmt/util.h" + +#include +#include +#include + +// Voucher(address,uint256,bytes) +#define VOUCHER CMT_ABI_FUNSEL(0x23, 0x7a, 0x81, 0x6f) + +// DelegateCallVoucher(address,bytes) +#define DELEGATE_CALL_VOUCHER CMT_ABI_FUNSEL(0x10, 0x32, 0x1e, 0x8b) + +// Notice(bytes) +#define NOTICE CMT_ABI_FUNSEL(0xc2, 0x58, 0xd6, 0xe5) + +// EvmAdvance(uint256,address,address,uint256,uint256,uint256,uint256,bytes) +#define EVM_ADVANCE CMT_ABI_FUNSEL(0x41, 0x5b, 0xf3, 0x63) + +#define DBG(X) debug(X, #X, __FILE__, __LINE__) +static int debug(int rc, const char *expr, const char *file, int line) { + if (rc == 0) { + return 0; + } + + if (cmt_util_debug_enabled()) { + (void) fprintf(stderr, "%s:%d Error %s on `%s'\n", file, line, expr, strerror(-rc)); + } + return rc; +} + +int cmt_rollup_init(cmt_rollup_t *me) { + if (!me) { + return -EINVAL; + } + + int rc = DBG(cmt_io_init(me->io)); + if (rc) { + return rc; + } + + cmt_merkle_init(me->merkle); + me->finish_leaf_count = UINT64_C(-1); + me->fromhost_data = 0; + memset(me->finish_outputs_merkle_root, 0, sizeof(me->finish_outputs_merkle_root)); + return 0; +} + +void cmt_rollup_fini(cmt_rollup_t *me) { + if (!me) { + return; + } + + cmt_io_fini(me->io); + cmt_merkle_fini(me->merkle); +} + +int cmt_rollup_emit_voucher(cmt_rollup_t *me, const cmt_abi_address_t *address, + const cmt_abi_u256_t *value, const cmt_abi_bytes_t *payload, uint64_t *index) { + if (!me) { + return -EINVAL; + } + if (!payload || (!payload->data && payload->length)) { + return -EINVAL; + } + + cmt_buf_t tx[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t wr[1] = {*tx}; + cmt_buf_t of[1]; + cmt_buf_t frame[1]; + + // clang-format off + if (DBG(cmt_abi_put_funsel(wr, VOUCHER)) + || DBG(cmt_abi_mark_frame(wr, frame)) + || DBG(cmt_abi_put_address(wr, address)) + || DBG(cmt_abi_put_uint256(wr, value)) + || DBG(cmt_abi_put_bytes_s(wr, of)) + || DBG(cmt_abi_put_bytes_d(wr, of, frame, payload))) { + return -ENOBUFS; + } + // clang-format on + + size_t used_space = wr->begin - tx->begin; + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_AUTOMATIC, + .reason = HTIF_YIELD_AUTOMATIC_REASON_TX_OUTPUT, + .data = used_space, + }}; + int rc = DBG(cmt_io_yield(me->io, req)); + if (rc) { + return rc; + } + + uint64_t count = cmt_merkle_get_leaf_count(me->merkle); + + rc = cmt_merkle_push_back_data(me->merkle, used_space, tx->begin); + if (rc) { + return rc; + } + + if (index) { + *index = count; + } + + return 0; +} + +int cmt_rollup_emit_delegate_call_voucher(cmt_rollup_t *me, + const cmt_abi_address_t *address, const cmt_abi_bytes_t *payload, uint64_t *index) { + if (!me) { + return -EINVAL; + } + if (!payload || (!payload->data && payload->length)) { + return -EINVAL; + } + + cmt_buf_t tx[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t wr[1] = {*tx}; + cmt_buf_t of[1]; + cmt_buf_t frame[1]; + + // clang-format off + if (DBG(cmt_abi_put_funsel(wr, DELEGATE_CALL_VOUCHER)) + || DBG(cmt_abi_mark_frame(wr, frame)) + || DBG(cmt_abi_put_address(wr, address)) + || DBG(cmt_abi_put_bytes_s(wr, of)) + || DBG(cmt_abi_put_bytes_d(wr, of, frame, payload))) { + return -ENOBUFS; + } + // clang-format on + + size_t used_space = wr->begin - tx->begin; + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_AUTOMATIC, + .reason = HTIF_YIELD_AUTOMATIC_REASON_TX_OUTPUT, + .data = used_space, + }}; + int rc = DBG(cmt_io_yield(me->io, req)); + if (rc) { + return rc; + } + + uint64_t count = cmt_merkle_get_leaf_count(me->merkle); + + rc = cmt_merkle_push_back_data(me->merkle, used_space, tx->begin); + if (rc) { + return rc; + } + + if (index) { + *index = count; + } + + return 0; +} + +int cmt_rollup_emit_notice(cmt_rollup_t *me, const cmt_abi_bytes_t *payload, uint64_t *index) { + if (!me) { + return -EINVAL; + } + if (!payload || (!payload->data && payload->length)) { + return -EINVAL; + } + + cmt_buf_t tx[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t wr[1] = {*tx}; + cmt_buf_t of[1]; + cmt_buf_t frame[1]; + + // clang-format off + if (DBG(cmt_abi_put_funsel(wr, NOTICE)) + || DBG(cmt_abi_mark_frame(wr, frame)) + || DBG(cmt_abi_put_bytes_s(wr, of)) + || DBG(cmt_abi_put_bytes_d(wr, of, frame, payload))) { + return -ENOBUFS; + } + // clang-format on + + size_t used_space = wr->begin - tx->begin; + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_AUTOMATIC, + .reason = HTIF_YIELD_AUTOMATIC_REASON_TX_OUTPUT, + .data = used_space, + }}; + int rc = DBG(cmt_io_yield(me->io, req)); + if (rc) { + return rc; + } + + uint64_t count = cmt_merkle_get_leaf_count(me->merkle); + + rc = cmt_merkle_push_back_data(me->merkle, used_space, tx->begin); + if (rc) { + return rc; + } + + if (index) { + *index = count; + } + + return 0; +} + +int cmt_rollup_emit_report(cmt_rollup_t *me, const cmt_abi_bytes_t *payload) { + if (!me) { + return -EINVAL; + } + if (!payload || (!payload->data && payload->length)) { + return -EINVAL; + } + + cmt_buf_t tx[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t wr[1] = {*tx}; + if (cmt_buf_split(tx, payload->length, wr, tx)) { + return -ENOBUFS; + } + + if (payload->data) { + memcpy(wr->begin, payload->data, payload->length); + } + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_AUTOMATIC, + .reason = HTIF_YIELD_AUTOMATIC_REASON_TX_REPORT, + .data = payload->length, + }}; + return DBG(cmt_io_yield(me->io, req)); +} + +int cmt_rollup_emit_exception(cmt_rollup_t *me, const cmt_abi_bytes_t *payload) { + if (!me) { + return -EINVAL; + } + if (!payload || (!payload->data && payload->length)) { + return -EINVAL; + } + + cmt_buf_t tx[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t wr[1] = {*tx}; + cmt_buf_t _[1]; + if (cmt_buf_split(tx, payload->length, wr, _)) { + return -ENOBUFS; + } + + if (payload->data) { + memcpy(wr->begin, payload->data, payload->length); + } + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_MANUAL, + .reason = HTIF_YIELD_MANUAL_REASON_TX_EXCEPTION, + .data = payload->length, + }}; + return DBG(cmt_io_yield(me->io, req)); +} + +static int cmt_rollup_get_rx(cmt_rollup_t *me, cmt_buf_t *lhs) { + cmt_buf_t rx[1] = {cmt_io_get_rx(me->io)}; + cmt_buf_t _[1]; + return cmt_buf_split(rx, me->fromhost_data, lhs, _); +} + +int cmt_rollup_read_advance_state(cmt_rollup_t *me, cmt_rollup_advance_t *advance) { + if (!me) { + return -EINVAL; + } + if (!advance) { + return -EINVAL; + } + + cmt_buf_t rd[1]; + if (cmt_rollup_get_rx(me, rd)) { + return -ENOBUFS; + } + cmt_buf_t frame[1]; + cmt_buf_t of[1]; + + // clang-format off + if (DBG(cmt_abi_check_funsel(rd, EVM_ADVANCE)) + || DBG(cmt_abi_mark_frame(rd, frame)) + || DBG(cmt_abi_get_uint(rd, sizeof(advance->chain_id), &advance->chain_id)) + || DBG(cmt_abi_get_address(rd, &advance->app_contract)) + || DBG(cmt_abi_get_address(rd, &advance->msg_sender)) + || DBG(cmt_abi_get_uint(rd, sizeof(advance->block_number), &advance->block_number)) + || DBG(cmt_abi_get_uint(rd, sizeof(advance->block_timestamp), &advance->block_timestamp)) + || DBG(cmt_abi_get_uint256(rd, &advance->prev_randao)) + || DBG(cmt_abi_get_uint(rd, sizeof(advance->index), &advance->index)) + || DBG(cmt_abi_get_bytes_s(rd, of)) + || DBG(cmt_abi_get_bytes_d(frame, of, &advance->payload.length, &advance->payload.data))) { + return -ENOBUFS; + } + // clang-format on + + return 0; +} + +int cmt_rollup_read_inspect_state(cmt_rollup_t *me, cmt_rollup_inspect_t *inspect) { + if (!me) { + return -EINVAL; + } + if (!inspect) { + return -EINVAL; + } + + cmt_buf_t rd[1]; + if (DBG(cmt_rollup_get_rx(me, rd))) { + return -ENOBUFS; + } + + inspect->payload.length = cmt_buf_length(rd); + inspect->payload.data = rd->begin; + return 0; +} + +static int accepted(union cmt_io_driver *io, uint32_t *n) { + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_MANUAL, + .reason = HTIF_YIELD_MANUAL_REASON_RX_ACCEPTED, + .data = *n, + }}; + int rc = DBG(cmt_io_yield(io, req)); + if (rc) { + return rc; + } + + *n = req->data; + return req->reason; +} + +static int revert(union cmt_io_driver *io) { + struct cmt_io_yield req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_MANUAL, + .reason = HTIF_YIELD_MANUAL_REASON_RX_REJECTED, + .data = 0, + }}; + return DBG(cmt_io_yield(io, req)); +} + +int cmt_rollup_finish(cmt_rollup_t *me, cmt_rollup_finish_t *finish) { + if (!me) { + return -EINVAL; + } + if (!finish) { + return -EINVAL; + } + + if (!finish->accept_previous_request) { + return revert(me->io); /* revert should not return! */ + } + + uint64_t leaf_count = cmt_merkle_get_leaf_count(me->merkle); + if (me->finish_leaf_count != leaf_count) { + cmt_merkle_get_root_hash(me->merkle, me->finish_outputs_merkle_root); + } + memcpy(cmt_io_get_tx(me->io).begin, me->finish_outputs_merkle_root, CMT_ABI_U256_LENGTH); + me->fromhost_data = CMT_ABI_U256_LENGTH; + int reason = accepted(me->io, &me->fromhost_data); + if (reason < 0) { + return reason; + } + finish->next_request_type = reason; + finish->next_request_payload_length = me->fromhost_data; + me->finish_leaf_count = leaf_count; + return 0; +} + +int cmt_gio_request(cmt_rollup_t *me, cmt_gio_t *req) { + if (!me) { + return -EINVAL; + } + if (!req) { + return -EINVAL; + } + + /* Accept empty id */ + if (!req->id && req->id_length) { + return -EINVAL; + } + + cmt_buf_t wr[1] = {cmt_io_get_tx(me->io)}; + cmt_buf_t _[1]; + if (cmt_buf_split(wr, req->id_length, wr, _)) { + return -ENOBUFS; + } + if (req->id) { + memcpy(wr->begin, req->id, req->id_length); + } + + struct cmt_io_yield rr[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_MANUAL, + .reason = req->domain, + .data = req->id_length, + }}; + + int rc = DBG(cmt_io_yield(me->io, rr)); + if (rc != 0) { + return rc; + } + me->fromhost_data = rr->data; + + cmt_buf_t rd[1]; + if (cmt_rollup_get_rx(me, rd)) { + return -ENOBUFS; + } + + req->response_data = rd->begin; + req->response_code = rr->reason; + req->response_data_length = rr->data; + return 0; +} + +int cmt_rollup_progress(cmt_rollup_t *me, uint32_t progress) { + cmt_io_yield_t req[1] = {{ + .dev = HTIF_DEVICE_YIELD, + .cmd = HTIF_YIELD_CMD_AUTOMATIC, + .reason = HTIF_YIELD_AUTOMATIC_REASON_PROGRESS, + .data = progress, + }}; + return DBG(cmt_io_yield(me->io, req)); +} + +int cmt_rollup_load_merkle(cmt_rollup_t *me, const char *path) { + return DBG(cmt_merkle_load(me->merkle, path)); +} + +int cmt_rollup_save_merkle(cmt_rollup_t *me, const char *path) { + return DBG(cmt_merkle_save(me->merkle, path)); +} + +void cmt_rollup_reset_merkle(cmt_rollup_t *me) { + cmt_merkle_reset(me->merkle); +} diff --git a/cartesi-tools/libcmt-sys/libcmt/src/util.c b/cartesi-tools/libcmt-sys/libcmt/src/util.c new file mode 100644 index 00000000..95a3137d --- /dev/null +++ b/cartesi-tools/libcmt-sys/libcmt/src/util.c @@ -0,0 +1,52 @@ +#include +#include +#include +#include +#include + +bool cmt_util_debug_enabled(void) { + static bool checked = false; + static bool enabled = false; + + if (!checked) { + enabled = getenv("CMT_DEBUG") != NULL; + checked = true; + } + + return enabled; +} + +int cmt_util_read_whole_file(const char *name, size_t max, void *data, size_t *length) { + int rc = 0; + + FILE *file = fopen(name, "rb"); + if (!file) { + return -errno; + } + *length = fread(data, 1, max, file); + int eof = (fseek(file, 0, SEEK_END) == 0) && (*length == (size_t) ftell(file)); + if (!eof) { + rc = -EIO; + } + if (fclose(file) != 0) { + rc = -errno; + } + return rc; +} + +int cmt_util_write_whole_file(const char *name, size_t length, const void *data) { + int rc = 0; + + FILE *file = fopen(name, "wb"); + if (!file) { + return -errno; + } + + if (fwrite(data, 1, length, file) != length) { + rc = -EIO; + } + if (fclose(file) != 0) { + rc = -errno; + } + return rc; +} diff --git a/cartesi-tools/libcmt-sys/src/lib.rs b/cartesi-tools/libcmt-sys/src/lib.rs new file mode 100644 index 00000000..a38a13a8 --- /dev/null +++ b/cartesi-tools/libcmt-sys/src/lib.rs @@ -0,0 +1,5 @@ +#![allow(non_upper_case_globals)] +#![allow(non_camel_case_types)] +#![allow(non_snake_case)] + +include!(concat!(env!("OUT_DIR"), "/bindings.rs")); diff --git a/cartesi-tools/testsi/Cargo.toml b/cartesi-tools/testsi/Cargo.toml new file mode 100644 index 00000000..2e9b5c0a --- /dev/null +++ b/cartesi-tools/testsi/Cargo.toml @@ -0,0 +1,23 @@ +[package] +name = "testsi" +version = "0.1.0" +license = "Apache-2.0" +edition = "2021" +description = "Host-side test harness that drives a stored Cartesi Machine rollup image" +publish = false + +[dependencies] +testsi-macros = { path = "./testsi-macros" } +types = { workspace = true } + +# Cartesi Machine v0.21 Rust bindings from cartesi/dave main, pinned by rev (no +# version requirement: from a git source the crate reports Dave's workspace +# version, not the bindings' 0.21.0). They link the prebuilt emulator named by +# LIBCARTESI_PATH / INCLUDECARTESI_PATH (exported by the dev flake and by CI's +# CM .deb install); `external_cartesi` makes a missing LIBCARTESI_PATH a build +# error instead of a source-checkout fallback. +cartesi-machine = { git = "https://github.com/cartesi/dave", rev = "79874c0a1b40678465daf533acc976c88301c6d4", features = ["external_cartesi"] } + +inventory = "0.3" +libtest-mimic = "0.6" +thiserror = "1.0" diff --git a/cartesi-tools/testsi/src/lib.rs b/cartesi-tools/testsi/src/lib.rs new file mode 100644 index 00000000..3888de37 --- /dev/null +++ b/cartesi-tools/testsi/src/lib.rs @@ -0,0 +1,7 @@ +pub mod machine; +pub mod test_runner; +pub mod types; + +pub use machine::{Machine, MachineBuilder}; +pub use test_runner::*; +pub use types::*; diff --git a/cartesi-tools/testsi/src/machine.rs b/cartesi-tools/testsi/src/machine.rs new file mode 100644 index 00000000..1ee738e7 --- /dev/null +++ b/cartesi-tools/testsi/src/machine.rs @@ -0,0 +1,374 @@ +use crate::types::{InputBuilder, OutputsForInput}; + +use cartesi_machine::{ + cartesi_machine_sys::CM_HTIF_CONSOLE_CMD_GETCHAR, + config::{machine, runtime::RuntimeConfig}, + constants::{break_reason, cmio::commands}, + types::cmio, +}; +use std::{ + ops::ControlFlow, + path::PathBuf, + time::{Duration, Instant}, +}; +use thiserror::Error; +use types::alloy_primitives::{Address, U256}; + +type Report = Vec; + +const DEFAULT_EXECUTION_TIMEOUT: Duration = Duration::from_secs(90); +const RUN_MCYCLE_INCREMENT: u64 = 1 << 28; + +#[derive(Debug, Error)] +pub enum TestsiMachineError { + #[error(transparent)] + Machine(#[from] cartesi_machine::error::MachineError), + #[error("machine halted unexpectedly at mcycle={mcycle}")] + Halted { mcycle: u64 }, + #[error("machine reached its per-input mcycle limit (mcycle overflow) at mcycle={mcycle}")] + McycleOverflow { mcycle: u64 }, + #[error("application raised tx exception: {message}")] + TxException { message: String }, + #[error( + "application rejected the input (manual rx_rejected); testsi does not revert \ + rejected inputs, so this machine cannot accept further inputs" + )] + InputRejected, + #[error("machine is not awaiting an input (expected a manual rx_accepted yield): {state}")] + NotAwaitingInput { state: String }, + #[error("machine execution timed out after {elapsed:?} (timeout {timeout:?})")] + ExecutionTimeout { + elapsed: Duration, + timeout: Duration, + }, +} + +type Result = std::result::Result; + +pub struct MachineBuilder { + cartesi_machine_path: PathBuf, + chain_id: usize, + dapp_address: Address, + input_index: usize, + no_console_putchar: bool, + execution_timeout: Duration, +} + +impl MachineBuilder { + pub fn load_from>(path: T) -> MachineBuilder { + Self { + cartesi_machine_path: path.into(), + chain_id: 1, + dapp_address: Address::ZERO, + input_index: 0, + no_console_putchar: true, + execution_timeout: DEFAULT_EXECUTION_TIMEOUT, + } + } + + pub fn at_chain(mut self, chain_id: usize) -> MachineBuilder { + self.chain_id = chain_id; + self + } + + pub fn deployed_at(mut self, dapp_address: Address) -> MachineBuilder { + self.dapp_address = dapp_address; + self + } + + pub fn with_input_count(mut self, input_index: usize) -> MachineBuilder { + self.input_index = input_index; + self + } + + pub fn no_console_putchar(mut self, no_console_putchar: bool) -> MachineBuilder { + self.no_console_putchar = no_console_putchar; + self + } + + pub fn with_execution_timeout(mut self, execution_timeout: Duration) -> MachineBuilder { + self.execution_timeout = execution_timeout; + self + } + + pub fn try_build(self) -> Result { + Machine::try_new(self) + } +} + +pub struct Machine { + cartesi_machine: cartesi_machine::machine::Machine, + builder: MachineBuilder, +} + +impl Machine { + pub fn try_new(builder: MachineBuilder) -> Result { + // v0.21 replaced `htif.no_console_putchar` with a console output + // destination; the default writes guest console output to stdout. + let runtime_config = if builder.no_console_putchar { + RuntimeConfig::quiet_console() + } else { + RuntimeConfig::default() + }; + + // Instantiate Machine + let cartesi_machine = { + let mut cm = + cartesi_machine::Machine::load(&builder.cartesi_machine_path, &runtime_config)?; + let c = cm.initial_config()?; + sanity_check_cm_config(&c); + cm + }; + + Ok(Self { + cartesi_machine, + builder, + }) + } + + pub fn advance_state( + &mut self, + input: InputBuilder, + ) -> Result<(OutputsForInput, Vec>)> { + let encoded_input = input.encode( + self.builder.chain_id, + current_input_index(self.builder.input_index), + self.builder.dapp_address, + ); + + // v0.21 accepts an advance-state response only at a manual + // rx_accepted yield, and requires the root hash of that pre-input + // state (the state a rejected input reverts to). testsi never + // reverts: a rejection is reported as `InputRejected`. + expect_awaiting_input(&mut self.cartesi_machine)?; + let revert_root_hash = self.cartesi_machine.root_hash()?; + self.cartesi_machine.send_cmio_response( + cmio::CmioResponseReason::Advance, + &encoded_input, + Some(&revert_root_hash), + )?; + advance_input_index(&mut self.builder.input_index); + + let mut outputs = OutputsForInput::default(); + let mut reports = Vec::new(); + let execution_started_at = Instant::now(); + + while let ControlFlow::Continue(_) = run_machine_increment( + &mut self.cartesi_machine, + &mut outputs, + &mut reports, + execution_started_at, + self.builder.execution_timeout, + )? {} + + Ok((outputs, reports)) + } + + pub fn inspect(&self) -> Result<()> { + panic!("testsi::Machine does not support inspect() yet") + } +} + +fn run_machine_increment( + cartesi_machine: &mut cartesi_machine::machine::Machine, + outputs: &mut OutputsForInput, + reports: &mut Vec, + execution_started_at: Instant, + execution_timeout: Duration, +) -> Result> { + // `run` takes an absolute mcycle target and rejects one already in the past. + let mcycle_end = cartesi_machine + .mcycle()? + .saturating_add(RUN_MCYCLE_INCREMENT); + let break_reason = cartesi_machine.run(mcycle_end)?; + + let control_flow = match break_reason { + break_reason::HALTED => { + return Err(TestsiMachineError::Halted { + mcycle: cartesi_machine.mcycle()?, + }); + } + // Fixed point: the input exhausted its mcycle budget (imcyclemax). + break_reason::MCYCLE_OVERFLOW => { + return Err(TestsiMachineError::McycleOverflow { + mcycle: cartesi_machine.mcycle()?, + }); + } + break_reason::REACHED_TARGET_MCYCLE => { + on_reached_target_mcycle(execution_started_at, execution_timeout)? + } + + break_reason::YIELDED_MANUALLY | break_reason::YIELDED_AUTOMATICALLY => { + handle_yield(cartesi_machine, outputs, reports)? + } + + break_reason::YIELDED_SOFTLY => { + panic!("testsi harness does not support softly yielded execution yet") + } + + _ => { + panic!("machine returned invalid break reason {break_reason}") + } + }; + + Ok(control_flow) +} + +fn on_reached_target_mcycle( + execution_started_at: Instant, + execution_timeout: Duration, +) -> Result> { + let elapsed = execution_started_at.elapsed(); + if elapsed >= execution_timeout { + return Err(TestsiMachineError::ExecutionTimeout { + elapsed, + timeout: execution_timeout, + }); + } + Ok(ControlFlow::Continue(())) +} + +fn handle_yield( + cartesi_machine: &mut cartesi_machine::machine::Machine, + outputs: &mut OutputsForInput, + reports: &mut Vec, +) -> Result> { + let request = cartesi_machine.receive_cmio_request()?; + + Ok(match request { + // Manual yield + cmio::CmioRequest::Manual(cmio::ManualReason::RxAccepted { + output_hashes_root_hash: _, + }) => ControlFlow::Break(()), + + cmio::CmioRequest::Manual(cmio::ManualReason::RxRejected) => { + return Err(TestsiMachineError::InputRejected); + } + + cmio::CmioRequest::Manual(cmio::ManualReason::TxException { message }) => { + return Err(TestsiMachineError::TxException { message }); + } + + cmio::CmioRequest::Manual(cmio::ManualReason::GIO { .. }) => { + panic!("unexpected GIO request") + } + + // Automatic yield + cmio::CmioRequest::Automatic(cmio::AutomaticReason::Progress { mille_progress: _ }) => { + ControlFlow::Continue(()) + } + cmio::CmioRequest::Automatic(cmio::AutomaticReason::TxOutput { data }) => { + outputs.push_encoded(&data); + ControlFlow::Continue(()) + } + cmio::CmioRequest::Automatic(cmio::AutomaticReason::TxReport { data }) => { + reports.push(data); + ControlFlow::Continue(()) + } + }) +} + +/// Fails unless the machine sits at the manual rx_accepted yield that v0.21 +/// requires before an advance-state response. +fn expect_awaiting_input(cartesi_machine: &mut cartesi_machine::machine::Machine) -> Result<()> { + if !cartesi_machine.iflags_y()? { + let state = if cartesi_machine.iflags_h()? { + "machine is halted".to_owned() + } else { + "machine is not at a manual yield".to_owned() + }; + return Err(TestsiMachineError::NotAwaitingInput { state }); + } + match cartesi_machine.receive_cmio_request()? { + cmio::CmioRequest::Manual(cmio::ManualReason::RxAccepted { .. }) => Ok(()), + other => Err(TestsiMachineError::NotAwaitingInput { + state: format!("{other:?}"), + }), + } +} + +fn sanity_check_cm_config(config: &machine::MachineConfig) { + // v0.21 keeps the HTIF feature switches as bitmasks in the iyield and + // iconsole CSRs, indexed by HTIF command. + let htif = &config.processor.registers.htif; + if htif.iyield & (1 << commands::YIELD_MANUAL) == 0 { + eprintln!("warning: machine config has manual yields disabled (htif.iyield)"); + } + if htif.iyield & (1 << commands::YIELD_AUTOMATIC) == 0 { + eprintln!("warning: machine config has automatic yields disabled (htif.iyield)"); + } + if htif.iconsole & (1 << CM_HTIF_CONSOLE_CMD_GETCHAR) != 0 { + eprintln!("warning: machine config has console getchar enabled (htif.iconsole)"); + } + + check_cmio_memory_range_config(&config.cmio.tx_buffer, "tx_buffer"); + check_cmio_memory_range_config(&config.cmio.rx_buffer, "rx_buffer"); +} + +fn check_cmio_memory_range_config(range: &machine::CmioBufferConfig, name: &str) { + assert!( + !range.backing_store.shared, + "cmio range {} cannot be shared", + name + ); +} + +fn current_input_index(input_index: usize) -> U256 { + U256::from(input_index) +} + +fn advance_input_index(input_index: &mut usize) { + *input_index = input_index + .checked_add(1) + .expect("testsi input index overflow"); +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn reached_target_mcycle_continues_before_timeout() { + let started_at = Instant::now(); + let timeout = Duration::from_secs(5); + + let result = on_reached_target_mcycle(started_at, timeout).expect("must continue"); + assert!(matches!(result, ControlFlow::Continue(()))); + } + + #[test] + fn reached_target_mcycle_returns_timeout_error_after_deadline() { + let timeout = Duration::from_millis(1); + let started_at = Instant::now() - Duration::from_secs(1); + + let err = on_reached_target_mcycle(started_at, timeout).expect_err("must time out"); + match err { + TestsiMachineError::ExecutionTimeout { elapsed, timeout } => { + assert!(elapsed >= timeout); + } + other => panic!("unexpected error: {other}"), + } + } + + #[test] + fn input_index_helpers_are_monotonic() { + let mut input_index = 7; + + assert_eq!(current_input_index(input_index), U256::from(7_u64)); + advance_input_index(&mut input_index); + assert_eq!(current_input_index(input_index), U256::from(8_u64)); + } +} + +// fn get_yield(machine: &cartesi_machine::Machine) -> Result<(isize, u32, u64)> { +// let cmd = machine.read_htif_tohost_cmd()? as isize; +// let data = machine.read_htif_tohost_data()?; +// +// let reason = data >> 32; +// let m16 = (1 << 16) - 1; +// let reason = reason & m16; +// let m32 = (1 << 32) - 1; +// let length = data & m32; +// +// Ok((cmd, reason as u32, length)) +// } diff --git a/cartesi-tools/testsi/src/test_runner.rs b/cartesi-tools/testsi/src/test_runner.rs new file mode 100644 index 00000000..02541a9d --- /dev/null +++ b/cartesi-tools/testsi/src/test_runner.rs @@ -0,0 +1,47 @@ +pub type TestResult = Result<(), libtest_mimic::Failed>; + +pub struct TestCase { + pub name: &'static str, + pub function: fn() -> TestResult, + pub ignore: bool, + pub kind: Option<&'static str>, +} + +inventory::collect!(TestCase); + +pub use inventory; +pub use libtest_mimic; +pub use testsi_macros::test_dapp; + +#[macro_export] +macro_rules! testsi_main { + () => { + fn main() { + let mimic_args = { + let mut a = testsi::libtest_mimic::Arguments::default(); + a.test_threads = Some(1); + a + }; + let mut trials: Vec<_> = testsi::inventory::iter:: + .into_iter() + .map(|c| { + let mut t = testsi::libtest_mimic::Trial::test(c.name, c.function) + .with_ignored_flag(c.ignore); + + if let Some(k) = c.kind { + t = t.with_kind(k); + } + + t + }) + .collect(); + + trials.sort_by(|a, b| match a.kind().cmp(b.kind()) { + std::cmp::Ordering::Equal => a.name().cmp(b.name()), + x => x, + }); + + testsi::libtest_mimic::run(&mimic_args, trials).exit(); + } + }; +} diff --git a/cartesi-tools/testsi/src/types.rs b/cartesi-tools/testsi/src/types.rs new file mode 100644 index 00000000..72b9106d --- /dev/null +++ b/cartesi-tools/testsi/src/types.rs @@ -0,0 +1,155 @@ +use std::ops::Index; + +use alloy_primitives::{Address, U256}; +use alloy_sol_types::SolCall; +use types::*; + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Output { + Voucher(Voucher), + Notice(Notice), +} + +impl Output { + pub fn abi_decode>(payload: &T) -> Self { + let payload = payload.as_ref(); + assert!( + payload.len() >= 4, + "encoded output must contain a 4-byte selector, got {} byte(s)", + payload.len() + ); + let selector = &payload[..4]; + if selector == Notice::SELECTOR.as_slice() { + Output::Notice(Notice::abi_decode(payload).expect("failed to decode notice")) + } else if selector == Voucher::SELECTOR.as_slice() { + Output::Voucher(Voucher::abi_decode(payload).expect("failed to decode voucher")) + } else { + panic!("unknown output selector: {selector:?}"); + } + } + + pub fn try_notice(&self) -> Option<&Notice> { + match self { + Self::Notice(n) => Some(n), + Self::Voucher(_) => None, + } + } + + pub fn expect_notice(&self) -> &Notice { + self.try_notice() + .unwrap_or_else(|| panic!("expected voucher {:?} to be a notice", self)) + } + + pub fn try_voucher(&self) -> Option<&Voucher> { + match self { + Self::Notice(_) => None, + Self::Voucher(v) => Some(v), + } + } + + pub fn expect_voucher(&self) -> &Voucher { + self.try_voucher() + .unwrap_or_else(|| panic!("expected notice {:?} to be a voucher", self)) + } +} + +#[derive(Clone, Debug, Default)] +pub struct OutputsForInput { + list: Vec, +} + +impl Index for OutputsForInput { + type Output = Output; + + fn index(&self, index: usize) -> &Self::Output { + &self.list[index] + } +} + +impl OutputsForInput { + pub fn push(&mut self, output: Output) { + self.list.push(output); + } + + pub fn push_encoded>(&mut self, encoded_output: &T) { + self.push(Output::abi_decode(encoded_output)); + } + + pub fn list(&self) -> &Vec { + &self.list + } + + pub fn notices(&self) -> Vec<&Notice> { + self.list.iter().filter_map(|x| x.try_notice()).collect() + } + + pub fn vouchers(&self) -> Vec<&Voucher> { + self.list.iter().filter_map(|x| x.try_voucher()).collect() + } +} + +#[derive(Clone, Debug)] +pub struct InputBuilder { + pub sender: Address, + pub prev_randao: U256, + pub block_number: U256, + pub block_timestamp: U256, + pub payload: Vec, +} + +impl InputBuilder { + pub fn from_address(sender: Address) -> Self { + Self { + sender, + prev_randao: U256::ZERO, + block_number: U256::ZERO, + block_timestamp: U256::ZERO, + payload: Vec::new(), + } + } + + pub fn at_block(mut self, block: usize) -> Self { + self.block_number = block.try_into().unwrap(); + self + } + + pub fn with_payload>(mut self, payload: &T) -> Self { + self.payload = payload.as_ref().into(); + self + } + + pub fn with_block_timestamp(mut self, block_timestamp: usize) -> Self { + self.block_timestamp = block_timestamp.try_into().unwrap(); + self + } + + pub fn encode(self, chain_id: usize, input_index: U256, dapp: Address) -> Vec { + let x = Input::new(( + U256::from(chain_id), + dapp, + self.sender, + self.block_number, + self.block_timestamp, + self.prev_randao, + input_index, + self.payload.into(), + )); + + x.abi_encode() + } + + pub fn payload(&self) -> &[u8] { + &self.payload + } +} + +#[cfg(test)] +mod tests { + use super::Output; + + #[test] + #[should_panic(expected = "encoded output must contain a 4-byte selector")] + fn output_decode_panics_cleanly_on_short_payload() { + let _ = Output::abi_decode(&[0_u8, 1, 2]); + } +} diff --git a/cartesi-tools/testsi/testsi-macros/Cargo.toml b/cartesi-tools/testsi/testsi-macros/Cargo.toml new file mode 100644 index 00000000..75d16bfb --- /dev/null +++ b/cartesi-tools/testsi/testsi-macros/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "testsi-macros" +version = "0.1.0" +license = "Apache-2.0" +edition = "2021" +publish = false + +[lib] +proc-macro = true + +[dependencies] +proc-macro2 = "1.0" +quote = "1.0" +syn = { version = "2.0", features = ["full"] } diff --git a/cartesi-tools/testsi/testsi-macros/src/lib.rs b/cartesi-tools/testsi/testsi-macros/src/lib.rs new file mode 100644 index 00000000..20e111a3 --- /dev/null +++ b/cartesi-tools/testsi/testsi-macros/src/lib.rs @@ -0,0 +1,114 @@ +use proc_macro::TokenStream; +use proc_macro2::TokenTree; +use quote::{quote, spanned::Spanned}; +use syn::{ + parse::{Parse, ParseStream}, + parse_macro_input, ItemFn, LitStr, Result as SynResult, +}; + +#[derive(Debug, Default)] +struct Args { + ignore: bool, + kind: Option, +} + +impl Parse for Args { + fn parse(input: ParseStream) -> SynResult { + let mut pa = Args::default(); + + while !input.is_empty() { + let t = input.parse()?; + match t { + TokenTree::Ident(i) if i == "ignore" => pa.ignore = true, + + TokenTree::Ident(i) if i == "kind" => match input.parse()? { + TokenTree::Group(g) => { + let arr: Vec<_> = g.stream().into_iter().collect(); + if arr.len() != 1 { + return Err(syn::Error::new( + g.span(), + "argument `kind` must have one argument", + )); + } + + pa.kind = Some(syn::parse2(g.stream())?); + } + + x => { + return Err(syn::Error::new( + x.span(), + format!("unrecognized kind `{x}`"), + )); + } + }, + + TokenTree::Punct(p) if p.as_char() == ',' => (), + + x => { + return Err(syn::Error::new( + x.span(), + format!("unrecognized argument `{x}`"), + )); + } + } + } + + Ok(pa) + } +} + +#[proc_macro_attribute] +pub fn test_dapp(args: TokenStream, input: TokenStream) -> TokenStream { + let parsed_args = parse_macro_input!(args as Args); + let test_fn = parse_macro_input!(input as ItemFn); + let name = &test_fn.sig.ident; + let test_name = name.to_string(); + + if !test_fn.sig.inputs.is_empty() { + return syn::Error::new( + test_fn.sig.__span(), + format!("test function `{}` must have no arguments", name), + ) + .into_compile_error() + .into(); + } + + let ignore = parsed_args.ignore; + let kind = quote_kind(parsed_args.kind.as_ref()); + + let expanded = quote! { + #test_fn + + testsi::inventory::submit! { + testsi::TestCase { name: #test_name, function: #name, ignore: #ignore, kind: #kind } + } + }; + + TokenStream::from(expanded) +} + +fn quote_kind(kind: Option<&LitStr>) -> proc_macro2::TokenStream { + if let Some(kind) = kind { + quote! { Some(#kind) } + } else { + quote! { None } + } +} + +#[cfg(test)] +mod tests { + use super::quote_kind; + use proc_macro2::Span; + use syn::LitStr; + + #[test] + fn quote_kind_defaults_to_none() { + assert_eq!(quote_kind(None).to_string(), "None"); + } + + #[test] + fn quote_kind_wraps_literal() { + let kind = LitStr::new("dapp", Span::call_site()); + assert_eq!(quote_kind(Some(&kind)).to_string(), "Some (\"dapp\")"); + } +} diff --git a/cartesi-tools/trolley/Cargo.toml b/cartesi-tools/trolley/Cargo.toml new file mode 100644 index 00000000..c9544a9a --- /dev/null +++ b/cartesi-tools/trolley/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "trolley" +version = "0.1.0" +license = "Apache-2.0" +edition = "2021" +description = "Safe Rust rollup-guest API over libcmt" +publish = false + +[dependencies] +libcmt-sys = { workspace = true } +types = { workspace = true } diff --git a/cartesi-tools/trolley/src/cmt.rs b/cartesi-tools/trolley/src/cmt.rs new file mode 100644 index 00000000..ce72f481 --- /dev/null +++ b/cartesi-tools/trolley/src/cmt.rs @@ -0,0 +1,212 @@ +use crate::{InputMetadata, RollupError, RollupRequest, RollupResult}; + +use core::ffi::c_void; +use std::mem::MaybeUninit; +use types::alloy_primitives::U256; + +pub struct RollupCmt { + r: libcmt_sys::cmt_rollup_t, +} + +impl RollupCmt { + pub fn try_new() -> RollupResult { + let mut r: MaybeUninit = MaybeUninit::uninit(); + let rc = unsafe { libcmt_sys::cmt_rollup_init(r.as_mut_ptr()) }; + cmt_ok("cmt_rollup_init", rc)?; + Ok(Self { + r: unsafe { r.assume_init() }, + }) + } + + pub fn new() -> Self { + Self::try_new().expect("failed to initialize rollup from libcmt") + } +} + +impl Default for RollupCmt { + fn default() -> Self { + Self::new() + } +} + +impl crate::Rollup for RollupCmt { + fn next_input(&mut self) -> RollupResult { + let mut finish = libcmt_sys::cmt_rollup_finish_t { + accept_previous_request: true, + next_request_type: 0, + next_request_payload_length: 0, + }; + + let rc = unsafe { libcmt_sys::cmt_rollup_finish(&mut self.r, &mut finish) }; + cmt_ok("cmt_rollup_finish(accept_previous_request=true)", rc)?; + + let request_type = finish.next_request_type as u32; + match request_type { + x if x == libcmt_sys::HTIF_YIELD_REASON_ADVANCE_STATE => { + let mut advance: MaybeUninit = + MaybeUninit::uninit(); + let rc = unsafe { + libcmt_sys::cmt_rollup_read_advance_state(&mut self.r, advance.as_mut_ptr()) + }; + cmt_ok("cmt_rollup_read_advance_state", rc)?; + let advance = unsafe { advance.assume_init() }; + + let metadata = InputMetadata { + chain_id: U256::from(advance.chain_id), + app_contract: advance.app_contract.data.into(), + msg_sender: advance.msg_sender.data.into(), + block_number: U256::from(advance.block_number), + block_timestamp: U256::from(advance.block_timestamp), + prev_randao: U256::from_be_bytes(advance.prev_randao.data), + index: U256::from(advance.index), + }; + let payload = payload_from_raw( + "advance.payload", + advance.payload.data, + advance.payload.length, + )?; + Ok(RollupRequest::Advance { metadata, payload }) + } + x if x == libcmt_sys::HTIF_YIELD_REASON_INSPECT_STATE => { + let mut inspect: MaybeUninit = + MaybeUninit::uninit(); + let rc = unsafe { + libcmt_sys::cmt_rollup_read_inspect_state(&mut self.r, inspect.as_mut_ptr()) + }; + cmt_ok("cmt_rollup_read_inspect_state", rc)?; + let inspect = unsafe { inspect.assume_init() }; + let payload = payload_from_raw( + "inspect.payload", + inspect.payload.data, + inspect.payload.length, + )?; + Ok(RollupRequest::Inspect { payload }) + } + _ => Err(RollupError::UnexpectedRequestType { request_type }), + } + } + + fn revert(&mut self) -> ! { + let mut finish = libcmt_sys::cmt_rollup_finish_t { + accept_previous_request: false, + next_request_type: 0, + next_request_payload_length: 0, + }; + + let rc = unsafe { libcmt_sys::cmt_rollup_finish(&mut self.r, &mut finish) }; + if rc != 0 { + panic!("cmt_rollup_finish(accept_previous_request=false) failed with rc={rc}"); + } + panic!("revert finished unexpectedly; expected libcmt to halt execution"); + } + + fn gio(&mut self, domain: u16, id: &[u8]) -> RollupResult<(u16, Vec)> { + if id.len() > u32::MAX as usize { + return Err(RollupError::LengthOverflow { + field: "gio.id", + len: id.len(), + max: u32::MAX as usize, + }); + } + + let mut req = libcmt_sys::cmt_gio { + domain, + id_length: id.len() as u32, + id: id.as_ptr() as *mut c_void, + response_code: 0, + response_data_length: 0, + response_data: std::ptr::null_mut(), + }; + + let rc = unsafe { libcmt_sys::cmt_gio_request(&mut self.r, &mut req) }; + cmt_ok("cmt_gio_request", rc)?; + + let response = payload_from_raw( + "gio.response_data", + req.response_data, + req.response_data_length as usize, + )?; + Ok((req.response_code, response)) + } + + fn emit_voucher(&mut self, voucher: &types::Voucher) -> RollupResult<()> { + let destination = voucher.destination; + let value = voucher.value.to_be_bytes(); + let mut index = 0; + + let rc = unsafe { + libcmt_sys::cmt_rollup_emit_voucher( + &mut self.r, + &libcmt_sys::cmt_abi_address { + data: **destination, + }, + &libcmt_sys::cmt_abi_u256 { data: value }, + &libcmt_sys::cmt_abi_bytes_t { + data: voucher.payload.as_ptr() as *mut c_void, + length: voucher.payload.len(), + }, + &mut index, + ) + }; + cmt_ok("cmt_rollup_emit_voucher", rc) + } + + fn emit_notice(&mut self, notice: &types::Notice) -> RollupResult<()> { + let mut index = 0; + let rc = unsafe { + libcmt_sys::cmt_rollup_emit_notice( + &mut self.r, + &libcmt_sys::cmt_abi_bytes_t { + data: notice.payload.as_ptr() as *mut c_void, + length: notice.payload.len(), + }, + &mut index, + ) + }; + cmt_ok("cmt_rollup_emit_notice", rc) + } + + fn emit_report(&mut self, report: &[u8]) -> RollupResult<()> { + let rc = unsafe { + libcmt_sys::cmt_rollup_emit_report( + &mut self.r, + &libcmt_sys::cmt_abi_bytes_t { + data: report.as_ptr() as *mut c_void, + length: report.len(), + }, + ) + }; + cmt_ok("cmt_rollup_emit_report", rc) + } +} + +impl Drop for RollupCmt { + fn drop(&mut self) { + unsafe { libcmt_sys::cmt_rollup_fini(&mut self.r) } + } +} + +fn cmt_ok(operation: &'static str, rc: i32) -> RollupResult<()> { + if rc == 0 { + Ok(()) + } else { + Err(RollupError::CmtCallFailed { + operation, + code: rc, + }) + } +} + +fn payload_from_raw( + field: &'static str, + data: *mut c_void, + length: usize, +) -> RollupResult> { + if length == 0 { + return Ok(Vec::new()); + } + if data.is_null() { + return Err(RollupError::InvalidPayloadPointer { field, len: length }); + } + Ok(unsafe { std::slice::from_raw_parts(data as *const u8, length) }.to_vec()) +} diff --git a/cartesi-tools/trolley/src/lib.rs b/cartesi-tools/trolley/src/lib.rs new file mode 100644 index 00000000..a08e8584 --- /dev/null +++ b/cartesi-tools/trolley/src/lib.rs @@ -0,0 +1,81 @@ +use core::fmt; +use types::alloy_primitives::{Address, U256}; + +pub mod cmt; + +#[derive(Debug)] +pub struct InputMetadata { + pub chain_id: U256, + pub app_contract: Address, + pub msg_sender: Address, + pub block_number: U256, + pub block_timestamp: U256, + pub prev_randao: U256, + pub index: U256, +} + +#[derive(Debug)] +pub enum RollupRequest { + Advance { + metadata: InputMetadata, + payload: Vec, + }, + Inspect { + payload: Vec, + }, +} + +pub type RollupResult = Result; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum RollupError { + CmtCallFailed { + operation: &'static str, + code: i32, + }, + UnexpectedRequestType { + request_type: u32, + }, + LengthOverflow { + field: &'static str, + len: usize, + max: usize, + }, + InvalidPayloadPointer { + field: &'static str, + len: usize, + }, +} + +impl fmt::Display for RollupError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::CmtCallFailed { operation, code } => { + write!(f, "{operation} failed with rc={code}") + } + Self::UnexpectedRequestType { request_type } => { + write!(f, "unknown request type from host: {request_type}") + } + Self::LengthOverflow { field, len, max } => { + write!(f, "{field} length overflow: {len} > {max}") + } + Self::InvalidPayloadPointer { field, len } => { + write!( + f, + "{field} returned null pointer with non-zero length {len}" + ) + } + } + } +} + +impl std::error::Error for RollupError {} + +pub trait Rollup { + fn next_input(&mut self) -> RollupResult; + fn revert(&mut self) -> !; + fn gio(&mut self, domain: u16, id: &[u8]) -> RollupResult<(u16, Vec)>; + fn emit_voucher(&mut self, voucher: &types::Voucher) -> RollupResult<()>; + fn emit_notice(&mut self, notice: &types::Notice) -> RollupResult<()>; + fn emit_report(&mut self, report: &[u8]) -> RollupResult<()>; +} diff --git a/cartesi-tools/types/Cargo.toml b/cartesi-tools/types/Cargo.toml new file mode 100644 index 00000000..e165ef0e --- /dev/null +++ b/cartesi-tools/types/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "types" +version = "0.1.0" +license = "Apache-2.0" +edition = "2024" +description = "Cartesi Rollups ABI types (inputs, outputs, portal deposits) shared by guest and host" +publish = false + +[dependencies] +alloy-sol-types = "1" +alloy-primitives = "1" diff --git a/cartesi-tools/types/src/lib.rs b/cartesi-tools/types/src/lib.rs new file mode 100644 index 00000000..40123270 --- /dev/null +++ b/cartesi-tools/types/src/lib.rs @@ -0,0 +1,236 @@ +pub use alloy_primitives; +pub use alloy_sol_types; + +use alloy_primitives::{Address, U256}; +use alloy_sol_types::sol; +use std::fmt; + +sol! { + /// @notice An advance request from an EVM-compatible blockchain to a Cartesi Machine. + /// @param chainId The chain ID + /// @param appContract The application contract address + /// @param msgSender The address of whoever sent the input + /// @param blockNumber The number of the block in which the input was added + /// @param blockTimestamp The timestamp of the block in which the input was added + /// @param prevRandao The latest RANDAO mix of the post beacon state of the previous block + /// @param index The index of the input in the input box + /// @param payload The payload provided by the message sender + /// @dev See EIP-4399 for safe usage of `prevRandao`. + #[derive(Debug, PartialEq, Eq)] + function EvmAdvance( + uint256 chainId, + address appContract, + address msgSender, + uint256 blockNumber, + uint256 blockTimestamp, + uint256 prevRandao, + uint256 index, + bytes calldata payload + ) external; + + /// @notice A piece of verifiable information. + /// @param payload An arbitrary payload. + #[derive(Debug, PartialEq, Eq)] + function Notice(bytes calldata payload) external; + + /// @notice A single-use permission to execute a specific message call + /// from the context of the application contract. + /// @param destination The address that will be called + /// @param value The amount of Wei to be transferred through the call + /// @param payload The payload, which—in the case of Solidity + /// contracts—encodes a function call + #[derive(Debug, PartialEq, Eq)] + function Voucher( + address destination, + uint256 value, + bytes calldata payload + ) external; + + /// @notice A single-use permission to execute a specific delegate call + /// from the context of the application contract. + /// @param destination The address that will be called + /// @param payload The payload, which—in the case of Solidity + /// libraries—encodes a function call + #[derive(Debug, PartialEq, Eq)] + function DelegateCallVoucher(address destination, bytes calldata payload) external; + + /// @notice Encode an Ether deposit. + /// @param sender The Ether sender + /// @param value The amount of Wei being sent + /// @param execLayerData Additional data to be interpreted by the execution layer + /// @return The encoded input payload + #[derive(Debug, PartialEq, Eq)] + function encodeEtherDeposit( + address sender, + uint256 value, + bytes calldata execLayerData + ) internal pure returns (bytes memory) { + return abi.encodePacked( + sender, // 20B + value, // 32B + execLayerData // arbitrary size + ); + } + + /// @notice Encode an ERC-20 token deposit. + /// @param token The token contract + /// @param sender The token sender + /// @param value The amount of tokens being sent + /// @param execLayerData Additional data to be interpreted by the execution layer + /// @return The encoded input payload + #[derive(Debug, PartialEq, Eq)] + function encodeERC20Deposit( + address token, + address sender, + uint256 value, + bytes calldata execLayerData + ) internal pure returns (bytes memory) { + return abi.encodePacked( + token, // 20B + sender, // 20B + value, // 32B + execLayerData // arbitrary size + ); + } + + /// @notice Encode an ERC-721 token deposit. + /// @param token The token contract + /// @param sender The token sender + /// @param tokenId The token identifier + /// @param baseLayerData Additional data to be interpreted by the base layer + /// @param execLayerData Additional data to be interpreted by the execution layer + /// @return The encoded input payload + /// @dev `baseLayerData` should be forwarded to `token`. + #[derive(Debug, PartialEq, Eq)] + function encodeERC721Deposit( + address token, + address sender, + uint256 tokenId, + bytes calldata baseLayerData, + bytes calldata execLayerData + ) internal pure returns (bytes memory) { + bytes memory data = abi.encode(baseLayerData, execLayerData); + return abi.encodePacked( + token, // 20B + sender, // 20B + tokenId, // 32B + data // arbitrary size + ); + } + + /// @notice Encode an ERC-1155 single token deposit. + /// @param token The ERC-1155 token contract + /// @param sender The token sender + /// @param tokenId The identifier of the token being transferred + /// @param value Transfer amount + /// @param baseLayerData Additional data to be interpreted by the base layer + /// @param execLayerData Additional data to be interpreted by the execution layer + /// @return The encoded input payload + /// @dev `baseLayerData` should be forwarded to `token`. + #[derive(Debug, PartialEq, Eq)] + function encodeSingleERC1155Deposit( + address token, + address sender, + uint256 tokenId, + uint256 value, + bytes calldata baseLayerData, + bytes calldata execLayerData + ) internal pure returns (bytes memory) { + bytes memory data = abi.encode(baseLayerData, execLayerData); + return abi.encodePacked( + token, // 20B + sender, // 20B + tokenId, // 32B + value, // 32B + data // arbitrary size + ); + } + + /// @notice Encode an ERC-1155 batch token deposit. + /// @param token The ERC-1155 token contract + /// @param sender The token sender + /// @param tokenIds The identifiers of the tokens being transferred + /// @param values Transfer amounts per token type + /// @param baseLayerData Additional data to be interpreted by the base layer + /// @param execLayerData Additional data to be interpreted by the execution layer + /// @return The encoded input payload + /// @dev `baseLayerData` should be forwarded to `token`. + #[derive(Debug, PartialEq, Eq)] + function encodeBatchERC1155Deposit( + address token, + address sender, + uint256[] calldata tokenIds, + uint256[] calldata values, + bytes calldata baseLayerData, + bytes calldata execLayerData + ) internal pure returns (bytes memory) { + bytes memory data = abi.encode(tokenIds, values, baseLayerData, execLayerData); + return abi.encodePacked( + token, // 20B + sender, // 20B + data // arbitrary size + ); + } +} + +sol! { + interface IERC20 { + #[derive(Debug, PartialEq, Eq)] + function transfer(address recipient, uint256 amount) external returns (bool); + } +} + +pub type Input = EvmAdvanceCall; +pub type Voucher = VoucherCall; +pub type Notice = NoticeCall; +pub type Erc20Transfer = IERC20::transferCall; + +pub const ERC20_DEPOSIT_PREFIX_BYTES: usize = 20 + 20 + 32; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Erc20Deposit { + pub token: Address, + pub sender: Address, + pub value: U256, + pub exec_layer_data: Vec, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Erc20DepositDecodeError { + PayloadTooShort { + expected_at_least: usize, + got: usize, + }, +} + +impl fmt::Display for Erc20DepositDecodeError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::PayloadTooShort { + expected_at_least, + got, + } => write!( + f, + "ERC-20 deposit payload too short: expected at least {expected_at_least} bytes, got {got}" + ), + } + } +} + +impl Erc20Deposit { + pub fn decode(payload: &[u8]) -> Result { + if payload.len() < ERC20_DEPOSIT_PREFIX_BYTES { + return Err(Erc20DepositDecodeError::PayloadTooShort { + expected_at_least: ERC20_DEPOSIT_PREFIX_BYTES, + got: payload.len(), + }); + } + + Ok(Self { + token: Address::from_slice(&payload[0..20]), + sender: Address::from_slice(&payload[20..40]), + value: U256::from_be_slice(&payload[40..72]), + exec_layer_data: payload[72..].to_vec(), + }) + } +} diff --git a/examples/canonical-app/Dockerfile b/examples/canonical-app/Dockerfile index f2f2069a..af5f3043 100644 --- a/examples/canonical-app/Dockerfile +++ b/examples/canonical-app/Dockerfile @@ -1,9 +1,11 @@ -# Stage 1: download `rollup-init` +# Stage 1: download `cartesi-init` # ----------------------------------------------------------------------------- FROM riscv64/debian:stable-slim AS extractor -ARG MACHINE_EMULATOR_TOOLS_VERSION=0.17.2 -ARG TOOLS_SHA512="4af9911a5a76738d526bfc2b5462cf96c9dee98ec8b23f3ca91ac4849d5761765f471b5e2e8779809bc4a26d2799f8e744622864fa549ada5941e21d999ff4be" +# Pairs with CARTESI_MACHINE_VERSION (toolchain-pins.env); keep the libcmt +# vendored in cartesi-tools/libcmt-sys/libcmt at the same release. +ARG MACHINE_EMULATOR_TOOLS_VERSION=0.18.0 +ARG TOOLS_SHA512="11a48b68545818a5e3b978e8c5339284ea7ac84acc44cc5b16550839c99066aaeec3c4107e043e653597613ebfb24fe0748984888af841341ce5bcb4dd01dae5" ADD https://github.com/cartesi/machine-guest-tools/releases/download/v${MACHINE_EMULATOR_TOOLS_VERSION}/machine-guest-tools_riscv64.deb /tmp/tools.deb diff --git a/examples/canonical-app/justfile b/examples/canonical-app/justfile index 74310db4..9aceb468 100644 --- a/examples/canonical-app/justfile +++ b/examples/canonical-app/justfile @@ -7,7 +7,10 @@ linux_kernel_filename := `grep '^CARTESI_LINUX_KERNEL_FILENAME=' ../../toolchain out_dir := "out" source_date_epoch := "0" -linux_kernel := out_dir + "/linux.bin" +# Cache the kernel under its pinned filename so a changed pin misses the cache +# and re-downloads (a generic name would verify a stale kernel against its own +# stale checksum). +linux_kernel := out_dir + "/" + linux_kernel_filename linux_kernel_sha512 := linux_kernel + ".sha512" linux_kernel_url := "https://github.com/cartesi/machine-linux-image/releases/download/" + linux_image_release + "/" + linux_kernel_filename linux_kernel_sha512_url := linux_kernel_url + ".sha512" diff --git a/tests/e2e/src/watchdog_compare.rs b/tests/e2e/src/watchdog_compare.rs index fcf6944c..ade60ddb 100644 --- a/tests/e2e/src/watchdog_compare.rs +++ b/tests/e2e/src/watchdog_compare.rs @@ -389,7 +389,7 @@ async fn prove_cm_inspect_genesis( let status = Command::new("cartesi-machine") .current_dir(workspace) - .arg("--no-rollback") + .arg("--no-revert") .arg(format!("--load={},sharing:none", machine_image.display())) .arg(format!( "--cmio-inspect-state=query:{},report:{}", diff --git a/toolchain-pins.env b/toolchain-pins.env index d4c8caf7..8d550548 100644 --- a/toolchain-pins.env +++ b/toolchain-pins.env @@ -10,13 +10,13 @@ XGENEXT2FS_SHA256_ARM64=e5aca81164b762bbe5447bacef41e4fa9e357fd9c8f44e519c520622 # Must match the cartesi-machine used to build canonical-machine-image-* tarballs # and the watchdog OCI image (in-process cartesi Lua module + CLI). -CARTESI_MACHINE_VERSION=v0.20.0 -CARTESI_MACHINE_SHA256_AMD64=46b2f37b889091df3b89a8909467935f8dd4a1426eeb0491b6a346a12f0c341c -CARTESI_MACHINE_SHA256_ARM64=27ea10571335ad174b75388e7de54a3d3434bd607554d8c0bdf6abca47ceae0d +CARTESI_MACHINE_VERSION=v0.21.0 +CARTESI_MACHINE_SHA256_AMD64=5f13034f43454c340062c677146daabe77c587cee1bd60342c95b8ad8f1463a3 +CARTESI_MACHINE_SHA256_ARM64=866f0bde2db53b9b8e6a6eac85e5ad4116338379fa25cdfcc5e4bf835f948bb1 # machine-linux-image release paired with CARTESI_MACHINE_VERSION (canonical-app justfile). -CARTESI_LINUX_IMAGE_RELEASE=v0.20.0 -CARTESI_LINUX_KERNEL_FILENAME=linux-6.5.13-ctsi-1-v0.20.0.bin +CARTESI_LINUX_IMAGE_RELEASE=v0.21.0 +CARTESI_LINUX_KERNEL_FILENAME=linux-6.5.13-ctsi-2-v0.21.0.bin # lua-cURLv3 is vendored in-tree under watchdog/third_party/lua-curl/src (no pin # to track -- the compiled bytes are the in-tree source). See its UPSTREAM file. From a04aa5bdfe7d49674608c94193494301464b3de2 Mon Sep 17 00:00:00 2001 From: gcdepaula Date: Tue, 22 Sep 2026 18:11:04 -0300 Subject: [PATCH 02/11] docs: record Cartesi Machine facts and the watchdog v2 plan docs/cartesi-machine.md owns the emulator behavior the canonical guest, watchdog and recovery depend on: all-or-nothing versions, the reference host semantics (revert on reject, sticky fixed points, inspect on a discarded snapshot), sharing modes and durable publication, clone fallback and measured costs, memory ranges and labels, hashing, and the CLI's traps as a test oracle. The watchdog v2 plan records the decisions: generic state sources (inspect or a labeled memory range), in-process execution with the reference snapshot semantics, SHA-256 digest comparison, a fetch-first tick, and a divergence latch with incident tooling, runbook and drills. --- docs/cartesi-machine.md | 133 ++++++++++++++ docs/plans/2026-09-watchdog-v2.md | 281 ++++++++++++++++++++++++++++++ 2 files changed, 414 insertions(+) create mode 100644 docs/cartesi-machine.md create mode 100644 docs/plans/2026-09-watchdog-v2.md diff --git a/docs/cartesi-machine.md b/docs/cartesi-machine.md new file mode 100644 index 00000000..82ba50dc --- /dev/null +++ b/docs/cartesi-machine.md @@ -0,0 +1,133 @@ +# Cartesi Machine Facts We Rely On + +The canonical application runs inside the Cartesi Machine (CM). The watchdog +drives it, the canonical guest is built for it, and cockroach recovery exports +from it. This page records the CM behavior those components depend on, +especially behavior that is easy to assume wrongly. Everything here holds for +the pinned version (`CARTESI_MACHINE_VERSION` in +[`toolchain-pins.env`](../toolchain-pins.env), currently v0.21.0) and was +checked against the emulator source at that tag. Re-check it on every bump. + +## Versions are all-or-nothing + +A stored machine records a config archive version and the emulator's machine +identifiers; loading rejects any mismatch. A CM bump therefore invalidates every +stored image, release tarball, and watchdog checkpoint: rebuild the canonical +images and re-`init` watchdog state directories. The kernel +(`machine-linux-image`) and guest tools (`machine-guest-tools`) are paired with +the emulator release and move with it. The development shell pins the same +emulator through the parent flake. + +## Rollup host semantics + +The reference host loop is the `cartesi-machine` CLI (`run_advance_state_epoch` +in `src/cartesi-machine.lua`); Dave's PRT client implements the same rules. +Any other host, including the watchdog, must match it exactly: + +| Guest outcome after an input | Host action | +|---|---| +| `RX_ACCEPTED` manual yield | commit the input's state | +| `RX_REJECTED` manual yield | restore the pre-input snapshot | +| `TX_EXCEPTION`, halt, any other manual yield, or mcycle overflow | fixed point: the machine stays there forever, with no revert | + +- The host reads the root hash, snapshots, then sends the input together with + that root as the *revert root hash*. The emulator refuses an advance unless + the machine sits at an `RX_ACCEPTED` yield and the revert root hash equals the + current root, so a host cannot silently continue from a rejected state. +- Each input gets an mcycle budget of 2^48 (`imcyclemax`). Exhausting it is a + fixed point, not a reject. At realistic speeds this takes days, so a stuck + guest shows up as a hung host long before the budget ends. +- An inspect query is legal only at an `RX_ACCEPTED` yield and runs on a + snapshot the host discards. Inspect execution is never part of canonical + history, and its single report is bounded by the 2 MiB CMIO TX buffer. + +A fixed point is permanent: once the canonical machine reaches one, no later +input is ever processed for that application. + +## Storage, sharing, and snapshots + +- **Loading** defaults to `SHARING_NONE`: every backing file is mapped + privately and never modified, under a shared `flock`. `SHARING_ALL` maps the + files shared and runs the machine in place on disk, under an exclusive + `flock`, so one directory cannot be open both ways at once. +- **`store`** writes every range in full as sparse files, refuses an existing + directory, and does **not** fsync. Durable publication uses `sync_stored` + (fsyncs files, the directory, and its parent), then `rename_stored` (atomic, + no-replace, fsyncs both parents). `remove_stored` also syncs the parent. +- **`clone_stored`** reflinks writable files (`clonefile` on macOS APFS, + `FICLONE` on Linux btrfs/XFS), hardlinks read-only ones, and falls back to a + sparse copy when the filesystem cannot link (`ENOTSUP`, `EXDEV`, `EPERM`). It + always works; it is cheap only on copy-on-write filesystems. +- **Snapshots come in two families.** An in-place (`SHARING_ALL`) machine + snapshots by closing, cloning its directory, and reopening; this is the CLI's + `--revert-mode=stored` and Dave's sling node. The CLI's default + `--revert-mode=fork` instead forks a JSON-RPC machine server process, which + refuses to fork a machine with shared ranges (both processes would see the + writes). We run the machine in-process and use the clone family only. + +Measured on APFS (Apple M5 Max) with a machine holding a 1 GiB NVRAM, 768 MiB +of it non-zero: + +| Operation | Cost | +|---|---| +| Write 768 MiB in place and compute the root hash | 0.68 s | +| Clone the whole stored machine | ~0.1 s, 64 KiB of new disk | +| Per input: root hash, close, clone, reopen, dirty 16 pages, commit or revert | ~50 ms; 100 inputs grew disk by ~40 MB | + +Every revert in that run restored exactly the recorded pre-input root hash. On +a filesystem without reflinks, each clone is a full sparse copy of the +machine. + +## Memory ranges for application state + +An application can keep canonical state in a labeled memory range instead of +answering an inspect query: + +- **NVRAM** (`--nvram=label:,...`) appears in the guest as `/dev/uioN`. It + has no filesystem layer and no page cache: guest writes through `mmap` are + immediately in the machine state. It requires the paired kernel with UIO + (ctsi-2 and later). +- **Flash drives** appear as `/dev/pmemN`. Guest writes go through the page + cache, so the application must flush (`fsync`, `msync(MS_SYNC)`, or + `O_DIRECT`) before finishing each input; otherwise the drive holds a stale + mix of pages. +- **A comparison range must be raw.** The CLI formats a flash drive that has no + data file (`mke2fs`) and mounts one that has; either puts filesystem metadata + the native engine cannot reproduce into the range. Declare state drives with + `mke2fs:false,mount:false`, or use an NVRAM. +- **Labels.** User labels are stored in the machine config (`flash_drive[i].label`, + `nvram[i].label`). The automatic names `flashdriveN` and `nvramN` exist only as + device-tree aliases; `cartesi.util.find_drive` matches user labels only. The + CLI's NVRAM and drive init lines call the guest-tools `nvram`/`flashdrive` + helpers, which are glibc binaries and do not run on a musl (Alpine) rootfs. +- Read a range with `machine:read_memory(start, length)`. The stored file layout + (`-.bin`) is an internal format; do not depend on it. + +## Hashing + +- `cartesi.sha256(data)` and `cartesi.keccak256(data)` hash a single Lua string + in one shot; there is no streaming hasher. Hashing a range in-process needs + the whole range as one string (about twice the range in peak memory). +- The machine's own hash tree uses keccak256 by default (sha256 is + configurable), with 32-byte leaves and 4 KiB pages. `get_node_hash(start, + ceil_log2(length))` returns a range's Merkle root at the cost of rehashing + dirty pages only, and `get_proof` ties it to the machine root hash. Any party + comparing against that root must reproduce the same tree. + +## The CLI as a test oracle + +The CLI is a script, not a module: it parses the global `arg` and ends in +`os.exit`, so it cannot be `require`d. Its building blocks can (`cartesi`, +`cartesi.util`, `cartesi.hash-tree`, `cartesi.jsonrpc`). When tests run the CLI +as a reference: + +- `--cmio-advance-state` checks the outputs Merkle root from genesis by default; + resuming from a mid-history checkpoint needs `check_outputs_merkle_root:false`. +- `--max-mcycle` reached mid-input exits 0 and stores the mid-input state. +- Exit codes do not distinguish outcomes (a halt with payload 0 exits 0); load + the stored machine and read its state instead, cross-checked with + `--final-hash`. +- An inspect query is delivered even to a machine at an exception yield, and + its own outcome is not reported. +- `--remote-spawn` leaves machine servers running unless `--remote-shutdown` is + also given. `--no-rollback` no longer exists; use `--no-revert`. diff --git a/docs/plans/2026-09-watchdog-v2.md b/docs/plans/2026-09-watchdog-v2.md new file mode 100644 index 00000000..20f5b480 --- /dev/null +++ b/docs/plans/2026-09-watchdog-v2.md @@ -0,0 +1,281 @@ +# Watchdog v2 + +**Status:** active plan (2026-09-22). Items marked *proposed* await +confirmation; everything else is decided. When the work lands, move durable +content to its owners (named in [Documentation](#documentation)) and reduce this +file to any remaining work. + +## Purpose + +The watchdog independently replays L1 inputs in the canonical Cartesi Machine +and checks that the sequencer's accepted checkpoint holds the same application +state. v2 changes three things: + +1. **Where the state comes from.** Requiring every application to answer an + inspect `state` query is not a reasonable contract, and inspect cannot carry + a state larger than one 2 MiB report. The only real client keeps its + canonical state `M` in a machine memory range (flash drive or NVRAM). +2. **How inputs execute.** The Lua host loop re-implemented rollup semantics + and got them wrong: no revert on reject, inspect run on the machine that is + then stored, and halt/exception treated as transient. CM v0.21 refuses to + feed an input after a reject without a revert, so the old loop cannot run on + the pinned emulator at all. +3. **What happens when it fires.** Detection alone is not enough. A divergence + needs a latch, preserved evidence, a documented playbook, tooling for each + step, and drills practiced before an incident. + +The [Cartesi Machine facts](../cartesi-machine.md) page records the emulator +behavior this design depends on. + +## Decisions + +### 1. A generic executable with two state sources + +The watchdog stays one generic executable. `init` persists the state source: + +- `inspect`: the query `state` must end at `RX_ACCEPTED` with exactly one + report. The toy wallet keeps using it. +- `range: