Skip to content

Latest commit

 

History

History
318 lines (221 loc) · 24.6 KB

File metadata and controls

318 lines (221 loc) · 24.6 KB

server.pyStellarRpcServer

StellarRpcServer exposes the Stellar RPC API over HTTP/JSON-RPC. Its job is to make the K semantics usable as a server. The compiled semantics are a one-shot interpreter — one process invocation per request, with no networking and no memory between runs — and StellarRpcServer is the long-running process wrapped around them: it keeps the HTTP socket open, decodes the XDR envelope, runs each request through the semantics, and returns whatever response.json the semantics produced. All RPC dispatch, receipt bookkeeping, ledger accounting, and response formatting happen in K — the server holds none of that state itself.

The server depends on three collaborators through the protocols in interfaces.py rather than on their concrete classes: an Encoder (the XDR → request-envelope decoder, TransactionEncoder), an Interpreter (the K runner, NodeInterpreter), and a Store (the io-dir reader/writer, ChainStore). A composition root wires the concrete classes together — see build_server in __main__.py — so a test can drive the server with fakes.


Class structure

class StellarRpcServer:
    interpreter: Interpreter  # the K runner (NodeInterpreter)
    encoder:     Encoder      # the XDR → request-envelope decoder (TransactionEncoder)
    store:       Store        # the io-dir reader/writer (ChainStore)
    io_dir:      Path         # the io-dir root, taken from the store
    state_file:  Path         # io_dir / 'state.kore', taken from the store

The per-item artifact directories (receipts/, traces/, ledgers/, requests/, wasms/, events/) belong to the Store, which owns the io-dir layout; the server reaches them through the store's methods rather than holding their paths. See ChainStore for the full layout.

The server is a plain http.server.HTTPServer (not pyk's JsonRpcServer). A BaseHTTPRequestHandler reads each POST body and calls _handle, which parses the JSON-RPC frame and delegates to handle_rpc.

JSON-RPC framing

The server implements the JSON-RPC 2.0 framing rules, including batch calls:

  • A single request object gets a single response object.
  • An array body is a batch: each element is validated and dispatched on its own, and the response is an array with one entry per answered element (matched by id; invalid elements each get an Invalid Request error with id: null). Batch elements run sequentially — the server is single-threaded by design, so a batch is equivalent to sending its elements one at a time. An empty array is answered with a single Invalid Request error object, per the spec.
  • A request without an id member is a notification: it is executed but never answered, not even with an error. A batch of only notifications (or a single notification) produces an empty response body.

Framing happens entirely in Python (_handle / _handle_batch / _handle_single); the K semantics see one request envelope per invocation regardless of how requests were framed on the wire.

The xdrFormat parameter

getTransaction and sendTransaction accept the spec's optional xdrFormat parameter. Only 'base64', the spec default, is supported: XDR fields in responses are always base64 strings. The alternative 'json' format (XDR rendered as JSON objects) is not implemented and is rejected with error -32602 and a message saying so; any other value is rejected with -32602 as well. The check runs before anything else, so a sendTransaction with a bad xdrFormat is rejected without executing the transaction.

handle_rpc(method, params, request_id) -> str

handle_rpc is the dispatch entry point; it returns the JSON-RPC response envelope as a string. You can call it without the HTTP layer, which is convenient for scripts and tests:

server = build_server(io_dir=Path('out'))
server.handle_rpc('sendTransaction', {'transaction': xdr})

For sendTransaction it builds the request envelope with encoder.build_tx_request and runs it with interpreter.run (skipping the <program> wasm injection when the hash already has a receipt, so a duplicate is not re-executed — the semantics answer DUPLICATE); for simulateTransaction it builds the envelope with encoder.build_simulate_request and runs it without committing; for the read-only methods (getHealth, getNetwork, getLatestLedger, getTransaction, getTransactions, getLedgers, getLedgerEntries, getEvents, traceTransaction) it builds a small envelope and runs it. For the history methods (getTransactions, getLedgers) and getEvents the server also validates the request parameters — pagination limits, ledger-window bounds, cursor format and exclusivity, filter/topic counts — because the JSON-RPC parameter-error path lives here. In every case the content of the response is produced by the semantics (node.md), not by Python — the exceptions are the failure fallback (below), the per-ledger XDR artifacts (_record_closed_ledger), the finished per-ledger event files (_finalize_events), and the XDR fields of the simulateTransaction response, which K cannot construct or serialize. Each call is logged to stderr.


Startup

server = build_server(
    io_dir=Path('out'),  # omit for a fresh temporary directory
    network_passphrase=Network.TESTNET_NETWORK_PASSPHRASE,
    host='localhost',
    port=8000,
)
server.serve()

build_server (in __main__.py) is the composition root: it resolves the io-dir, constructs the NodeInterpreter, TransactionEncoder, and ChainStore, and injects them into the server. io_dir defaults to None, in which case build_server creates a fresh temporary directory (tempfile.mkdtemp) and runs against that — a throwaway chain that starts empty on every launch and leaves the working directory untouched. Pass an explicit io_dir to keep the state in a known place.

At construction the ChainStore prepares the io dir, where state.kore lives at io_dir / 'state.kore':

  • state.kore absentinterpreter.empty_config() produces the initial idle K configuration (a blank-slate state with no accounts, contracts, or storage) and the store writes it; metadata.json is seeded with {"latest_ledger": 0}.
  • state.kore present — it is used as-is, and metadata.json is seeded only if missing. This lets you resume a previous session (ledger counter and stored receipts included) or start against a pre-built state.

In both cases the store creates the receipts/, traces/, ledgers/, requests/, wasms/, and events/ directories if they do not already exist, because the K file-system hooks write into them but cannot create them.

Once the socket is bound, serve logs three lines to stderr: whether it is starting from a fresh state (an empty io-dir) or resuming an existing one (with the latest ledger), the io-dir path, and the listening address. Instruction tracing is always on, so every transaction the semantics run produces a trace. (Tracing only produces records for contract invocations.)


State lifecycle

state.kore and metadata.json hold the persistent chain state; per-transaction receipts and traces live under receipts/ and traces/; the server and semantics also exchange a few transient files per request. See architecture.md for the complete io-dir layout.

startup (state.kore absent):
          → empty_config() → state.kore ; metadata.json {latest_ledger:0}
          → create receipts/ traces/ ledgers/ requests/ wasms/ events/

per successful transaction:
          → the semantics run the steps (trace → traces/trace_<hash>.jsonl),
            write receipts/receipt_<hash>.json, bump latest_ledger in metadata.json,
            and write response.json
          → NodeInterpreter persists the new state.kore
          → the server writes ledgers/ledger_<seq>.json (the ledger→tx index entry,
            with the ledger-header XDR artifacts)

per failed (stuck) transaction:
          → no response.json is produced; state.kore and metadata.json are left unchanged
          → the server writes a FAILED receipts/receipt_<hash>.json (see below)

Because these artifacts live on disk, the server can be stopped and restarted without losing the world state, the ledger counter, or the stored receipts.


RPC methods

All methods are answered by the K semantics and follow the Stellar RPC specification — except traceTransaction, which is a komet-specific extension (see below) — including its serialization quirks: ledger sequence numbers and protocolVersion are JSON numbers, while close-time fields (latestLedgerCloseTime, oldestLedgerCloseTime, createdAt) are int64s serialized as JSON strings holding a decimal integer, matching what real stellar-rpc emits.

getHealth

{ "status": "healthy", "latestLedger": 4, "latestLedgerCloseTime": "1716000000", "oldestLedger": 0, "oldestLedgerCloseTime": "0", "ledgerRetentionWindow": 5 }

The node retains every ledger since genesis, so oldestLedger is always 0 and the retention window covers all ledgers so far. Per-ledger close times are not recorded: the latest close time is approximated by the request time and the oldest by the epoch.

getNetwork

{ "passphrase": "Test SDF Network ; September 2015", "protocolVersion": 22 }

friendbotUrl is omitted (not null) because the node runs no friendbot — matching real stellar-rpc's omitempty serialization.

getLatestLedger

getLatestLedger returns the current ledger sequence (the latest_ledger from metadata.json), which increments by 1 per successfully committed transaction. The id is a deterministic per-ledger stand-in for the ledger-header hash, derived from the sequence (see #ledgerId in node-semantics.md).

{ "id": "1715609f7c74...", "protocolVersion": 22, "sequence": 4 }

getVersionInfo

getVersionInfo reports the komet-node package version as the RPC server version and the komet package (the K semantics of Soroban executing the transactions) as the "Captive Core". komet-node is a Python package, so no commit hash or build timestamp is baked in at build time — those fields are all-zeros / epoch placeholders with the spec-correct types. protocolVersion is a JSON number.

{ "version": "0.1.0", "commitHash": "0000...0000", "buildTimestamp": "1970-01-01T00:00:00", "captiveCoreVersion": "komet 0.1.86 (K semantics of Soroban)", "protocolVersion": 22 }

getFeeStats

getFeeStats returns constant fee distributions: komet-node has no fee market, so every statistic is the network minimum inclusion fee of 100 stroops over an empty sample (transactionCount: "0", ledgerCount: 0). Matching real stellar-rpc, every distribution field except ledgerCount is a JSON string holding a decimal number; latestLedger is a JSON number read live from metadata.json.

{ "sorobanInclusionFee": { "max": "100", "min": "100", "mode": "100", "p10": "100", ..., "p99": "100", "transactionCount": "0", "ledgerCount": 0 }, "inclusionFee": { ... }, "latestLedger": 4 }

sendTransaction

sendTransaction submits a base64-encoded XDR transaction envelope.

Execution model: real Stellar RPC was designed around a mempool and ledger close, so the API requires sendTransaction to return PENDING and have clients poll getTransaction. komet-node has no mempool — the semantics execute the transaction immediately — but it still returns PENDING to stay compatible with the two-step pattern.

Response:

{ "hash": "<64-char hex>", "status": "PENDING", "latestLedger": 5, "latestLedgerCloseTime": "1716000000" }

Resubmitting a transaction whose hash already has a receipt returns status DUPLICATE without re-executing it: the ledger does not advance and the stored receipt is untouched.

A transaction that decodes as XDR but cannot be processed (e.g. it contains an operation the semantics do not support) is rejected at admission time with status ERROR and an errorResultXdr carrying a txMALFORMED TransactionResult — mirroring how real stellar-rpc reports core's admission rejections. Such a transaction never reaches the ledger: no receipt is stored (getTransaction stays NOT_FOUND) and the ledger does not advance. Undecodable XDR remains a plain -32602 Invalid params error, as in real stellar-rpc. TRY_AGAIN_LATER is never returned: it signals mempool backpressure, and komet-node executes synchronously without a mempool, so the condition it reports cannot arise.

simulateTransaction

simulateTransaction takes a base64-encoded XDR transaction envelope (which may be unsigned) containing exactly one InvokeHostFunction operation and executes it as a dry run: the invocation runs against the current world state, but nothing is committed — no receipt, no trace, no ledger bump, and the resulting configuration is discarded (interpreter.run(..., commit=False)).

Response (success):

{
  "latestLedger": 4,
  "minResourceFee": "100000",
  "results": [ { "xdr": "<base64 SCVal XDR>", "auth": [] } ],
  "transactionData": "<base64 SorobanTransactionData XDR>"
}

results[0].xdr is the host function's return value. The K semantics report it as a JSON-encoded ScVal (see the simulateTransaction section of node.md), and the server serializes it to SCVal XDR (scval_from_json in scval.py) — K has no XDR encoder and no base64 hook, so this is the one read path where Python builds part of the response content. minResourceFee is a synthetic constant and transactionData an empty-footprint, all-zero-resources placeholder, because the semantics do not meter resources. auth is always empty (authorization recording is not implemented).

Response (failure — the invocation trapped, the target contract does not exist, the transaction is not a single InvokeHostFunction, or the host function is an upload/deploy, which cannot be simulated yet):

{ "error": "<reason>", "latestLedger": 4 }

Failures are reported in the result body, matching real stellar-rpc; only an undecodable transaction parameter is a JSON-RPC error (-32602). A failed simulation likewise commits nothing. The optional events, restorePreamble, and stateChanges response fields and the resourceConfig/authMode/xdrFormat parameters are not supported.

traceTransaction (komet-specific extension)

traceTransaction is not part of the Stellar RPC specification — it exists only on komet-node, and clients must not expect it from real Stellar RPC endpoints. It keeps its plain name rather than a vendor-prefixed one (komet_traceTransaction): the official spec has no method of that name and none is announced, so there is no collision to avoid, and renaming would break every existing client for no gain. If stellar-rpc ever claims the name, the method will be renamed with a prefix.

traceTransaction retrieves the instruction trace of a previously submitted transaction. It takes a hash parameter (the same one getTransaction takes) and returns the trace that sendTransaction stored for that transaction. The result is a JSON array with one record per executed WebAssembly instruction (empty when the transaction ran no instructions), or null when no transaction with that hash exists.

[
  {"pos": 3, "instr": ["const", "i32", 1048576], "stack": [], "locals": {}, "mem": null}
]

getTransaction

getTransaction reads the hash's receipts/receipt_<hash>.json file. The hash parameter must be a 64-character hex string; anything else is rejected with -32602 Invalid params (this and traceTransaction share the validation).

Status Meaning
NOT_FOUND No receipts/receipt_<hash>.json for that hash
SUCCESS Transaction executed successfully
FAILED Transaction was submitted but got stuck in the semantics

SUCCESS response:

{
  "status": "SUCCESS", "applicationOrder": 1, "feeBump": false,
  "envelopeXdr": "<base64 XDR>", "resultXdr": "<base64 XDR>", "resultMetaXdr": "<base64 XDR>",
  "ledger": 5, "createdAt": "1716000000",
  "latestLedger": 5, "latestLedgerCloseTime": "1716000000",
  "oldestLedger": 0, "oldestLedgerCloseTime": "0"
}

The ledger-range fields (latestLedger through oldestLedgerCloseTime) are present on every response, NOT_FOUND included. Every ledger contains exactly one transaction, so applicationOrder is always 1; fee-bump envelopes are not supported, so feeBump is always false. resultXdr is a base64 TransactionResult (code txSUCCESS or txFAILED) and resultMetaXdr a base64 TransactionMeta v3; when the transaction invoked a contract, its return value is reported as sorobanMeta.returnValue inside the meta. Both are synthesised by the server (result_xdr.py) from the envelope and the return value the semantics recorded in the receipt — see node-semantics.md. Fees and ledger-entry change sets in these structs are zero/empty (komet-node does not track them). resultMetaXdr is omitted on FAILED receipts — a failed run rolls back and produces no meta. The receipt carries no trace — use traceTransaction with the same hash to fetch it.

Receipts are an internal format, not a stable one: receipts written by older komet-node versions stored ledger as a string and lack applicationOrder/feeBump, and getTransaction returns stored receipts verbatim. Start from a fresh io-dir after upgrading rather than resuming one with old receipts.

getTransactions

getTransactions returns the transactions in a ledger range, in chain order. Params: startLedger (number; mutually exclusive with a cursor), pagination {cursor, limit} (limit 1–200, default 50), and xdrFormat (base64 only; json is rejected with -32602). The records are joined from the per-ledger index files (ledgers/ledger_<seq>.json) and the stored receipts. Failed transactions never close a ledger, so they do not appear in the history.

{
  "transactions": [
    { "status": "SUCCESS", "txHash": "<64-char hex>", "applicationOrder": 1, "feeBump": false,
      "envelopeXdr": "<base64 XDR>", "ledger": 5, "createdAt": 1716000000 }
  ],
  "latestLedger": 5, "latestLedgerCloseTimestamp": 1716000000,
  "oldestLedger": 0, "oldestLedgerCloseTimestamp": 0,
  "cursor": ""
}

Serialization follows real stellar-rpc: ledger sequences and the top-level close times are JSON numbers, and per-transaction createdAt is a number too (unlike the singular getTransaction, where it is a string — an upstream quirk). resultXdr/resultMetaXdr are populated from the stored receipt (real base64 XDR for a contract invocation) and omitted only when a receipt carries none. Each ledger holds exactly one transaction on this node, so applicationOrder is always 1. The cursor is a TOID-style stringified integer (ledger << 32 | applicationOrder << 12) naming the page's last transaction when the page is full, and empty otherwise; resume by passing it as pagination.cursor (without startLedger).

getLedgers

getLedgers returns the closed ledgers in a range and takes the same parameters as getTransactions. Each record comes straight from the ledger's index file; headerXdr (a LedgerHeaderHistoryEntry) and metadataXdr (a LedgerCloseMeta) are built by the server when the ledger closes, since K cannot construct XDR, and the ledger hash is the SHA-256 of the header XDR — unique per ledger and chained through previousLedgerHash.

{
  "ledgers": [
    { "hash": "<64-char hex>", "sequence": 5, "ledgerCloseTime": "1716000000",
      "headerXdr": "<base64 XDR>", "metadataXdr": "<base64 XDR>" }
  ],
  "latestLedger": 5, "latestLedgerCloseTime": 1716000000,
  "oldestLedger": 0, "oldestLedgerCloseTime": 0,
  "cursor": ""
}

Per-ledger ledgerCloseTime is a string holding a decimal number (matching real stellar-rpc's Go ,string encoding), while the top-level close times are numbers. The cursor is the last returned ledger sequence, stringified, under the same full-page rule as getTransactions.

getLedgerEntries

getLedgerEntries takes keys (an array of up to 200 base64-encoded LedgerKey XDR strings; required) and an optional xdrFormat (only "base64" is supported — "json" is rejected with -32602). It returns the entries found for the supported key types — ACCOUNT, CONTRACT_DATA (both the contract-instance entry and persistent/temporary storage), and CONTRACT_CODE — the ones the K world state tracks. Keys that do not resolve (unknown, or of an untracked type) are not an error; they are simply absent from entries.

Response (latestLedger and lastModifiedLedgerSeq are JSON numbers; liveUntilLedgerSeq appears only on Soroban entries):

{
  "entries": [
    { "key": "<base64 LedgerKey>", "xdr": "<base64 LedgerEntryData>",
      "lastModifiedLedgerSeq": 4, "liveUntilLedgerSeq": 4095 }
  ],
  "latestLedger": 4
}

This is the one method whose response the server post-processes: the semantics look the keys up in the K state and answer with intermediate JSON entries, and ledger_entries.py re-encodes them as LedgerEntryData XDR (K cannot produce XDR). Because the K state keeps uploaded wasm parsed, the server stores the raw bytes under wasms/<hash>.wasm at upload time and reattaches them to CONTRACT_CODE entries. The semantics do not track per-entry modification ledgers, so lastModifiedLedgerSeq reports the current ledger; ACCOUNT entries carry the real balance but synthesised constants for the remaining required fields (sequence number 0, master weight 1, no signers).

getEvents

getEvents returns the contract events emitted in a ledger window — startLedger (inclusive) to endLedger (exclusive) — optionally narrowed by up to five filters (type, contractIds, topics with */** wildcards) and paginated with pagination.cursor / pagination.limit (default 100). A cursor replaces startLedger/endLedger and resumes strictly after the last returned event.

Events are captured during sendTransaction: the semantics intercept the contract_event host function and stage each event's topics and data, and after a successful run the server converts the staged records into one finished events/events_<ledger>.json per ledger (_finalize_events — the base64 SCVal XDR, strkey, and TOID encodings are XDR work K cannot do). The K getEvents rules scan, filter, and paginate those files. Parameter validation lives in _get_events_envelope; xdrFormat: "json" is not supported and is rejected with -32602.

Response:

{
  "latestLedger": 4,
  "events": [
    {
      "type": "contract", "ledger": 4, "ledgerClosedAt": "2024-05-18T04:00:00Z",
      "contractId": "C...", "id": "0000000017179873280-0000000000",
      "inSuccessfulContractCall": true, "txHash": "<64-char hex>",
      "topic": ["<base64 SCVal XDR>"], "value": "<base64 SCVal XDR>"
    }
  ],
  "cursor": "0000000021474836480-0000000000"
}

system events are never emitted (the semantics have no source of them), and events carrying values with no staging representation (maps, errors, 256-bit ints) are dropped with a warning rather than served with fabricated XDR.


Failure fallback

A failed transaction leaves the semantics stuck without writing response.json, so interpreter.run returns None. For sendTransaction the server then synthesises the response in Python: it writes a FAILED receipts/receipt_<hash>.json (so a later getTransaction finds it) with a txFAILED resultXdr and no resultMetaXdr, without bumping the ledger, and returns PENDING. ledger on a FAILED receipt pins the latest ledger at failure time (the chain does not advance) and createdAt the submission time, with the same encoding as on SUCCESS receipts. A stuck simulateTransaction run is answered with a result-level {error, latestLedger} and leaves no artifacts behind.

On the success path the server also post-processes the receipt the semantics wrote: _attach_result_xdr replaces the receipt's internal returnValue field (a JSON-encoded SCVal, or null) with the spec-mandated resultXdr/resultMetaXdr base64 XDR, which K cannot construct (skipped for a duplicate resubmission, whose receipt is already final). These are the only response contents the server builds itself.


CLI

komet-node [--host HOST] [--port PORT] [--io-dir DIR]
Flag Default Description
--host localhost Bind address
--port 8000 Port
--io-dir a fresh temp dir Directory holding every artifact (state.kore, metadata.json, receipts/, traces/, ledgers/, requests/, wasms/)