perf(where): remove per-event interpretive overhead from the MQL predicate tree

[1linkovdim]

Why

MQLQuery.matches is invoked once per event, per active query, per tagging stage. In a merged fleetwide CPU profile of the Mantis agent fleet, the corekaas AbstractAckableTaggingStage accounts for 15.00% of agent CPU, of which 10.75% is inside MQLQuery.matches. Splitting the self time inside that subtree:

	share of fleet CPU
java.util.regex (actual matching work)	4.42%
Clojure interpretive runtime (plumbing)	4.03%

That second row is the target here. It is not the comparison a query asks for — it is trampolining, seq allocation, per-event fn? tests and per-event operator dispatch inside the compiled predicate tree. It is also disjoint from #15, which attacks the regex row by rewriting .*(a|b).* patterns into contains checks; the two changes compose.

Nothing here changes what a query means.

What was happening once per event that only needs to happen once per query

1. with-meta on a function. On the JVM, clojure.lang.AFunction.withMeta cannot tag in place. It returns an anonymous AFunction$1 extends RestFn whose entire body is:

protected Object doInvoke(Object args) {
    return AFunction.this.applyTo((ISeq) args);
}

So every call to a tagged function goes invoke(datum) → RestFn.invoke → allocate an ArraySeq → doInvoke → AFn.applyTo → AFn.applyToHelper → the real function. Five frames and an allocation, per operand, per node, per event. In the profile this shows up as RestFn.invoke 11.28% cumulative, AFunction$1.doInvoke 10.46%, AFn.applyToHelper 10.37%.

Both the operand constructors in operands.cljc and the predicates in where.cljc were tagged this way.

• Operands genuinely need their metadata — select.cljc reads :name, :as, :distinct off them — so they now carry it on a small MetaFn deftype (new io.mantisrx.mql.fnmeta) that implements IFn directly and IObj for round-tripping. meta, with-meta and fn? behave exactly as before (MetaFn implements the clojure.lang.Fn marker interface, which is what clojure.core/fn? tests).
• The {:clause :where} tag on the predicates is read by nothing. Its only consumer is util/ops->clause-map, which has no callers anywhere in the repo. It is simply dropped.

2. Operator dispatch. check-predicate decided MQL's nil policy per event by comparing the operator function against the = and not= vars. The operator is fixed when the query compiles, so the policy is resolved there instead. check-predicate itself is left in place, public and unchanged, for API compatibility — the compiler just no longer calls it.

3. The fn? tests on each operand, likewise resolved at compile time.

4. property->fn built a one-element key seq and reduced over it for every lookup. properties/get-in is 2.95% of fleet CPU cumulative, essentially all of it reached from property->fn, and 1.83% of fleet CPU is reduce/seq machinery wrapped around a single map lookup. The single-property case — the overwhelming majority of real queries — now closes over the key and calls properties/get directly. properties/get itself now tests for List and RandomAccess with two literal instance? forms rather than (every? #(instance? % m) [List RandomAccess]), which was allocating a vector and running a seq traversal on every lookup.

Predicate results are deliberately not coerced to boolean: ==+ and ==* return the truthy value from some, and server.clj's make-matcher already wraps the tree in (comp boolean ...).

Correctness

New mql-jvm/src/test/clojure/io/mantisrx/mql/test_where_pred.clj pins behaviour by differential comparison: it keeps a reference implementation of the old binary-expr->pred built on check-predicate, and asserts the new compiled predicate agrees with it across every comparison operator (=, ==, !=, <>, <, >, <=, >=) crossed with nil operands, absent properties and constant operands — plus star-binary-expr->pred, non-boolean ==+/==* results, operand metadata round-tripping (including apply, map, and re-tagging through with-meta), nested and indexed property lookups, and end-to-end where clauses through eval-mql.

Ran 58 tests containing 341 assertions.
0 failures, 0 errors.

(main is 50 tests / 97 assertions before the new namespace.)

Benchmark

No benchmark sources are committed — the harness below is throwaway scaffolding, and I did not want to add a JMH plugin to this build on the strength of one change. It is reproduced in full here so the numbers can be checked.

before is this harness run against main; after is this branch. Run with -f 2 -wi 3 -i 5 -r 1s -w 1s -prof gc on Zulu JDK 17.0.17, JMH 1.36, on a laptop; ± is JMH's 99.9% confidence interval and B/op is gc.alloc.rate.norm.

benchmark	before ns/op	after ns/op	delta	B/op before	B/op after
equality — country == 'US'	75.27 ± 1.85	25.12 ± 2.00	−66.6%	248	56
conjunction — 3 terms anded	242.93 ± 20.15	135.95 ± 0.60	−44.0%	856	280
disjunction — 3 terms ored	241.97 ± 3.25	155.77 ± 2.66	−35.6%	856	280
nestedProperty — e['request']['path'] == ...	111.85 ± 3.32	58.81 ± 4.24	−47.4%	328	56
regexMatch — path ==~ /.*api.*ping/	118.63 ± 2.05	61.74 ± 1.39	−48.0%	392	256

35–67% faster and 2–4.4× less garbage per matched event. Note regexMatch retains 256 B/op — that is the regex work itself, which is #15's territory, not this change's.

Caveat: laptop numbers, JMH auto-detected compiler blackholes, and shorter iteration counts than you would want for a final figure.

How to reproduce

Two things are worth knowing before wiring this up:

• Do not go through the MQLServer AOT class. Its :gen-class uses the default - prefix, so the static parse/parses stubs look for server/-parse, which server.clj does not define; calling them throws UnsupportedOperationException. The working entry point is the Clojure fn io.mantisrx.mql.jvm.interfaces.server/make-query, reached through clojure.java.api.Clojure.
• A query that fails to parse still yields a Query, and its matcher accepts everything. My first regexMatch run measured 9 ns/op because I had written =~; MQL's regex operator is ==~. The harness therefore asserts at setup that each query accepts a populated datum and rejects an empty map, so a parse failure cannot masquerade as a fast benchmark.

Apply this to the build — me.champeau.jmh:jmh-gradle-plugin:0.7.2 on the root buildscript classpath, plugin and config in mql-jvm:

// build.gradle, buildscript { dependencies { ... } }
classpath 'me.champeau.jmh:jmh-gradle-plugin:0.7.2'

// mql-jvm/build.gradle
apply plugin: 'me.champeau.jmh'

dependencies {
  jmh 'com.fasterxml.jackson.core:jackson-databind:2.9.9'
}

jmh {
  // The predicate tree is invoked once per event per active query, so the
  // interesting figures are per call latency and per call allocation.
  benchmarkMode = ['avgt']
  timeUnit = 'ns'
  fork = 2
  warmupIterations = 5
  iterations = 5
  profilers = ['gc']
}

Then mql-jvm/src/jmh/java/io/mantisrx/mql/MQLMatchBenchmark.java:

package io.mantisrx.mql;

import clojure.java.api.Clojure;
import clojure.lang.IFn;
import io.mantisrx.mql.jvm.core.Query;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;

/**
 * Measures {@link Query#matches(Object)}, the operation a tagging stage performs once per event
 * per active subscription. Queries are parsed in setup, exactly as they are in production, so the
 * measured cost is purely the compiled predicate tree walking one event.
 */
@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(2)
public class MQLMatchBenchmark {

    private Query equality;
    private Query conjunction;
    private Query disjunction;
    private Query nested;
    private Query like;

    private Map<String, Object> datum;

    private static Query compile(String query) {
        IFn require = Clojure.var("clojure.core", "require");
        require.invoke(Clojure.read("io.mantisrx.mql.jvm.interfaces.server"));
        IFn makeQuery = Clojure.var("io.mantisrx.mql.jvm.interfaces.server", "make-query");
        return (Query) makeQuery.invoke("bench", query);
    }

    /**
     * A query which fails to parse still yields a Query, whose matcher accepts everything. Verify
     * up front that each benchmarked query really does discriminate, so a silent parse failure
     * cannot masquerade as a fast benchmark.
     */
    private static Query verify(String query, Map<String, Object> matching) {
        Query q = compile(query);
        if (!q.matches(matching) || q.matches(new HashMap<String, Object>())) {
            throw new IllegalStateException("query did not compile to a discriminating predicate: " + query);
        }
        return q;
    }

    @Setup
    public void setup() {
        Map<String, Object> request = new HashMap<>();
        request.put("path", "/api/v1/ping");
        request.put("method", "GET");

        datum = new HashMap<>();
        datum.put("country", "US");
        datum.put("status", 500L);
        datum.put("app", "nfweb");
        datum.put("path", "/api/v1/ping");
        datum.put("esn", "NFANDROID2-PRV-P-SAMSUNG");
        datum.put("request", request);

        equality = verify("select * from stream where country == 'US'", datum);
        conjunction = verify(
                "select * from stream where country == 'US' and status >= 400 and app == 'nfweb'", datum);
        disjunction = verify(
                "select * from stream where country == 'CA' or country == 'MX' or country == 'US'", datum);
        nested = verify("select * from stream where e['request']['path'] == '/api/v1/ping'", datum);
        like = verify("select * from stream where path ==~ /.*api.*ping/", datum);
    }

    @Benchmark
    public void equality(Blackhole bh) {
        bh.consume(equality.matches(datum));
    }

    @Benchmark
    public void conjunction(Blackhole bh) {
        bh.consume(conjunction.matches(datum));
    }

    @Benchmark
    public void disjunction(Blackhole bh) {
        bh.consume(disjunction.matches(datum));
    }

    @Benchmark
    public void nestedProperty(Blackhole bh) {
        bh.consume(nested.matches(datum));
    }

    @Benchmark
    public void regexMatch(Blackhole bh) {
        bh.consume(like.matches(datum));
    }
}

Build on JDK 17 (Gradle 7.6.6 here does not accept 25) and run ./gradlew :mql-jvm:jmh. To diff two revisions, build :mql-jvm:jmhJar on each and run the two jars directly: java -jar build/libs/mql-jvm-*-jmh.jar -f 2 -wi 3 -i 5 -r 1s -w 1s -prof gc -rf json.

Notes for reviewers

• check-predicate, where-clause->fn, having-clause->fn, search-condition->pred and boolean-term->pred are unchanged.
• fnmeta.cljc is a .cljc with a :cljs branch that falls straight back to with-meta, so the ClojureScript build is unaffected.
• Two with-meta calls in operands.cljc (property-with-as->fn, distinct-operand->property) are intentionally left as-is: they re-tag an already-wrapped MetaFn, so they go through IObj.withMeta and keep the fast path.
• Numbers above come from a merged fleet profile dated 2026-08-11. perf(optimization): rewrite the regex->string-op rules to cover real queries #15's description quotes 7.71% / 16.00% for the same frames where this pull gets 5.52% / 10.75% — same shape, different base window (and likely --max-nodes truncation). Worth pinning one window before comparing the two.