Skip to content

[Benchmark]: Measure policy evaluation overhead #57

Description

@rian-be

Summary

Benchmark the overhead of policy evaluation in the core mutation runtime.

Goal

Measure the cost of evaluating policies under execution patterns that most affect the core engine: synchronous policies, asynchronous policies, and multiple policies evaluated together.

Problem

Policy evaluation is one of the main extension points in the core runtime, and its cost can change as policy orchestration evolves. Without dedicated benchmark, it is difficult to tell whether change affects the policy pipeline, the engine execution path, or the interaction between multiple policies.

This benchmark should make policy overhead visible without mixing it with governance provider behavior or unrelated example code.

Scope

  • Benchmark synchronous policy evaluation overhead
  • Benchmark asynchronous policy evaluation overhead
  • Benchmark multiple policies evaluated in single runtime pass
  • Keep the benchmark focused on core runtime policy evaluation
  • Reuse existing policy benchmark helpers where they already exist
  • Make the benchmark scenarios explicit enough to compare sync and async paths over time

Design Expectations

  • Sync and async policy paths should be measured separately.
  • Multi policy evaluation should expose aggregate overhead rather than hiding it in single combined number.
  • The benchmark setup should be minimal and deterministic.
  • If helper policies are needed, they should stay inside the benchmark project.
  • Benchmark names should clearly indicate the evaluation mode being measured.

Suggested Measurements

  • one synchronous allow policy
  • one asynchronous allow or deny policy
  • several policies with mixed priorities in single evaluation pass

Acceptance Criteria

  • The benchmark suite measures both sync and async policy evaluation overhead
  • Multi policy evaluation is benchmarked as distinct scenario
  • The benchmark output makes the evaluation mode obvious
  • The benchmark remains centered on the core runtime policy pipeline

Non-Goals

  • This issue does not change policy semantics
  • This issue does not add governance specific policy scenarios
  • This issue does not design generic policy authoring framework
  • This issue does not change runtime behavior

Notes

This issue is child of the benchmark umbrella #55.

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkBenchmark coverage and performance measurement changes

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions