Kelvran

LLM gateway with an embedded multi-layer cache, plus a standalone agent-evaluation system.

Kelvran is a unified AI infrastructure platform built around two complementary systems: gateway, a self-hosted LLM gateway with an embedded multi-layer cache that sits between your application and any model provider, cutting latency and cost on repeated calls; and evals, a standalone agent-evaluation engine (in development) that will score the quality, correctness, and regressions of what your agents actually produce.

Route and cache with one. Grade the results with the other. Or run both together as one pipeline.

flowchart LR
    A[Your Application] -->|OpenAI-compatible API| B[kelvran/gateway]
    B --> C{Multi-Layer Cache}
    C -->|hit — return cached response| A
    C -->|miss| D[(LLM Providers<br/>OpenAI · Anthropic · etc.)]
    D --> C
    B -.->|request/response traces| E[kelvran/evals — planned]
    E -.->|quality scores, regression flags| F[Eval Reports / CI Gate]

gateway handles the request path in real time: route → check cache → call a provider on miss → populate cache → return. evals will run asynchronously against the traces gateway emits (or against any agent output fed to it directly) and turn them into scores you can gate a release on.

• gateway: core routing + multi-layer cache
• evals: initial scoring engine (v0.1)
• evals: CI-gate mode (fail a build on regression)
• Shared trace format between gateway and evals
• Client SDKs (once there's a stable API worth wrapping)
• Hosted docs site

• 🐛 Found a bug? Open an issue
• 📖 Community standards: see this repo for the org-wide CODE_OF_CONDUCT, CONTRIBUTING, and SECURITY policy that every Kelvran repo inherits.