Estimate Ollama model cost from reference pricing and aggregate cache hits #88

Summary

Ollama and Ollama Cloud usage reports input/output tokens but does not report cached-input token counts. CodexHub should still calculate an API-equivalent model cost estimate when the underlying open model has authoritative published token pricing.

This estimate is distinct from the user's actual Ollama bill:

• local Ollama may have no API charge but does have local compute cost;
• Ollama Cloud is billed through plans/GPU usage rather than a fixed token tariff;
• the displayed dollar value represents the equivalent cost at the underlying model publisher's official API rates.

The UI must label that distinction instead of showing $0.00 or presenting the estimate as an Ollama invoice amount.

Current evidence

• Gateway telemetry records Ollama input/output token usage.
• Runtime metadata for ollama/glm-5.2 has pricing: null.
• The same underlying model already has reference pricing under another canonical identity, for example zai/glm-5.2.
• estimate_usage_cost() already supports estimating cached tokens with the aggregate observed cache-hit ratio when a Provider does not report cache usage.
• Ollama requests currently fail earlier at pricing lookup, so that cache-estimation path never produces a cost.

Pricing resolution

Resolve Ollama model pricing in this order:

1. explicit user/Provider pricing override;
2. exact Ollama model metadata with an authoritative source;
3. canonical underlying-model alias mapped to the model publisher's official API pricing;
4. otherwise leave the request unpriced.

Examples must use an explicit alias/identity table rather than fuzzy display-name matching. Pricing metadata must record its source URL, retrieval/version date, currency, and that it is a reference estimate.

Cache estimation

Because the Ollama API does not provide cache-hit token counts:

• use the aggregate cache-hit ratio observed from cache-reporting traffic;
• compute the ratio over the same active date window/filter as the cost estimate;
• apply it only when the selected reference pricing has a cached-input rate;
• expose the ratio and number of estimated requests in cost_label/coverage details;
• never label estimated cached tokens as directly observed;
• define an explicit fallback when no aggregate cache ratio exists (recommended: price all input as uncached and label the fallback).

Billing semantics

Keep billing model separate from the reference estimate:

• per_token: estimated/actual from published input/cache/output rates;
• subscription or gpu_time: actual bill is not derivable from tokens, but an API-equivalent reference estimate may be shown;
• local_compute: no provider API charge; API-equivalent estimate is optional and must not be called actual cost;
• unknown: no dollar estimate.

Recommended UI wording for Ollama:

API-equivalent estimate based on the underlying model's published pricing; cached input estimated using the aggregate cache-hit rate. Not your Ollama bill.

Acceptance criteria

• Add canonical underlying-model identity/alias metadata for supported Ollama models.
• Reuse authoritative publisher pricing rather than inventing an Ollama token rate.
• Support explicit user/Provider price overrides with higher precedence.
• Ollama requests with resolvable reference pricing contribute to the cost estimate.
• Ollama cached input uses the aggregate cache-hit ratio because the API does not report cache hits.
• Aggregate cache ratio uses the same active date window/filter as the estimate.
• The UI exposes the estimated cache ratio and marks the result as API-equivalent/reference cost.
• Actual Ollama billing and API-equivalent estimates remain separate concepts.
• Unknown models remain unpriced and are included in pricing coverage from Show pricing coverage for partial Gateway cost estimates #87.
• Local Ollama and Ollama Cloud remain distinguishable.
• Add tests for canonical alias resolution, override precedence, aggregate-cache estimation, no-ratio fallback, and unknown models.

Related

• Show pricing coverage for partial Gateway cost estimates #87 defines unknown and partial cost coverage behavior.
• Refresh OpenAI pricing and support context/service-tier rates #86 covers OpenAI tiered token pricing.

Primary references

• https://ollama.com/pricing
• Use each underlying model publisher's official pricing page as the price source.

Relevant code

• src-tauri/src/models.rs
• src-tauri/src/gateway/telemetry.rs
• frontend/src/components/StackedUsageChartShell.tsx