Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.6.0] - 2026-10-22

### Added

- **`captures.rounds` in `evalshift.yaml`.** `first` (the default) or `all` sets
Expand Down
2 changes: 1 addition & 1 deletion DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ hosted (opt-in) run history, diffs, PR gates

The suite is the crux, so the capture SDK is the recommended way to build one: it records real production runs to disk and `evalshift capture sync` promotes them into golden suites. Hand-written suites are fully supported — see [The golden suite](#the-golden-suite).

- **Package name:** `evalshift` · **CLI entry point:** `evalshift` · **version:** 1.5.0
- **Package name:** `evalshift` · **CLI entry point:** `evalshift` · **version:** 1.6.0
- **Python:** >= 3.11 · **License:** Apache-2.0 · **Status:** stable
- **Local-first.** Runs, scores, stats, and reports all happen on your machine under `.evalshift/`. The only network calls are the model API calls you asked for — and, if you opt in, pushes to the hosted service and reads from a capture bucket you own (`captures.store`).
- **Four pieces:** CLI (this doc), SDK, GitHub Action, hosted server — each with its own machine-readable reference for AI tools. See [Ecosystem and AI-tool references](#ecosystem-and-ai-tool-references).
Expand Down
6 changes: 3 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1100,8 +1100,8 @@ used them). Take it knowingly: `--force` rebuilds every promoted case whose capt
(not after `capture clean` or on a fresh checkout, where promoted captures are not fetched) from
its capture, which drops hand edits and `provenance: reviewed` marks. Reviewed cases, stale ones
and ones promoted under another name (`capture promote --as`) are left out of the count. An
evalshift older than the release that added `rounds` rejects the key, so update a CI pin
(`evalshift-version:`) before adding it.
evalshift older than 1.6.0 rejects the key, so update a CI pin (`evalshift-version:`) before
adding it.

Setting either `max_age` or `max_cases` turns [bounded suites](#bounded-suites) on. Both work
without `store`: they govern how captures become suites, whether the captures came from disk or
Expand All @@ -1112,7 +1112,7 @@ because it would silently mean seconds. So is a `max_age` above 100 years, which
"no limit": `captures.max_age: Value error, must be at most 36500d (100 years); omit
captures.max_age for no age limit`.

CLI versions before 1.3.0 reject the `captures:` key, and versions before this release reject
CLI versions before 1.3.0 reject the `captures:` key, and versions before 1.6.0 reject
`max_age` and `max_cases` (the config is strict). Bump the CLI, and a pinned GitHub Action's
`evalshift-version`, before adding them.

Expand Down
2 changes: 1 addition & 1 deletion llms-full.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# evalshift (CLI) — complete reference for AI tools

Canonical hosted copy: https://www.evalshift.dev/cli-llms-full.txt
Package: evalshift (PyPI) | CLI entry point: evalshift | version: 1.5.0
Package: evalshift (PyPI) | CLI entry point: evalshift | version: 1.6.0
Python: >=3.11 | license: Apache-2.0 | status: stable
Install: pip install evalshift (or: uv pip install evalshift)

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "evalshift"
version = "1.5.0"
version = "1.6.0"
description = "Run your prompts on two LLMs and find out, with statistical confidence, what regressed."
readme = "README.md"
license = "Apache-2.0"
Expand Down
Loading