Skip to content

feat(api): update API spec from langfuse/langfuse a7fa476 - #1830

Merged
wochinge merged 1 commit into
mainfrom
api-spec-bot-a7fa476-32558842575-1
Aug 22, 2026
Merged

feat(api): update API spec from langfuse/langfuse a7fa476#1830
wochinge merged 1 commit into
mainfrom
api-spec-bot-a7fa476-32558842575-1

Conversation

@langfuse-bot

@langfuse-bot langfuse-bot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Greptile Summary

The PR regenerates the unstable evaluation API from the latest upstream specification.

  • Adds multi-evaluator assignments to evaluation rules, including create, update, and response models.
  • Adds default evaluator mappings and optional bounds for numeric evaluator outputs.
  • Updates evaluator identity and lifecycle contracts, removing evaluator scope and allowing assignment cleanup during deletion.
  • Updates synchronous and asynchronous generated clients and package exports consistently.

Confidence Score: 5/5

The PR appears safe to merge with no concrete blocking or independently actionable non-blocking issues identified.

The generated request serialization, response models, exports, and synchronous and asynchronous client surfaces are internally consistent with the updated multi-evaluator API contract.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Client["SDK caller"] --> RuleRequest["Evaluation-rule request"]
  RuleRequest --> Assignments["One or more evaluator assignments"]
  Assignments --> Evaluator["Evaluator family"]
  Assignments --> Mapping["Default or rule-specific mapping"]
  Evaluator --> Latest["Latest evaluator version"]
  Mapping --> Target["Observation or experiment target"]
Loading

Reviews (1): Last reviewed commit: "feat(api): update API spec from langfuse..." | Re-trigger Greptile

Context used:

@langfuse-bot
langfuse-bot requested a review from wochinge August 22, 2026 07:11
@github-actions

Copy link
Copy Markdown

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — this is an auto-generated Fern spec-sync PR; the flagged issues are docstring-only nits with no functional impact.

What was reviewed: verified the new/changed generated types (EvaluationRuleReadMapping, EvaluationRuleEvaluatorAssignment, CreateEvaluationRuleWithEvaluatorsRequest, PublicNumericEvaluatorOutputScoreDefinition, etc.) import cleanly and are internally consistent with the removal of EvaluatorScope; checked the __init__.py lazy-import tables (_dynamic_imports/__all__) stay in sync with the added/removed symbols; confirmed the evaluator/mapping deprecated-alias fields were consistently made optional alongside the new evaluators array across client, raw_client, and type files.

Extended reasoning...

Overview

This PR is a mechanical Fern-generated sync of the Python SDK's unstable API client from an upstream API definition (langfuse/langfuse@a7fa476). It removes the EvaluatorScope concept (project vs. managed), introduces multi-evaluator assignments on evaluation rules (evaluators array replacing the single evaluator/mapping pair), adds a stable evaluator id semantic across versions, adds bounded numeric score definitions, and raises the active-rule limit from 50 to 500. All 36 changed files are under langfuse/api/unstable/, i.e. generated client/model code, not hand-written business logic.

Security risks

None identified. There is no auth, crypto, or permission logic in this diff — it's pydantic model definitions and generated HTTP client wrappers for an already-unstable, versioned API surface.

Level of scrutiny

Low-to-medium is appropriate here: this is routine, repeatable spec-sync tooling output (the repo already has a prior commit with an identical PR title pattern), the actual behavior is delegated to the server-side API, and correctness is primarily about type/import consistency, which I verified directly by importing the changed modules. The two issues surfaced by the bug hunt are both docs-only (stale docstring prose referencing deleted EvaluatorScope semantics, and a pre-existing wrong import path convention in examples) with explicitly no runtime effect.

Other factors

No CODEOWNERS restriction applies to these paths. No prior claude[bot] review exists on this PR to reconcile against. I ran a syntax/AST check and a live import of all newly added and structurally changed types (assignment types, read-mapping type, evaluator types) and confirmed no import errors or circular-import issues from the EvaluatorScope removal.

Comment thread langfuse/api/unstable/evaluators/types/evaluator.py
Comment thread langfuse/api/unstable/evaluation_rules/types/evaluation_rule.py
@wochinge
wochinge enabled auto-merge (squash) August 22, 2026 07:22
@wochinge
wochinge merged commit d22213a into main Aug 22, 2026
32 of 34 checks passed
@wochinge
wochinge deleted the api-spec-bot-a7fa476-32558842575-1 branch August 22, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants