Skip to content

Evals: cryptic error when spans not yet propagated in CloudWatch #666

Description

@notgitika

Description

When running evals, if the spans have not yet propagated in CloudWatch, the CLI surfaces a raw API error that is cryptic and unhelpful. It should instead display a user-friendly message explaining the delay.

Steps to Reproduce

  1. Run an agent invocation
  2. Immediately run evals against that invocation (within ~1-2 minutes)
  3. Observe the error when spans are not yet available

Expected Behavior

The CLI should display a clear, user-friendly message such as:

"Spans not found — it takes 2-3 minutes to propagate in CloudWatch. Please try again shortly."

Actual Behavior

A cryptic, raw API error is shown that does not help the user understand the issue or what to do next.

CLI Version

N/A

Operating System

macOS

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions