Skip to content

feat: wire observability client to harness and evals - #2182

Open
nborges-aws wants to merge 2 commits into
refactorfrom
harness-and-eval-wiring
Open

feat: wire observability client to harness and evals#2182
nborges-aws wants to merge 2 commits into
refactorfrom
harness-and-eval-wiring

Conversation

@nborges-aws

@nborges-aws nborges-aws commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Description

Extend shared observability abstraction to Batch Evaluation and Harness. Plus, add endpoint qualifier support to Runtime observability commands.

  • adds log stream reader to the shared CloudWatch client using GetLogEvents
  • moves CloudWatch logic hosted in batch eval stream (e.g. pagination, normalization) to the shared CloudWatch client
  • resolves Harness resources to their underlying Runtime IDs for observability
  • reuse Harness runtime resolution in existing Eval agent resolution
  • adds harness logs and harness traces list/get functionality
  • adds optional --qualifier flag to Runtime and Harness (falls back to default endpoint)
  • updates shared trace output handling and test coverage for CloudWatch, Eval, Harness, and Runtime wiring

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • bun run test (2809 pass, 0 fail)
  • I ran npm run test:unit and npm run test:integ
  • I ran npm run typecheck
  • I ran npm run lint
  • If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Stack created with GitHub Stacks CLIGive Feedback 💬

@github-actions github-actions Bot added the size/xl PR size: XL label Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from 648c527 to 7c4f499 Compare September 3, 2026 16:16
@github-actions github-actions Bot added size/l PR size: L and removed size/xl PR size: XL labels Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Sep 3, 2026
@codecov-commenter

codecov-commenter commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.20000% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.01%. Comparing base (831025c) to head (b1ddb37).

Files with missing lines Patch % Lines
src/handlers/harness/logs/index.tsx 44.11% 19 Missing ⚠️
src/handlers/harness/traces/get/index.tsx 65.38% 9 Missing ⚠️
src/handlers/harness/traces/list/index.tsx 62.50% 9 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           refactor    #2182      +/-   ##
============================================
- Coverage     97.09%   97.01%   -0.09%     
============================================
  Files           544      549       +5     
  Lines         37834    38028     +194     
============================================
+ Hits          36736    36893     +157     
- Misses         1098     1135      +37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from 7c4f499 to 9564d5a Compare September 3, 2026 17:21
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from 9564d5a to cbc10fc Compare September 3, 2026 18:03
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from cbc10fc to 6c2d90f Compare September 3, 2026 19:35
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from 6c2d90f to d95292b Compare September 3, 2026 20:12
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from d95292b to 45e3cb9 Compare September 3, 2026 20:24
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
Base automatically changed from traces-scaffold to refactor September 3, 2026 20:38
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from 45e3cb9 to fff91c9 Compare September 3, 2026 20:38
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@nborges-aws
nborges-aws force-pushed the harness-and-eval-wiring branch from fff91c9 to b1ddb37 Compare September 3, 2026 21:28
@github-actions github-actions Bot added size/l PR size: L and removed size/l PR size: L labels Sep 3, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Sep 3, 2026
@AlexanderRichey

Copy link
Copy Markdown
Contributor

Looks like this can't be merged yet?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l PR size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants