Repository navigation
agent_simulation: add JobMetrics.Turn.text - #1874
Merged
u9g merged 1 commit intoOct 19, 2026
Merged
Conversation
A job's transcript reached only the simulator's logs, so a client reading the job (lk agent simulate export, a CI check) could see each turn's timings and scores but not the words that were scored.
🦋 Changeset detectedLatest commit: f97eec3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Base automatically changed from
jason/simulation-job-call-facts
to
jason/simulation-job-failure-reason
October 19, 2026 22:25
Contributor
Author
|
Combined into #1867, which carries this PR's commit. It shows as merged only because the commit landed on its base branch; nothing has reached main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
texttoSimulationRun.JobMetrics.Turn: the words of each turn, as the simulation's judges read them. That's the speaker's own words, or, when those are unknown, the words as the other side heard them. The agent's own words are unknown when it has no RemoteSession, e.g. over SIP.Today a job reports one row per turn, with timings and scores, but not what was said. The transcript the judges scored only reaches the simulator's logs. So anyone reading a job through the API, such as
lk agent simulate exportor a CI check, can see that a turn was slow or interrupted but not what it said. They also can't check that the judged transcript has the right shape, for example one caller line per turn and each reply after the line it answers.Stacked on #1868, which is stacked on #1867.