Skip to content

fix(mistral): drop eager response serialization, use allowlist metadata pass-through#604

Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
mainfrom
fix/mistral-drop-eager-serialization
Jul 21, 2026
Merged

fix(mistral): drop eager response serialization, use allowlist metadata pass-through#604
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
mainfrom
fix/mistral-drop-eager-serialization

Conversation

@starfolkai

@starfolkai starfolkai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Aligns the Mistral integration with .agents/skills/sdk-integrations/SKILL.md, following the same pattern as #594 (litellm) and #583 (anthropic). Audit turned up two issues:

  • Excess serialization. _normalize_mistral_multimodal_value (which recursively calls value.model_dump(mode="python", by_alias=True)) was invoked 25 times across tracing.py. Two sites did real waste:

    • Every non-streaming response finalizer (_finalize_completion_response, _finalize_conversation_response, _finalize_ocr_response, _finalize_embeddings_response) full-dumped the response just to read a handful of fields (id, model, choices/outputs/pages, usage).
    • _build_request_metadata walk-and-dumped every metadata value (temperature, top_p, tools, response_format, stop, …) — none of which carry attachments.

    Both are redundant: bt_safe_deep_copy in logger.log_internal already model_dumps pydantic values at log time. Replaced with attribute reads through the existing _get_value helper (already dict-or-pydantic aware and handles Mistral's Unset sentinel — no new helper needed).

  • Aggregators. _aggregate_completion_events / _aggregate_conversation_events / _aggregate_transcription_events / _aggregate_speech_events pre-dumped usage / segments into the intermediate dict. _parse_usage_metrics already accepts pydantic, and bt_safe_deep_copy handles the output. Dropped.

  • Tool-span walkers. _completion_tool_calls, _conversation_tool_outputs, _log_completion_tool_spans, _log_conversation_tool_spans switched to _get_value so they walk raw pydantic responses instead of assuming pre-dumped dict shape.

Multimodal walker retained where attachment materialization matters (_start_span input path, _append_delta_content / _merge_tool_calls streaming assembly, per-field conversation-output accumulators).

Metadata is still allowlist-per-surface — no changes to what fields we capture. Comments were not excessive; nothing to trim.

Net: −36 LOC in tracing.py, one dead helper file removed (_response_data_to_metadata, _conversation_response_data_to_metadata, _conversation_outputs_data, _ocr_output_data collapsed into the pydantic-aware equivalents).

Test plan

  • No new mocks/fakes. The two existing narrow fakes (_PlainResponse for the plain-Python usage path, and lambda fail-injectors for error-propagation tests) fall under the SKILL's "narrow error injection / provider-independent helpers" carve-out. All provider-behavior coverage stays on the 30+ @pytest.mark.vcr cassette-backed tests.
  • No cassette re-recording. HTTP behavior is unchanged; only in-process span shaping was touched.
  • cd py && nox -s "test_mistral(latest)" — 45/45 passed.
  • cd py && nox -s "test_mistral(1.12.4)" — 43/43 passed (2 pre-existing skips for speech API which doesn't exist on 1.12.4).
  • ruff check + ruff format --check on touched files — clean.

Two existing unit tests were updated because their assertions hinged on the old eager-serialization behavior:

  • test_wrappers_ignore_usage_when_response_normalization_failstest_wrappers_read_usage_from_plain_python_responses. The old test asserted metrics were dropped for plain-Python responses — that only happened because .model_dump() failed on non-pydantic objects and we bailed out. The new code correctly reads usage via getattr; assertion is now positive (metrics are present).
  • test_aggregate_completion_events_merges_tool_calls_and_content: aggregated["usage"] is now the raw pydantic UsageInfo (dumps at log time, not in the aggregator); switched to getattr(..., "total_tokens").

🤖 Generated with Claude Code

Created by abhijeet

Slack thread

…ta pass-through

Align the Mistral integration with `.agents/skills/sdk-integrations/SKILL.md`
following the same pattern as #594 (litellm) and #583 (anthropic):

- **Response finalizers** no longer call `_normalized_mistral_dict(response)`
  to full-`model_dump` the response just to read a few fields. Read `choices`
  / `outputs` / `pages` / `usage` / `id` / `model` directly via the existing
  `_get_value` helper (dict-or-pydantic aware, handles Mistral's `Unset`
  sentinel). `bt_safe_deep_copy` in `logger.log_internal` already
  `model_dump`s pydantic values at log time.
- **`_build_request_metadata`** stops walk-and-dumping every metadata value
  (`temperature`, `top_p`, `tools`, `response_format`, `stop`, …). These
  don't carry attachments; pass through and let bt_json handle it.
- **Streaming aggregators** stop pre-dumping `usage` / `segments` into the
  intermediate dict; `_parse_usage_metrics` already accepts pydantic and
  the log-time deep-copy handles the output.
- **Tool-span walkers** (`_completion_tool_calls`, `_conversation_tool_outputs`,
  `_log_completion_tool_spans`, `_log_conversation_tool_spans`) switched to
  `_get_value` so they work on raw pydantic responses.

The multimodal walker is retained where attachment materialization matters:
the `_start_span` input path, `_append_delta_content` / `_merge_tool_calls`
streaming assembly, and per-field accumulators for conversation outputs.

Metadata is still allowlist-per-surface — no changes there.

Test updates (no new mocks, no cassette re-recording):
- `test_wrappers_ignore_usage_when_response_normalization_fails` →
  `test_wrappers_read_usage_from_plain_python_responses`. The old test
  asserted metrics were dropped for plain-Python responses — that was a
  side-effect of `.model_dump()` failing on non-pydantic objects. The new
  behavior correctly reads usage via `getattr`; assert the positive.
- `test_aggregate_completion_events_merges_tool_calls_and_content`:
  `aggregated["usage"]` is now the raw pydantic `UsageInfo` (dumps at log
  time, not in the aggregator); switched to `getattr(..., "total_tokens")`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit a095270 into main Jul 21, 2026
82 checks passed
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) deleted the fix/mistral-drop-eager-serialization branch July 21, 2026 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants