Add an opt-in plaintext message trace for conformance tooling

.claude/skills/house-patterns/SKILL.md

@@ -93,7 +93,12 @@ the descriptions in `docs/internals.md`:
   the codec gate
   `SENDSPIN_ENABLE_OPUS` only in `src/decoder.h`, `src/decoder.cpp`,
   `src/player_role.cpp`, `src/source_role.cpp`, `src/source_task.cpp`, and
-  the source's Opus encoder list in `cmake/sources.cmake`. Examples must
+  the source's Opus encoder list in `cmake/sources.cmake`; the host-only
+  message trace gate `SENDSPIN_ENABLE_MESSAGE_TRACE` only in
+  `include/sendspin/config.h`, `src/connection.h`, `src/connection.cpp`,
+  `src/connection_manager.cpp`, `src/noise_transport.h`,
+  `src/noise_transport.cpp`, and its own source list in `cmake/sources.cmake`
+  (no Kconfig entry). Examples must
   build with roles (and Opus) disabled:
   role and opus usage in `examples/` is guarded like an external consumer
   would guard it. Verify this by grepping the example for the role's types

.github/workflows/ci.yml

@@ -104,8 +104,9 @@ jobs:
 
       - name: Configure CMake
         # BUILD_EXAMPLES=OFF so the test job configures only sendspin + tests and skips the
-        # example-only FetchContent deps (e.g. FTXUI).
-        run: cmake -B build-tests -DENABLE_WERROR=ON -DSENDSPIN_BUILD_TESTS=ON -DENABLE_SANITIZERS=ON -DBUILD_EXAMPLES=OFF .
+        # example-only FetchContent deps (e.g. FTXUI). The message trace is on here and in the TSan
+        # job; the build and no-opus jobs build it off, as consumers get it.
+        run: cmake -B build-tests -DENABLE_WERROR=ON -DSENDSPIN_BUILD_TESTS=ON -DENABLE_SANITIZERS=ON -DSENDSPIN_ENABLE_MESSAGE_TRACE=ON -DBUILD_EXAMPLES=OFF .
 
       - name: Build tests
         run: |
@@ -190,7 +191,7 @@ jobs:
       - name: Configure CMake
         # ENABLE_TSAN instead of ENABLE_SANITIZERS: TSan cannot be combined with ASan/UBSan, and
         # the concurrency tests only report a data race when built this way.
-        run: cmake -B build-tsan -DENABLE_WERROR=ON -DSENDSPIN_BUILD_TESTS=ON -DENABLE_TSAN=ON -DBUILD_EXAMPLES=OFF .
+        run: cmake -B build-tsan -DENABLE_WERROR=ON -DSENDSPIN_BUILD_TESTS=ON -DENABLE_TSAN=ON -DSENDSPIN_ENABLE_MESSAGE_TRACE=ON -DBUILD_EXAMPLES=OFF .
 
       - name: Build tests
         run: |

CLAUDE.md

@@ -30,6 +30,7 @@ The library provides `SendspinClient` as the main public API. It handles the ful
 - `RecordStore` (`record_store.h`): pairing records and the Pairing PSK (configured, stored, or generated), seeded from the client config and the persistence provider
 - `Inbox` / `InboxSlot` / `GenerationSlot` (`inbox.h`): single-mutex mailbox for all main-loop-bound cross-thread state - atomic topic bitmask polled lock-free by `loop()`, plus a fixed event ring for ordered lifecycle events
 - `RoleTeardown` / `TeardownTracker` (`teardown_tracker.h`): every role's teardown generation and the main-loop half of a role's teardown
+- `MessageTracer` (`message_trace.h`): one connection's handle on the opt-in plaintext message trace (`SendspinClientConfig::message_trace`), built only with `SENDSPIN_ENABLE_MESSAGE_TRACE`
 - `SendspinTimeFilter` (`time_filter.h`): 2D Kalman filter for NTP-style time sync
 - `SendspinTimeBurst` (`time_burst.h`): burst-based time message coordinator, one per connection
 - `SendspinDecoder` (`decoder.h`): FLAC/PCM decoder wrapper, plus Opus when built with `SENDSPIN_ENABLE_OPUS`
@@ -38,7 +39,7 @@ The library provides `SendspinClient` as the main public API. It handles the ful
 
 Roles are added to the client at runtime via `add_player()`, `add_metadata()`, etc. Each role receives a `SendspinClient*` at construction time and uses it to access shared services (time sync, state publishing, message sending). The consumer provides behavior by implementing listener interfaces (`PlayerRoleListener`, `MetadataRoleListener`, etc.) and setting them via `set_listener()`. Required callbacks are pure virtual; optional callbacks have default no-op implementations. The client dispatches messages to roles via null-pointer checks on role pointers.
 
-Roles can be disabled at compile time via `SENDSPIN_ENABLE_*` cmake options (host build) or Kconfig entries (ESP-IDF build). When a role is disabled, its source files are not compiled and its `add_*()` declaration, accessor, and `unique_ptr` member are removed from `client.h`. Role `#ifdef` guards live in exactly two places in the library: `cmake/sources.cmake` (source lists) and `include/sendspin/client.h`, `src/client.cpp` and `src/client_dispatch.cpp` (dispatch points); the codec gate `SENDSPIN_ENABLE_OPUS` (cmake option / Kconfig entry, default on) lives only in `src/decoder.h`, `src/decoder.cpp`, `src/player_role.cpp`, `src/source_role.cpp`, and `src/source_task.cpp`, plus the `SENDSPIN_SOURCE_OPUS_SOURCES` list (`src/source_encoder_opus.cpp`) in `cmake/sources.cmake`. Examples and tests guard their own role and opus usage like any consumer. micro-flac is only linked when the player role is enabled; micro-opus only with `SENDSPIN_ENABLE_OPUS` and the player or source role enabled.
+Roles can be disabled at compile time via `SENDSPIN_ENABLE_*` cmake options (host build) or Kconfig entries (ESP-IDF build). When a role is disabled, its source files are not compiled and its `add_*()` declaration, accessor, and `unique_ptr` member are removed from `client.h`. Role `#ifdef` guards live in exactly two places in the library: `cmake/sources.cmake` (source lists) and `include/sendspin/client.h`, `src/client.cpp` and `src/client_dispatch.cpp` (dispatch points); the codec gate `SENDSPIN_ENABLE_OPUS` (cmake option / Kconfig entry, default on) lives only in `src/decoder.h`, `src/decoder.cpp`, `src/player_role.cpp`, `src/source_role.cpp`, and `src/source_task.cpp`, plus the `SENDSPIN_SOURCE_OPUS_SOURCES` list (`src/source_encoder_opus.cpp`) in `cmake/sources.cmake`. The host-only message trace gate `SENDSPIN_ENABLE_MESSAGE_TRACE` (cmake option, default off, no Kconfig entry) lives only in `include/sendspin/config.h`, `src/connection.h`, `src/connection.cpp`, `src/connection_manager.cpp`, `src/noise_transport.h`, `src/noise_transport.cpp`, and the `SENDSPIN_MESSAGE_TRACE_SOURCES` list (`src/message_trace.cpp`). Examples and tests guard their own role and opus usage like any consumer. micro-flac is only linked when the player role is enabled; micro-opus only with `SENDSPIN_ENABLE_OPUS` and the player or source role enabled.
 
 The consuming platform (e.g., ESPHome) supplies the listener implementations plus `SendspinNetworkProvider` and the optional `SendspinPersistenceProvider`/`SendspinClientListener` providers; `docs/integration-guide.md` has the full wiring, including a minimal working example.
 
@@ -95,6 +96,7 @@ Core source files in `src/` have no `#ifdef ESP_PLATFORM` guards; all platform d
 - **Host (CMake)**: `cmake -B build && cmake --build build`. Uses ccache when installed (`SENDSPIN_USE_CCACHE`, default on when standalone). Fetches dependencies via FetchContent: ArduinoJson, noise-c, IXWebSocket, micro-flac for the player role, and micro-opus for the player or source role when `SENDSPIN_ENABLE_OPUS` is on.
 - **Tests**: `cmake -B build-tests -DSENDSPIN_BUILD_TESTS=ON -DENABLE_SANITIZERS=ON -DBUILD_EXAMPLES=OFF .`, then `cmake --build build-tests --target sendspin_tests` and `ctest --test-dir build-tests --output-on-failure`.
 - **ThreadSanitizer tests**: `cmake -B build-tsan -DSENDSPIN_BUILD_TESTS=ON -DENABLE_TSAN=ON -DBUILD_EXAMPLES=OFF .`, then `cmake --build build-tsan --target sendspin_tests` and `TSAN_OPTIONS=halt_on_error=1 ctest --test-dir build-tsan --output-on-failure`. `ENABLE_TSAN` applies the thread sanitizer to every target, including the fetched dependencies, and cannot be combined with `ENABLE_SANITIZERS`.
+- **Message trace**: `-DSENDSPIN_ENABLE_MESSAGE_TRACE=ON` compiles in `SendspinClientConfig::message_trace` and `tests/test_message_trace.cpp`; CI's test and TSan jobs build it on, the build and no-opus jobs off.
 - **Stack budgets**: the ESP task stack defaults in `config.h` are static call-graph bounds from `tools/stack_usage/` (recipe and per-task figures in its README); re-derive them, and update the README table with them, after a change that deepens a task's call chain.
 - **ESP dependencies**: ArduinoJson, noise-c, esp_websocket_client, micro-flac, micro-opus (with `SENDSPIN_ENABLE_OPUS`), esp_http_server, mbedtls, pthread, esp_ringbuf, esp_hw_support
 - **Host dependencies**: ArduinoJson, noise-c, micro-flac, micro-opus (with `SENDSPIN_ENABLE_OPUS`), IXWebSocket, pthreads

CMakeLists.txt

@@ -161,6 +161,10 @@ else()
     # patent-encumbered; see the player and source role specs.
     option(SENDSPIN_ENABLE_OPUS "Enable Opus in the player and source roles" ON)
 
+    # Plaintext trace of every protocol message (SendspinClientConfig::message_trace), for
+    # conformance tooling. Host only: the ESP-IDF build has no switch for it.
+    option(SENDSPIN_ENABLE_MESSAGE_TRACE "Enable the plaintext protocol message trace" OFF)
+
     include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/host.cmake)
 
     # Get sources
@@ -193,6 +197,9 @@ else()
             list(APPEND SENDSPIN_ALL_SOURCES ${SENDSPIN_SOURCE_OPUS_SOURCES})
         endif()
     endif()
+    if(SENDSPIN_ENABLE_MESSAGE_TRACE)
+        list(APPEND SENDSPIN_ALL_SOURCES ${SENDSPIN_MESSAGE_TRACE_SOURCES})
+    endif()
 
     # Create the library: core sources, enabled role sources, host networking (via host.cmake)
     add_library(sendspin STATIC ${SENDSPIN_ALL_SOURCES})
@@ -222,6 +229,9 @@ else()
     if(SENDSPIN_ENABLE_SOURCE)
         target_compile_definitions(sendspin PUBLIC SENDSPIN_ENABLE_SOURCE)
     endif()
+    if(SENDSPIN_ENABLE_MESSAGE_TRACE)
+        target_compile_definitions(sendspin PUBLIC SENDSPIN_ENABLE_MESSAGE_TRACE)
+    endif()
 
     # Host build options
     option(ENABLE_WERROR "Treat warnings as errors" OFF)

cmake/sources.cmake

@@ -107,6 +107,13 @@ function(sendspin_get_sources BASE_DIR)
         PARENT_SCOPE
     )
 
+    # The plaintext message trace, appended on host when SENDSPIN_ENABLE_MESSAGE_TRACE is on
+    set(SENDSPIN_MESSAGE_TRACE_SOURCES
+        ${BASE_DIR}/src/message_trace.cpp
+
+        PARENT_SCOPE
+    )
+
     # ESP-IDF only sources: networking layer deeply coupled to ESP-IDF APIs
     set(SENDSPIN_ESP_SOURCES
         ${BASE_DIR}/src/esp/server_connection.cpp

docs/conventions.md

@@ -36,12 +36,13 @@ checklists in `.claude/skills/` apply these standards to a diff.
   provider poll, nothing else (while the client is stopped it only frees the
   outbound attempts `stop()` left parked): do not add mutex-protected endpoints
   or atomics that `loop()` polls for work. The protocol task calls no
-  listener, a role thread calls only the data-path callbacks
-  `docs/integration-guide.md` names for it (`on_audio_write()` on the sync
-  task, `on_image_decode()` on the artwork decode thread, the visualizer data
-  callbacks on its drain thread), and only the main loop calls the persistence
-  and network providers: everything else a consumer hears it hears from the
-  main loop's drain. A thread that must wait for the main loop to
+  listener (its one consumer callback is the host-only
+  `SendspinClientConfig::message_trace`), a role thread calls only the
+  data-path callbacks `docs/integration-guide.md` names for it
+  (`on_audio_write()` on the sync task, `on_image_decode()` on the artwork
+  decode thread, the visualizer data callbacks on its drain thread), and only
+  the main loop calls the persistence and network providers: everything else a
+  consumer hears it hears from the main loop's drain. A thread that must wait for the main loop to
   have called a listener waits for a grant the drain publishes (the
   high-performance grant), never for the main loop itself.
 - The Inbox event ring is for ordered lifecycle events only (stream start and
@@ -148,9 +149,14 @@ checklists in `.claude/skills/` apply these standards to a diff.
   and the codec gate
   `SENDSPIN_ENABLE_OPUS` only in `src/decoder.h`, `src/decoder.cpp`,
   `src/player_role.cpp`, `src/source_role.cpp`, `src/source_task.cpp`, and
-  the source's Opus encoder list in `cmake/sources.cmake`; consumers,
-  including the examples, guard their own role and codec usage (see Public
-  API).
+  the source's Opus encoder list in `cmake/sources.cmake`. The host-only
+  message trace gate `SENDSPIN_ENABLE_MESSAGE_TRACE` lives only in
+  `include/sendspin/config.h`, `src/connection.h`, `src/connection.cpp`,
+  `src/connection_manager.cpp`, `src/noise_transport.h`,
+  `src/noise_transport.cpp`, and its own source list in
+  `cmake/sources.cmake` (`src/message_trace.cpp`); it has no Kconfig entry.
+  Consumers, including the examples, guard their own role and codec usage
+  (see Public API).
 - Logging uses the `SS_LOG*` macros; allocation uses the `platform_malloc`
   family with an explicit `MemoryLocation` choice where it matters.
 - Code that only builds on one platform still keeps the other platform's build

docs/integration-guide.md

@@ -1185,6 +1185,7 @@ Most listener callbacks fire on the main loop thread (the thread calling `client
 | `ArtworkRoleListener::on_image_decode()` | Dedicated artwork decode thread |
 | `ArtworkRoleListener::on_image_display()` | Main loop thread |
 | `VisualizerRoleListener` data callbacks (`on_loudness()`, `on_beat()`, `on_f_peak()`, `on_spectrum()`, `on_peak()`) | Dedicated visualizer drain thread |
+| `SendspinClientConfig::message_trace` (built with `SENDSPIN_ENABLE_MESSAGE_TRACE`) | Protocol task |
 | All other listener methods | Main loop thread |
 
 `PlayerRole::notify_audio_played()` is thread-safe and is designed to be called from an audio output callback thread.
@@ -1310,6 +1311,16 @@ Available options (all `ON` by default):
 
 When `SENDSPIN_ENABLE_PLAYER` is `OFF`, micro-flac is not fetched, and micro-opus is fetched only for the source role. `SENDSPIN_ENABLE_OPUS=OFF` drops micro-opus alone, for products that cannot ship Opus (see the patent note in the player and source role specs); the player then refuses `OPUS` entries in `audio_formats`, and the source refuses an `OPUS` codec in its config.
 
+### Message trace (host only)
+
+`SENDSPIN_ENABLE_MESSAGE_TRACE` (`OFF` by default) adds `SendspinClientConfig::message_trace`, a callback that receives every protocol message's plaintext (inbound after decryption, outbound before encryption, plus the cleartext text frames), for conformance tooling and interop debugging. The ESP-IDF build has no switch for it. The callback runs on the protocol task and must not block. The pairing secrets a client sends read `"<redacted>"`; `SendspinTracedMessage` in `config.h` documents the fields.
+
+```cpp
+config.message_trace = [](const SendspinTracedMessage& message) {
+    // message.plaintext is valid only during this call: copy what you keep
+};
+```
+
 ### ESP-IDF (Kconfig)
 
 Role flags are exposed via Kconfig under `Component config → sendspin-cpp`:
@@ -1375,6 +1386,7 @@ X25519 keypair and read back via `client.client_id()` after `start()`.
 | `pairing_psk_locations` | `std::vector<std::string>` | `{}` | Where the operator can find the pairing token the device shipped with: any of `"device"`, `"leaflet"`, `"operator"`. Advertised as the informational `locations` hint on the `pairing_psk` descriptor in `client/hello`; empty omits the hint, see [The locations hint](#the-locations-hint). |
 | `static_pairing_code_locations` | `std::vector<std::string>` | `{}` | Where the operator can find the static pairing code the device shipped with, same values as above. Advertised on the `static_pairing_code` descriptor in `client/hello`; empty omits the hint. |
 | `json_arena_size` | `size_t` | `2048` | Size in bytes of a fixed internal-RAM scratch buffer that backs every JSON document the protocol task works with: the parse of each incoming protocol message and every message it builds, instead of PSRAM. Costs this many bytes of internal RAM permanently but removes PSRAM traffic from the protocol task on every message. The arena holds one document at a time: a parsed message is released before the reply it triggers (the `client/state` after a `server/activate`, a pairing reply, a re-handshake's second message) is built. The default covers one steady-state protocol message, including the FLAC stream-start header; a document larger than the budget on its own, such as a large track-metadata message, falls back to PSRAM (those arrive only once per song). That holds on a 32-bit target, where ArduinoJson allocates each variant pool as one 1,024-byte block. Every freed block is wiped, so a document that held key material leaves none behind. Set to `0` to send every document to PSRAM. On host there is no PSRAM distinction, so the arena is just a fixed scratch buffer, and since a variant pool alone is 4 KB there, the default fits no document's pool on host. |
+| `message_trace` | `SendspinMessageTraceFn` | empty | Only in a build with `SENDSPIN_ENABLE_MESSAGE_TRACE` (host only): called with every protocol message's plaintext, on the protocol task. See [Message trace (host only)](#message-trace-host-only). |
 
 ---