Wipe the session's plaintext buffers when they grow and when it ends

CLAUDE.md

@@ -95,7 +95,7 @@ script/clang-tidy.sh      - clang-tidy over src/ using build/compile_commands.js
 - Library target `sendspin_server_cpp` (alias `sendspin_server_cpp::sendspin_server_cpp`); CMake options are prefixed `SENDSPIN_SERVER_`. `SENDSPIN_SERVER_ENABLE_WERROR` (default off) makes warnings in this project's targets errors; CI builds with it on, so local builds should too. `SENDSPIN_SERVER_EMBEDDED_PROFILE` (default off) builds with `src/profile.h`'s embedded limits, as the ESP-IDF component (`cmake/esp_idf.cmake`) always does.
 - Plain: `cmake -B build && cmake --build build`, which builds `examples/` too (`SENDSPIN_SERVER_BUILD_EXAMPLES`, default on when top-level). Uses ccache when installed (`SENDSPIN_SERVER_USE_CCACHE`, default on when top-level). `SENDSPIN_SERVER_ENABLE_DISCOVERY` (default on) builds the mDNS backend: dns_sd on macOS, avahi-client through pkg-config on Linux (`libavahi-client-dev`). `SENDSPIN_SERVER_ENABLE_OPUS` (default on) builds the Opus encoder and the source role's Opus decoder; off compiles them out (`src/frame_encoder.cpp` and `src/source_decoder.cpp` are the places it is guarded) and is refused with the tests on.
 - Tests never use the network's mDNS: `tests/test_recorder.h` `local_config()` turns advertising and browsing off and listens on a port the system picks, and `test_server_discovery.cpp` supplies its own `DiscoveryProvider` through `ServerConfig::discovery`.
-- Tests: `cmake -B build-asan -DSENDSPIN_SERVER_ENABLE_WERROR=ON -DSENDSPIN_SERVER_BUILD_TESTS=ON -DSENDSPIN_SERVER_ENABLE_SANITIZERS=ON`, then `cmake --build build-asan` and `ctest --test-dir build-asan --output-on-failure`. The suites whose assertions depend on players keeping up with real time carry the CTest label `realtime` (`tests/CMakeLists.txt`); CI runs them on their own and the rest in parallel (`-LE realtime -j`). The tests also build `server_cli`'s WAV and FLAC readers, feeders, `--one-group`, timing figures and log and `--web` dashboard; `tests/temp_dir.h` gives a test a temporary directory, and `tests/job_view.h` a copy of what a group's pipelines read of a host view. `tests/test_ports.h` hands out loopback ports for test peers, and holds one that nothing listens on for a dial that must be refused; `tests/raw_client.h` is the raw test client (noise-c and Beast, either suite, scriptable misbehavior), `tests/silent_listener.h` a listener that holds the server's dials in their upgrade or handshake, `tests/test_recorder.h` a recording listener, an in-memory provider and `LocalServer`, a server started with them, `tests/wire_messages.h` the spec's binary message types and fields as the tests decode them, `tests/flac_reader.h` decodes a FLAC stream as a player gets it, `tests/opus_reader.h` an Opus stream, with the test tones and an SNR to compare lossy audio by, `tests/player_sink.h` holds the test signal and a real-time audio output for sendspin-cpp players, `tests/peer.h` what each in-process sendspin-cpp client shares (its network provider and blob store, its config and the URL the server dials, its main loop run on the test thread, the client_id at a URL), and `tests/trust_fixtures.h` distinct PSKs and the client record a server saved.
+- Tests: `cmake -B build-asan -DSENDSPIN_SERVER_ENABLE_WERROR=ON -DSENDSPIN_SERVER_BUILD_TESTS=ON -DSENDSPIN_SERVER_ENABLE_SANITIZERS=ON`, then `cmake --build build-asan` and `ctest --test-dir build-asan --output-on-failure`. The suites whose assertions depend on players keeping up with real time carry the CTest label `realtime` (`tests/CMakeLists.txt`); CI runs them on their own and the rest in parallel (`-LE realtime -j`). The tests also build `server_cli`'s WAV and FLAC readers, feeders, `--one-group`, timing figures and log and `--web` dashboard; `tests/temp_dir.h` gives a test a temporary directory, and `tests/job_view.h` a copy of what a group's pipelines read of a host view. `tests/test_ports.h` hands out loopback ports for test peers, and holds one that nothing listens on for a dial that must be refused; `tests/raw_client.h` is the raw test client (noise-c and Beast, either suite, scriptable misbehavior), `tests/silent_listener.h` a listener that holds the server's dials in their upgrade or handshake, `tests/test_recorder.h` a recording listener, an in-memory provider and `LocalServer`, a server started with them, `tests/wire_messages.h` the spec's binary message types and fields as the tests decode them, `tests/flac_reader.h` decodes a FLAC stream as a player gets it, `tests/opus_reader.h` an Opus stream, with the test tones and an SNR to compare lossy audio by, `tests/player_sink.h` holds the test signal and a real-time audio output for sendspin-cpp players, `tests/peer.h` what each in-process sendspin-cpp client shares (its network provider and blob store, its config and the URL the server dials, its main loop run on the test thread, the client_id at a URL), `tests/trust_fixtures.h` distinct PSKs and the client record a server saved, and `tests/freed_blocks.h` counts the heap blocks freed still holding a secret, through the sanitizer's free hook (a test that needs it skips itself without ASan or TSan).
 - Soak: `tests/soak.cpp` (its trace checker in `tests/soak_wire_checker.h`, tested in the default suite), target `sendspin_server_cpp_soak` with its own entry point, registered as the CTest test `Soak` (label `soak`), which skips itself unless `SENDSPIN_SERVER_SOAK_SECONDS` is set (`SENDSPIN_SERVER_SOAK_SEED` picks the script); README.md has the command.
 - CI also builds the library with Opus and discovery off and memory statistics on; as a subproject (`tests/consumer/`); and as the ESP-IDF component (`tests/esp_idf/`: `idf.py -C tests/esp_idf set-target esp32s3`, then `idf.py -C tests/esp_idf build`).
 - Memory statistics: `SENDSPIN_SERVER_MEMORY_STATS` (default off; `CONFIG_SENDSPIN_SERVER_MEMORY_STATS` in the component's `Kconfig`) logs the heaps and the tasks' stacks at the server's start (on the thread calling `start()`) and, on the core thread, at a connection's start and end and a group's playback (`profile::MEMORY_STATS`); the tests count log lines and expect it off.

docs/conventions.md

@@ -46,7 +46,7 @@ The design standards that code in this repository is held to. This document is n
 ## Secrets
 
 - **Held in wiping types, wiped after use.** A private key or PSK lives in a type that wipes it when destroyed (`Identity`, `HandshakePsk`, `ClientRecord`, the ephemeral key a `NoiseInitiator` keeps), and a buffer that held one (a blob loaded or encoded for the provider) is wiped with `secure_zero()` once used. A secret is never formatted into a string.
-- **Pairing secrets the same way.** A pairing code or token from the host, the CPace run and the values it unwraps are held in wiping types (`PairingAttempt`, `PairingPskToken`, the code's buffer on its way to core), and the buffers a pairing message was read from are wiped once it is parsed, whether or not it passed validation, spare capacity included, as is the reassembly buffer when a fragment sequence fails or the reassembler is destroyed. The message trace never sees one: `redact_pairing_secrets()` replaces each secret field a client sends before the plaintext is traced.
+- **Pairing secrets the same way.** A pairing code or token from the host, the CPace run and the values it unwraps are held in wiping types (`PairingAttempt`, `PairingPskToken`, the code's buffer on its way to core), and the buffers a pairing message was read from are wiped once it is parsed, whether or not it passed validation, spare capacity included, as are the decrypted message's buffer and the reassembly buffer when a message fails to decrypt, a fragment sequence fails, the session ends or either is destroyed; either grows through `secure_reserve()`, which wipes the block it leaves. The message trace never sees one: `redact_pairing_secrets()` replaces each secret field a client sends before the plaintext is traced.
 - **Persisted before use.** A credential the server will reference in later handshakes (a long-term PSK) is referenced only once the provider reports it durable; until then the record as it was applies (`ServerCore::store_long_term_psk()`).
 - **Removed at once, written until durable.** A removed long-term PSK or approval (an unpairing, a revocation) stops applying at once, and a record write no caller waits on that the provider fails is written again until it lands or the server stops (`ServerCore::FailedWrites`), so a restart does not restore it once the provider recovers before the stop.
 

src/crypto/crypto.h

@@ -55,6 +55,20 @@ inline void secure_zero_storage(std::vector<uint8_t>& storage, size_t bytes) {
     storage.resize(size);
 }
 
+/// @brief Reserves room for `capacity` bytes in a byte vector, as reserve() does, but wipes the
+/// block it leaves first with secure_zero_storage(), as far as its size or `written`, whichever is
+/// further; the block freed holds none of the contents
+inline void secure_reserve(std::vector<uint8_t>& storage, size_t capacity, size_t written) {
+    if (capacity <= storage.capacity()) {
+        return;
+    }
+    std::vector<uint8_t> grown;
+    grown.reserve(capacity);
+    grown.assign(storage.begin(), storage.end());
+    secure_zero_storage(storage, std::max(written, storage.size()));
+    storage.swap(grown);
+}
+
 constexpr size_t SHA256_DIGEST_SIZE = 32;
 
 /// @brief Streaming SHA-256

src/fragmentation.cpp

@@ -94,6 +94,13 @@ Reassembler::Outcome Reassembler::accept(std::span<const uint8_t> plaintext) {
         if (this->buffer_.size() - 1 + data.size() > this->max_message_bytes_) {
             return this->fail(Outcome::TOO_LARGE);
         }
+        const size_t needed = this->buffer_.size() + data.size();
+        if (needed > this->buffer_.capacity()) {
+            // At least doubled, as insert() would, within the cap
+            const size_t doubled =
+                std::min(2 * this->buffer_.capacity(), this->max_message_bytes_ + 1);
+            secure_reserve(this->buffer_, std::max(needed, doubled), this->written_);
+        }
         this->buffer_.insert(this->buffer_.end(), data.begin(), data.end());
         this->written_ = std::max(this->written_, this->buffer_.size());
     }

src/session.cpp

@@ -57,6 +57,10 @@ Session::Session(const Identity& identity, const SessionSettings& settings, Sess
       queues_(settings.queue_caps),
       credentials_(credentials) {}
 
+Session::~Session() {
+    this->wipe_scratch();
+}
+
 void Session::start(int64_t now_us) {
     this->deadline_us_ = now_us + this->settings_.step_timeout_us;
 }
@@ -205,6 +209,9 @@ void Session::on_transport_message(MessageKind kind, std::span<const uint8_t> pa
         this->finish(SessionEnd::CLEARTEXT_IN_TRANSPORT);
         return;
     }
+    // Emptied first so a growth copies nothing it would overwrite
+    this->scratch_.clear();
+    secure_reserve(this->scratch_, payload.size(), this->scratch_written_);
     this->scratch_.assign(payload.begin(), payload.end());
     this->scratch_written_ = std::max(this->scratch_written_, this->scratch_.size());
     if (!this->ciphers_ || !this->ciphers_->decrypt(this->scratch_)) {
@@ -233,12 +240,9 @@ void Session::on_transport_message(MessageKind kind, std::span<const uint8_t> pa
             }
             return;
         case Reassembler::Outcome::MALFORMED:
-            // The reassembler wiped its buffer; the fragment here may carry part of a secret too
-            this->wipe_scratch();
             this->finish(SessionEnd::MALFORMED_FRAGMENTS);
             return;
         case Reassembler::Outcome::TOO_LARGE:
-            this->wipe_scratch();
             this->finish(SessionEnd::MESSAGE_TOO_LARGE);
             return;
     }
@@ -1000,6 +1004,10 @@ void Session::finish(SessionEnd end) {
     }
     this->end_ = end;
     this->phase_ = SessionPhase::CLOSED;
+    // A pairing message's plaintext may be left: a fragment pending, or a message AES-GCM decrypted
+    // in place before its tag check failed
+    this->wipe_scratch();
+    this->reassembler_.wipe();
     this->initiator_.reset();
     this->rekeys_.clear();
     // A re-handshake that completed has no frozen activation left to write

src/session.h

@@ -131,6 +131,12 @@ class Session {
     /// session
     Session(const Identity& identity, const SessionSettings& settings,
             SessionTrace* trace = nullptr, CredentialSource* credentials = nullptr);
+    /// @brief Wipes the decrypted message's buffer
+    ~Session();
+    Session(const Session&) = delete;
+    Session& operator=(const Session&) = delete;
+    Session(Session&&) = delete;
+    Session& operator=(Session&&) = delete;
 
     /// @brief Starts the AWAIT_CLIENT_INIT timeout once the transport is connected. Called once,
     /// before anything else.

tests/CMakeLists.txt

@@ -161,8 +161,9 @@ set_source_files_properties(test_group_audio.cpp PROPERTIES COMPILE_OPTIONS -fno
 # White-box: test_session.cpp queues a control message behind the initial server/activate through
 # Session::queue_json(), since the server queues no other control message there. It reads
 # Session::scratch_, the buffer_ of Session::reassembler_ and of a Reassembler of its own to their
-# capacity to show a pairing message's PSK, or a longer message's bytes, wiped from them, which
-# nothing the session sends or reports shows.
+# capacity to show a pairing message's PSK, or a longer message's bytes, wiped from them, and
+# reads the capacity of the first two to show a longer message grew them, which nothing the session
+# sends or reports shows.
 # test_server_core.cpp reads ServerCore::targets_ (a target's phase: ServerCore::RetryArmed, a
 # PendingDial) to show a dial waiting out its backoff, and a lost instance's target gone with its
 # wait, and a dial that waited not started once no longer due,

tests/freed_blocks.h

@@ -0,0 +1,125 @@
+// Copyright 2026 Sendspin Contributors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/// @file freed_blocks.h
+/// @brief FreedBlockScan, which counts the heap blocks the test thread frees while it is alive that
+/// still hold a secret, through the sanitizer's free hook; only in a build with ASan or TSan
+
+#pragma once
+
+#if defined(__SANITIZE_ADDRESS__) || defined(__SANITIZE_THREAD__)
+#define SENDSPIN_TEST_FREE_HOOKS 1
+#elif defined(__has_feature)
+#if __has_feature(address_sanitizer) || __has_feature(thread_sanitizer)
+#define SENDSPIN_TEST_FREE_HOOKS 1
+#endif
+#endif
+
+#include <pthread.h>
+
+#include <atomic>
+#include <cstddef>
+#include <cstdint>
+#include <string_view>
+
+#ifdef SENDSPIN_TEST_FREE_HOOKS
+#if __has_include(<sanitizer/allocator_interface.h>)
+#include <sanitizer/allocator_interface.h>
+#else
+// GCC's runtimes export these but install no header declaring them
+extern "C" {
+int __sanitizer_install_malloc_and_free_hooks(void (*malloc_hook)(const volatile void*, size_t),
+                                              void (*free_hook)(const volatile void*));
+size_t __sanitizer_get_allocated_size(const volatile void* pointer);
+}
+#endif
+#endif
+
+namespace freed_blocks {
+
+/// The secret the live scan looks for; nullptr while none is alive
+inline std::atomic<const std::string_view*> needle{nullptr};
+/// The thread whose frees the live scan counts
+inline std::atomic<pthread_t> scanning_thread{};
+inline std::atomic<int> holding{0};
+
+#ifdef SENDSPIN_TEST_FREE_HOOKS
+// Runs before the allocator takes the block back, so its bytes are still there. A container's
+// spare capacity may be poisoned, so the block is read only in this uninstrumented function.
+// Another thread's frees are not counted, and only the scanning thread ends the scan, so the
+// secret it views outlives every read of it.
+__attribute__((no_sanitize_address, no_sanitize_thread)) inline void on_free(
+    const volatile void* pointer) {
+    if (pointer == nullptr || pthread_equal(pthread_self(), scanning_thread.load()) == 0) {
+        return;
+    }
+    const std::string_view* secret = needle.load();
+    if (secret == nullptr) {
+        return;
+    }
+    const void* block = const_cast<const void*>(pointer);
+    const size_t size = __sanitizer_get_allocated_size(block);
+    const auto* data = static_cast<const uint8_t*>(block);
+    for (size_t start = 0; start + secret->size() <= size; ++start) {
+        size_t matched = 0;
+        while (matched < secret->size() &&
+               data[start + matched] == static_cast<uint8_t>((*secret)[matched])) {
+            ++matched;
+        }
+        if (matched == secret->size()) {
+            holding.fetch_add(1);
+            return;
+        }
+    }
+}
+
+inline void on_malloc(const volatile void* /*pointer*/, size_t /*size*/) {}
+#endif
+
+}  // namespace freed_blocks
+
+/// Counts the blocks the calling thread frees while it is alive that hold `secret`. One at a time.
+class FreedBlockScan {
+public:
+    /// @brief Whether the build has the hook; a test that needs it skips itself otherwise
+    static bool available() {
+#ifdef SENDSPIN_TEST_FREE_HOOKS
+        // The sanitizer takes a few hook pairs and never removes one: installed once per process
+        static const bool installed = __sanitizer_install_malloc_and_free_hooks(
+                                          &freed_blocks::on_malloc, &freed_blocks::on_free) != 0;
+        return installed;
+#else
+        return false;
+#endif
+    }
+
+    explicit FreedBlockScan(std::string_view secret) : secret_(secret) {
+        freed_blocks::holding.store(0);
+        freed_blocks::scanning_thread.store(pthread_self());
+        freed_blocks::needle.store(&this->secret_);
+    }
+    ~FreedBlockScan() {
+        freed_blocks::needle.store(nullptr);
+    }
+    FreedBlockScan(const FreedBlockScan&) = delete;
+    FreedBlockScan& operator=(const FreedBlockScan&) = delete;
+
+    /// @brief The blocks freed holding the secret so far
+    int holding() const {
+        return freed_blocks::holding.load();
+    }
+
+private:
+    std::string_view secret_;
+};