Add a loopback input on macOS through a process tap

CMakeLists.txt

@@ -7,7 +7,8 @@ project(sendspin-cpp-cli
     LANGUAGES CXX
 )
 
-# The CoreAudio capture source asks for microphone access through AVFoundation, which is Objective-C.
+# The CoreAudio capture source asks for the microphone through AVFoundation and taps output
+# through CATapDescription, both Objective-C.
 if(APPLE)
     enable_language(OBJCXX)
 endif()
@@ -301,9 +302,10 @@ if(SENDSPIN_CLI_COREAUDIO_ENABLED)
         src/coreaudio_permission.mm
         src/coreaudio_sink.cpp
         src/coreaudio_source.cpp
+        src/coreaudio_tap.mm
     )
     # -Wno-error: blocks and Apple's headers are outside what -Wpedantic was chosen for.
-    set_source_files_properties(src/coreaudio_permission.mm PROPERTIES
+    set_source_files_properties(src/coreaudio_permission.mm src/coreaudio_tap.mm PROPERTIES
         COMPILE_OPTIONS "-fobjc-arc;-Wno-error")
     target_link_libraries(sendspin-cli-core PUBLIC
         "-framework CoreAudio" "-framework AudioToolbox" "-framework CoreFoundation"

README.md

@@ -71,7 +71,7 @@ sendspin-cli -n den --input default
 | `-n, --name <name>` | The friendly name a server displays. Defaults to this host's name. |
 | `-o, --output <device>` | Which sound card to play through. `-l` lists this host's devices and what they accept. |
 | `--audio-format <codec:rate:depth:channels>[,...]` | Formats to offer first, in priority order, e.g. `flac:48000:24:2,pcm:48000:24:2`. Everything else the player normally offers still follows, and a server uses the first format it can encode, so it can still fall back to a later one. This sets a preference, not a restriction. |
-| `--input <device>` | Audio to stream to the server: a microphone or line-in, e.g. `default`, `hw:1,0`, `pulse`, `pipewire:<node>` or `portaudio`, or `coreaudio` on macOS; or `loopback` for what this host plays. `-l` lists them. Off unless given. See [Stream a microphone or line-in](https://github.com/Sendspin/sendspin-cpp-cli/wiki/Advanced-Usage#stream-a-microphone-or-line-in) and [Stream what this host plays](https://github.com/Sendspin/sendspin-cpp-cli/wiki/Advanced-Usage#stream-what-this-host-plays). |
+| `--input <device>` | Audio to stream to the server: a microphone or line-in, e.g. `default`, `hw:1,0`, `pulse`, `pipewire:<node>` or `portaudio`, or `coreaudio` on macOS; or `loopback` for what this host plays (on macOS without the player's own playback, on Linux with it). `-l` lists them. Off unless given. See [Stream a microphone or line-in](https://github.com/Sendspin/sendspin-cpp-cli/wiki/Advanced-Usage#stream-a-microphone-or-line-in) and [Stream what this host plays](https://github.com/Sendspin/sendspin-cpp-cli/wiki/Advanced-Usage#stream-what-this-host-plays). |
 | `--allow-unpaired` | Let a server play without pairing with this player first. Off by default. |
 | `--pairing-code <8 digits>` | A fixed code a server can pair with, confirmed with `sendspin-cli pair confirm`. Without it the player shows a fresh 6-digit code per attempt. |
 | `-s, --server <host[:port]>` | Connect out to a server rather than waiting to be discovered, or `mdns:[<name>]` to discover one over mDNS. Turns off the mDNS advertisement. |

docs/wiki/Advanced-Usage.md

@@ -140,6 +140,28 @@ so the server gets its own audio back. The player logs a warning at startup unle
 names a different PulseAudio or PipeWire sink, or is `null` or `stdout`. Loop back a sink the player
 does not play to.
 
+On macOS 14.2 or later, `loopback` captures everything the Mac plays through a CoreAudio
+process tap, and `loopback:<index|name>` captures one output device, numbered and named as
+`-o coreaudio:` names it. Both leave out sendspin-cli's own playback, so a player on the
+same Mac is never sent back to the server.
+
+```bash
+sendspin-cli -n den --input loopback
+sendspin-cli -n den --input "loopback:MacBook Pro Speakers"
+```
+
+A virtual loopback device installed by another app is an ordinary input:
+`--input "coreaudio:<device>"`. If `-o` plays into that same device, or into a Multi-Output
+Device that includes it, the player logs a warning at startup, since the server would get its own audio back.
+
+macOS asks once for **System Audio Recording**. It names the app sendspin-cli is started
+from: the terminal, or `sendspin-cli` itself under `launchd`. If the answer is no, macOS
+gives the tap silence rather than an error, so the player keeps running, and logs a
+warning after ten seconds of nothing but silence. Allow it under **System Settings →
+Privacy & Security → Screen & System Audio Recording → System Audio Recording Only**.
+macOS does not ask again by itself after a refusal; `tccutil reset AudioCapture` makes it
+ask on the next start.
+
 ### Microphone permission on macOS
 
 macOS asks for microphone access the first time `--input` opens a CoreAudio or PortAudio

docs/wiki/Configuration.md

@@ -53,7 +53,7 @@ already does that.
 | `product-name` | `--product-name` | the product name `client/hello` reports | `sendspin-cli` |
 | `server` | `-s`, `--server` | `<host>[:<port>]`, a `ws://` URL, or `mdns:[<name>]` | none — wait to be discovered |
 | `port` | `--port` | the port this player's own WebSocket server listens on | `8928` |
-| `input` | `--input` | a capture device to stream to the server: `default`, `hw:1,0`, `alsa:<device>`, `pulse[:<source>]`, `pipewire[:<node>]`, `coreaudio[:<device>]` on macOS, `portaudio[:<device>]`, `loopback[:<sink>]`, `tone`, `null`; see [Stream a microphone or line-in](Advanced-Usage#stream-a-microphone-or-line-in) and [Stream what this host plays](Advanced-Usage#stream-what-this-host-plays) | none — nothing is captured |
+| `input` | `--input` | a capture device to stream to the server: `default`, `hw:1,0`, `alsa:<device>`, `pulse[:<source>]`, `pipewire[:<node>]`, `coreaudio[:<device>]` on macOS, `portaudio[:<device>]`, `loopback[:<sink>]` (on macOS `loopback[:<device>]`), `tone`, `null`; see [Stream a microphone or line-in](Advanced-Usage#stream-a-microphone-or-line-in) and [Stream what this host plays](Advanced-Usage#stream-what-this-host-plays) | none — nothing is captured |
 | `buffer-ms` | `--buffer-ms` | audio the output backend keeps queued, 10–2000 | `100` |
 | `audio-format` | `--audio-format` | preferred formats, comma-separated in priority order: `codec:rate:depth:channels[,...]`, e.g. `flac:48000:24:2,pcm:48000:24:2`; offered first in that order, with the rest of the advertised list still behind them, so a server that cannot encode them falls back — a preference, not a restriction; refuses to start if the advertised list does not carry every one — it carries a single channel count — and an `opus` entry at anything but 48000/16 and at most 2 channels is refused outright | none — device-derived order |
 | `output-delay` | `--output-delay` | latency this endpoint's hardware adds after the audio port, 0–5000. `static-delay` / `--static-delay` is still accepted as the old name | `0` |

packaging/sendspin-cli.Info.plist.in

@@ -14,5 +14,7 @@
     <string>@PROJECT_VERSION@</string>
     <key>NSMicrophoneUsageDescription</key>
     <string>sendspin-cli streams this microphone or line-in to your Sendspin server when run with --input.</string>
+    <key>NSAudioCaptureUsageDescription</key>
+    <string>sendspin-cli streams what this Mac plays to your Sendspin server when run with --input loopback.</string>
 </dict>
 </plist>

src/audio_source.cpp

@@ -28,6 +28,7 @@
 #include "pulse_source.h"
 #endif
 #ifdef SENDSPIN_CLI_HAVE_COREAUDIO
+#include "coreaudio_device.h"
 #include "coreaudio_source.h"
 #endif
 #ifdef SENDSPIN_CLI_HAVE_PORTAUDIO
@@ -66,10 +67,14 @@ constexpr bool HAVE_PIPEWIRE = true;
 #else
 constexpr bool HAVE_PIPEWIRE = false;
 #endif
-constexpr bool HAVE_LOOPBACK = HAVE_PULSE || HAVE_PIPEWIRE;
+#ifdef SENDSPIN_CLI_HAVE_LOOPBACK
+constexpr bool HAVE_LOOPBACK = true;
+#else
+constexpr bool HAVE_LOOPBACK = false;
+#endif
 
-#if defined(SENDSPIN_CLI_HAVE_PULSE) || defined(SENDSPIN_CLI_HAVE_PIPEWIRE)
-/// The loopback source on the first sound server that answers: PipeWire, then PulseAudio.
+#ifdef SENDSPIN_CLI_HAVE_LOOPBACK
+/// The loopback source on the first backend that can capture: PipeWire, PulseAudio, CoreAudio.
 std::unique_ptr<AudioSource> make_loopback_source(const std::string& device, std::string& error) {
     std::string tried;
 #ifdef SENDSPIN_CLI_HAVE_PIPEWIRE
@@ -86,7 +91,14 @@ std::unique_ptr<AudioSource> make_loopback_source(const std::string& device, std
     }
     tried += (tried.empty() ? "" : "; ") + ("PulseAudio: " + pulse_error);
 #endif
-    error = "--input loopback: no sound server answered (" + tried + ")";
+#ifdef SENDSPIN_CLI_HAVE_COREAUDIO
+    std::string coreaudio_error;
+    if (CoreAudioTap::supported(coreaudio_error)) {
+        return std::make_unique<CoreAudioSource>(device, true);
+    }
+    tried += (tried.empty() ? "" : "; ") + ("CoreAudio: " + coreaudio_error);
+#endif
+    error = "--input loopback found nothing to capture through (" + tried + ")";
     return nullptr;
 }
 #endif
@@ -308,8 +320,8 @@ bool resolve_input_spec(const std::string& spec, InputSpec& out, std::string& er
 
     if (prefix == "loopback") {
         if (!HAVE_LOOPBACK) {
-            error = "--input loopback needs the PipeWire or PulseAudio backend, and this build has "
-                    "neither, so --input takes only: " +
+            error = "--input loopback needs the PipeWire, PulseAudio or CoreAudio backend, and "
+                    "this build has none of them, so --input takes only: " +
                     input_backend_list();
             return false;
         }
@@ -377,6 +389,48 @@ bool may_capture_own_playback(const DeviceSpec& output, const std::string& loope
     return true;
 }
 
+#ifdef SENDSPIN_CLI_HAVE_COREAUDIO
+bool coreaudio_input_hears_output(const std::string& input, const std::string& output) {
+    InputSpec captured_spec;
+    DeviceSpec played_spec;
+    std::string error;
+    if (!resolve_input_spec(input, captured_spec, error) ||
+        captured_spec.backend != SourceBackend::CoreAudio ||
+        !resolve_device_spec(output, played_spec, error) ||
+        played_spec.backend != SinkBackend::CoreAudio) {
+        return false;
+    }
+    AudioDeviceID captured = kAudioObjectUnknown;
+    AudioDeviceID played = kAudioObjectUnknown;
+    if (!resolve_ca_device(captured_spec.device, CaDirection::Input, captured, error) ||
+        !resolve_ca_device(played_spec.device, CaDirection::Output, played, error)) {
+        return false;
+    }
+    // A hardware device's input is a microphone or line-in, not what the device plays.
+    if (ca_u32_property(captured, kAudioDevicePropertyTransportType,
+                        kAudioObjectPropertyScopeGlobal) != kAudioDeviceTransportTypeVirtual) {
+        return false;
+    }
+    if (captured == played) {
+        return true;
+    }
+    // A Multi-Output Device plays into each of its sub-devices.
+    const AudioObjectPropertyAddress address =
+        ca_address_of(kAudioAggregateDevicePropertyActiveSubDeviceList);
+    UInt32 size = 0;
+    if (AudioObjectGetPropertyDataSize(played, &address, 0, nullptr, &size) != noErr) {
+        return false;
+    }
+    std::vector<AudioObjectID> subdevices(size / sizeof(AudioObjectID));
+    if (subdevices.empty() || AudioObjectGetPropertyData(played, &address, 0, nullptr, &size,
+                                                         subdevices.data()) != noErr) {
+        return false;
+    }
+    subdevices.resize(size / sizeof(AudioObjectID));
+    return std::find(subdevices.begin(), subdevices.end(), captured) != subdevices.end();
+}
+#endif
+
 std::unique_ptr<AudioSource> make_audio_source(const std::string& spec, StreamFormat& format,
                                                std::string& error) {
     InputSpec resolved;
@@ -413,7 +467,7 @@ std::unique_ptr<AudioSource> make_audio_source(const std::string& spec, StreamFo
 #endif
             break;
         case SourceBackend::Loopback:
-#if defined(SENDSPIN_CLI_HAVE_PULSE) || defined(SENDSPIN_CLI_HAVE_PIPEWIRE)
+#ifdef SENDSPIN_CLI_HAVE_LOOPBACK
             source = make_loopback_source(resolved.device, error);
             if (!source) {
                 return nullptr;
@@ -520,6 +574,16 @@ void print_capture_devices(std::FILE* out) {
                  "until sendspin-cli restarts; while --input portaudio runs, that goes for\n"
                  "-o portaudio's device too.\n");
 #endif
+#ifdef SENDSPIN_CLI_HAVE_COREAUDIO
+    std::fprintf(out,
+                 "\nLoopback (--input loopback[:<index|name>]):\n"
+                 "--input loopback captures everything this Mac plays, and --input\n"
+                 "loopback:<index|name> one output device, through a process tap (macOS 14.2 or\n"
+                 "later). sendspin-cli's own playback is left out. macOS asks once for System\n"
+                 "Audio Recording; refused, the capture is silent.\n"
+                 "\nCoreAudio output devices on this host:\n");
+    CoreAudioSource::list_loopback_devices(out);
+#endif
 #if !defined(SENDSPIN_CLI_HAVE_ALSA) && !defined(SENDSPIN_CLI_HAVE_PULSE) &&         \
     !defined(SENDSPIN_CLI_HAVE_PIPEWIRE) && !defined(SENDSPIN_CLI_HAVE_COREAUDIO) && \
     !defined(SENDSPIN_CLI_HAVE_PORTAUDIO)

src/audio_source.h

@@ -32,6 +32,12 @@
 #include <string>
 #include <thread>
 
+#if defined(SENDSPIN_CLI_HAVE_PULSE) || defined(SENDSPIN_CLI_HAVE_PIPEWIRE) || \
+    defined(SENDSPIN_CLI_HAVE_COREAUDIO)
+/// Some backend in this build can capture what the host plays.
+#define SENDSPIN_CLI_HAVE_LOOPBACK
+#endif
+
 namespace sendspin_cli {
 
 /// Bit depths the source role takes, in the order a capture device is asked for them.
@@ -150,6 +156,12 @@ bool input_pcm_is_reachable(const std::string& pcm);
 /// True unless `output` provably plays somewhere other than `looped_sink` (empty: the default).
 bool may_capture_own_playback(const DeviceSpec& output, const std::string& looped_sink);
 
+#ifdef SENDSPIN_CLI_HAVE_COREAUDIO
+/// True if --input `input` captures a virtual CoreAudio device, such as a loopback driver, that
+/// -o `output` also plays to, directly or as part of a Multi-Output Device.
+bool coreaudio_input_hears_output(const std::string& input, const std::string& output);
+#endif
+
 /// Settles `format` for a sound server, which converts: only a format no stream can carry changes.
 void settle_server_capture_format(StreamFormat& format);
 

src/cli.cpp

@@ -952,7 +952,7 @@ void print_usage(std::FILE* out, const char* prog) {
     std::fprintf(out, "                Capture device to stream to the server: a microphone or\n");
     std::fprintf(out, "                line-in. Read the way -o is, with <backend> one of: %s\n",
                  input_backend_list().c_str());
-#if defined(SENDSPIN_CLI_HAVE_PULSE) || defined(SENDSPIN_CLI_HAVE_PIPEWIRE)
+#ifdef SENDSPIN_CLI_HAVE_LOOPBACK
     std::fprintf(out, "                --input loopback streams what this host plays, and\n");
     std::fprintf(out, "                --input loopback:<sink> what one sink plays\n");
 #endif

src/coreaudio_device.cpp

@@ -204,7 +204,7 @@ std::vector<AudioDeviceID> ca_enumerate_devices(CaDirection direction) {
 }
 
 bool resolve_ca_device(const std::string& device, CaDirection direction, AudioDeviceID& out,
-                       std::string& error) {
+                       std::string& error, const std::string& typed) {
     const std::string noun = noun_of(direction);
     if (device.empty()) {
         const AudioDeviceID fallback = ca_default_device(direction);
@@ -223,7 +223,8 @@ bool resolve_ca_device(const std::string& device, CaDirection direction, AudioDe
         return false;
     }
 
-    const std::string spec = std::string(flag_of(direction)) + " coreaudio:" + device;
+    const std::string spec =
+        typed.empty() ? std::string(flag_of(direction)) + " coreaudio:" + device : typed;
     if (is_device_index(device)) {
         // strtoull saturates rather than wrapping into a plausible index.
         const unsigned long long value = std::strtoull(device.c_str(), nullptr, 10);

src/coreaudio_device.h

@@ -59,8 +59,10 @@ AudioDeviceID ca_default_device(CaDirection direction);
 std::vector<AudioDeviceID> ca_enumerate_devices(CaDirection direction);
 
 /// Resolves a CoreAudio device spec: empty is the default, digits an index, else a unique name.
+/// @param typed The spec as the user wrote it, for error text; empty for `<flag>
+/// coreaudio:<device>`.
 bool resolve_ca_device(const std::string& device, CaDirection direction, AudioDeviceID& out,
-                       std::string& error);
+                       std::string& error, const std::string& typed = {});
 
 /// Maps a stream format onto the interleaved signed little-endian PCM an AUHAL unit takes.
 bool ca_asbd_for(uint32_t sample_rate, uint8_t channels, uint8_t bits_per_sample,