feat: wire MS-RDPEAI capture into Windows client and ActiveX

[Marc-André Moreau (mamoreau-devolutions)]

Add the client MS-RDPEAI capture path on top of hardened RDPSND playback: connector CFG + static channel wiring, CPAL PCM capture backend, ironrdp-client --audio-capture, and ActiveX AudioCaptureRedirectionMode.

PCM capture only accepts encode formats that match the Open capture stream, rejects non-16-bit capture (Data PDU size contract), and gates the capture backend behind ironrdp-rdpsnd-native/capture.

Depends on #1648 (playback).

[Copilot]

Pull request overview

Adds native audio playback hardening and end-to-end MS-RDPEAI microphone capture, including protocol, client, configuration, and ActiveX integration.

Changes:

• Stabilizes RDPSND negotiation and adds volume/Wave handling.
• Introduces the ironrdp-rdpeai protocol crate and CPAL capture backend.
• Wires audio capture through connector, client configuration, and ActiveX settings.

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
ffi/src/connector/config.rs	Defaults capture off.
crates/ironrdp/examples/screenshot.rs	Updates connector configuration.
crates/ironrdp-web/src/session.rs	Defaults web capture off.
crates/ironrdp-testsuite-extra/tests/e2e.rs	Updates test configuration.
crates/ironrdp-testsuite-extra/tests/client_config.rs	Tests audio-mode mapping.
crates/ironrdp-testsuite-core/tests/session/connection_activation.rs	Updates session fixture.
crates/ironrdp-testsuite-core/tests/rdpsnd/server.rs	Updates format-matching tests.
crates/ironrdp-testsuite-core/tests/rdpsnd/client.rs	Expands playback negotiation tests.
crates/ironrdp-testsuite-core/tests/connector/autodetect.rs	Updates connector fixture.
crates/ironrdp-rdpsnd/src/server.rs	Reuses shared format matching.
crates/ironrdp-rdpsnd/src/pdu/mod.rs	Adds negotiation matching API.
crates/ironrdp-rdpsnd/src/client.rs	Stabilizes formats and Wave handling.
crates/ironrdp-rdpsnd-native/src/lib.rs	Exports capture backend.
crates/ironrdp-rdpsnd-native/src/cpal.rs	Adds formats and volume scaling.
crates/ironrdp-rdpsnd-native/src/capture.rs	Implements CPAL microphone capture.
crates/ironrdp-rdpsnd-native/examples/cpal.rs	Updates playback example.
crates/ironrdp-rdpsnd-native/Cargo.toml	Adds RDPEAI dependency.
crates/ironrdp-rdpeai/src/pdu.rs	Implements RDPEAI PDUs.
crates/ironrdp-rdpeai/src/lib.rs	Defines the new crate API.
crates/ironrdp-rdpeai/src/client.rs	Implements the capture state machine.
crates/ironrdp-rdpeai/README.md	Documents RDPEAI support.
crates/ironrdp-rdpeai/Cargo.toml	Defines the new crate.
crates/ironrdp-connector/src/lib.rs	Adds capture configuration.
crates/ironrdp-connector/src/connection.rs	Sets INFO_AUDIOCAPTURE.
crates/ironrdp-client/src/rdp.rs	Wires the capture DVC.
crates/ironrdp-client/src/config.rs	Adds capture builder settings.
crates/ironrdp-client/Cargo.toml	Extends the sound feature.
crates/ironrdp-cfg/src/lib.rs	Adds audiocapturemode.
crates/ironrdp-activex/src/control.rs	Maps ActiveX capture settings.
crates/ironrdp-activex/README.md	Documents audio behavior.
Cargo.lock	Locks the new crate.
ARCHITECTURE.md	Registers the RDPEAI crate.
.agents/skills/commit-scope/SKILL.md	Adds the RDPEAI commit scope.

[Marc-André Moreau (mamoreau-devolutions)]

Addressed review feedback and split the XXL change into a stack:

1. feat: add AUDIO_INPUT protocol crate #1645 — ironrdp-rdpeai protocol crate (+ testsuite coverage, publish = false / test = false)
2. This PR — RDPSND playback hardening + CPAL capture backend + connector/client/cfg/ActiveX wiring (base: copilot/rdpeai-crate)

Copilot findings (sink clear on restart, open handshake, packet bounds, FormatChange ack, dead fields, Formats allocation) are fixed on the tip.

Please review #1645 first, then this PR once CI settles.

Note

LLM-assisted content (no human feedback).

[Mariusz Białończyk (manio)]

Marc-André Moreau (@mamoreau-devolutions) Much thanks for this! Finally I was able to record something using my microphone (but I had to make this change beforehand):

diff --git a/crates/ironrdp-client/src/config.rs b/crates/ironrdp-client/src/config.rs
index 699e2932..a3fe2f34 100644
--- a/crates/ironrdp-client/src/config.rs
+++ b/crates/ironrdp-client/src/config.rs
@@ -291,7 +291,7 @@ impl Default for ChannelConfig {
             #[cfg(feature = "sound")]
             sound: true,
             #[cfg(feature = "sound")]
-            audio_capture: false,
+            audio_capture: true,
             #[cfg(feature = "clipboard")]
             clipboard: ClipboardType::Enable,
             #[cfg(feature = "rdpdr")]

Then in Windows/settings/audio I clicked to record a sample, then connected with wlfreerdp and clicked play - and I was able to hear what I said to my mic in IronRDP, yay! So I can confirm that microphone capture is working.

But at the same time no idea why playback is not working (not even sure if it was before). I am using Arch with pulseaudio. It should work?

[Marc-André Moreau (mamoreau-devolutions)]

Marc-André Moreau (@mamoreau-devolutions) Much thanks for this! Finally I was able to record something using my microphone (but I had to make this change beforehand):

diff --git a/crates/ironrdp-client/src/config.rs b/crates/ironrdp-client/src/config.rs
index 699e2932..a3fe2f34 100644
--- a/crates/ironrdp-client/src/config.rs
+++ b/crates/ironrdp-client/src/config.rs
@@ -291,7 +291,7 @@ impl Default for ChannelConfig {
             #[cfg(feature = "sound")]
             sound: true,
             #[cfg(feature = "sound")]
-            audio_capture: false,
+            audio_capture: true,
             #[cfg(feature = "clipboard")]
             clipboard: ClipboardType::Enable,
             #[cfg(feature = "rdpdr")]

Then in Windows/settings/audio I clicked to record a sample, then connected with wlfreerdp and clicked play - and I was able to hear what I said to my mic in IronRDP, yay! So I can confirm that microphone capture is working.

But at the same time no idea why playback is not working (not even sure if it was before). I am using Arch with pulseaudio. It should work?

I'm glad that it at least partially works for you, and we should revisit current command-line options to avoid the need to override the configuration like that. The pull request you're commenting on may contain cross-platform code but it's been only tested by wiring the platform-specific code on Windows. I haven't checked the status of platform-specific audio code on Linux, but my guess it may need a bit of love, but probably nothing an AI agent can't figure out. You're welcome to give it a try

[Mariusz Białończyk (manio)]

I'm glad that it at least partially works for you, and we should revisit current command-line options to avoid the need to override the configuration like that

I'd be happy to take care of this one, so if possible, just leave it to me - it should be a pretty easy patch...

So, just to summarize where I'm at with this so far, here’s what I know:

1. I know how to enable the microphone, and that part works.
2. Audio playback in the latest version (master branch) doesn’t work - there isn’t even a stream showing up in the mixer.
3. A few versions back, it does work in the sense that the stream shows up in the mixer, which is already the first sign of some specific regression… but there’s still no actual sound.
4. I already have a preliminary quick-and-dirty fix that makes the audio playback working! basically by disabling Opus in the features. I still need to dig into this properly and figure out what’s actually going on.

If I figure something out, I’ll probably open a new issue/PR with the changes to not mess with this PR discussion. I hope you're ok with mention you Marc-André Moreau (@mamoreau-devolutions) ?

[Marc-André Moreau (mamoreau-devolutions)]

I'm glad that it at least partially works for you, and we should revisit current command-line options to avoid the need to override the configuration like that

I'd be happy to take care of this one, so if possible, just leave it to me - it should be a pretty easy patch...

So, just to summarize where I'm at with this so far, here’s what I know:

1. I know how to enable the microphone, and that part works.
2. Audio playback in the latest version (master branch) doesn’t work - there isn’t even a stream showing up in the mixer.
3. A few versions back, it does work in the sense that the stream shows up in the mixer, which is already the first sign of some specific regression… but there’s still no actual sound.
4. I already have a preliminary quick-and-dirty fix that makes the audio playback working! basically by disabling Opus in the features. I still need to dig into this properly and figure out what’s actually going on.

If I figure something out, I’ll probably open a new issue/PR with the changes to not mess with this PR discussion. I hope you're ok with mention you Marc-André Moreau (@mamoreau-devolutions) ?

Yes just tag me on the PRs, I'll try to fast track them

[Mariusz Białończyk (manio)]

for the reference:

1. is addressed in feat(viewer): add /microphone command-line switch #1790
2. and
3. in fix(client): restore Noop RDPDR channel fallback, fixing audio regres… #1791
4. (disabling opus compilation as a workaround) is not needed after fix(client): restore Noop RDPDR channel fallback, fixing audio regres… #1791

There is still one issue left. Microphone is available on the very first capture, not subsequent ones (probably device is closed or something like this) but I'll try to address this in separate PR... (Update: #1797)