Skip to content

Add bidirectional audio support for HomeKit cameras#1981

Open
andersbaekgaard wants to merge 1 commit intoAlexxIT:masterfrom
andersbaekgaard:feature/homekit-bidirectional-audio
Open

Add bidirectional audio support for HomeKit cameras#1981
andersbaekgaard wants to merge 1 commit intoAlexxIT:masterfrom
andersbaekgaard:feature/homekit-bidirectional-audio

Conversation

@andersbaekgaard
Copy link

@andersbaekgaard andersbaekgaard commented Dec 14, 2025

Summary

  • Add bidirectional audio support for HomeKit cameras (speaker/talkback functionality)
  • Audio path: WebRTC/Browser mic → FFmpeg transcoding (OPUS→AAC-ELD) → HomeKit speaker via SRTP
  • Implements connection tracking to prevent duplicate consumer-track pairs
  • Adds accept-audio parameter to FFmpeg for waiting on external audio sources
  • Thread-safe SRTP writes with mutex protection

Requirements

  • FFmpeg must be compiled with libfdk_aac for AAC-ELD encoding (required for HomeKit speaker audio)

Test plan

  • Tested with Aqara G4 doorbell - microphone works in Home app
  • Verified audio latency is optimal for the device (~3s jitter buffer is device firmware, not go2rtc)
  • Test with other HomeKit cameras that support bidirectional audio

Implemented by Anders Baekgaard in cooperation with Anthropic Claude Opus 4.5
https://claude.ai/

Enable speaker/talkback functionality for HomeKit cameras:
- Audio path: WebRTC/Browser mic → FFmpeg (OPUS→AAC-ELD) → HomeKit SRTP
- Connection tracking to prevent duplicate consumer-track pairs
- FFmpeg `accept-audio` parameter for external audio sources
- Thread-safe SRTP writes with mutex protection

Tested with Aqara G4 doorbell.

Implemented by Anders Baekgaard in cooperation with Anthropic Claude Opus 4.5
https://claude.ai/
@AlexxIT
Copy link
Owner

AlexxIT commented Dec 15, 2025

Thanks! The implementation is quite complex and introduces a lot of confusion into the core.
I won't accept it in this form, but I will try to understand the idea and adapt it when I have time.

And I really don't like the idea of ​​using custom ffmpeg. This is one of the block factors for implementing two-way for HomeKit cameras.

@AlexxIT AlexxIT added the doubt label Dec 15, 2025
@andersbaekgaard
Copy link
Author

Thank you for the prompt feedback.

I understand your concern about complexity, the PR got much more complex than I expected when first looking into it. I could not get streams matching to work for the aac-eld transcoder producer, due to archtiectural limitation. With your insight, you could maybe make it simpler.

Note that there is also a simple bug fix.

@kombihacke
Copy link

Would really love to see this integrated. Cannot get my Aqara G410 running with 2 way audio. Thankful for any advice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants