Hacker News new | ask | show | jobs
by ncmncm 1518 days ago
Pipewire is driven by the need to handle synchronized media from multiple sources delivered to multiple sinks in real time with minimal, constant latency. The reference problem is cars, which often have a dash cam, back-up cam, dash video display, instrument panel display, bluetooth phone connection, radio receiver, satellite receiver, audio amplifier, dash microphone, backseat video display with audio, seatbelt alarm, and more besides, that all have to be managed sensibly with minimal engagement.

Handling audio without video fails if they have to be kept in sync.

Anyway that's my outsider's impression.
1 comments
stragies 1518 days ago
So, a glue layer for

* (LAN-)devices, most a source/sink/control for various Audio/Video streams.

* Those devices (can) share/use any HW (Microphones/Speakers/V4L-devices) available

* programs producing/consuming sound/video, to be "exported" as individual sinks/sources

* Eventing support to dynamically change media routing

Your stated "reference problem" (Car AV) is (imho) not the most user-"relatable" scenario (after all, who has "hackable car"), a more common scenario I'd see:

* In several of the rooms of the dwelling there are

* Devices large/small with several connection types

  * LAN (PCs, TVs, AV-HW, MediaPlayers, ...) with protocols including e.g.

    * Well-known programs (VLC, KODI) with their "own" protocol

    * UPNP/DLNA/AirPlay/Miracast/PulseAudio/RTSP/...

    * SIP/WebRTC devices/clients (Doorbells, VoIP-Phones, VideoConf)

    * Devices with Vendor-specific protocols. e.g LG WebOS

  * USB connection to one of the above (WebCam, Android-devices, phones)

  * Wifi-P2P (Some of LAN devices have this too, for e.g. screencasting)

  * Bluetooth BTLE/Classic (devices can be stationary/mobile/transient)
I would like to enable these usecases: * Send [emulated] [second] screen from PC to any Display surface

* View any desktop/UI/program/device on any other display surface

* Control all the above from PCs/Phones/Tablets, or Events (Doorbell -> Camera PiP @TV)

* Route any audio/video source to my speech-recognition/video-conf/VoIP Software

* Use Android device [connected to USB[ of another Device]] as 8+ devices: Camera F/R, Screen I/O, Audio I/O Headphone/Speaker/BT Microphone/App-Output

* Use "hook"-scripts for eventing/device control (TV might need some command to switch to HDMI3)

I realize most of the above can be probably be done using some GST/FFMPEG glue scripts/code and some systemd dependencies, but I'm haven't found quickly relatable examples of: * "Run this command/config to add an ffpmeg commandline as virtual camera"

* "Use this config to represent your TV as another display output device"

* "Run this command On DevA to launch a GUI-binary on DevB, and make this "stream" available as VideoSource in Browser@DevA"

* "Use the Webcam on DevA, bluetooth headsets via DevB for input, TV1 as VideoOutput, AudioOut on AVR, runnings VideoConf on DevC"

* "Do this to implement access-control" (Bob can see Input "foo", but Alice can't)

link
ncmncm 1518 days ago
Cars are a hard but representative problem, with intrinsic advantages:

  - Multiple data flows
  - Limited available attention
  - Need for "just works"
  - Need for trivial override
  - Funding available
  - Strong demand for solution
link