Support jack

[richard-uk1]

Would you consider a PR for jack support (it would be behind a feature for linux)?

[knappador]

@derekdreery did you get a PR going? I obviously don't have merge access on this repo but am willing to help work on a fork that targets sound server API's.

[richard-uk1]

I haven't done any work on this yet.

[mitchmindtree]

One important consideration yet to be settled upon is the way in which we allow for supporting multiple possible audio backends per platform (e.g. alsa, pulse or jack on linux, wdm or asio on windows, etc). Right now, CPAL assumes just one per backend per platform (WDM on windows, ALSA on linux, CoreAudio on macos).

I posted some ideas regarding this at #204 which I still think might be a good direction, I'm hoping to get some time to flesh this out a little more in the near future. There's also an ASIO PR #221 which makes some adjustments to the module system to make this multiple-backend-per-platform support possible.

Anyway, just wanted to mention this as something to keep in mind!

[knappador]

#204 (comment)

The "host" is supposed to be cpal because cpal is supposed to be as low-level as possible.

The only reason to go below a sound server is IMO to be a sound server or an embedded device library. It's probably worth checking that perception before I get the idea to work on importing my work on PulseAudio.

Is the best solution ultimately going to make an abstraction layer over devices and then an abstraction layer over sound servers where the two are independent? I expect the answer cannot be a clear yes or no because some platforms might not have one or the other. If it can't be either, it almost has to be both.

[x37v]

Would you consider a PR for jack support (it would be behind a feature for linux)?

@derekdreery Jack works on osx and windows as well.

[ErikNatanael]

I thought I'd have a look at integrating the Rust jack bindings over the weekend. Give me a shout if you have tips/tricks/WIPs/want to collaborate on this!

[richard-uk1]

@ErikNatanael awesome - I've got quite a lot of work on this weekend, but any questions you have post them here. :)

[ErikNatanael]

Starting to get a grip on the different abstractions in CPAL and JACK and how they might fit together. One of the significant differences between JACK and most other Hosts/Devices is that a program can create an arbitrary number of JACK clients, each of which can have an arbitrary number of input and output ports (regardless of what sound card is used). This raises a number of questions (big and small), let me know if you have opinions or ideas!

1. In CPAL input and output streams are separate whereas in JACK a single ProcessHandler (similar to a Stream) can have both input and output ports. A single JACK client only has one ProcessHandler. I haven't benchmarked it, but it seems to me like sending data between input and output streams would add at least the buffer size in latency compared to dealing with them in the same process (which is significant for DMIs for example). So I wonder, is there any way with the current CPAL architecture to combine the input and output streams so that they can live inside a singleProcessHandler which would first run the input stream and then the output stream?
2. One workaround would be creating separate JACK clients for inputs and outputs which should work, but possibly introduce some overhead.
3. I see no real benefit of offering to starting multiple difference Devices (although that's certainly possible), but on the other hand to start a Device with any number of channels. This can be done using StreamConfig to manually set the channel count. However, there is a theoretically infinite number of correct/available SupportedStreamConfigs. I say either we provide only a single default stereo config and leave it up to the user to set the channel count for their purpose, or provide SupportedStreamConfigs for some common configurations (1, 2, 4, 6, 8, 16, 32, 64 channels?). Another option is to count the number of ports started by the system and match that as a default
4. Normally, connecting straight to a sound card (e.g. using ALSA) the outputs of the device correspond to the actual outputs. With JACK you have to rout the audio yourself (which is part of its great power). Some programs (e.g. SuperCollider) automatically connect the main outputs to the system outputs and the main inputs to the system, but others (e.g. effects) prefer to leave the client unconnected. It's also possible to use the JACK API to connect/disconnect ports while the client is running. For CPAL, would it make sense to expose this functionality (patching ports) or just treat the JACK client more similarly to a physical device with automatic connections to system inputs/outputs? To be consistent with other Hosts, maybe having the ports unconnected could be an option in the Host with the default behaviour being to connect them to the system (~soundcard).

[mitchmindtree]

Hey @ErikNatanael !

1. In CPAL input and output streams are separate whereas in JACK a single ProcessHandler (similar to a Stream) can have both input and output ports.
2. One workaround would be creating separate JACK clients for inputs and outputs which should work, but possibly introduce some overhead.

Yes CPAL still needs an abstraction for duplex streams. See this issue. I think for now we might need a unique client for both input and output, but once we land duplex stream support we can support this properly. We have a similar issue in both the CoreAudio and ASIO backends. It's still theoretically possible to avoid introducing latency between input and output by ensuring the input stream user callback is called first, but it's certainly not as nice a user experience as having a proper duplex stream API.

3. However, there is a theoretically infinite number of correct/available SupportedStreamConfigs.

Yes this is another known issue with CPAL's API at the moment. The current setup is certainly not a scalable or long-term solution, however it's also tricky determining what the right API is for checking device capabilities when each platform is so different in this regard. See #368 and #203 . I think it makes sense to expose a set of common configurations for now. There's also a set of COMMON_SAMPLE_RATES in the bottom of lib.rs aimed at assisting with this workaround.

4. To be consistent with other Hosts, maybe having the ports unconnected could be an option in the Host with the default behaviour being to connect them to the system (~soundcard).

Yes this sounds like a nice solution!

Thanks so much for diving into this @ErikNatanael!

[ErikNatanael]

Yes CPAL still needs an abstraction for duplex streams. See this issue. I think for now we might need a unique client for both input and output, but once we land duplex stream support we can support this properly.

Good to know this is a known issue. I'll add some info on how this work in JACK to that issue.

We have a similar issue in both the CoreAudio and ASIO backends. It's still theoretically possible to avoid introducing latency between input and output by ensuring the input stream user callback is called first, but it's certainly not as nice a user experience as having a proper duplex stream API.

Nice, I'll do my best to ensure the input stream gets called first when I get there then.

Yes this is another known issue with CPAL's API at the moment. The current setup is certainly not a scalable or long-term solution, however it's also tricky determining what the right API is for checking device capabilities when each platform is so different in this regard. See #368 and #203 . I think it makes sense to expose a set of common configurations for now. There's also a set of COMMON_SAMPLE_RATES in the bottom of lib.rs aimed at assisting with this workaround.

Cool, I'll do that then. Yeah, it's tricky when the backends are so different! The sample rate is set when starting the jack server and not changed by the clients so that one won't change here at least.

Thanks so much for diving into this @ErikNatanael!

Not a bother, though don't count your chickens before they hatch ;)

[ErikNatanael]

I've run into a problem I can't quite figure out even from reading the other Host implementations: how to store the data callbacks in order to call them from the function called by JACK.

The JACK library works like this: when activating a Client you provide it with something implementing the ProcessHandler trait which requires the method called by JACK each audio iteration (fn process(&mut self, _: &Client, _process_scope: &ProcessScope) -> Control). When process() is called by JACK it needs access to the data callback so I tried storing it in a Box<dyn FnMut(&Data) + Send + 'static>, but that doesn't compile because

`(dyn for<'r> std::ops::FnMut(&'r mut Data) + std::marker::Send + 'static)` cannot be shared between threads safely

Here it is: https://github.com/ErikNatanael/cpal/blob/jack-host/src/host/jack/stream.rs#L187

I'm generally confused about how the data callbacks get access to whatever data model the audio process needs so that probably doesn't help either :D

I created a pull request to track progress here: #389

[hugovdm]

#389 was merged on Oct 1. Thanks @ErikNatanael ! And works for me...

(Helps me obtain much lower latency - e.g. testing via the feedback.rs example, setting LATENCY_MS to 20.0 works fine, with my JACK configuration - 128 frames/period.)

[mitchmindtree]

Closed via #389. Thanks again @ErikNatanael!