Switch from ALSA to PulseAudio

[krisgesling]

Switches all ALSA commands to use PulseAudio

• Sets input and output devices as default devices in PA
• Provides a new selection menu for non-standard devices using the output of pactl info list sinks/sources short
• Set volume using PA
• As the devices are now set as defaults, we no longer need to set custom play_type_cmdline settings for each device, hence these mycroft.conf settings have been removed.
• Simplifies the speak function as Mimic can now output audio directly, reducing chance of tmp file issues

This fixes the issues raised in PR #150 that are present in the newer RPi kernel.

Description of how to validate or test this PR

1. Fire up a fresh copy of Picroft or run mycroft-wipe
2. Apply new files
3. Set up as normal - should just be a nice experience...

If you need to run the wizard again as if it was fresh:

touch first_run
rm .setup_choices
. auto_run.sh

CLA

• Yes

[forslund]

Reading through the code everything looks great. I'm wondering if the audio_setup.sh script should be updated as well to refer to pactl / pacmd in the comments instead of the alsa / amixer ones?

[krisgesling]

Thanks - I didn't see those.

I've put both this and PR #154 into a release candidate image available here:
https://downloads.mycroft.ai/releases/picroft/rc/

It doesn't include these comment updates, and still has the mycroft-setup-wizard alias. But is functionally equivalent.

If anyone is watching along it would be great to get some independent validation of these updates.

[j1nx]

Couple of things as feedback, from going through the same process for OVOS a while ago;

Did you recompile mimic1 with pulse support? As the default uses ALSA.
The default mycroft.volume skills controls the ALSA master, so that might need some changes as well.

Quickest solution is to also force ALSA through Pulse by;
https://github.com/OpenVoiceOS/OpenVoiceOS/blob/develop/buildroot-external/rootfs-overlay/etc/asound.conf

[krisgesling]

Thanks @j1nx - great feedback!

I haven't recompiled Mimic - TBH I just switched it over, it seemed to work so I called it a day. I have added an issue to Mimic for us to do this properly as I think we should be moving more consistently over to PA. For the moment I'll add the OVOS asound.conf

The ALSA stuff in the Volume Skill has also been on my radar to tackle for a while. I started to extract things out yesterday - see the WIP PR above. Interested in any thoughts on this approach as a bridge to these moving to the HAL.

[forslund]

Mycroft doesn't use mimic to output the audio so for "normal" usage it won't really affect anything that it's currently built with alsa, running from the commandline may be affected.

[j1nx]

Last tip which is the default within OVOS (less sensitive for; no output issues)

pactl load-module module-combine-sink sink_name=combined
pactl set-default-sink combined

This will send the sound output to all found sinks. So with that you will have sound output to both 3.5mm jack and HDMI (and any other attached sinks for that matter)

[forslund]

Oh, that is really cool!

[mcdonc]

mimic appears to already be compiled for pulseaudio, or at least an strace of the output of "mimic -t hello" shows it looking up pulseaudio config files and libraries?

[j1nx]

Could be, but the linked library is wrong.
https://github.com/OpenVoiceOS/OpenVoiceOS/blob/develop/buildroot-patches/0101-Update-mimic-to-latest-1.3.0.1-version.patch#L55
(there is also a old PR at themimic repo, although not sure if it still can be applied)

I then configure it with "--prefix="/usr/local" --with-audio=pulseaudio"

However like Ake said, if it is only used to generate WAV files it should be fine regardsless, but perhaps your chop off issue is related.

[mcdonc]

However like Ake said, if it is only used to generate WAV files it should be fine regardsless, but perhaps your chop off issue is related.

Right.

(@jinx is talking about my woes at https://community.mycroft.ai/t/picroft-headphones-no-output-sound/10852/16)

It appears that the "speak" command calls the mimic command-line executable, so it doesn't take the same codepath as the mycroft audio process does when it does TTS, it seems. That said, even if I do mycroft-speak hello; mycroft-speak hello which it appears does take the same codepath as TTS output by skills, I hear "lo, hello". And if I do mycroft-speak hello; mycroft-speak hello; sleep 5; mycroft-speak hello, I hear "lo, hello <5 sec pause> lo". So whatever issue this is appears to be independent of mimic-vs-mycroft, or at least both exhibit a similar issue.

In any case, here's ldd mimic in this branch (doesn't appear to link to any PA libs):

linux-vdso.so.1 (0xbefd3000)
/usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so => /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so (0xb6f0e000)
libttsmimic_lang_all_langs.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_all_langs.so.0 (0xb6efc000)
libttsmimic_lang_all_voices.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_all_voices.so.0 (0xb6eea000)
libttsmimic.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic.so.0 (0xb6ead000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6e2b000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6cdd000)
libttsmimic_lang_usenglish.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_usenglish.so.0 (0xb6cb6000)
libttsmimic_lang_cmu_grapheme_lang.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_grapheme_lang.so.0 (0xb6ca1000)
libttsmimic_lang_cmu_indic_lang.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_indic_lang.so.0 (0xb6c8d000)
libttsmimic_lang_cmulex.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmulex.so.0 (0xb6bcf000)
libttsmimic_lang_cmu_indic_lex.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_indic_lex.so.0 (0xb6bba000)
libttsmimic_lang_cmu_grapheme_lex.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_grapheme_lex.so.0 (0xb6ac1000)
libttsmimic_lang_cmu_us_kal.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_us_kal.so.0 (0xb6957000)
libttsmimic_lang_cmu_time_awb.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_time_awb.so.0 (0xb65c4000)
libttsmimic_lang_cmu_us_kal16.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_us_kal16.so.0 (0xb61e6000)
libttsmimic_lang_cmu_us_awb.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_us_awb.so.0 (0xb5f29000)
libttsmimic_lang_cmu_us_rms.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_us_rms.so.0 (0xb5bf9000)
libttsmimic_lang_cmu_us_slt.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_cmu_us_slt.so.0 (0xb593a000)
libttsmimic_lang_vid_gb_ap.so.0 => /home/pi/mycroft-core/mimic/lib/libttsmimic_lang_vid_gb_ap.so.0 (0xb0e4d000)
libicui18n.so.63 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.63 (0xb0bfd000)
libicuuc.so.63 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.63 (0xb0a77000)
libicudata.so.63 => /usr/lib/arm-linux-gnueabihf/libicudata.so.63 (0xaf079000)
libasound.so.2 => /usr/lib/arm-linux-gnueabihf/libasound.so.2 (0xaef98000)
/lib/ld-linux-armhf.so.3 (0xb6f23000)
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xaee51000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xaee24000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xaedfa000)
libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xaede7000)
librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xaedd0000)

[mcdonc]

This is painful and insane but the solution at https://askubuntu.com/a/1310141 works.

Play a 5 minute inaudible 20khz wav file via paplay over and over in the background while mycroft runs. It keeps the usb speaker from idling. Every syllable uttered by Mycroft is played while this is happening.

I wish disabling suspend-on-idle worked instead, but it does not.

I don't have a dog. I'm sure he would not like it.

[mcdonc]

I kind of wonder if it would behoove us here to emit pactl unload-module module-suspend-on-idle by default here, given that we know that suspension is a problem with many devices. If that doesn't sound reasonable, maybe we could add it as a comment to the sample audio_setup.sh above?

[mcdonc]

It's good that the user may hear some output as a result of this being called, but using mimic esp when it is not compiled for pulseaudio doesn't tell us much about whether e.g. mycroft-speak or the codepath that mycroft actually uses to do TTS will work for them, will it?

[ConorIA]

I have been following this issue lately for a picroft with the AIY v1 voice hat. All is working, except for sound. Mycroft is constantly "yelling" (it is embarrasing when I am in a work call and all of a sudden mycroft pipes in with "I'm sorry I don't understand". I am curious whether we would expect the AIY-asound.conf file or the OVOS asound.conf to reside at /etc (though I didn't really find either helped volume). Is this a quirk of the hardware, or am I missing a step somewhere?

[krisgesling]

I am curious whether we would expect the AIY-asound.conf file or the OVOS asound.conf to reside at /etc (though I didn't really find either helped volume). Is this a quirk of the hardware, or am I missing a step somewhere?

Hey, I don't have an AIY kit so can't help there, but for the ALSA config (asound.conf) /etc/asound.conf is (at least one) recommended location: https://www.alsa-project.org/wiki/Setting_the_default_device