Sounds don't play in the editor and animations aren't able to change streams

[ZeEndy]

Are you using the provided fork of Godot?
Yes

Godot-steam-audio version
0.1.0

Steam-Audio version
4.5.0

Operating System
Godot v4.2.2.rc.mono unknown - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 3070 Ti (NVIDIA; 31.0.15.4629) - 12th Gen Intel(R) Core(TM) i5-12600KF (16 Threads)

Describe the bug
When using a Audio clip in the AnimationPlayer and combining it with AnimationTree blending the audio volume doesn't properly transition from animations. Confirmed to only happen in the fork with steam audio

To Reproduce
Steps to reproduce the behavior:

1. Create 3 animations with AnimationPlayer and add 1 AudioStreamPlayer3D or SteamAudioPlayer.
2. In all animations add a sound effect.
3. Put 3 different sound effects for each animation
4. Create a AnimationTree with a BlendTree root.
5. Add a BlendSpace2D with all the animations.

Expected behavior
Expected beheivour is how its in the 4.2.1 and 4.3dev where the sound's volume gets blended between animations

Example project, code snippet, or screenshots
This video demonstrates how it behaves in both the 4.3dev and 4.2.1
https://github.com/stechyo/godot-steam-audio/assets/29980512/fb3047c8-db7f-4e00-9981-269d976d9785

While this one shows how it behaves in the current Steam Audio version
https://github.com/stechyo/godot-steam-audio/assets/29980512/2d134814-bfd7-4e75-b8b7-ab28bd2bc66f

NOTE:
I had to compile this version on my own because I use mono so it could just be a me thing but as far as I can tell it doesn't look like it

[ZeEndy]

It appears that there could be something wrong with my compile of the fork since I needed to make .NET version.

2024-01-25.20-18-39.mp4

Testing the same project on the pre-compiled version of the fork it seems to work on it.

2024-01-25.20-19-28.mp4

I'm guessing its most likely just a me thing where I messed something up while compiling but it also could be something is messed up in the .NET version somehow?? I'm adding a minimal reproduction project to test it yourself.

SteamAudioBlendingIssueTest.zip

[stechyo]

Hi, sorry I didn't have time to reply beforehand.

1. Can't see the videos on your second comment, is it just on my end?
2. If you compiled the engine yourself you probably did so with MSVC since you're on Windows, try to compile the extension as well yourself. Both engine and extension are compiled with MinGW, and I don't know if having different compilers between them causes problems.
3. I'll check the reproduction project, thanks. But please note that, depending on how this blending is done, it's somewhat likely that fixing this issue requires a full restructuring of the extension or it just isn't possible without messing around with the engine code.

[stechyo]

With the minimal reproduction project you sent:

• It works on my machine
• Changing the AudioPlayer to a SteamAudioPlayer makes the sound stop working (this is probably because the player isn't really configured to work inside the editor, only when the game starts

This is likely an issue with your compilation w/r/t native audio player, but I don't think the SteamAudioPlayer is going to work with blend spaces as-is. From what I see, you're basically setting stream on animations, but this extension does not like that see this. This can probably be fixed, but there's the additional issue of animation stream blending, for which I'd have to spend some quality time reading how animation works in the engine to understand if it's even feasible to support this.

In sum, there are three issues here:

• Animations setting stream don't work (probably not an über-hard fix)
• SteamAudioPlayer does nothing in the editor (requires some changes, prolly doable assuming that having SteamAudio effects in the editor isn't required)
• Animations blending stream doesn't work (no idea if it's fixable)

[ZeEndy]

* Animations blending `stream` doesn't work (no idea if it's fixable)

While I'm not knowledgeable in the inner workings of the engine the only thing I could point to is this commit as its the one that made audio blending possible
godotengine/godot@e5752fd#diff-47d98bbc16960ace8510b2cfa795da0e87601cc3f22a6f2e5155cfe01c750952R1528
I guess the only way would be to modify the AnimationTree so that it mimicks what it normally does to the AudioPlayer. I'm just spitballing since I dont really know how the engine handles polyphony and blending but it looks like a audio object gets spawned and just adjusts its volume depending on the blend.

[stechyo]

Yeah I understand... AudioStreamPlayers have several stream_playbacks, which of course aren't exposed to GDExtension, meaning we'd have to muck around in the engine code some more... This is not a "won't fix" but a "won't fix" any time soon, I really don't want to distance the fork away from the engine even more. Sorry. As for animations in general, I may take a look.

[ZeEndy]

Unfortunate, but completely understandable as I've already seen the whole issue with getting audio methods exposed has been happening for 5 years already. The only hope here is that the extension gets popular enough to warrant some of the necessary changes to allow extensions like this to exist properly without the need for a custom fork.