Question about loop points handling for SFX from RIFF WAV sample chunk

[klei1984]

Hello @Wohlstand ,

I am porting an MS-DOS HMI S.O.S v3 based game to SDL2 backend. The game uses only digital audio samples for music, sound effects and voice over. The audio samples are stored in RIFF WAV files. Out of ~300 samples about ~80 has a "smpl" chunk at the end of the audio file right after the "data" chunk. Loop points are not only used for music tracks, but also for many sound effects.

E.g. to move a unit from one location to another the game first ramps up the unit's movement speed, keeps a constant maximum speed, and then finally slows down quickly to a stand still. This visual effect is also simulated in the related sound effect for unit movement. First there is a section for the speed up, then comes the part that is looped which is played as long as the unit moves at full speed and then comes an after loop part which is additionally faded out during timed playback.

In the past couple of years there were many improvements in SDL2_mixer thanks to you and SDL-Mixer-X with respect to loop point support in various formats, but as far as I can tell the feature is only available for music tracks.

What I gathered from the various bits and pieces of documentation, change history and community discussions it seems to me that even if I would try to load the special SFX samples manually into three dedicated Mix_chunks (startup, loop, fade out) there is no way to play these parts in a seamless manner one after the other without undesired pauses in the audio stream. Please correct me if I am wrong.

With your deep understanding of SDL Mixer's inner workings and keeping the roadmap of the library in mind, do you see any chance that what I would need to mimic HMI S.O.S could be pulled off by SDL Mixer now (latest master) or in the foreseeable future? Or would it be better to look for an alternative audio mixer as this will not be feasible for a long time to come for sure.

Thanks for your kind response in advance.

[Wohlstand]

A good idea, I also thought about it. That would require the change of the public Mix_Chunk structure that will break the ABI compatibility:
https://github.com/WohlSoft/SDL-Mixer-X/blob/master/include/SDL_mixer.h#L157-L163
However, there are two ideas on how to make that safely:

• Make the sort of "header" inside of the raw data block that detects the sub-structure inside with all loop points and makes such behavior
• Make the separated "chunks" API calls that will use the new structure and keep the old API being untouched for the ABI compatibility. At least, there are only calls with Mix_Chunk* argument, all other calls can be used for the same easily.

[klei1984]

I quickly went through some of the relevant source files like mixer.c, music.c and music_wav.c to see how loop points are realized for music playback. I did not know that there could be multiple loop point start/stop pairs (or maybe I just misunderstood something). This further complicates things. Fortunately for my use case each sound sample has only a single loop point start and stop position.

Any ways I do not dare to say that based on my sloppy analysis I could comprehend much from the design, but what I did notice is that the mixer uses the callback based pull mechanism to feed audio data to the SDL audio device and the music uses a streaming buffer or something comparable in the interface backend for wav files. As far as I can tell if the chunk of data before the loop point stop position does not fill up the expected buffer space or chunk size further data is added from the loop point start position; basically the WAV_GetSome callback is called in a loop as long as the mixer needs further data or the end of stream is reached (loops finished, done flag set).

In the game where I need to support loop points music samples are looped if the sound track shuffle feature is disabled or when only a single sound track or music track is found on the file system and there is no need to switch music tracks. So the typical use case for music looping in the game is to loop forever or do not loop at all. Which behavior to take is known to the game at the time when the music playback is requested by the sound manager.

On the other hand the SFX specific use case is a bit different within the same game. SFX samples are fully loaded into RAM, but the game does not know how long the SFX needs to be looped and the game itself does not convert the audio format, that is left for S.O.S to be done. So when it comes to loop count, the game again does not care to calculate anything, it again just lets the sample loop forever. When the sound manager finally schedules the SFX sample to play it just flips on a lets loop from now on flag, and feeds sound data to S.O.S accordingly. Then later another API call tells the sound manager that it is time to stop looping in which case it lets the audio progress from loop point stop to the actual end of the SFX sample and when that is finished, the sample related memory is finally freed.

With respect to your proposals based on my use cases and my very basic current understanding of SDL Mixer I think the second option would be simpler to do. A new set of APIs with a new extended Mix_Chunk structure variant that could be kept opaque for users as well. The only doubt I have is that normal Mix_chunks are fully loaded samples, but Mix_LoadWav and the likes cannot read loop point "smpl" chunks from RIFF WAV files and even if they would be able to load them there is no place to store loop point data within the original structure. So not only a special SFX with loop points support would be necessary, but also the various load audio files would need alternatives.

[klei1984]

Hello @Wohlstand , would you be so kind to join the SDL discord server? https://discord.com/invite/BwpFGBWsv8

[Wohlstand]

By the way, there is a work on SDL3 on-going, and once will be enough time, I'll roll up the SDL3 version of MixerX. And, possibly, there is time to add new features into Mix_Chunk structure, or even, make it being opaque, so, it will be an ABI-safe and can be freely updated without breaking of any ABIs. @sezero, @slouken, what do you think about the Mix_Chunk structure in the SDL3-based Mixer? I mean, the case where is an API/ABI incompatibility gets introduced. Just for your info: the main topic is adding loop points support for the Mix_Chunk too.

[slouken]

Making the mix chunk structure opaque is definitely a possibility for SDL3_mixer.