Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Audio effects #9640

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Add Audio effects #9640

wants to merge 4 commits into from

Conversation

gamblor21
Copy link
Member

@gamblor21 gamblor21 commented Sep 17, 2024

Foundation to add audio effects to CircuitPython as discussed in issue #8974.

This PR aims to create initial modules, some base effects and utilities to help create future effects. The ones included will serve as a template for future creations.

To do:

  • Create modules for filters, delays, dynamics and reverbs
  • Create 2 or 3 effects (echo, chorus and TBD open to suggestions)
  • Test effects against audio sources that are not signed, 16-bit
  • Add "mix" setting for how much of the effect is applied.
  • Look into adding dynamic range compression (like synthio)
  • Look into BlockInput for inputs so they can be varied by the synth/time

Looking for feedback on the API and anything else that comes up.

When I have time I plan to document how to create your own effects using the ones provided.

Sample code to run:

i2s_bclk, i2s_lclk, i2s_data = board.GP20, board.GP21, board.GP22
audio = audiobusio.I2SOut(bit_clock=i2s_bclk, word_select=i2s_lclk, data=i2s_data)
mixer = audiomixer.Mixer(voice_count=1, channel_count=1, sample_rate=44100, buffer_size=2048)
audio.play(mixer)

synth = synthio.Synthesizer(channel_count=CHANNELS, sample_rate=44100)
amp_env = synthio.Envelope(attack_time=0.02, attack_level=1, sustain_level=.9, release_time=0.1)
synth.envelope = amp_env
n1 = synthio.Note(261)

echo1 = audiodelays.Echo(delay_ms=500, decay=0.7, buffer_size=1024, channel_count=CHANNELS, sample_rate=44100)
echo1.play(synth)
mixer.voice[0].play(echo1)

synth.press(n1)
time.sleep(0.25)
synth.release(n1)

@dcooperdalrymple
Copy link

I'm still digging into the code, but my first instinct is to have the delay_ms property just be a float in seconds as delay instead (ie: rather than 500 ms use 0.5 s). The calculation that is being used to allocate the buffer uses floating point regardless.

Speaking of buffer sizes, it may be nice to explore having a fixed buffer size and altering the rate of playback of that buffer depending on the delay setting. This would be similar to a bucket-brigade style delay pedal and require some more advanced interpolation, etc. I think it be worth exploring an "Analog" delay effect of this nature.

Copy link
Member

@tannewt tannewt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the initial PR! This is very exciting.

}
MP_DEFINE_CONST_FUN_OBJ_KW(audiodelays_echo_play_obj, 1, audiodelays_echo_obj_play);

//| def stop_voice(self, voice: int = 0) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//| def stop_voice(self, voice: int = 0) -> None:
//| def stop(self) -> None:

Comment on lines 143 to 145
if (decay > 1 || decay < 0) {
mp_raise_ValueError(MP_ERROR_TEXT("decay must be between 0 and 1"));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +26 to +27
{ MP_QSTR_delay_ms, MP_ARG_INT | MP_ARG_KW_ONLY, {.u_int = 50 } },
{ MP_QSTR_decay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_obj = MP_OBJ_NULL} },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect you'll want delay and decay to be BlockInputs so they can be varied.

//| BlockInput = Union["Math", "LFO", float, None]
//| """Blocks and Notes can take any of these types as inputs on certain attributes
//|
//| A BlockInput can be any of the following types: `Math`, `LFO`, `builtins.float`, `None` (treated same as 0).

Copy link

@dcooperdalrymple dcooperdalrymple Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As much as I'd like to see BlockInput on delay, it would cause a reallocation of the buffer on each update. I imagine that wouldn't be great for performance. However, I do think that would be a great feature for a fixed buffer delay effect.

void common_hal_audiodelays_echo_set_delay_ms(audiodelays_echo_obj_t *self, uint32_t delay_ms) {
    self->delay_ms = delay_ms;
    self->echo_buffer_len = self->sample_rate / 1000.0f * self->delay_ms * (self->channel_count * (self->bits_per_sample / 8));

    self->echo_buffer = m_realloc(self->echo_buffer, self->echo_buffer_len);

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed delay is fine. A future refinement could also specify a "max_delay" that does the alloc and "delay" lets you move within that range.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could adjust your initial allocation to be larger if given a non-fixed value. That way you don't need to re-allocate when the value changes. You just don't use all of the buffer for short delays.

I also think it's good to show how to use BlockInput so that further effects use it too. It is more complicated, but it'd be good to show how to use. It is super powerful IMO.

Comment on lines +1975 to +1978
#: shared-module/audioeffects/Echo.c
msgid "The sample's sample rate does not match the effect's"
msgstr ""

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably better to drop "the effect's" and "the mixer's" so we can share these messages.

@gamblor21
Copy link
Member Author

I'm still digging into the code, but my first instinct is to have the delay_ms property just be a float in seconds as delay instead (ie: rather than 500 ms use 0.5 s). The calculation that is being used to allocate the buffer uses floating point regardless.

I'm not too worried at this point as I'm using Echo basically to test out ideas/concepts. Is there a reason you would want time in a float of seconds vs more precise?

Same thought on the BlockInput for the buffer size. You are right if it could change a maximum size/current size would have to be done to prevent constant reallocation.

Speaking of buffer sizes, it may be nice to explore having a fixed buffer size and altering the rate of playback of that buffer depending on the delay setting. This would be similar to a bucket-brigade style delay pedal and require some more advanced interpolation, etc. I think it be worth exploring an "Analog" delay effect of this nature.

I think this would be good for another effect. I have tossed about the idea of a generic delay buffer that other effects could use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants