Ease in/out audio when starting/stopping playback

[DarkDefender]

Currently when stopping or starting playback there is no ease in or out of the sound level (that I know of).

This usually leads to audio pops if the sound level is not zero when starting or stopping playback.

I'm wondering if there could be some ramping implemented on the audaspace side (in just a few milliseconds to eliminate these pops).

[neXyon]

The SoftwareDevice::mix method that is used in pretty much all backends except for OpenAL already uses the version of Mixer::mix that can ramp the volume per sound. SoftwareDevice::mix could be extended with a global, multiplicative ramping volume and used in the backend to put a few more samples in the ringbuffer with the ramp when audio is stopped and ramped back up the first few samples when starting playback.

If you want to implement this, you're more than welcome. Maybe with a prototype in one backend/device before extending it to all of them?

[DarkDefender]

I'll just post an update as I did some minor experiments.

The main use case for this was when using the "Audio scrubbing" feature in Blender. Currently it plays back around one sound buffer or so per seek when scrubbing. Because this leads to a lot of disconnected audio being played back, there is a lot of clicks and pops while you are doing it.

When I looked at this now, I noticed that we don't really stop and start the sound devices while scrubbing and instead keep them open for as long as we are still scrubbing and playing back audio. Which makes sense.

This means that the audio ease in/out has to be on the playback queue level.
I did a quick and dirty hack to add a 1ms ease in/out for every queued audio buffer and it improved the perceived sound quality by a lot when scrubbing:

diff --git a/extern/audaspace/src/respec/Mixer.cpp b/extern/audaspace/src/respec/Mixer.cpp
index 15872fbcff2..7db4e6c2e3c 100644
--- a/extern/audaspace/src/respec/Mixer.cpp
+++ b/extern/audaspace/src/respec/Mixer.cpp
@@ -96,9 +96,20 @@ void Mixer::mix(sample_t* buffer, int start, int length, float volume_to, float
 
        length = (std::min(m_length, length + start) - start);
 
+       int ramp_limit = 45; // ~1ms for 48 kHz
+       volume_to = 1.0f;
+       volume_from = 0.0f;
+
        for(int i = 0; i < length; i++)
        {
-               float volume = volume_from * (1.0f - i / float(length)) + volume_to * (i / float(length));
+               float volume = 1.0f;
+               if (i < ramp_limit) {
+                       volume = volume_from * (1.0f - i / float(ramp_limit)) + volume_to * (i / float(ramp_limit));
+               }
+               if (length - i < ramp_limit) {
+                       int left = length - i;
+                       volume = volume_from * (1.0f - left / float(ramp_limit)) + volume_to * (left / float(ramp_limit));
+               }
 
                for(int c = 0; c < m_specs.channels; c++)
                        out[(i + start) * m_specs.channels + c] += buffer[i * m_specs.channels + c] * volume;

There are still some pops but this eliminated 99% ish of the clicks and pops during my test. Increasing the ease in/out time above 1ms didn't really improve the situation much and sometimes actually made it worse. So I think that around 1ms is probably the sweet spot here.

This hack solution is of course not a viable solution as now every queued sound buffer has ease in/out.

@neXyon I think this might need to be implemented on a sound handle level where we would introduce two things:

1. Functionality to add ease in/out volume ramps over a span of the handle. In the scrubbing case, it would make sense to have something like "start playback at the current location with an ease in and then after 0.5s of playback ease out".
2. If playback of a handle is stopped/paused before it has reached the end, the handle will queue up one final audio buffer but immediately ease out at the start of this final buffer. (to make sure we eases out immediately even if the user has a big buffer size.)
   This is for the case where the user scrubs again before the current scrubbing audio clip has finished playing back.

Does that make any sense to you? Does it seem like a good approach?

[neXyon]

Hmm, I thought a little more about this and I think now that Blender uses C++ audaspace, there might be a nice way to implement this. You could add a class that derives from IReader that does the fading semi-automatically and then you can add this reader on top of the SequenceReader that is created for the playback handle of a scene. The basic idea is as follows:

• Fade in on playback can be done automatic, it just needs a bool member that tells it whether playback started or not.
• Fade out for stopping needs to be set externally, since readers don't know when they will be stopped. Since a timed stopping is already implemented for scrubbing, this could be a simple method of this new class that sets a "timer" when the fadeout should happen and that time is externally set to that. Similarly for stopping playback in Blender, this could also toggle the fadeout and stop with a same timed mechanism, a little later. Only disadvantage is, that to start playback then means, that seeking backward may be necessary. I don't know if that is necessary or this can be done somehow else to stop more precisely. Stopping is definitely the more difficult part. Stopping resets the strated bool member, so on playback the fade in happens automatically again.
• On seek there could even be a crossfade implementation instead of fading out and in again. That should be fairly simple, it's only necessary to read a bit more after seek and store this data for the fade. This could be done purely automatically in the class.

So thinking about this, I can say the hard part is definitely stopping, not starting. I don't think that your point 2 would work very well and this may well be the cause for still existing clicks and pops in your prototype.