Add SoX as method in pitch_shift

[Laubeee]

You point to SoX (and others) in your docs - why not have them implemented with some checks whether SoX (lib) and sox (pip) are installed, which the user would have to do manually to use them. See also https://github.com/marl/pysox

Currently what I do is I copied your PitchShift class and adapted it and then use that. The results are faster and tend to be better, especially for singing (and could be even faster using quick=True).

import random
import numpy as np
from numpy.typing import NDArray

import sox

from audiomentations.core.transforms_interface import BaseWaveformTransform


class SoxPitchShift(BaseWaveformTransform):
    """Pitch shift the sound up or down without changing the tempo"""

    supports_multichannel = True

    def __init__(self, min_semitones: float = -4.0, max_semitones: float = 4.0, p: float = 0.5, quick: bool = False):
        """
        :param min_semitones: Minimum semitones to shift. A negative number means shift down.
        :param max_semitones: Maximum semitones to shift. A positive number means shift up.
        :param p: The probability of applying this transform
        """
        super().__init__(p)
        if min_semitones < -24:
            raise ValueError("min_semitones must be >= -24")
        if max_semitones > 24:
            raise ValueError("max_semitones must be <= 24")
        if min_semitones > max_semitones:
            raise ValueError("min_semitones must not be greater than max_semitones")
        self.min_semitones = min_semitones
        self.max_semitones = max_semitones
        self.quick = quick

    def randomize_parameters(self, samples: NDArray[np.float32], sample_rate: int):
        super().randomize_parameters(samples, sample_rate)
        if self.parameters["should_apply"]:
            self.parameters["num_semitones"] = random.uniform(
                self.min_semitones, self.max_semitones
            )

    def apply(self, samples: NDArray[np.float32], sample_rate: int) -> NDArray[np.float32]:
        tfm = sox.Transformer()
        tfm.pitch(self.parameters["num_semitones"], quick=self.quick)
        return tfm.build_array(input_array=samples, sample_rate_in=sample_rate)

I think it'd be quite easy to implement two additional methods sox and sox_quick with this.

[iver56]

Neat! Thanks for sharing. How much faster is sox than the other alternatives? Can you share an audio example (e.g. singing, as you mentioned) that demonstrates how much better it sounds with sox compared to signalsmith-stretch?

One of the reasons sox was not considered as audiomentations dependency earlier was the desire to have first-class Windows compatibility (alongside Mac and Linux). In earlier years, sox had limited or spotty Windows compatibility. Has the situation changed recently?

[Laubeee]

Here's a short example https://drive.google.com/drive/folders/1yjCUhFGL4clnDyXBE6v6JHybeJE8qqzK?usp=sharing

Speedup: I guess that is quite HW-specific. It seems to be ~1.5x faster on my machine with quick=False and will soon test it on a GPU cluster in the context of a larger training.

I am actually on windows. But you are right, both SoX and sox/pysox do not really mention to officially support windows. So far it seems to work well for me but I haven't tested a lot, just pitch shift and resampling (via soxr). However, given it can be implemented as an optional dependency via conditional import, I think that'd be okay?

One thing I noted is that SoX pitch shifting does not seem cope well if the samples are malformed, i.e. I had a float32 array that was an average of two int16 channels (making it mono) and because my model anyway normalizes it I didn't bother dividing by int16.max, and pitch-shifting with audiomentations had no issues with that but sox apparently does not seem to produce useful output. I didn't dig into it deeper though.

[Laubeee]

Some additional details

• I noticed that sox doesn't seem to guarantee that the resulting sample count will remain exactly the same, so I added a check and either pad with a 0 or cut off the additional sample, since this will break my batch processing. Running a few thousand random shifts I never experienced more than one missing or additional sample.

def apply(self, samples: NDArray[np.float32], sample_rate: int) -> NDArray[np.float32]:
        tfm = sox.Transformer()
        tfm.pitch(self.parameters["num_semitones"], quick=self.quick)
        shifted = tfm.build_array(input_array=samples, sample_rate_in=sample_rate)
        
        # Pad or cut samples to match original length. To our knowledge the maximum difference is 1.
        if shifted.shape[0] < samples.shape[0]:
            return np.pad(shifted, (0, samples.shape[0] - shifted.shape[0]))
        return shifted[:samples.shape[0]]

• According to pysox the WSOLA algorithm is used.
• The quick option passes -q which is described as:

By default, linear searches are used to find the best overlapping points; if the optional -q parameter is given, tree searches are used instead, giving a quicker, but possibly lower quality, result

[Laubeee]

I'm realizing I was comparing to the old librosa vocoder. The newer sounds much better - I can't make out audible differences between sox and signalsmith_stretch.
A 10 minute file on my machine is now shifted in 2.1s (~2.7-2.8s before), whereas its 1.7s with sox. So it seems sox is still a bit faster, as for quality perhaps a more quantitative and/or systematic comparison is needed to make proper statement.