dew.data.processors
Turns raw audio into the arrays a batch carries.
This runs inside grain’s workers, so the device only ever sees ready tensors.
transformers is imported when a step runs, not on import. Captions are
tokenized by the condition encoder that reads them (dew.inputs).
| Name | Summary |
|---|---|
Sampled | States the sampling rate a feature extractor expects its audio at. |
FeatureExtractor | A Hugging Face audio feature extractor, as AutoFeatureExtractor loads one: it turns waveforms into the arrays its model reads, the first of which it names first. |
sampling_rate | The rate a feature extractor reads its audio at: the one it states (Sampled), or 16 kHz. |
LogMel | A waveform’s log mel frames [frames, n_mels] float32, from transformers’ shared STFT and mel filters (audio_utils, which its feature extractors compute with): centred frames of a reflect-padded waveform under a periodic Hann window, HTK mel filters without a norm, the magnitude to power, floored at floor before the natural log. |
Sampled
Section titled “Sampled”class Sampled(Protocol)States the sampling rate a feature extractor expects its audio at.
Whisper’s and wav2vec2’s extractors do. One that states none is read at 16 kHz, the rate every released speech model here is trained on.
FeatureExtractor
Section titled “FeatureExtractor”class FeatureExtractor(Protocol)A Hugging Face audio feature extractor, as AutoFeatureExtractor loads one: it turns
waveforms into the arrays its model reads, the first of which it names first.
sampling_rate
Section titled “sampling_rate”def sampling_rate(extractor: FeatureExtractor) -> intThe rate a feature extractor reads its audio at: the one it states (Sampled), or 16 kHz.
LogMel
Section titled “LogMel”class LogMel( sampling_rate: int, n_fft: int, hop_length: int, win_length: int, n_mels: int, f_min: float = 0.0, f_max: float | None = None, power: float = 1.0, floor: float = 1e-07,)A waveform’s log mel frames [frames, n_mels] float32, from transformers’ shared STFT and mel
filters (audio_utils, which its feature extractors compute with): centred frames of a
reflect-padded waveform under a periodic Hann window, HTK mel filters without a norm, the
magnitude to power, floored at floor before the natural log. That is torchaudio’s
MelSpectrogram at its defaults and power, as MiMo-V2.6’s audio tokenizer reads it at 24 kHz and
128 mels (MiMo-Audio-Tokenizer’s mel_spectrogram, MiMoAudioTokenizer.mel).