Voice Distortion Tool

Using the functions to distort the voice

There are five main functions that could be called to process a certain audio.

Requirements of input

The input format should be .wav. For .m4a or .mp4 files, it's required to run the preprocessing function in advance to transform it into .wav format.

There is no need to convert inputs if you are using run_me file to produce all the graphs for analysis, since we have already included all the required input files in \inputfiles with .wav format.

Introduction of functions

For each function, it takes one parameter as input (except Gender_style_conversion(), specified below), that is to say, the path of the audio file. The output is then written in \output path of this folder with the name of [function name].wav.

If you provide empty parameter, for example, filtering(), it will fallback to process the sample audio of our choice.

Effect	Fallback sample audio to process	Default style
Bandpass filter	Soft music	Drop < 300 Hz $\land$ > 3000 Hz
Equalizer	Soft music	Low band 1.5x, high band 0.6x
Chorus	Cappella female solo	See description below
Robotic distortion	Voice of speaking	See description below
Gender style conversion	Voice of speaking	Feminine effect

filtering()

This is a direct DFT bandpass filter.

This function takes the DFT of the entire signal at once (fft), builds a binary frequency-domain mask that is 1 between 300 Hz and 3000 Hz and 0 everywhere else, multiplies the spectrum by that mask (zeroing out all energy outside the passband), then recovers the time-domain signal with ifft. This is the most direct application of the convolution theorem: multiplication in the frequency domain equals convolution with a rectangular filter in the time domain.

graphicEqualizer()

Processes the signal in overlapping 1024-sample frames (75% overlap, Hamming window). For each frame it computes the FFT, then applies three different scalar gains to three frequency regions defined as index fractions of N: low band × 1.5, mid band × 1.0, high band × 0.6. The scaled spectrum is sent through ifft and the frames are reassembled via overlap-add.

roboticdistortion()

Same 1024-sample / 75%-overlap STFT structure as the equalizer. For each frame it computes the FFT, circularly shifts the entire spectrum by a small number of bins, and then add to all phases a random phase jittering or resetting all phases to zero (uncomment one of the lines in the function to switch between them). The magnitude envelope is preserved while all phase information is destroyed or jittered. After ifft this produces the characteristic flat, metallic robotic sound.

chorus()

This main function calls a function which is a realization of phase vocoder: pitchShift(x, fs, semitones, windowLength, overlap). It executes the following steps:

• stft decomposes the signal into frames, giving a time-frequency grid of complex values
• For each frame and bin, the instantaneous frequency deviation is estimated from the phase difference between consecutive frames.
• To shift pitch by a ratio $\alpha = 2^{(semitones/12)}$, the synthesis phase is accumulated at a scaled rate: $\phi_{syn} = \phi_{syn,prev} + \alpha·\delta \phi + \frac{2\pi·k·hop·(\alpha−1)}{fs}$. This stretches or compresses the instantaneous frequency of every bin by $\alpha$ without changing the playback speed
• istft reconstructs the time-domain signal from the modified phases and original magnitudes

chorus.m calls pitchShift eight times with different semitone values (−9, −8, −0.22, −0.10, 0, +0.10, +0.22, +0.35), delays each voice, and mixes them. It also runs a final DFT-domain EQ (makeWarmAndThick) that boosts 120–350 Hz and 350–800 Hz and rolls off above 3500 Hz, using a single whole-signal fft/ifft with a smooth frequency-dependent gain curve.

Gender_style_conversion()

This function takes two variables instead of one: the path to the file to be processed, and the target effect (masculine or feminine).

Call it as Gender_style_conversion(inputFile, targetStyle).

The core of this function is similar to chorus().

This function converts vocal character between feminine and masculine styles through three successive DFT-based processing steps.

• Pitch shift with pitchShift function. Instantaneous frequency per bin is tracked from inter-frame phase differences and accumulated at a scaled rate, then the signal is resampled back to original length. Applied in small stages (≤1.5 semitones each) to avoid artefacts
• Formant scaling. STFT magnitude per frame is split into a smooth spectral envelope movmean and harmonic detail. The envelope is warped by interpolating at scaled bin indices (simulating vocal tract length change), then recombined with the detail before ISTFT
• Voice EQ. Single whole-signal fft, frequency-dependent scalar gain applied per bin (boost body/warmth for masculine, boost presence for feminine), then ifft.

Main conclusions from spectrum analysis

Using the Visualize_comparison() function, we compute and visualize the comparison of average magnitude every 10 Hz, between original signals and processed signals.

General conclusions from the graph:

• The energy are mainly concentrated between 300-3000 Hz, which is typical for vocal and music.
• The 10 Hz bin-averaging makes the graph not spiky and readable.
• For the domain that we do not modify the frequency distribution, the original signal average overlaps with processed signal average, confirming the retaining of original signal.

Conclusions per effect:

• For filter(), there is a clear cut at 300 Hz and 3000 Hz, where magnitude of original signal presents and of processed signal is missing, confirming our band pass filtering is effective.
• For equalizer(), the original more concentrated magnitude distribution becomes higher in lower frequencies and lower in higher frequencies, confirming the execution of gain among frequencies is effective.
• For Gender_style_conversion(), we inspect the general spectrum shifted to the left for feminine to masculine conversion and to the right for masculine to feminine, confirming our intention of moving the magnitude as a whole on the frequency domain worked out.
• For chorus(), the signal before and after processing overlap largely, which is expected since we did not modify the distribution of magnitude on frequency domain.
• For robotic(), the magnitude spectrum is redistributed into sharp harmonic spikes (a comb-like structure), but the magnitude plot alone understates the effect. The robotic character comes mainly from zeroing the phase, which a magnitude-only DFT cannot display.

Robust analysis

Noise robustness

The idea

We investigated the effect of different levels of noise on the output of our functions.

To produce graphs for analysis, for each effect we:

• Process the clean signal once to get a baseline output.
• Add white Gaussian noise to the input at four signal-to-noise ratios snrLevels = [30, 20, 10, 0] dB, so the noise grows from barely-there up to as loud as the signal itself. Noise power is set from the measured signal power (addNoise).
• Run the same effect on each noisy version and overlay all the output magnitude spectra on one plot.

Reading the plot

Each figure overlays the output spectrum of the clean run (black) with the outputs at SNR_in = 30, 20, 10, 0 dB. The closer a coloured curve stays to the black one, the less the added noise survived processing. Where curves separate in the regions the effect is supposed to suppress, that gap shows noise leaking through.

What the results show

Effect	Characteristics	Conclusions
Chorus	The variance is spreaded evenly among the frequency domain. The lines of different colors overlap largely.	Robustness to noise of chorus effect is high in general.
Robotic	The lines overlap largely at low frequencies, but varied largely in the higher frequencies.	Robustness to noise is fairly good at low frequencies but not good at high frequencies.
Equalizer	The lines overlap largely at low frequencies, but varied largely in the higher frequencies.	Robustness to noise is fairly good at low frequencies but not good at high frequencies.
Filter	The variance is spreaded evenly among the frequency domain. The lines of different colors overlap largely.	Robustness to noise of chorus effect is high in general.
Gender(M-F)	The lines overlap largely at low frequencies, but varied largely in the higher frequencies.	Robustness to noise is fairly good at low frequencies but not good at high frequencies.
Gender(F-M)	The lines varied largely everywhere.	Robustness to noise is not good among the frequency domain.

Note that these results are produced with 30-sec recordings. To ensure the run_me file could be executed within a short time, we cut them into 5 second snippets. However, the shorter version file produces graphs that suggest low robustness. In order to reproduce the graphs in our video, add _long to each of the file name.

Which conclusions became invalid?

• Energy concerntration. With a SNR near 0, the spectrum is flattened into a broadband.

Other conclusions remain valid.

Resolution

The idea

Every effect here is built on the DFT, so a natural question is: does the answer depend on the sampling rate we happen to run at? A digital signal sampled at Fs can only represent frequencies up to the Nyquist limit Fs/2.

To check the robustness against downsampling, for each effect we:

• Take the native signal and downsample it by an integer factor M with resample (which applies a polyphase anti-aliasing low-pass first, so we don't fold high frequencies back into the band).
• Run the exact same effect at the reduced rate Fs/M.
• Compute an averaged magnitude spectrum over the whole output, so the comparison reflects the entire signal rather than one arbitrary frame.
• Compare against the full-rate (M=1) result.

We keep the factor list short (factors = [1, 4]) in order to reduce run time.

Reading the two subplots

• Top: overlaid spectum. The original (M=1) and the downsampled (M=4) magnitude spectra on the same axes. Each curve simply stops at its own Nyquist: the original reaches Fs/2, the downsampled one ends at Fs/(2M). Where the two overlap, they should lie almost on top of each other.
• Bottom: deviation. The downsampled spectrum minus the original (interpolated onto the same grid), in dB, over the shared band.

What the results show

Because the existence of alising, the magnitude of frequencies above Nyquist frequency are added to the lower frequencies. These are the same for every effect.

Which conclusions became invalid?

• Clear edge cutting at 300 and 3000 Hz for filtering() function does not remain valid, because it requires a high sampling rate to keep it accurate.

Web app rewritten in JavaScript

We have rewritten the sound effectors in JavaScript, embedded to a HTML website, with the assitance from generative AI models.

It could be run from opening the file voice_distortion_tool_vn.html.