audio effectsprocessing · 2019. 2. 15. · impression of space (echo, reverb) increasing perceived...

1

Audio Effects ProcessingVesa Välimäki, Fabián Esqueda & Benoit Alary

ELEC-E5620 Audio Signal Processing

15.02.2019

DEMO:Pink Noise GeneratorsEero Lehtimäki & Uljas Pulkkis

2

2

Course Schedule in 2019 (Periods III, IV)

Välimäki, Parker, Esqueda & Alary15.2.2019

0. General issues (Vesa & Benoit) 11.1.20191. History and future of audio DSP (Vesa) 18.1.20192. Digital filters in audio (Vesa) 25.1.20193. Audio filter design (Vesa) 1.2.20194. Analysis of audio signals (Vesa) 8.2.20195. Audio effects processing (Vesa) 15.2.2019* No lecture (Evaluation week for Period III) 22.2.20196. Synthesis of audio signals (Fabian) 1.3.20197. Reverberation and 3-D sound (Benoit) 8.3.20198. Physics-based sound synthesis (Vesa) 15.3.20199. Sampling rate conversion (Vesa) 22.3.201910. Audio coding (Vesa) 29.3.2019

3

Outline• Echo / Delay• Flanging and Phasing• Chorus • Pitch shifting / Time stretching• Dynamic processing (compression & expansion)• Other effects

Demo• Beat-Aligning Looper• Pink Noise Generators


4

3

What is an audio effect? Any kind of audio signal processing applied to a

captured or synthesized sound for creative purposes

Possible purposes: Impression of space (echo, reverb) Increasing perceived size of a sound (chorus) Introducing movement into a static sound (flanging, phasing) Altering timbre (distortion) Altering dynamics (compression, limiting)

5

Echo/Delay One of the simplest and earliest audio effects

Initially they were made using tape loops Digital version very simple

Delay line with feedback Filtering or distortion can be added to the feedback loop Extra taps can be added for more complex pattern Real-time implementation using “circular buffer”.*

Sound example taken from: http://en.wikipedia.org/wiki/Delay_%28audio_effect%29*Good reference: The Audio Programming Book by R. Boulanger & V. Lazzarini


4

Bucket-Brigade Devices (BBDs)• Tape machines are expensive! • BBDs are discrete-time analog delay lines• Invented by F. Sangster and K. Teer at the Phillips Research Labs in 1968.• Input signal is sampled in time and passed through a series of capacitors

and switches.• Charge in each capacitor is passed to subsequent stage at a rate determined

by clock.

http://www.electrosmash.com/mn3007-bucket-brigade-devices


7

BBDs (cont’d)• CLK 1 and 2 in anti-phase configuration.• BBD flangers typically have a single 1024-stage unit.• Number of stages fixed. Delay length determined by clock’s rate.


8

5

BBDs (cont’d)• Although analog, BBD delay samples input signal and we must

adhere to Sampling Theorem• Appropriate anti-aliasing and anti-imaging filters required at input

and output, respectively.• Signal-to-noise ratio (SNR) is typically poor.• To ameliorate this, BBD is preceded by compressor and succeeded

by expander (compander).• Very smart but not so intuitive design!

http://ant-s4.unibw-hamburg.de/dafx/paper-archive/2005/P_155.pdf

BBD Flanger


9

Flanging Invented by Les Paul (1915-2009) in 1945, but the name

came from John Lennon in 1966 (http://en.wikipedia.org/wiki/Flanging) Original analog method for flanging

Copy the same sound on two open-reel tapes Play the 2 tapes on 2 synchronized tape machines Touch the flange of one tape reel to slow it down Get a nice “wooshing” phase-cancellation effect


10

6

Flanging: Analog-Era Realization

http://www.audiotechnology.com/tape-flanging-in-the-new-world/


11

Flanging Sounds Familiar

Many everyday cases C. Huygens (1693): the

sound of a fountain has a pitch when it reflects from a staircase

Moving and hissing sound source (or listener moving)

Jet airplane flying over a city Direct sound and its echo

Time-varying delay


12

7

Digital Flanger – Naive Version

A copy of the signal is fed through a variable digital delay line and added to the original

Produces a time-varying comb filter Magnitude response contains many uniformly spaced, moving

notches


13

Digital Flanger – Naive Version with LFO

Delay-line length is modulated with a Low Frequency Oscillator (LFO) Slow modulation frequency, approx. 0.1 Hz – 10 Hz

Pink noise Pink noise

E-gtr examples by Timo Hiekkanen and Tuukka Lyly, TKK, 2007

E-Guitar

Drums Drums

E-Guitar


14

8

Digital Flanger – Thru Zero

Problem with naive implementation Dry and delayed signal never coincide exactly

Solution: Add a static delay l to the ‘undelayed’ path, which is about half of the max value of m


15

Interpolated Variable Delay Line

In flanging, the delay-line length must vary smoothly to avoid discontinuities and clicks Otherwise “zipper noise” is produced

A fractional delay is needed Usually an FIR interpolation filter

z-1x(n)

h(0) h(1)

y(n)

z-1 z-1

h(N)h(2) ...


16

9

Delay Line with Linear Interpolation

• For digital audio effects, linear interpolation may be sufficiently good– The two-tap FIR is a mild lowpass filter, which varies with d

dd1

)(nx

)(ˆ)( dMnxny

1zMz


17

Flanging – The Movie

Flanging - No Interpolation(notches move in steps)

Flanging - Linear Interpolation(smooth sliding with filtering)


18

10

Phasing Allpass-filtered signal is added to the original signal

Originally an analog electronic version of flanging (“a poor man’s flanger”)

Usually a series of allpass filters Each allpass filter is of low order, e.g., first or second order Phase shift of each allpass is modulated with LFO


19

Phasing - Notches Notches are generated at frequencies where the phase

response of filter chain is multiple of –π (or –180˚) For example, four 2nd-order allpass filters in cascade → 4 notches Change of coefficients moves the notches

Magnitude response of the overall system

Frequency

Phase response of the allpass chain

Frequency


20

11

The Allpass Filter RevisitedFilters• The z-domain transfer function of a digital allpass filter is given by

• Parameter a1 determines break frequency.

• The phase response of a single allpass and several cascaded units is then:


One allpass filter Several allpass filters

21

Phaser with 10 Allpass Filters• Topology similar to that of MXR Phase 100 Pedal (5 notches, i.e. 10 filters).

• DC Blocker at the input.

• Naive approach; parameter a1 fully modulated. Not so useful!

a1 = 0.1 a1 = 0.9

100 1k 10k 20kFrequency (Hz)

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

Mag

nitu

de (d

B)


-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

Mag

nitu

de (d

B)


22

12

Phaser Measurements• Time-varying behavior of pedal can be measured using an allpass chirp train

• Modulation patterns and notch locations can be extracted from measurements

• Case Study: Fame Sweet Tone Phaser (MXR 100 clone)

R. Kiiski, F. Esqueda and V. Välimäki, “Time-variantgray-box modeling of a phaser pedal”, DAFx-16.


23

Gray-Box Phaser Model• Same topology as previous example.

• Measurement-based, modulation of a1 restricted to frequencies of interest.

• Measurement also exhibited LFO waveform.

Sound examples by Ricardo Falcón and Aleksi Myöhänen, ASP 2017

Original100 1k 10k 20k

Frequency (Hz)

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

Mag

nitu

de (d

B)


-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

Mag

nitu

de (d

B)

a1 = –0.8 a1 = –0.4


24

13

Gray-Box Phaser Model


R. Kiiski, F. Esqueda and V. Välimäki, “Time-variantgray-box modeling of a phaser pedal”, DAFx-16.

25

Flanging vs. Phasing

Flanging Variable time-delay Short delay (< 10ms) Hundreds of notches Notches harmonically related Number of notches is time-

varying

Phasing Variable phase shift Very short delay Few notches (1-10) Notches not harmonic Notches can be individually

modulated Number of notches is fixed


26

14

Flanging or Phasing?

Flanger Phaser Phaser

Flanger Phaser Flanger


27

DEMO:Beat-Aligning LooperJon & Petteri

28

15

Chorus The goal: make one source sound like many sources

Useful for vocals and electrical instrument sounds Very similar structure to flanger and echo effects


29

Famous Chorus Examples

Chorus Unit: EHX Small Clone

Chorus Unit: Boss Chorus CE-2


30

16

Chorus Implementations (1) One choice: multiple feedforward paths with modulated

delay-lines (Orfanidis, 1996) Modulation waveforms may be sinewaves or lowpass-filtered

noise (“random walk” signal)


31

Chorus Implementations (2) “Industry standard” (Dattorro, 1997)


32

17

Chorus Implementations (3) The “industry standard” structure is cheap to implement

Use one for each stereo channel, or more Generalized allpass-comb filter

Becomes an allpass filter, when delays and coefficients are equal

Negative feedback is used for flattening the spectrum (“white chorus”)

For clean effect, allpass fractional delay filter must be used for the variable delay, not linear interpolation


33

Chorus Implementations (4) The industry standard structure can produce many effects

Vibrato: blend = 0, feedback gain = 0, large modulation depth Flanger: small delay (< 10 ms) Doubling (double tracking) when blend = feedforward gain, feedback

gain = 0, large delay (> 10 ms), with random modulation Echo: feedback or feedforward gain is zero; a lowpass filter is inserted

in the non-zero path; delay is large (> 50 ms) Stereo effects: modulating sine waves out of phase or in quadrature

for the 2 channels

Crazy cartoon-like effects!Great reference for this:

Udo Zölzer’s DAFX Book.


34

18

Chorus vs. Flanging Flanging

Small time delays (5ms) Separate signals perceived Min. delay approx 5 ms Notches usually

undesirable


35

Pitch Shifting

• Multiply all the frequencies with a constant value• Can be done with resampling (reading a buffer faster),

but it changes the playback speed as well– 1.5 x

36

19

Time Stretching using Phase Vocoder• Use the short-time Fourier transform• Reconstruct the signal with a larger

hop size• Makes a good pitch shifter

– Pitch up and time stretch to preserve time

• What happens to the phase?

https://cycling74.com/tutorials/the-phase-vocoder-%E2%80%93-part-i

38

Time Stretching

• Stretching the duration of a signal while retaining the pitch

39

20

Sound microscope

Damskägg, E.-P.; Välimäki, V. Audio Time Stretching Using FuzzyClassification of Spectral Bins. Appl. Sci.2017, 7, 1293.

• Decomposes sound into threeclasses:

1. Tonal

2. Transients

3. Noise

40

Sound Microscope

• Tonal layer:– Must restore

phase continuity

41

21

Sound Microscope

• Transient layer:– Must avoid repeating

or smearing

42

Sound Microscope

• Noise layer:– Must avoid

windowing artefacts

43

22

Dynamic range processing

44

Dynamics Processing Compressor reduces the dynamic range of an audio signal

Pop music, radio and TV broadcasting, live PA systems Special case: Limiter, which saturates at a certain max. level

Expander increases the dynamic range For example to reduce background noise in silent passages Special case: Gate, which mutes the signal below a threshold

Compressor

x0 x (dB)

Limitery0

y (dB)

Expander

x (dB)

Gate

y0

y (dB)

x0


45

23

Time-Domain View of Compression When signal level gets high, the gain is reduced

automatically 2 parameters: attack time TA and release time TR


46

Digital Feed-forward Compressor Signal level detector

Temporal envelope: average power over a short time interval Gain must not be changed instantaneously (aliasing can occur)

Gain computer Gain G is adjusted based on signal level (power)

Level detector


47

24

Level Detector Full-wave rectification (abs) and temporal averaging

For example, a leaky integratory(n) = (1 – a1) |x(n)| + a1 y(n – 1)where a1 = 1 – ε (such as a1 = 0.99)


48

Compression of Musical Signals After compressing the signal, the overall gain is

increased by applying make-up gain Many potential uses

Maximizing loudness, like in the recent “loudness war” Controlling transients/emphasizing decay


49

25

Multiband Compression• Split audio signal into sub-bands for improved performance.• Avoid “pumping” effect.

Demo by Tae Ho Kim and Elias Raninen, ASP-2016

comp

comp

comp

comp


50

Gating and Limiting of Musical Signals Limiting is an extreme version of a compressor

Signal value is not allowed to exceed a certain level Mainly used to maximize loudness

Gating mutes audio when below a certain threshold amplitude Used to e.g. remove background noise between notes Can also be used creatively (e.g. gate chord sound based on

amplitude of high-hats)


51

26

Gating and Limiting Examples

The outcome can sound very loud but lacking natural dynamics Original music file Gated (< -20 dB) Hard limited (> -10 dB) Gated (< -40 dB) & limited (> -20dB)

Audio examples by Mika Luukkanen, TKK, 2004


52

Audio Antiquing

• Render a new recording to sound aged For example, imitate the lo-fi sound of LP,

gramophone, or phonograph recordings• Simulate degradations with signal processing techniques

(González, thesis 2007; Välimäki et al., JAES 2008) Local degradations: clicks and thumps (low-frequency pulses) Global degradations: hiss, wow, distortion, limited dynamic range, frequency band limitations, resonances

© 2013 Vesa Välimäki

53

27

Audio Antiquing Example

1. CD (original)2. Phonograph cylinder (new – best quality)3. Phonograph cylinder (worn)


54

Phonograph and Gramophone Simulation


55

• Insert local and global disturbances Local disturbances: clicks, thumps, tracking errors (editing) Global (all samples affected): background noise, pitch variation,

distortion, coloration (e.g. bandlimiting, acoustic horn modeling), dynamic range limitation

28

Other Effects (not exhaustive) Vocoder

Technique from speech coding misused for effect. Uses large banks of bandpass filters to analyse the spectral envelope of a

sound and apply it to another sound Sounds like robot voice

Frequency shifting (do not confuse with pitch-shifting) Shifts all frequencies by a fixed amount additively, hence ruining harmonic

relationships Can also be used to produce very rich chorus sounds

Autotune (1998-) Highly popular pitch corrector with quantization (“Cher”) Uses LPC and interpolation, or spectral techniques


56

Other Effects (not exhaustive) Wah-wah

Center frequency of a resonator is modulated with envelope follower, control pedal, or LFO

Filtering (telephone sound, resonances etc.) Enhancer

Harmonic distortion of only high frequencies to increase brightness

Distortion and tube-amp modeling effects Reverberations effects Spatial audio effects Stereo expansion, 3-D sound etc. More?


57

29


Resources

58

Literature – Effects E.-P.Damskägg and V. Välimäki, ”Audio time stretching using fuzzy classification

of spectral bins,” Applied Sciences, 2017, 7, 1293. J. Dattorro, “Effect design—part 2: Delay line modulation and chorus,” J. Audio

Eng. Soc., vol. 45, no. 10, pp. 764–788, Oct. 1997. Available online at:http://www.stanford.edu/~dattorro/research.html

R. Dobson, A Dictionary of Electronic & Computer Music Technology. OxfordUniversity Press, 1992.

W. M. Hartmann, “Flanging and phasing,” J. Audio Eng. Soc., vol. 26, no. 6, pp.439–443, June 1978.

S. J. Orfanidis, Introduction to Signal Processing. Prentice-Hall, 1996. Section8.2 (“Digital Audio Effects”), pp. 355-388.

G. D. White, The Audio Dictionary (2nd ed.). University of Washington Press,1991.

D. Giannoulis, M. Massberg and J. D. Reiss, ”Digital dynamic range compressordesign – A tutorial and analysis”, J. Audio Eng. Soc., vol. 60, no. 6, June 2012,pp. 399–408


59

30

Literature – Effects (2) • R. Kiiski, F. Esqueda & V. Välimäki, “Time-variant gray-box modeling of a phaser

pedal,” in Proc. 19th Int. Conf. Digital Audio Effects (DAFx-16), pp. 31–38, Brno,Czech Republic, Sept. 2016.

• V. Välimäki, S. González, J. Parviainen, and O. Kimmelma, ”Digital audioantiquing – Signal processing methods for imitating the sound quality ofhistorical recordings,” Journal of the Audio Engineering Society, vol. 56, no. 3,pp. 115–139, March 2008.


60

audio effectsprocessing · 2019. 2. 15. · impression of space (echo, reverb) increasing perceived...

Documents