Applied ACOI/st ics 30 (1990) 219-245

An Investigation of Vocal Vibrato for Synthesis t

Robert Maher* & James Beauchamp

Computer Music Project, School or Music. Univcrsitv or Illinois at Urbana-Champaign,2136 vlusic Bldg. 1114 W. Nevada. Urbana. Illinois 61XOI. USA

ABSTRACT

High-fidclitv singing svnthcsis require: careful consideration of the propertiesand character oinatural ribrato. The research rcporird here craluutcs sonicor111C characteristics otiocal vibrato and presents a nCl\' panncd-wavetablcsvnthcsis 1I1C1!zoi!. The singing lone 'a' lias recorded digitallv and anolvzcdforbass, tenor. a!10and soprano 1'Oin'.I, providing timc-iariant measurements ofWII[ili1 II de , frequency and phase [or each or the ioicc partials. Formant'tracings' provide a noiel ntethodjor cxamining roculspcctra during iihrato.The significanc« or ribrato II 'aiefo I'll1 parameters and the role of spectrummodulation (due 10 partial amplitudcIluct uationst dl/ring th« i.ihrato cvcl«lI'ilS illl'n1iga1ed jn rcsvntliesis of the singing 10llCI FOIII modified anal vsisdata. Informal listening 10 cxaniplcs produced b v the panncd-warctahlc,11'11 thesis mode! indica1cd Iha 1 inclusion or typica! random flue tua Iions ofvibrato rate. vibrato depth and nOlllina!.llIlIgjiTijlleniT rcsultcd ill no qualitvpreference OITr example: with constant ialues. lnclusion of vibrato-inducedspectrum modulation rcsultcr! ill a substantial intprorctnent ot.er examplesbaring 1'011.1/([1/1 spectra.

1 INTRODUCTION

Vocal vibrato is an essential aspect of trained singing. Vibrato is employedfor emphasis and timbral variety in many singing situations and provides avaluable added dimension for expressive control. Indeed, the degree of

~ (PACS) Subject Classification numbers: 43.75.Rs 43.75.Hc 4375.Wx.* Present address: Department or Electrical Engineering. Lnivcrsiiy of Nebraska, Lincoln,Nebraska 6X5XX-0511. LSA.

219clpplied Acoustics 0003-6X2X.90 503·50 ( ISlSlO Elsevier SCience Publishers Ltd. England.Printed in Great Britain

220 Roher! Muhcr. Lames Bcuuchamp

control over vibrato may be a good indication of a singer's skill andflexibility, Other observations suggest that vibrato may help mask any smallunintentional errors in singing pitch, I The importance of vibrato has manyimplications for singing synthesis methods: vibrato must be treated withcare if a convincing result is desired.

Our work on vocal vibrato began as an ofTshoot of a more generalinvestigation of time-variant spectral analysis methods for musicalinstrument sounds, Most techniques employed for time-variant analysis usea bank of fixed-frequency bandpass analysis filters to separate an inputsignal into its presumably harmonic partials. The presence of vibrato in theanalyzed tone may cause unwanted 'cross-talk' between the analysis filters.i.e. a partial may appear in the passband of two or more analysis filtersduring one vibrato cycle (see Fig. 1). Additional problems appear if a validestimate of the time-varying amplitude. phase and frequency (phasederivative) of each partial is desired. The fundamental time-bandwidthproduct constraining the minimum analysis filter bandwidth to beinversely proportional to the observation time interval must also beconfronted.

In an attempt to circumvent some of the problems inherent in the fixed­band approach we have adopted a time-variant analysis procedure based ona speech analysis synthesis method of McAulay and Quatieri.? This methodhas already been found to provide useful results for the analysis of musicalsignals. 3 '-+ The method assumes a priori that the input signal can herepresented adequately as the sum ofa finite number of possibly inharmonicsinusoids with appropriate time-varying amplitude and frequency (or phase)modulation. This model has proved to be accurate for representing most

Spectral Components Pass bands of Filter Bank

8 f1

7 f1

6 f1

f1

<lJD~

4 f 5 f1 1

Frequency

Fig. 1. A Iixcd-b.md harmonic analyzer employs a hank or hand pass filters centered at the

harmonic frcqucru ics. Vibrato causes thc spectral components to shirt hack and forth in

frequency, resulting in a misalignment ('cross-talk'j between the analysis filters.

All inrestigation otiocal iibrato [or .l1'11I171'.lil 221

Input signal

Block #1t tt Block #2 t

t Block #3 tetc.

For each Input block:

Window

Data Block

.r-:Zero-pad

and

FFT

log

magnitude

magnitude,

dB

Frequency

Fig. 2. Summary or the McAulav Oua ticri analysis procedure. The input signal is

segmented into overlapping unn lvxis blocks. or 'fru mcs' The log-magnitude spectrum ofeachblock is computed. and the location and height or spectral peaks arc identified.

musical sounds, Moreover, we have found it to he an improvement overfixed-hand methods for the analysis of vibrato tones since it is possible totrack changing frequencies. thereby avoiding the inter-hand cross-talkproblem.

In our version of the McAulay-Quatieri procedure (Fig, 2). the digitizedinput signal is divided into windowed. overlapping time segments or'frames', For each frame (zero-padded by a factor of two or more) wecompute the complex spectrum and the log-magnitude spectrum. fromwhich all spectral peaks are identified, The model assumes that each spectralpeak is due to the presence of an underlying sinusoidal partial at the locationof the peak, We then estimate the magnitude. frequency and phase of eachpartial from the log-magnitude and complex spectral data. The peaks aretracked from frame to frame and connected into frequency 'tracks', Because

222 Robert Maher, JOllies Brauchanu:

Peak tracking algorithm

~

- ---

.....2 3 4 5

..etc...

Analysis Frame Number

StructureNode:

AMPFREDPHASE

NEXT --..PREY --..FORWARD --..BACKWARD --.. Typical peak list structure

...etc...

Fig. 3. The peaks [rom one analysis frame arc connected with corresponding peaks inprevious and subsequent Ir.uncs. A data structure is constructed containing the amplitude.frequency and phase measured at each peak. and a xct 01' pointers to each adjacent clement

(linkcd-Iist l

only those partials whose amplitudes exceed a certain threshold are retained.the number of tracks present can vary considerably from frame to frame.Thus. frequency tracks are continually being born and dying as the spectralcontent of the input signal changes from instant to instant. The collection offrequency tracks provides a connected mesh of amplitude. frequency andphase estimates for each partial as a function of time (Fig. 3).

The corresponding synthesis method is essentially a sum-of-sinusoidsadditive synthesis procedure. but polynomial interpolation of the analyzedphase data is employed to provide a smoothly varying phase function fromframe to frame for each partial component. The technique is well suited forinharmonic sounds.v" We have obtained useful results for tones containingvibrato and even for analysis/synthesis of polyphonic music (Maher"]. Ourimplementation of the McAulay Quatieri procedure will be referred to asthe MQ technique for the remainder of this paper.

411 inrestieat unr oirocul rihmlo!o}'ITIII!lcsis 223

ResonantStructu re

Sound Output

Acoustic Pulse

Generator

----':~__M-Lungs

Esophagus

Soft palate _.-.-...... Nasal cavity/'.:=::::::::::::~:::- Mouth

Tongue ----\~~~e:--_=_:- TeethPharynx LipsEpiglottisVocal cords------

----­Diaphragm

(al (hi

Fig. 4. A schematic model for the vocal apparatus. (a) Cutallay diagram of the human voc.ilsystem (after Rossing et al.''). (h) Simple block diagram consisling of an acoustic pulse

generator (lungs and voc.il cord-: and a resonant structure tphuryn«. oral cax itv. ctc.l.

Several approaches to singing synthesis have been reported (cf Refs 1 and6--8). All of the methods attempt to capture the formant properties of thesinging voice. either by direct simulation of the human vocal apparatus or bymimicking the voice spectrum itself. In general. the human vocal system hasbeen treated as a multi-resonant structure (the pharynx and oral cavity)excited by an acoustic pulse generator (the vocal folds and breathingapparatus). as shown in Fig. 4. In this model the output of the vocal system ischaracterized by regions of spectral emphasis (formants) due to the implicitconvolution of the vocal fold signal with the resonant impulse response of

the vocal tract. Thus the goal has been to identify and match the salientspectral properties of the human singing voice.

In the case of vibrato most reports indicate. perhaps tacitly. that theresonant character of the vocal tract remains FIXED while the excitationfrom the vocal folds changes frequency in some quasi-sinusoidalmanner. 1

•tJ

.1

() If the amplitude of each partial in the source spectrum of thevocal folds is considered to be nearly constant throughout the vibrato cycle.the output spectrum of the singing voice will show frcq uency modulationdue to activity of the vocal folds and amplitude modulation due to thesource partials being swept back and forth through the vocal tractresonances. Although trained singers. particularly sopranos. are able to shirt

224 Roher! .Haher. JOllies Beauch amp

the spectral location of certain formants to coincide with a source partial, 11

it is reasonable to assume that formant positions remain fixed as partialfrequencies fluctuate according to a vibrato pattern, causing the amplitudesof the partials to also vary in time.

This vibrato-induced amplitude modulation of the partials leads to atime-varying magnitude spectrum, or 'spectrum modulation', occurring atthe vibrato rate. We did not know in advance whether spectrum modulationmight be a vital ingredient for a convincing synthesis, because the efTects ofnatural vibrato on timbre perception have not been evaluated bypsychoacoustic testing. 1

2 Therefore, it seemed pruden t to in vestiga te thisaspect of natural vibrato in our experimental synthesis models. Also, sinceprevious reports have stressed the importance of random variations invibrato. I. C we decided to include in our study an analysis of the deviationsfrom strict periodic behavior found in vocal vibrato waveforms.

The work reported here is a preliminary evaluation of various parametersof vibrato in singing. The results arc intended to guide further work in thisarea, particularly in economical synthesis techniques. In several respects theapproach and conclusions of this paper can be viewed as independentextensions and refinements of the work reported by Bennett.'

2 RESEARCH PLAN

Four singers (soprano. alto, tenor and bass) were recorded individually whilesinging the vowel 'a' at a comfortable output level and range of pitches. Allsingers had received some formal training. but none was a professionalvocalist. No special instructions were given to the singers. except to relax andsing naturally.

The recordings were made in a nonreverberant room with a high-qualityprofessional microphone. The soprano recordings were obtained using aSony rCM-50l ES digital tape system. while the alto, tenor and bassrecordings were obtained from a previous study. 1" Several example tonesfrom each singer were then rerecorded digitally at a 20 kHz sample rate usingthe Sound Conversion and Storage System (SCSS). built at the University orIllinois. 14 Once on the SCSS. the digitized tones were available as 'samplefiles' for processing and analysis using software on a general purpose digitalcomputer (an IBM Model 125 RT PC workstation).

The sample flies were analyzed first using the MQ procedure. Next. afundamental frequency tracking technique was employed to identify andextract those partials which were closest to being the harmonics of the signalwithin each frame. The frequency domain histogram method reported bySchroeder 1 S was chosen for fundamental freq uency tracking since the

All illres/igarioll o! ioia! ribruto lor svntlu-si-

method operates on a list of partial frequencies, automatically pro , idcd hythe MQ analysis procedure.

Since the fundamental Irequencys.tr) varied in time. we assumed that thefrequency of the kth partial would vary according to

(1 )

We should emphasize that this frequency extraction process docs notinherently restrict partials to he perfect harmonics. hut we expect they V\ ouldbe for most noripercussive musical instrument and vocal sounds. Anexample of the raw output of the MQ analysis procedure is shO\\ n in Fig.S(a). The quasi-harmonic form extracted after tracking the fundamental isgiven in Fig. SIb).

Finally, the quasi-harmonic voice partials were assembled as a data tileconsisting of the initial phase of each partial followed by the amplitude andfrequency change measurements obtained for each partial <It each ana lysi-,frame. A time domain signal could be resynthesized dircct lv from theoriginal quasi-harmonic data or from modified <Ina lysis d.ua.

We investigated the significance of various i imc-varving vibratoparameters by generating several synthetic tones based on the MQ a na lyxi-,data. A list of resynthesis alterations is given in Table 1. The MQ proceduresupplies information on the amplitude and frequency evolution of eachpartial, so it was convenient to perform independent modilications of thesedomains. For example, frequency variations could he omitted V\ hileretaining amplitude variations, and vice versa.

The importance of spectrum modulation due to vibrato was 111\ cstigatedby resynthesizing tones with (I) measured amplitude fl uctuat ions for eachpartial, but with constant partial frequencies replacing the measuredfrequency oscillations. and (2) measured frequency oscillations. hut withconstant (time average) partial amplitudes.

TABLE IResynthesis Alterations Applied to Anulvsi-, f)ala lo r luch SUI1!C V()\lc]

(ai No modifications (direct resynthesis Irom the 'V1Q al1ahSISI(h) Replacement oflrcqucncy vibr.uo ofeach partial with a 'L'omp()sik" I ibr.uo \lalelnrm(c) Additive rcsynt hcsis with fixed number ofquaci-h.umonic partials (partials hL't\leen I)

and 4 SOO JIz)(di Elimination of partial amplitude Iluctu.uions Isvnlhesls with Iivcd p~lrtl~Ii .unpliiudc-,

and measured individual partial vibr.uos:(e) Elimination of partial Frequency vibruiox (svni hcsi, \11th li\ed perkcl haIJl1llI1IL"

frequencies and rncusurcd modul.uion on each partial)In Combination of (d) and (e) (Ii\ed wuvctorm synthesisl(gi Combin.uion of (h) and (dl

III illu's/iga/ioll o] coca! ribrato [tir svnthcsi: 227

The effect of possible inharmonicity in vibrato tones was examined byreplacing the individual partial vibratos with a 'composite' frequencyvibrato waveform. The composite vibrato waveform was obtained byaveraging the frequency waveforms measured for each partial. Thecontribution of each partial to the composite vibrato was weighted by itsamplitude; thus, strong partials contributed more to the composite thanweak partials. For a given analysis frame. the composite frequency is givenby

I'

L ilkU~ - kf;Jk<I« + k_=:l __y, _

L {/kk 1

(2)

where

P = the number of partialsilk = measured amplitude of partial f..:

t. = measured frequency of partial k/0 = assumed fixed target frequency.

The results were examined via several graphic display routines and audioplayback. Evaluation was informal and intended to guide further research incomputer music synthesis. Emphasis was on determining guidelines forvibrato generation that would be appropriate for inclusion in varioussynthesis algorithms.

To test the utility of the measured parameters we developed a simplesoftware synthesis algorithm having a favorable tradeoff betweencomputational efficiency and synthesis quality.

3 RESLLTS AND DISCUSSION

We found direct resynthesis from the original MQ analysis data (where alltracks are retained and phase interpolation is used) to be of excellent quality.The results were nearly indistinguishable from the original recordings, basedon informal auditions by several critical listeners who were not biased byfamiliarity with the analysis method nor the research goals. Examples werealso generated from the analysis data by simple additive resynthesisemploying a fixed number of quasi-harmonic. time-varying partials. Whilewe found the initial attack quality to be inferior to the result obtained by thedirect resynthesis method. the simple resynthesis provided good results forthe sustained portions of the sung vowels.

The tones were further altered by removing partial amplitude fluctu­ations, frequency fluctuations (or both), and by replacing individual partial

228 Robert :'v1011<'1", }OIllCS Beauchanip

TABLE 2Rating of Resynthesis Examples in Order of Decreasing Fidelity

(I) Onginal recordingI::') Unaltered MQ synthesis(3) Simple additive synthesis of quasi-harmonic partials(4) Resynthesis of (3) with the composite frequency vibr.uo used on all partials(5) Resynthesis of (3) without amplitude fluctuations on the partiuls(6) Resynthesis of (41 without amplitude tluct u.uions on the partials(7) Resynthesis of (31 without trcqucucy vibr.u o(81 Resynthesis of (31 without frequency vibrato or amplitude fluctuations

frequencies with frequencies harmonically related to the compositefundamental frequency (defined by eqn (2)), We used informal listening torank the several resynthesis examples in order of decreasing fidelity, asshown in Table 2, The largest degradation in quality was noted betweenmethods 6 and 7, The differences between methods I, 2 and 3 were veryslight and methods 5 and 6 were essentially indistinguishable, However.methods 5 and 6 (constant amplitude partials) provided substantiallydifferent quality from the similar methods 3 and 4, which included amplitudefluctuations.

Table 3 summarizes several measured characteristics of vibrato for thevocal tones used in this investigation. We note the reasonable consistency ofthese parameters. Vibrato rate is most consistent, being between 5·0 and5·7 Hz. The alto tone had the smallest vibrato depth (± 2·5'Yt,) and thesmallest dB ripple over all partials. The ripple of the RMS amplitude (givenby the square root of the sum of the squares of the partial amplitudes) issubstantially smaller than the maximum for the individual partials becausethe maxima and minima ofditTerent partials are frequently out of phase andtend to compensate for one another.

3.1 Vibrato differences among partials

One aspect of interest to us was the possibility of time-varyinginharmonicity during the vibrato cycle. Due to the frequency-dependentphase response of the vocal tract. the partials of a singing voice couldundergo a frequency-dependent group delay (i.e, phase derivative) relative toone another. As the fundamental and other partial frequencies oscillateduring the vibrato cycle, they could be inharrnonic with respect to each otherin direct relation to the product of the vibrato rate and the difference of thegroup delays for each two frequencies compared. In fact. some of ouranalyses ofmusical instrument tones with vibrato (e.g. violin and oboe tones)have revealed significant momentary inharmonicities. On the other hand,

All llIl'i'sllg(/lloll of toca! IlhUI!O lor ITII I/WI II 229

TABLE 3Some Typical Measurements of Vibrato Characteristics in Sung Vowels. Vowel Sound 'a'

J'olee III/sica! tibrrn» tihrut«, .If(/I. rippl« Ifl/.I . rippl,:

pinh r(/Ie depl h o] au : purtiul of R .'vIS amplitude(H~I lill dHi lill dB!

-------- .._----

Soprano C;:::5 1554-4 Hz) 57 j~ .5 (j,o 10 .\

Alto D4 129371171 5h ~ :!-)oo 4Fcnor G.I (llJhO Iii) 5·3 +4'~()() I] OS

B,ISS C3 (130XII/I 'iO 4·5°'0 S 05

assuming that the vocal system can he represented by a standard sourcefixed-filter model having about five fairly broad formant resonances, I h weexpect that the group delay variation will be minimal (unlike the case of theviolin. which is characterized by numerous overlapping narrow bandresonances). indicating that inhurmonicity effects due to vibrato should heinsignificant in vocal tones. However. another effect. due to the possibilityfor modulation of the vocal tract parameters during vibrato, therebyaffecting the individual partial frequencies differently, could not be ruled out.Thus. we chose to perform an analysis of time-varying inharmonicity. Theresults were that the analyses revealed some subtle inharmonicitics. but theeffects were of the same order as our analyzer's inherent frequency accuracylimitation of approximately 1 H7 and appeared to be too small to beperceivable. Indeed. informal listening tests of (ones synthesized with and

I I I

2.'+ 2.7 3.0I

0.3

0.0'++H :f\ 0.03+

R 0.02tM 0.01

io.ootiD-o.O I 71F-O.02~Ii,0.03t!

-o.ot.t~

-0.05~-O.OE> ]L.--+--+-1--;-1--i--- >-~'-+-I--1__+-__

0.0 O.E> 0.8 1,2 1.5 1,8 2.1

T I ,''-: E (SEC J

Fig. 6. '\ormalized measured trcqucnc. dn i.u ion for c<lch "I ihc partials of at) pica] ,tltotone, shown overlapped. !\ormali/c'd dnl":II)IlS culcul.ucd h\ (f --k!..llkU. where / =

measured Ircqucncv in liz. /,.==assulllc'd l'lllldall1cilLli l rcq ucncv III Hz and k~partial

number II. O. lnIC'i'cri.

-

230Roher( :\Iaher. Jallles Beauchalllp

without the detected inharmonicities indicated that the differences werebarely distinguishable. Thus, we conclude that inhannonicity effects can beignored in vocal vibrato tones. Visually, this seems obvious from Fig. 6,which shows superimposed normalized frequency measurements for anumber of partials of a typical alto tone.

The results also clearly show a complex relationship betwccn the time­varying fundamental frequency and the individual partial envelopes. Theprominence and qualitative compJexity of the amplitude fluctuations IS

TIM ESE C J

la)

rIM E ( SEc i

Fig. 7. Typical timc-variant spcctra for four Sll1gcrs using vihrato: la) soprano. C5: Ih) alto.D4: (c/ tenor. GJ: (d) hass. C3. Al1lplitlllk (linear scalc) is the vertical dimension. Quasi­

harmonic data \\e1'C c'.tractcd from the lull MQ analysis.

,jll investigation of 1'11('0/ iibruto ior ,I vntlusi, 231

illustrated by the time-variant analysis data for four typical vocal tonesshown in Fig. 7.

The phase relationship between the time-varying fundamental frequencyand the amplitude fluctuation of an individual partial can he used to identifythe position of that partial relative to a vocal tract resonance in the followingways: 1

(i) If the amplitude ofa partial is ill phase with the frequency vibrato (i.c.it increases in amplitude as the frequency increases). the partial is onthe low side of a resonance peak (see Fig. Sla)).

TIM E ( ~ E C )

(el

0.0TIM E ( SEC )

(Ll)

Fig. 7 contd.

A

R

'1 ?7c)G -r IIi

5 2250 ~ I;

~

:8DO

P !351]

1\/

/

I

1I

/1

1

I

I

1I

/

I

I

/{

{

o. :+

0.2+------+----+----- -+-----+ ~---+-- --+-

0.3 0.4 0.5 c.s 0.7 0.8 0.9

M [ (SEC)

(a)

-0

1.0

(h)

---+---j0.9 1. 0

,:] --¥!'- ---- --+------+ --- --+----+-----+------i--------+----+-0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

~ I ME [SECI

8SC~-iI

8500~

7[,

50

1' ', .: 1\ YJr\I-I i r I \ I' Ii t, IH 1:>800 --+- I \., . ./ I 'I / \ I '\ I \ .. /\ r,A I II / \j \/ \! V Y I \ r

R SSSG i Ai :\) M IV\ r \ I

M 5100 ~- I ! \ / I I \: I I I

~ ~250 - :: I I /', / i \ IA " ',I I I J \ r

\ I I I \ I \M 3400 +Ii \ f \ \ \ {, I'! I I I \ I I I

P 2550 ~ 1 \ / / I \ \ I, \ I \ I \ --I \ i

!700 --+-,1 \ / \ J \ /

\-.r 1./ \ I'J

: lCJU "T

H 900 I \ 'II / \

P- I {

ROC I \ {

R I

7CJO T{

b FlUC -- ,

SOO~ ,r

"

D-j'------+ - - --+------+-----+----+-----+----+1~-__+__--_+__---"O. 0.1 0.2 0.3 O.~ 0.5 0.& 0.7 0.8 0.9 1.0

TIM E [SEC:

hg. K RC'latllllhhlp between Ircqucncy or partial and a vocal resonance. Solid line is.unpl.iudc. dotled line IS Irequene\. (a) Partial located on low-frequency shoulder ofre\l\n:lllCc' pc.ik : .implu udc Iluctu.uion-, ill phus: wi t h vibrato. (hi Partial located on high­Irc'qucnc , <houldcr 01'rc-on a nee pea k: a III piii udc fluct ua tions I i(() degrees (II/I otplursc with. ibr.u» iL'1 Parual loc.uo.l ncar rcxon.mcc peak ,11' trough: amplitude fluctuations atlll'ice the

\ ibrato rate.

All im'nligarioll oj iocal tibruto [or svnthcsis 233

(ii) If the amplitude ofa partial is out ofphase with the frequency vibrato,the partial is on the high side of a resonance peak (see Fig. 8(b)).

(iii) If the amplitude of a partial contains a component changing at twicethe ribratofrequencv, the partial is swept across a resonance peak ortrough (see Fig. 8(c)).

The amplitude versus frequency behavior can be observed more clearlywhen the data are plotted as the locus of [frequency. amplitude] coordinatesof each partial during vibrato. Figure 9 shows the spectral envelopes of thesung 'a' vowels for each of the four singers as traced by the partials duringseveral vibrato cycles. The striking feature of these 'scribble-plots' is the easewith which the spectral position and shape of several formants may beidentified. Although the excitation spectrum from the vocal folds and thefrequency response of the vocal tract have not been separated here. themulti-resonant character of the output spectrum is obvious. Unfortunately,with increasing fundamental frequency. the spacing of the voice partialsincreases. This provides a coarser sampling of the underlying spectralenvelope and a corresponding decrease is formant information availablefrom the plots. as evidenced hy the data from the female singers in Fig. 9(a)and (b).

What is the perceptual importance of the amplitude f1uctuations duringthe vibrato cycle') Although the tracing of formant characteristics due tovibrato might indicate a means for a listener to identify more features of thespectral envelope-thereby increasing the ease of vowel identification-thishas not heen fully confirmed. I 0.1 S Perhaps most significant is that tirnbralvariation occurs throughout the vibrato cycle. That periodic timbralfluctuation is perceptually obvious (in the absence of frequency vibrato) wasdemonstrated in our resynthesis examples where freq uency vibrato had beenremoved. leaving only the spectrum modulation to he heard.

The perceptual importance of the amplitude fluctuations for convincingsynthesis was also noted from our resynthesis examples where the partialamplitudes were held constant. In general, it was apparent that a certainwarmth and natural quality found in the resyntheses including amplitudef1uctuations was missing from the frequency-modulation-only examples.Also. some listeners noted that the inclusion of spectrum modulationseemed to aid perceptual fusion. With the amplitude fluctuations omitted,the upper partials produced a 'whistling effect' and did not blend well withthe lower partials. It is possible that some of the unnatural character ofcertain economical synthesis methods such as complex frequencymodulation (FM)' -might be attributed to the unnaturally constant partialamplitude behavior of the synthesized tones.

We conclude that the time variation of vowel timhre due to frequency

234 Robert Alahcr, Janu-» Bcuuihamp

vibrato provides a richness and sonic variety characteristic of good singingquality and can profitably be included in models for vocal vibrato synthesis.

3.2 Analysis of vibrato waveforms

The vibrato waveforms obtained from the singing examples were analyzedto evaluate their sta tistica I properties. The long-term (1-2 s) average

8[i

,.::0- r78 ~

F 38

)~\~ t 'V~

I ,

'( lr1-:"":J

70--(a I

, r

1200 18CO 2'tOD SOLJrJ-+

rt f~

,I V\,\Ii l0III, \0-

-+- --

,

---f---------t---+--­bOOCJ

10

5EJ ,-.j.-

'to .•

Y I Hz)

Ib)

Fig, 9. Spectral tracings for rom singers using vibrato. Each partial traced its amplitudeversus frequency value over several vibrato cycles: (a) soprano. ("5: (b) alto. D4: (c) tenor. G3:

(ei, bass, C3.

All invrstivation of ioca! vibrato /01' svntlicsi: 23S

spectrum of each vibrato waveform was computed, and a time-variantanalysis using fixed-band filters centered at the nominal vibrato frequencyand its harmonics was performed,

Analyses of the vibrato waveforms (sec examples. Fig. 10) showed nearlysinusoidal frequency changes, with amplitudes (vibrato depths) varyingbetween ± 2(1<) and ± 5% of the fundamental frequencies. The vibratowaveforms also exhibited a low frequency 'drift' above and below the targ«!

73

11 b3rr,p

59

T 49U

D

E39

g29

13 ----t-' - -----t---- ---r-- - -+- -- --j- - --j---- ---,

o bOO 1200 1800 2~OO 3000 3SDD 42CD 480e 5400 boDeiRE 0_1 E ;'. C: y (Hz

lei

(ell

Fig. 9 contil.

236

bOOT590

~ 550

1 5'+01

; 530 I

520+ II

Slot500 - I

0.0 0.2

315

Robert Maher . James Beaurhump

~ \ ~\ ~ V I J

I I· -----+-----+------+----+---+- ~-+-------i0.'+ 0.5 0.8 1.0 1.2 1.'+ 1.5 1.8 2.0

TIME (SEC)

lal

285 -- 1-- - --+- --0.0 ll.3 O.b 0.9

309 -l-I

F I '; 3llSt i

~ 303~1~. 1 I

c 300+co II1 -'97 ,I

rI C 11

H Iz 29'+ t!

2GB

1-, c, . ~)

Tli"1E (SEl:)

(hi

11. B 2.1

t­

'+

, I

\1,

,--~. 7 D.ll

Fig. 10. Vibrato waveforms for four singers: (al soprano: (hi uh o: ((I tenor: lell hass: (el Jong­

term average spcctrur» Cor tenor \ ibr.uo waveform.

Ie)

---+--1

1.8 2.0

-- +--- ----+ ---- --t

1.2 1.4 1.b

~ 20 1 ~ A (\ ~ A I~ I,I~ 0 t ~ A~ ::: I1

1

\ II \1 II 1'\ I' I'I I \ (\ \ / \ / '\. / \1\ 1/ II /1/: '1\ ' \ II/I \~ 1,1 III~ 192 , \ I I, i !, / \ / \

~ I 89~ ,! II \ \ I

II I" \'J1 ~\'II\\JI 'l! I VI/I 8E., \ i \Ji' \) \j \) ~(183~ rj ~

180 -ll--- -+ ----,- -f- -- -+- ------+-- --t-U.O 0.2 U.'+ 0.[, 0.8 1.0

TIME (SEC)

(dl

HI II i, l(

, ~

j"i,-1

r' , , \, ! '", \ / \\ \

,

I, f,I )!

'\! '\/ \J \'J ~,

1

I I - ---,----------,

CJ. l+ lJ.b c.n 1.;-; l.t) :.It l.h 1.8 (J.n

-;:ME ,SECj

I'+IJ-I'l'

lJ8,I

!' \1'1

13G

III', I

c ),I' 13'+c:uu 132o,1L' lJO -,

:1'j

1 128I,

r'i I~c112["Z

12 4 -

122 -

120

0.0 U.2

'+.0

fREQL;t>"CY (,-17;

Fig. I() contd.

Robert AIalicr. James Bcaucluunp

fundamental frequency with an extent on the order of ± I% of thefundamental (I % ;:::; 17 cents). The drift appeared to be random and, asindicated by the long-term spectral average of the vibrato waveform(example shown in Fig. IO(e)). had a bandwidth of one-third the nominalvibrato rate for the male singers and approximately one-half the nominalrate for the female singers. The vibrato rate itself varied over the duration ofthe note, with maximum deviation of ± lCVYc, of the nominal vibrato rate. Anexpression for the fundamental frequency of a sung tone with vibrato is

flU) =fo + dl(r) d(1) = 0lrl(r)

dIU) = d(r) + tl/Ul sin (lip) + (!)o) = frequency deviation,~[

11,(1) = 2n I/\JI + O,,",,(T)j dr = phase of fundamental (3)IL! 0

tl/(I) =f~[1 + 11 2"2(I)J = vibrato depth in Hz

lid!) = 2n f'/;(T)dTo

where

rk = random numbers in range -I to + 1. with specified bandwidthilk = extent scaling of random variationsf", = nominal vibrato depth

1,(, = nominal vibrato rate10 = target fundamental frequency 111 Hz.

Equation (3) could also be written with the frequency parameters expressedas fractions of the target fundamental frequency 1;1' Parameter measure­ments for four vowel tones are given in Table 4.

It is important to realize the significance of the iariations in vibrato rate.

TABLE 4Tvpical Vibrato Parumcrcr-, and Random Vu n.u ions

I [!ICC Random .lcptl,iariation a,

Random dritt U I Random iibra tnrat: rariat ion il,

FYICnl

(U" o!idBandwidth Extrnt

(/h I"" o/j"J

BandwidthIH~)

FYICIII

o! t, ,lBandividtli

( II~)

Soprano .10 ') 0·6 50 K 5-Alto .10 .IS 07 2·5 K .~

Tcnor 15 5·5 I 35 10 3-5Bass 25 5 1 .1·5 10 .I

All inrcst ivut ion of local rihruto fIJ'- svnthcsis 2.)'1

vibrato depth and fundamental frequency. Although a trained singer canadjust the Target values of these parameters. the 'random' variations aboutthe desired values do not appear to he controlled by the singer in anyconscious manner. I The maximum measured deviation from the targetvalues may give some indication of the 'looseness' of the vocal controlsystem. The singer evidently uses feed hack from the sensory and hearingsystems to adjust the vocal fold apparatus in the direction of target valueswhenever the random drift is greater than some tolerable amount."? The± 17 cents drift extent measured here for solo vocal tones with vibrato is

comparable to the ± 15 cents frequency deviations between singers in a choirreported hy Ternstrom and Sundberg.?" It is unclear whether this agreementrepresents a typical limitation of the vocal.auditory feedback system. In anycase. it has been reported tha t incl usion of appropria te ra ndom f uct ua tionsin singing synthesis is vital to avoid a mechanical. unnatural sound.':"

3.3 Computer model for singing synthesis with vibrato

A proposed block diagram for 'natural' vocal vihrato is given in Fig. 11. Themodel has two main parameters: vibrato rate(" and vibrato depthr, Theseparameters are augmented by lowpass random fluctuations. and a final low­frequency drift term is added. The design parameters of the randomvariations are based on the results of our analysis effort. The vibrato blockdiagram is appropriate for inclusion in many existing synthesis models.

Our final experiment involved a vibrato and singing synthesis algorithmincorporating the important features of the measured data. Althoughseveral sophisticated singing synthesis techniques have been reported (cfReferences), those methods allowing control over partial amplitudes requiresubstantial setup and computation time. Instead. we chose to reduce thecomputation load in a manner appropriate for a general-purpose softwaresynthesis program (e.g. Music 4( 2 1

). Thus, what the algorithm lacked insophistication, it gained in simplicity and economy.

The computation required to synthesize a sustained signal can be reducedby the use of a )varetable. A wavetable is typically a pre-calculated. singlecycle of the desired waveform which is repetitively read from memory. Thetable lookup operation has been used in many software synthesis methods,and is very efficient in terms of computation load. 2 2 The fundamentalfrequency of the output signal can he varied by changing the effective rate atwhich the wavetable is accessed. Also, the overall amplitude envelope of thesignal can be varied with multiplicative scaling of the data obtained from thewavetable. However. a single. fixed wavetablc does not allow independentamplitude control of the individual partials comprising the storedwaveform. In order to simulate the spectrum modulation present in vocal

240 Robert Moher, James Beaucliamp

l.owuas s No'seMaxAmpc,/, i x cr r

ofco3.SHZ

Sl~usoljal 05C' ato"

Target,::undamenta i

: cCQucncy

}----.... vibr at o Out

:_OW;:JJSS ""Jo'seMax Amo ° ./- 10% of 'v

fco3,SHZ

210

207

20'+

Lowoass l,o'sOMax Ar--,o 0 ./- IS% of FA

fcoS.SHZ

(a)

Fr 201

I~ I~ I~ 1\ II I~e

fl r (\ 1\ f~ClLJ 198enc 195" i' I I \1 192 I \ !

il~~~~~~! \ ! \! In

H189

~I \ i \ Iz I I I I

18E. IV V ~183

1800,0 0.3 O.E. 0.9 1.2 1. S 1.8 2.1 2.'+ 2.7 3.CJ

TIM[ (SEC=J

(hi

Fig. I I. Block diagram for vocal vibrato simulation. (a) The nominal vibrato rate and

dcpihr, vary in a quasi-random manner. A low-frequency drift of the target fundamental isalso present. (h) Example of vibrato waveform generated hy the process in (a).

All imcstigation of ioca! ribrato [or svnthcsis 241

(4)

vibrato, our approach was to use two wave tables, each being scanned at thesame rate (frequency), but with different amplitude scaling. The outputsignal was computed as a time-varying weighted sum of the individualoutputs of the two wavetables.

The pair of wavetables were generated using the time-varying spectrumobtained for one of the vocal vibrato tones as the source of two "target'spectra. The amplitude of each partial measured at the extreme positii:cfrequency excursion during a vibrato cycle was noted, and a fixed waveformtable (one cycle) was computed using the partial amplitudes as Fourier seriescoefficients. The other table was generated corresponding to the extremenegative frequency excursion of the same vibrato cycle.

A 'blending' or 'panning' function :x(t) can be defined as

df(f)-min :df(f):

:x(t) = nl~IXT(~): - min :df(f):

where dl(t) is given by eqn (3), and min t! and mux t-j are the extremenegative and positive frequency excursions. respectively. For example. :x(t) isequal to I when df(t) is at its positive peak and equal to 0 when df(f) is at itsnegative peak. Further, defining \\'!(OJfl as the waveform corresponding tothe peak positive frequency excursion during the vibrato cycle and 11"2(oJf)

corresponding to the peak negative frequency excursion. the resultingsynthesis output becomes

(5)

where IIt(f) is given by eqn (3).

Thus, by "panning' between the wavetables in synchronous with afrequency vibrato waveform the output consists of a perfect spectral matchto the original at the vibrato extremes and an approximation at frequenciesin between. The spectral evolution created by this simple method cannothandle the situation in which a partial traverses a spectral peak or troughduring a vibrato cycle, but the spectrum modulation is simulated in areasonable way.

In our investigation, the main synthesis parameters were the fundamental(carrier) frequency and amplitude. and the vibrato (modulator! frequencyand amplitude. The block diagram of Fig. II was used to synthesize thevibrato waveform and to control the blend between the two synthesiswavetables. A diagram for the complete synthesis algorithm is shown inFig. 12.

This simple panned-wavetable synthesis method produced good-qualitysynthetic sung vowel sounds. We were surprised to find that syntheses oftones 2-3 s in duration containing the random vibrato variations of Fig. 11did I/O[ sound superior to examples containing only perfect. sinusoidal

242 Robert Maher, James Beauchamp

Wavetable #1

(high frequency)

VibratoGenerator

Wavetable #2

(lOW frequency)

Scale Intorangeo to 1

Fig. 12. Block diagram for panncd-wavetable synthesis scheme. The contribution of eachwavctable to the output is determined by the instantaneous vibrato extent. An overall

amplitude envelope is applied to the signal.

vibrato. Our informal listening detected differences between 'straight'sinusoidal vibrato examples and examples with random vibrato variations,but did not express a strong preference. A satisfactory explanation of thisresult-given the reports of Bennett, 1 Chowning 7 and others regarding theimportance of random vibrato fluctuations-will require further study.However, it is possible that the timbre fluctuations resulting from thepanned-wavetable synthesis provide a sufficiently strong perceptual cue forsinging that the assistance of simulated random variations of the vibratowaveform is not essential.

The synthesized vowel tones were only of useful quality for about twosemitones above and below the target frequency, implying the need for atleast three sets of wavetables per octave range. Also, the method did notinclude attack cues (e.g. portamento, onset noise, etc.), which may benecessary for a truly convincing solo synthesis. Thus, we may be satisfiedwith the simple method for use in synthesizing sustained vocal sounds, butnot for rapid, exposed solo passages. We feel that the simple algorithmrepresents a useful computation/quality tradeoff, nonetheless, and is a goodplatform for further research.

An investigation of' weal ribrato Fir synthesis

4 AREAS FOR FURTHER RESEARCH

243

During the course of the work reported here, we came across severalunanswered questions appropriate for further study.

(1) The behavior of vibrato in the vicinity of a transition from one noteto another-particularly in legato singing-is not considered in ourcurrent model. However, some of our preliminary work in this areaindicates that the singer systematically advances or delays the vibratorelative to the time of transition in order to perform a change fromone note to the next so as to join particular points on successivevibrato cycles. A more extensive investigation will be necessary toconfirm or refute this preliminary observation.

(2) The formant tracings (Fig. 9) obtained from the vibrato measure­ments suggest a means to design accurately a complex filter for use inthe standard source-filter synthesis model. 8 The resulting filter couldbe designed to match the measured spectral contour instead of beingbased on approximate values for resonance frequencies, bandwidths,etc.

(3) Many composers using electronic media are not interested in merelyrecreating existing acoustic timbres. For these composers, synthesismodels retaining certain qualities of a natural sound source areneeded. For this purpose, the timbral characteristics of acousticinstruments, including the singing voice, must be catalogued andrepresented in a convenient, intuitive form.

(4) Vibrato generation based on musical context remains an ad hocprocedure. The standard guidelines for vocal vibrato are learned bysingers without the benefit of written rules, so research is necessary ifa useful descriptive model is desired. The need for random variationsof the vibrato parameters and the importance of amplitudefluctuations also merit further study.

5 CONCLUSION

Based on our experiments, we draw the following conclusions:

(i) The amplitude fluctuation of a partial during the vibrato cycle variesin form, amplitude and phase according to the partial's positionwithin the vocal tract resonances. The location and other charac­teristics of the formants can be aided by examination of thisamplitude variation.

(ii) The partials of vocal tones are almost perfectly harmonic during

244 Roher! Maher. James Beauchamp

vibrato, i.e. a single vibrato waveform accurately characterizes thefrequency variations of all partials when singing.

(iii) The vibrato waveform is typically nearly sinusoidal. However, itsamplitude (vibrato depth), rate (vibrato frequency) and average value(nominal sung frequency) all appear to vary in a random fashion.From our measurements, the model and guidelines of Fig. 11 can beidentified.

(iv) For singing synthesis, our informal evaluation indicates theperceptual importance of fluctuations of the partial amplitudesduring vibrato, Inclusion of the fluctuations tends to add warmth tothe sounds and improve their perceptual fusion.

(v) Although previous reports have mentioned the importance ofrandom variations of the vibrato waveform, we were surprised to findthat high-quality synthesis by the panned-wavetable algorithm ofFig. 12, using the vihrato generator of Fig, 1L did not reveal anynotable improvement over parmed-wavetable synthesis with fixedsine wave vibrato.

ACKNOWLEDGEMENTS

This work was supported in part by a National Science FoundationGraduate Fellowship and by the University of Illinois Research Board. Thehelp and cooperation of the singers is greatly appreciated. The authors wishto acknowledge John C. O'Neill for collecting some of the vocal tones andthe staff of the University of Illinois Computer Music Project for theirtechnical assistance. We also would like to acknowledge the helpfulcomments of a reviewer in improving the clarity of several sections of thispaper,

REFERENCES

1. Bennett. G .. Singing synthesis in electronic mUSIc, Proc. S',\4AC /')81(Stockholm Music Acoustic Conference, 1981), YoL 33, Royal SwedishAcademy of Music' 1981. pp, 34 5(}

, McAulay, R. J. & Quatieri, T. E. Speech analysis synthesis based on asinusoidal representation. IEEE Trans. Acuust .. Speech. Signal Processing,ASSP-34(4)(Aug.) (1986) 74454.

J Smith. J. & Serra. X., PARSHL: An analysis/synthesis program for non­harmonic sounds based on a sinusoidal representation. lCAfC-/,)87:Proceedings ofthe International Computer Musi: Conference. Computer MusicAssociation. San Francisco. CA. 1987, pp. 290 7.

All imestivation otrocal ribrato for svnthesi: 245

4. Maher, R. C; An approach for the separation of voices in composite musicalsignals. PhD dissertation, University of Illinois, Urbana, 1989.

5. Serra, X.. A computer model for bar percussion Instruments. ICAtC-19M:Proceedings ofthe International Computer Music Conference. Computer MusicAssociation, San Francisco, CA. 1986. pp, 257 62.

6. Roder. X., Time-domain formant-wave-function synthesis. Comput. Music 1..8(1984) 9 14.

7. Chowning, L Computer synthesis of the singing voice. Proc. S'M A C 191'1,()(Stockholm Music Acoustic Conference, 1980), Vol. :33. Royal SwedishAcademy of Music, Stockholm. !980. pp. 4 13.

8. Sundberg. L Synthesis of singing. Swed. 1. Musicol.. 60(1) (1978) 107 12.9. Rossing. T.. Sundberg, J. & Ternstrorn. S.. Acoustic comparison of soprano solo

and choir singing. 1. Acollsr. Soc. Anier., 82(3)(Sept.) (1987) 830 6.10. Sundberg, J., Vibrato and vowel identification. Arch. Acoust.. 2 (1977) 25766.11. Sundberg. J., Singing and timbre. Proc. ."l'MAC /977 (Stockholm Music

Acoustic Conference. 1977), Vol. 17. Royal Swedish Academy of Music,Stockholm. 1977. pp. 57 88.

12. Bloothooft, G. & Plornp, R .. The timbre of sung vowels. 1. Acoust. 5'01'. AII/er ..84(3)(Sept.) (1988) 847 60.

13. O'Neill, 1. C. Computer analysis and synthesis of" a sung vowel. DMADissertation, Department of Music. U nivcrsi ty of Illinois, Urbana-Champaign.Illinois, 1984,

14. Hebel, K., A machine-independent sound conversion sound storage system.ICA1C-1985: Proceedings oj the International COll1jiurer Music Coniercnce.Computer Music Association. San Francisco. CA. 1985.

15. Schroeder, M .. Period histogram and product spectrum: new methods forfundamental-frequency measurements. 1. Acoust . Soc. AII/er., 43(4) (April)(1968) 829-34.

16. Rodet. X. & Bennett. G .. Synthcse de la Voix Chantcc par Ordinateur. Conf desJourness d'Etudes 1980, Festival International du Son, 1981.

17. Oncley, P. B.. Frequency. amplitude, and waveform modulation in the vocalvibrato. 1. Acoust. Soc. A mer.. 49(I)(Jan.) (1971) 136.

18. McAdams, S. & Rodet. X., The role of FM-induced AM in dynamic spectralprofile analysis. In Basic Issues in Hearing. ed. H. Duifhuis, J. Horst & H. Wit.Academic Press. London, 1988.

19. Elyn, 1. R., An analysis of the onset characteristics of the fundamentalfrequency of sung tones. PhD Dissertation, Department of Speech and HearingScience. University of Illinois, Urbana-Champaign, 1977.

20. Ternstrorn, S. & Sundberg, 1., Intonation precision of choir singers. 1. ACOII.II.

Soc. Amer.. 84(1 )(July) (1988) 59 69.21. Beauchamp, 1. W., New computer music facilities at the University of Illinois at

Urbana-Champaign. ICMC-1985: Proceedings oj the International Compute:Music Cantrrcn::«. Computer Music Association. San Francisco. CA. 1985. pp.407 14.

22. Dodge, C. & Jerse, T., Computer Music: Svnthesis. Composition, andPerformance. Schirmer Books, New York, 1985.