presented at the 2004 comp.dsp conference mark allie consulting llc

Mark Allie comp.dsp 2004 1

Signal to Noise and Numeric Range issues for Direct Form I & II IIR Filters on Modern Analog

Devices and TI Digital Signal Processors

Presented at the 2004 comp.dsp conference

Mark Allie Consulting LLC.


Main Topics

• Filter configurations Direct Form I and II and the transposes are investigated.

• This presentation contains a summary of the issues of signal to noise and numeric range of Direct Form IIR filtering.

• Design criteria combined with sensible dsp choices are presented.


Direct Form I, II and Transposes Filter Assumptions.

• Unity gain 2nd order filter sections (biquads) are used.• The transfer function of the biquad is given in equations 1 & 2.• The recursive coefficients are scaled to yield a0 = 1.

• The filters are implemented with all terms being added.

mkyankxbkyM

mm

N

nn

10

M

m

mm

nN

nn

Za

Zb

zX

zYzH

1

0

11

2


Filter Forms

Z-1

Z-1

Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n) y(n)

Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n) y(n)

Z-1

Z-1

Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n) y(n)

Z-1

Z-1

b0

b1

b2

-a1

-a2

y(n)x(n)

Direct Form I

Direct Form I Transposed

Direct Form II

Direct Form II Transposed

w(n)

w(n-1)

w(n-2)


Matlab Design Example

• Because all products are summed the Matlab generated recursive coefficients a1 and a2 must be negated for use in the filter.

• For example a Butterworth 2nd order low pass filter with a cutoff frequency of 0.01 * Fs has the coefficients:

• b0 = 0.00094469184384• b1 = 0.00188938368768• b2 = 0.00094469184384• a0 = 1.00000000000000• a1 = -1.91119706742607 use -a1 or 1.91119706742607 • a2 = 0.91497583480143 use -a2 or -0.91497583480143


Topics of Interest

• Overflow in ideal and real filters.• Noise issues.• Noise Compensation issues.


Two Types of Overflow

• The final result overflows.• Intermediate results overflow.

• Overflow of the result.• Unity gain filters can have outputs that are greater than

±1 with a bounded input.


Ideal Frequency Response


Ideal Time Response


Internal Overflow with Fixed Point Math

• Modulo math is your friend.• Modulo math is the default when not in saturation mode.• Jackson’s Rule: Any number of additions and/or

subtractions may occur. Intermediate results and operands may fall into any modulo. As long as the final result is made to fall into the first modulo by design, it will be representable in two’s complement at the chosen wordlength, and a valid result.

• Once overflow detection is applied to the result the correct sign will be reflected in the saturated value.


Noise Issues

• Noise is generated by truncating high resolution results.• The noise generated by truncation often resembles the

spectrum of the signal being filtered.• The generated noise is most harmful when it re-

circulates through the recursive coefficients.• Force correlated truncation noise to be random (white)

by using dither.• If the truncation process is deterministic then correction

can be applied to the filter.• Fixed point truncation is deterministic.• Floating point truncation is not deterministic.


Fixed Point Truncation

AccN bits

In a fixed point processor the accumulator has at least 2 times the number of bits as in the native data word.

N bits N bitsNative Data Word

Any time the accumulated result needs to be stored it must be truncated.

It is assumed that data needs to be stored for multiplying, delaying and sending to a converter.

Truncation is performed after the lsb of the highest word of N bits.

N bits N bitshi

lo

Truncate

null


Floating Point Truncation

• The data word in a floating point processor is composed of an exponent and a mantissa.

• The number of bits used to represent a mantissa is not directly reduced.

• The number of bits used to represent one mantissa is usually indirectly reduced when two numbers are added.

• This occurs because the exponents of floating point numbers must be equal to add them.

• There is no indirect truncation when the exponents are already equal.

• The number of bits lost is dependant on the ratio of the 2 numbers being added.


Floating Point Truncation

• For example: When adding 1 to 1/256, 7 bits are truncated because of shifting when the exponents are matched.

• The truncation associated with floating point addition is not deterministic in the case of Direct Form IIR filters.

• The number of bits truncated can be different for each addition.

• The number of bits truncated depends on the coefficients and input signal statistics.

• Truncation errors can not be corrected.

• They usually must be accepted!

• If the recursive products can be kept larger than the direct products then the recursive data path will not be truncated very much.


Noise compensation

• Fixed point compensation is accomplished through an error feedback signal.

• What is the error signal? e(n) = y(n) – yt(n).

• Floating point compensation can not be performed.• So how bad is this problem?


How is the error term generated?

N bits N bitsy(n) is 2N bits.

yt(n) is N bits.

e(n) = y(n) - yt(n).

N bits XXXX

y(n), yt(n) and e(n) are signed quantities.

e(n) has a sign.

Is it correct to use the bit pattern in the lower half of the register as e(n)?

The pattern is correct for e(n) but what is the sign of e(n)?

For 2's complement math the sign of e(n) is always positive.

Use unsigned multiplies for e(n).

???? N bits

0000... N bits

yt(n)QN bits N bits

hi

lo

Truncate

e(n)


Direct Form Implementation Analysis

• Overload considerations.1. Input scaling.

2. Modulo math.

• Noise generation considerations.1. Truncation noise.

2. Terms affected.


Z-1

Z-1

Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n) y(n)

Direct Form I

N

N

N

2N

2N

2N

2N

2N

2N

N

N

Q

N(lo)

N(hi)yt(n)

e(n)

Fixed Point:Modulo math useful. One summer before saturator.No Input scaling required to prevent internal overflow.One truncation feedback error term.Truncation noise affects recursive terms.Truncation noise negligible for N large enough.

Floating point:No input scaling required.No overflow concerns.Truncation error is coefficient dependant and not

correctible.Truncation noise affects recursive terms.Number of bits in mantissa critical.

N

N

N

N

N

N

Qyt(n)

)(1

1)(

1 12

11

12

11

22

110 zE

zazazX

zaza

zbzbbzYt


Floating Point Truncation Analysis

xn*b0

xn-1*b1 yn-2*a2xn-2*b2 yn-1*a1

5 products feeding the summer.

This means there are 4 summations performed per sample period (Ts).

Take the ratio of the ABS of 2 terms being summed*.

Find the maximum ratio for the 4 sets of addends for each Ts.

This indicates the number of truncated bits for each Ts.

* The chosen addend terms can be optimized.

yn


Floating Point Truncation Analysis Example


Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n)

y(n)

Direct Form II

Q

2N

2N

2N

2N

2N

e1(n)

2N

Q

N(hi) N(hi)

N(lo)

e2(n)

w(n)

w(n-1)

w(n-2)

N

Fixed Point:Modulo math not useful. There are 2 sums

separated by a saturator.Input scaling required to prevent internal overflow.One truncation feedback error term.Truncation noise affects all terms.Truncation noise negligible for N large enough.


correctible.Truncation noise affects all terms.Number of bits in mantissa critical.

N(lo)N

N

N N

N

N

yt(n)


Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n)

y(n)

Z-1

Z-1

Direct Form I Transposed

2N

2N

N

Q

Q

N(hi)

N(hi)

N

2N

2N

2N

Q

Q

Q

N(hi)

N(hi)

N(hi)

e1(n)

e2(n)

e3(n)

e4(n)

e5(n)

Q

Fixed Point:Modulo math not useful. There are 2 sums

separated by a saturator.Input scaling required to prevent internal overflow.Multiple truncation feedback error terms.Truncation noise affects all terms.Truncation noise negligible for N large enough.


correctible.Truncation noise affects all terms.Number of bits in mantissa critical.

N

N

yt(n)


Z-1

Z-1

b0

b1

b2

-a1

-a2

y(n)

x(n)

Direct Form II Transposed

N 2NN

Q

Q

2NN

Q

N(hi)

N(hi)

N(hi)

yt(n)

Qyt(n)

e1(n)

e2(n)

2NN

2NN

2NN

2NN

2NN

2NN

N(lo)

N(lo)

N(lo)

e3(n)

Fixed Point:Modulo math useful. One summer before saturator.No Input scaling required to prevent internal overflow.Multiple truncation feedback error terms*.Truncation noise affects recursive terms.Truncation noise negligible for N large enough.

* One if Q1 and Q2 are removed. Dattorro


correctible.Truncation noise affects recursive terms.Number of bits in mantissa critical.


Tabulated Results

Fixed Point DFI DFII DFIT DFIIT

Floating Point DFI DFII DFIT DFIIT

Input scaling + + + + + +

Modulo math + +

Truncation noise + + + - - - -Performs well with 32 bit mantissa + + + + + +

Error compensation + + + - - - -

+ Good Attribute + Qualified Good Attribute - Poor Attribute


Error Compensation

• Fixed Point Only.• Direct Forms with 1 error term.• Attempt to correct noise by feeding back the error signal.• Evaluate DFI based on Dottorro and Wilson.


Error Compensation

Z-1

Z-1

Z-1

Z-1

b0

b1

b2

-a1

-a2

x(n) y(n)

Direct Form I with 2nd order error feedback

N

N

N

2N

2N

2N

2N

2N

2N

N

N

Q

N(lo)

N(hi)yt(n)

e(n)

Z-1

Z-1

k1

k2

2N

2N


Trivial Noise Shaping with the Error Term

Z-1

Z-1

-a1

-a2

y(n)2N

2N

2N

N

N

Q

N(lo)

N(hi)yt(n)

e(n)

Z-1

Z-1

k1

k2

2N

)(1

1)(

1 12

11

12

11

22

110 zE

zazazX

zaza

zbzbbzYt

)(1

1)(

1 12

11

22

11

12

11

22

110 zE

zaza

zkzkzX

zaza

zbzbbzYt

2N

For k1 = 1 and k2 = 0 the noise is attenuated at dc and low frequencies.

Trivial k

k1 k2 Region θ+2 -1 0 Twice-2 -1 π Twice 0 +1 0 and π+1 -1 π/3 Twice-1 -1 2π/3 Twice+1 0 0 Once-1 0 π Once

yt(n) = y(n) - e(n).


Optimal Noise Shaping with the Error Term

Z-1

Z-1

-a1

-a2

y(n)2N

2N

2N

N

N

Q

N(lo)

N(hi)yt(n)

e(n)

Z-1

Z-1

k1

k2

2N

)(1

1)(

1 12

11

12

11

22

110 zE

zazazX

zaza

zbzbbzYt

)(1

1)(

1 12

11

22

11

12

11

22

110 zE

zaza

zkzkzX

zaza

zbzbbzYt

2N

Optimal k

k1 = -a1

k2 = -a2

yt(n) = y(n) - e(n).

)()(1 1

21

1

22

110 zEzX

zaza

zbzbbzYt


2N Word Equivalent to 2nd Order Error Compensated Filter.

Wilson:If k1 = -a1 and k2 = -a2 it is equivalent to implementing a 2N bit recursive data path biquad without error feedback compensation.

Z-1

Z-1

-a1

-a2

y(n)Q

yt(n)

2N

2N

2N

3N

3N

3N

N(lo) e(n)


Is This Good News?

• Yes DFI and DFIIT both work.• A processor with 32 bit integer native word size

capabilities doesn’t need truncation error compensation.• Some modern floating point processors can do this

inexpensively and fast.• Some modern fixed point processors can do this

inexpensively and more slowly.


Floating Point?

• All DF IIR filters can work.• Complicated analysis and addend evaluation. • Floating point processors with extended precision results

may work well. • Analog devices has processors with 32 bit mantissa

capabilities.• Fixed point processing at 32 bits a sure bet.


References

• J. Dattorro, “The Implementation of Recursive Digital Filters for High Fidelity Audio,” J. Audio Eng. Soc., vol 36, pp 851-878 (1988 Nov)

• R. Wilson, “Filter Topologies,” J. Audio Eng. Soc., vol 41, pp 667-678 (1993 Sept)

• S.P. Lipshitz, R.A. Wannamaker and J. Vanderkooy, “Quantization and Dither: A Theoretical Study,” J. Audio Eng. Soc., vol 40, pp355-375 (1992 May)

presented at the 2004 comp.dsp conference mark allie consulting llc

Documents