fast fourier transformjordicf/teaching/ap2/pdf/14_fast... · 2020. 2. 7. · •using the master...
TRANSCRIPT
![Page 1: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/1.jpg)
Fast Fourier Transform
Jordi Cortadella and Jordi Petit
Department of Computer Science
![Page 2: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/2.jpg)
Sum of sinusoids
Fast Fourier Transform © Dept. CS, UPC 2
![Page 3: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/3.jpg)
FFT application in Signal Processing
Fast Fourier Transform © Dept. CS, UPC 3
Converting a signal: time domain ↔ frequency domain
![Page 4: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/4.jpg)
Fourier series
• Periodic function 𝑓(𝑡) of period 1:
𝑓 𝑡 =𝑎02+
𝑛=1
∞
𝑎𝑛 cos(2𝜋𝑛𝑡) +
𝑘=1
∞
𝑏𝑛 sin(2𝜋𝑛𝑡)
• Fourier coefficients:
𝑎𝑛 = 2න0
𝑇
𝑓(𝑡) cos(2𝜋𝑛𝑡) 𝑑𝑡, 𝑏𝑛 = 2න0
𝑇
𝑓(𝑥) sin 2𝜋𝑛𝑡 𝑑𝑡
• Fourier series is fundamental for signal analysis (to move from time domain to frequency domain, and vice versa)
Fast Fourier Transform © Dept. CS, UPC 4
![Page 5: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/5.jpg)
Why Fourier Transform?
Fast Fourier Transform © Dept. CS, UPC 5
![Page 6: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/6.jpg)
Discrete Fourier Transform (DFT)
A method for converting a sequence of 𝑁complex numbers 𝑥0, 𝑥1, … , 𝑥𝑁−1 to a new sequence of 𝑁 complex numbers:
𝑋𝑘 =
𝑛=0
𝑁−1
𝑥𝑛𝑒−2𝜋𝑖𝑘𝑛/𝑁
such that
𝑥𝑛 =1
𝑁
𝑘=0
𝑁−1
𝑋𝑘𝑒2𝜋𝑖𝑘𝑛/𝑁
Fast Fourier Transform © Dept. CS, UPC 6
![Page 7: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/7.jpg)
Polynomials: coefficient representation
Fast Fourier Transform © Dept. CS, UPC 7
![Page 8: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/8.jpg)
Polynomials: point-value representation
• Fundamental Theorem (Gauss): A degree 𝑛 polynomial with complex coefficients has exactly 𝑛 complex roots.
• Corollary: A degree 𝑛 − 1 polymonial 𝐴(𝑥) is uniquely identified by its evaluation at 𝑛 distinct values of 𝑥.
Fast Fourier Transform © Dept. CS, UPC 8
![Page 9: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/9.jpg)
Polynomials: point-value representation
Fast Fourier Transform © Dept. CS, UPC 9
![Page 10: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/10.jpg)
Lagrange formula: example
Fast Fourier Transform © Dept. CS, UPC 10
𝑃 𝑥 = { −1,−1 , 0, −2 , 2,2 }
𝑃 𝑥 = −𝑥 𝑥 − 2
3− 2
𝑥 + 1 𝑥 − 2
−2+ 2
𝑥 + 1 𝑥
6
𝑃 𝑥 = 𝑥2 − 2
![Page 11: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/11.jpg)
Conversion between both representations
Fast Fourier Transform © Dept. CS, UPC 11
representation addition multiplication evaluation
coefficient O(𝑛) O(𝑛2) O(𝑛)
point-value O(𝑛) O(𝑛) O(𝑛2)
𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1
evaluation
interpolationCoefficient representation Point-value representation
Could we have an efficient algorithm to move from coefficientto point-value representation and vice versa?
![Page 12: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/12.jpg)
From coefficients to point-values
Fast Fourier Transform © Dept. CS, UPC 12
𝑝 𝑥 = 𝑎0 + 𝑥 𝑎1 + 𝑥 𝑎2 + 𝑥 𝑎3 +⋯+ 𝑥 𝑎𝑛−1 + 𝑥𝑎𝑛 ⋯
Horner’s rule:
![Page 13: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/13.jpg)
Evaluation by divide-and-conquer• Credits: based on the intuitive explanation by Dasgupta,
Papadimitriou and Vazinari, Algorithms, McGraw-Hill, 2008.
• We want to evaluate 𝐴(𝑥) at 𝑛 different points. Let us choose them to be positive-negative pairs: ±𝑥0, ±𝑥1, … , ±𝑥 Τ𝑛 2−1
• The computations for 𝐴(𝑥𝑖) and 𝐴(−𝑥𝑖) overlap a lot.
• Split the polynomial into odd and even powers
3 + 4𝑥 + 6𝑥2 + 2𝑥3 + 𝑥4 + 10𝑥5 = 3 + 6𝑥2 + 𝑥4 + 𝑥 4 + 2𝑥2 + 10𝑥4
• The terms in parenthesis are polynomials in 𝑥2:
𝐴 𝑥 = 𝐴𝑒 𝑥2 + 𝑥𝐴𝑜(𝑥2)
Fast Fourier Transform © Dept. CS, UPC 13
![Page 14: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/14.jpg)
Evaluation by divide-and-conquer
• The calculations needed for 𝐴 𝑥𝑖 can be reused for computing 𝐴 −𝑥𝑖 .
𝐴 𝑥𝑖 = 𝐴𝑒 𝑥𝑖2 + 𝑥𝑖𝐴𝑜 𝑥𝑖
2
𝐴 −𝑥𝑖 = 𝐴𝑒 𝑥𝑖2 − 𝑥𝑖𝐴𝑜 𝑥𝑖
2
• Evaluating 𝐴 𝑥 at 𝑛 paired points
±𝑥0, ±𝑥1, … , ±𝑥 Τ𝑛 2−1
reduces to evaluating 𝐴𝑒 𝑥 and 𝐴𝑜(𝑥) at just𝑛/2 points: 𝑥0
2, ⋯ , 𝑥𝑛/2−12
Fast Fourier Transform © Dept. CS, UPC 14
![Page 15: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/15.jpg)
Evaluation by divide-and-conquer
Fast Fourier Transform © Dept. CS, UPC 15
+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥𝑛/2−1 −𝑥𝑛/2−1⋯Evaluate: 𝐴 𝑥degree ≤ 𝑛 − 1
Evaluate:𝐴𝑒 𝑥 and 𝐴𝑜(𝑥)degree ≤ 𝑛/2 − 1
𝑥02 𝑥1
2 𝑥𝑛/2−12⋯
𝑇 𝑛 = 2 ⋅ 𝑇 𝑛/2 + O 𝑛 = O(𝑛 log 𝑛)
If we could recurse, we would get a running time:
But can we recurse?
![Page 16: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/16.jpg)
Evaluation by divide-and-conquer
Fast Fourier Transform © Dept. CS, UPC 16
+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥𝑛/2−1 −𝑥𝑛/2−1⋯Evaluate: 𝐴 𝑥degree ≤ 𝑛 − 1
Evaluate:𝐴𝑒 𝑥 and 𝐴𝑜(𝑥)degree ≤ 𝑛/2 − 1
𝑥02 𝑥1
2 𝑥𝑛/2−12⋯
The problem: ? We need 𝑥02 and 𝑥1
2 to be a plus-minus pair.But a square cannot be negative !
![Page 17: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/17.jpg)
Selection of the evaluation points
Fast Fourier Transform © Dept. CS, UPC 17
+𝑥0 −𝑥0 +𝑥1 −𝑥1 +𝑥2 −𝑥2 +𝑥3 −𝑥3
𝑥02 𝑥1
2 𝑥22 𝑥3
2
𝑥04 𝑥2
4
𝑥08
+ 𝑖 − 𝑖
+1
+1 −1
+1 −1
+1 −1
−𝑖+𝑖
+𝑖 −𝑖 + −𝑖 − −𝑖
Note:
𝑖 = ±1
21 + 𝑖
−𝑖 = ±1
2(1 − 𝑖)
![Page 18: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/18.jpg)
Selection of the evaluation points
Fast Fourier Transform © Dept. CS, UPC 18
+ 𝑖
− 𝑖
+1−1
+𝑖
−𝑖
+ −𝑖
− −𝑖
![Page 19: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/19.jpg)
Complex numbers: review
Fast Fourier Transform © Dept. CS, UPC 19
𝑟
𝜃
𝑎
𝑏
Imaginary
Real
𝑧 = 𝑎 + 𝑏𝑖 𝑧 = 𝑟(cos 𝜃 + 𝑖 sin 𝜃) = 𝑟𝑒𝑖𝜃
Polar coordinates: (𝑟, 𝜃)
Length: 𝑟 = 𝑎2 + 𝑏2
Angle 𝜃 ∈ 0,2𝜋 : cos 𝜃 =𝑎
𝑟, sin 𝜃 =
𝑏
𝑟
𝜃 can always be reduced modulo 2𝜋
Some examples:
Number −1 𝑖 5 + 5𝑖
Polar coords (1, 𝜋) (1, 𝜋/2) (5 2, Τ𝜋 4)
![Page 20: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/20.jpg)
Complex numbers: multiplication
Fast Fourier Transform © Dept. CS, UPC 20
(𝑟1, 𝜃1)
(𝑟2, 𝜃2)
(𝑟1𝑟2, 𝜃1 + 𝜃2)
𝑟1, 𝜃1 × 𝑟2, 𝜃2 = (𝑟1𝑟2, 𝜃1 + 𝜃2)
For any 𝑧 = 𝑟, 𝜃 :
−𝑧 = (𝑟, 𝜃 + 𝜋), since −1 = 1, 𝜋
If 𝑧 is on the unit circle, then 𝑧𝑛 = (1, 𝑛𝜃)
![Page 21: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/21.jpg)
Complex numbers: the 𝑛th roots of unity
Fast Fourier Transform © Dept. CS, UPC 21
Angle 2𝜋/𝑛
4𝜋/𝑛
2𝜋
𝑛+ 𝜋
Solutions to the equation 𝑧𝑛 = 1(𝑛 = 16)
Solutions are 𝑧 = 1, 𝜃 ,for 𝜃 a multiple of 2𝜋/𝑛
All roots are plus-minus paired:
− 1, 𝜃 = (1, 𝜃 + 𝜋)
![Page 22: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/22.jpg)
Roots of unity for 𝑛 = 8
Fast Fourier Transform © Dept. CS, UPC 22
+ 𝑖
− 𝑖
+1−1
+𝑖
−𝑖
+ −𝑖
− −𝑖
![Page 23: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/23.jpg)
Divide-and-conquer step
Fast Fourier Transform © Dept. CS, UPC 23
Evaluate 𝐴(𝑥) at 𝑛th roots of unityEvaluate 𝐴𝑒(𝑥) and 𝐴𝑜 𝑥at Τ𝑛 2 nd roots of unity
![Page 24: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/24.jpg)
Divide-and-conquer steps
Fast Fourier Transform © Dept. CS, UPC 24
![Page 25: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/25.jpg)
FFT algorithm
function FFT(𝑨,𝝎)Inputs: 𝑨 = (𝒂𝟎, 𝒂𝟏, … , 𝒂𝒏−𝟏), for 𝒏 a power of 2
𝝎: A primitive 𝒏th root of unity
Output: 𝑨 𝟏 , 𝑨 𝝎 , 𝑨 𝝎𝟐 , … , 𝑨 𝝎𝒏−𝟏
if 𝝎=1: return 𝐴
𝑨𝒆 𝝎𝟎 , 𝑨𝒆 𝝎𝟐 , … , 𝑨𝒆(𝝎𝒏−𝟐) = FFT(𝑨𝒆, 𝝎
𝟐)
𝑨𝒐 𝝎𝟎 , 𝑨𝒐 𝝎𝟐 , … , 𝑨𝒐(𝝎𝒏−𝟐) = FFT(𝑨𝒐, 𝝎
𝟐)
for 𝒌 = 𝟎 to 𝒏 − 𝟏:𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎
𝟐𝒌)
return 𝑨 𝟏 , 𝑨 𝝎 , 𝑨 𝝎𝟐 , … , 𝑨(𝝎𝒏−𝟏)
Fast Fourier Transform © Dept. CS, UPC 25
![Page 26: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/26.jpg)
FFT algorithm
for 𝒌 = 𝟎 to 𝒏 − 𝟏: 𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎𝟐𝒌)
Fast Fourier Transform © Dept. CS, UPC 26
Example (𝒏 = 𝟖):
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔4 = 𝐴𝑒 𝜔8 + 𝜔4𝐴𝑜 𝜔8
𝐴 𝜔5 = 𝐴𝑒 𝜔10 + 𝜔5𝐴𝑜 𝜔10
𝐴 𝜔6 = 𝐴𝑒 𝜔12 + 𝜔6𝐴𝑜 𝜔12
𝐴 𝜔7 = 𝐴𝑒 𝜔14 + 𝜔7𝐴𝑜 𝜔14
𝝎𝟎
𝝎𝟏
𝝎𝟐
𝝎𝟑
𝝎𝟒
𝝎𝟓
𝝎𝟔
𝝎𝟕
𝝎𝟒 = 𝝎𝟏𝟐 𝝎𝟐 = −𝝎𝟔
![Page 27: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/27.jpg)
FFT algorithm
for 𝒌 = 𝟎 to 𝒏 − 𝟏: 𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎𝟐𝒌)
Fast Fourier Transform © Dept. CS, UPC 27
Example (𝒏 = 𝟖):
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔4 = 𝐴𝑒 𝜔8 + 𝜔4𝐴𝑜 𝜔8
𝐴 𝜔5 = 𝐴𝑒 𝜔10 + 𝜔5𝐴𝑜 𝜔10
𝐴 𝜔6 = 𝐴𝑒 𝜔12 + 𝜔6𝐴𝑜 𝜔12
𝐴 𝜔7 = 𝐴𝑒 𝜔14 + 𝜔7𝐴𝑜 𝜔14
𝝎𝟒 = 𝝎𝟏𝟐 𝝎𝟐 = −𝝎𝟔
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔4 = 𝐴𝑒 𝜔0 − 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔5 = 𝐴𝑒 𝜔2 −𝜔1𝐴𝑜 𝜔2
𝐴 𝜔6 = 𝐴𝑒 𝜔4 − 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔7 = 𝐴𝑒 𝜔6 − 𝜔3𝐴𝑜 𝜔6
![Page 28: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/28.jpg)
FFT algorithm
for 𝒌 = 𝟎 to 𝒏 − 𝟏: 𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎𝟐𝒌)
Fast Fourier Transform © Dept. CS, UPC 28
Example (𝒏 = 𝟖):
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔4 = 𝐴𝑒 𝜔0 − 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔5 = 𝐴𝑒 𝜔2 −𝜔1𝐴𝑜 𝜔2
𝐴 𝜔6 = 𝐴𝑒 𝜔4 − 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔7 = 𝐴𝑒 𝜔6 − 𝜔3𝐴𝑜 𝜔6
FFT shuffling
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔4 = 𝐴𝑒 𝜔0 − 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔5 = 𝐴𝑒 𝜔2 −𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔6 = 𝐴𝑒 𝜔4 − 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔7 = 𝐴𝑒 𝜔6 − 𝜔3𝐴𝑜 𝜔6
![Page 29: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/29.jpg)
FFT algorithm
for 𝒌 = 𝟎 to 𝒏 − 𝟏: 𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐(𝝎𝟐𝒌)
Fast Fourier Transform © Dept. CS, UPC 29
Example (𝒏 = 𝟖):
𝐴 𝜔0 = 𝐴𝑒 𝜔0 + 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔4 = 𝐴𝑒 𝜔0 − 𝜔0𝐴𝑜 𝜔0
𝐴 𝜔1 = 𝐴𝑒 𝜔2 + 𝜔1𝐴𝑜 𝜔2
𝐴 𝜔5 = 𝐴𝑒 𝜔2 −𝜔1𝐴𝑜 𝜔2
𝐴 𝜔2 = 𝐴𝑒 𝜔4 + 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔6 = 𝐴𝑒 𝜔4 − 𝜔2𝐴𝑜 𝜔4
𝐴 𝜔3 = 𝐴𝑒 𝜔6 + 𝜔3𝐴𝑜 𝜔6
𝐴 𝜔7 = 𝐴𝑒 𝜔6 − 𝜔3𝐴𝑜 𝜔6
for 𝒌 = 𝟎 to 𝒏/𝟐 − 𝟏:
𝑨 𝝎𝒌 = 𝑨𝒆 𝝎𝟐𝒌 +𝝎𝒌𝑨𝒐 𝝎𝟐𝒌
𝑨 𝝎𝒌+𝒏𝟐 = 𝑨𝒆 𝝎𝟐𝒌 −𝝎𝒌𝑨𝒐 𝝎𝟐𝒌
![Page 30: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/30.jpg)
FFT algorithm
function FFT(𝒂,𝝎)Inputs: 𝒂 = (𝒂𝟎, 𝒂𝟏, … , 𝒂𝒏−𝟏), for 𝒏 a power of 2
𝝎: A primitive 𝒏th root of unity
Output: 𝒂 𝟏 , 𝒂 𝝎 , 𝒂 𝝎𝟐 , … , 𝒂 𝝎𝒏−𝟏
if 𝝎=1: return a // 𝒏 = 𝟏, 𝒂 has only one element
𝒔𝟎, 𝒔𝟏, … , 𝒔 Τ𝒏 𝟐−𝟏 = FFT( 𝒂𝟎, 𝒂𝟐, … , 𝒂𝒏−𝟐 , 𝝎𝟐)
𝒔𝟎′ , 𝒔𝟏
′ , … , 𝒔 Τ𝒏 𝟐−𝟏′ = FFT 𝒂𝟏, 𝒂𝟑, … , 𝒂𝒏−𝟏 , 𝝎𝟐
for 𝒌 = 𝟎 to Τ𝒏 𝟐 − 𝟏: // FFT shuffling𝒓𝒌 = 𝒔𝒌 +𝝎𝒌𝒔𝒌
′
𝒓𝒌+ Τ𝒏 𝟐 = 𝒔𝒌 −𝝎𝒌𝒔𝒌′
return (𝒓𝟎, 𝒓𝟏, … , 𝒓𝒏−𝟏)
Fast Fourier Transform © Dept. CS, UPC 30
![Page 31: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/31.jpg)
FFT: asymptotic complexity• The runtime of the FFT can be expressed as:
𝑇 𝑛 = 2 ⋅ 𝑇 Τ𝑛 2 + O 𝑛
• Using the Master Theorem we conclude:
Runtime FFT 𝑛 = O(𝑛 log 𝑛)
• Gilbert Strang (MIT, 1994):
“the most important numericalalgorithm of our lifetime”.
• Reference: Cooley, James W., and John W. Tukey, 1965,“An algorithm for the machine calculation of complex Fourier series,” Math. Comput. 19: 297-301.
Fast Fourier Transform © Dept. CS, UPC 31
![Page 32: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/32.jpg)
Unfolding the FFT
Fast Fourier Transform © Dept. CS, UPC 32
𝐹𝐹𝑇 Τ𝑛 2
𝑎0𝑎2
𝑎𝑛−2
⋮
𝐹𝐹𝑇 Τ𝑛 2
𝑎1𝑎3
𝑎𝑛−1
⋮
𝑟𝑘
𝑟𝑘+ Τ𝑛 2+
+
𝜔𝑘
𝜔𝑘+ Τ𝑛 2
![Page 33: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/33.jpg)
Fast Fourier Transform © Dept. CS, UPC 33
Unfolding the FFT (butterfly diagram)𝑎0
𝑎4
𝑎2
𝑎6
𝑎1
𝑎5
𝑎3
𝑎7
𝐴(𝜔0)
𝐴(𝜔1)
𝐴(𝜔2)
𝐴(𝜔3)
𝐴(𝜔4)
𝐴(𝜔5)
𝐴(𝜔6)
𝐴(𝜔7)
4
4
4
4
4
4
2
2
6
6
4
2
3
7
6
5
1
000
001
010
011
100
101
110
111
000
100
010
110
001
101
011
111
![Page 34: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/34.jpg)
Why is it called a butterfly diagram?
Fast Fourier Transform © Dept. CS, UPC 34
![Page 35: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/35.jpg)
Conversion between both representations
Fast Fourier Transform © Dept. CS, UPC 35
representation addition multiplication evaluation
coefficient O(𝑛) O(𝑛2) O(𝑛)
point-value O(𝑛) O(𝑛) O(𝑛2)
𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1
evaluation
interpolationCoefficient representation Point-value representation
values = FFT( coefficients , 𝜔)
O(𝒏 𝐥𝐨𝐠𝒏)
![Page 36: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/36.jpg)
From point-values to coefficients
Fast Fourier Transform © Dept. CS, UPC 36
![Page 37: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/37.jpg)
From point-values to coefficients
Fast Fourier Transform © Dept. CS, UPC 37
![Page 38: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/38.jpg)
Conversion between both representations
Fast Fourier Transform © Dept. CS, UPC 38
representation addition multiplication evaluation
coefficient O(𝑛) O(𝑛2) O(𝑛)
point-value O(𝑛) O(𝑛) O(𝑛2)
𝑎0, 𝑎1, ⋯ , 𝑎𝑛−1 𝑥0, 𝑦0 , ⋯ , 𝑥𝑛−1, 𝑦𝑛−1
evaluation
interpolationCoefficient representation Point-value representation
values = FFT( coefficients , 𝜔)
coefficients =1
𝑛FFT( values , 𝜔−1)
O(𝒏 𝐥𝐨𝐠𝒏)
![Page 39: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/39.jpg)
Polynomial multiplication
Input: Coefficients of two polynomials 𝐴(𝑥) and 𝐵(𝑥),of degree 𝑑𝐴 and 𝑑𝐵, respectively. Let 𝑑 = 𝑑𝐴 + 𝑑𝐵.
Output: The product 𝐶 = 𝐴 ⋅ 𝐵.
1. Selection: Pick 𝜔 = (1, Τ2𝜋 𝑛), such that 𝑛 ≥ 𝑑 + 1 and 𝑛 is a power of two.
2. Evaluation (FFT): Compute 𝐴 1 , 𝐴 𝜔 , 𝐴 𝜔2 , … , 𝐴 𝜔𝑛−1 . Compute 𝐵 1 , 𝐵 𝜔 , 𝐵 𝜔2 , … , 𝐵 𝜔𝑛−1 .
3. Multiplication: Compute 𝐶 𝜔𝑘 = 𝐴 𝜔𝑘 ⋅ 𝐵(𝜔𝑘), for all 𝑘 = 0,… , 𝑛 − 1.
4. Interpolation (inverse FFT): Recover 𝐶 𝑥 = 𝑐0 + 𝑐1𝑥 + 𝑐2𝑥
2 +⋯+ 𝑐𝑑𝑥𝑑.
Fast Fourier Transform © Dept. CS, UPC 39
![Page 40: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/40.jpg)
Example: from values to coefficients• Let us consider a polynomial:
𝑃 𝑥 = 𝑎0 + 𝑎1𝑥 + 𝑎2𝑥2 + 𝑎3𝑥
3
• We have 𝑛 = 4 and 𝜔 = 𝑖. Let us assume that the evaluation at four different points is:
𝑃 1 = 2𝑃 𝑖 = 1 − 𝑖𝑃 −1 = 4𝑃 −𝑖 = 1 + 𝑖
• We want to calculate the coefficients (𝑎0, 𝑎1, 𝑎2, 𝑎3) using the inverse FFT, i.e.,
[a0, a1, a2, a3] =1
4FFT( 2, 1 − 𝑖, 4, 1 + 𝑖 , 𝜔−1)
Fast Fourier Transform © Dept. CS, UPC 40
![Page 41: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/41.jpg)
Example: from values to coefficients
Fast Fourier Transform © Dept. CS, UPC 41
FFT( 2,1 − 𝑖, 4,1 + 𝑖 , 𝜔 = −𝑖)
FFT( 2,4 , 𝜔 = −1) FFT( 1 − 𝑖, 1 + 𝑖 , 𝜔 = −1)
FFT( 2 ,𝜔 = 1) FFT( 4 ,𝜔 = 1) FFT( 1 − 𝑖 , 𝜔 = 1) FFT( 1 + 𝑖 , 𝜔 = 1)
2 4 1 − 𝑖 1 + 𝑖
𝑟0 = 2 + 4 = 6𝑟1 = 2 − 4 = −2
𝑟0 = 1 − 𝑖 + 1 + 𝑖 = 2𝑟1 = 1 − 𝑖 − 1 − 𝑖 = −2𝑖
𝑟0 = 6 + 2 = 8𝑟2 = 6 − 2 = 4𝑟1 = −2 + −𝑖 −2𝑖 = −4𝑟3 = −2 − −𝑖 −2𝑖 = 0
𝑘 = 0 ቊ
𝑘 = 1 ቊ
𝑎0, 𝑎1, 𝑎2, 𝑎3 =1
4[8,−4,4,0] 𝑃 𝑥 = 𝑥2 − 𝑥 + 2
![Page 42: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/42.jpg)
Example: from coefficients to values
Fast Fourier Transform © Dept. CS, UPC 42
FFT( 2,−1,1,0 , 𝜔 = 𝑖)
FFT( 2,1 , 𝜔 = −1) FFT( −1,0 , 𝜔 = −1)
FFT( 2 ,𝜔 = 1) FFT( 1 ,𝜔 = 1) FFT( −1 ,𝜔 = 1) FFT( 0 ,𝜔 = 1)
2 1 −1 0
𝑟0 = 2 + 1 = 3𝑟1 = 2 − 1 = 1
𝑟0 = −1 + 0 = −1𝑟1 = −1 − 0 = −1
𝑟0 = 3 − 1 = 2𝑟2 = 3 + 1 = 4𝑟1 = 1 + 𝑖 ⋅ −1 = 1 − 𝑖𝑟3 = 1 − 𝑖 ⋅ −1 = 1 + 𝑖
𝑘 = 0 ቊ
𝑘 = 1 ቊ
𝑦0, 𝑦1, 𝑦2, 𝑦3 = [2,1 − 𝑖, 4,1 + 𝑖]𝑦0, 𝑦1, 𝑦2, 𝑦3 = FFT( 𝑎0, 𝑎1, 𝑎2, 𝑎3 , 𝜔 = 𝑖)
𝑃 𝑥 = 𝑥2 − 𝑥 + 2
![Page 43: Fast Fourier Transformjordicf/Teaching/AP2/pdf/14_Fast... · 2020. 2. 7. · •Using the Master Theorem we conclude: RuntimeFFT𝑛=O(𝑛log𝑛) •Gilbert Strang (MIT, 1994):](https://reader033.vdocuments.net/reader033/viewer/2022051918/600b05f420e1ac2a39685f4b/html5/thumbnails/43.jpg)
Exercises
1. Consider the polynomials 1 + 𝑥 − 2𝑥2 + 𝑥3 and −1 + 𝑥2:– Choose an appropriate power of two to execute the FFT for the
polynomial multiplication. Find the value of 𝜔.
– Give the result of the FFT for 𝑥2 − 1(no need to execute the FFT).
2. Consider the polynomials −1 + 2𝑥 + 𝑥2 and 1 + 2𝑥:– Choose an appropriate power of two to execute the FFT.
Find the value of 𝜔.
– Calculate their point-value representation using the FFT (execute the FFT algorithm manually).
– Calculate the product of the point-value representations.
– Execute the inverse FFT to obtain the coefficients of the product.
Fast Fourier Transform © Dept. CS, UPC 43