![Page 1: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/1.jpg)
Li-Wei Kang and Chun-Shien LuInstitute of Information Science, Academia Sinica
Taipei, Taiwan, ROC{lwkang, lcs}@iis.sinica.edu.tw
April 20092009 IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP2009, Taipei, Taiwan, ROC)
Distributed Compressive Video Sensing
![Page 2: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/2.jpg)
Distributed Compressive Video Sensing April 24, 2009 2
Distributed Source Coding
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Vanishing error probabilityfor long sequences
No errors
[Slepian and Wolf, 1973]
![Page 3: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/3.jpg)
Distributed Compressive Video Sensing April 24, 2009 3
Distributed Video Coding
“Motion JPEG”
Decoder
“Motion JPEG”
Encoder
X’X
Wyner-ZivInterframe Decoder
Wyner-ZivIntraframe Encoder
Side Information
Y
[Girod, 2006]
![Page 4: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/4.jpg)
Distributed Compressive Video Sensing April 24, 2009 4
Distributed Video Coding• The statistical dependency between X and Y
Laplacian distribution
,2
ii YXii eYXp
ii YX
2
![Page 5: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/5.jpg)
Distributed Compressive Video Sensing April 24, 2009 5
Compressive Sensing
• When data is sparse/compressible, one can directly acquire a condensed representation with no/little information loss
• Random projection will work
[Baraniuk, 2008]
![Page 6: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/6.jpg)
Distributed Compressive Video Sensing April 24, 2009 6
Compressive Sensing
• Directly acquire “compressed” data• Replace samples by more general “measurements”
[Baraniuk, 2008]
![Page 7: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/7.jpg)
Distributed Compressive Video Sensing April 24, 2009 7
Compressive Sensing
• y = Фx = ФΨθ = Aθ
y Ф Ψ θ
x = Ψθ
A = ФΨ
N×1N×N
M×NM×1
[Baraniuk, 2008]
![Page 8: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/8.jpg)
Distributed Compressive Video Sensing April 24, 2009 8
Measurement Matrix
• Scrambled block Hadamard ensemble (SBHE)partial block hadamard transform and random column
permutation
Ф = QMWPN
L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled hadamard ensemble,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).
![Page 9: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/9.jpg)
Distributed Compressive Video Sensing April 24, 2009 9
Signal Reconstruction
• The convex unconstrained optimization problem
• Can be seen as a maximum a posteriori criterion for estimating θ from
y = A θ + n, where n is white Gaussian noise
122A
21min
y
![Page 10: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/10.jpg)
Distributed Compressive Video Sensing April 24, 2009 10
Signal Reconstruction
• Signal recovery from random measurementsGradient projection for sparse reconstruction (GPSR)Two-step iterative shrinkage/thresholding algorithm (TwIST)Orthogonal matching pursuit (OMP)
M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. of Selected Topics in Signal Processing, vol. 1,no. 4, pp. 586-597, Dec. 2007.
J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Processing, vol. 16, no. 12, pp. 2992-3004, Dec. 2007.
T. Blumensath and M. E. Davies, “Gradient pursuits,” IEEE Trans. on Signal Processing, vol. 56, June 2008.
![Page 11: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/11.jpg)
Distributed Compressive Video Sensing April 24, 2009 11
Distributed Compressive Video Sensing
• Measurement matrix Ф: scrambled block Hadamard ensemble (SBHE)
• Sparse basis matrix Ψ: DWT• Video signal sensing (encoder): general random
projection• Video signal recovery (decoder)
Key frame: GPSR with default settingsCS frame
side information generation (motion compensated interpolation)GPSR with the proposed initialization and the proposed
termination criteria
![Page 12: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/12.jpg)
Distributed Compressive Video Sensing April 24, 2009 12
Distributed Compressive Video Sensing
CS measurementyt = Ф xt
Each frame xtMeasurement vector (compressed frame) yt
Measurement vector ytfor each non-key frame
Initialization by SI generation
Reconstructed previous key frames
GPSR optimization
Stopping criteria (a)-(c)Non-stop Stop
ttx ~~ Reconstructed non-key frame tx~
Compressive video sensing
Video signal recovery
![Page 13: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/13.jpg)
Distributed Compressive Video Sensing April 24, 2009 13
Distributed Compressive Video Sensing
• At the decoder, for a CS frame xt = Ψθt
its side information St = ΨθSt can be generated from its previous reconstructed key frames
• Proposed initializationinitial solution at the 0-th iteration:
• α(xt, St): the Laplacian parameter of (xt- St)
,~ 0Stt ,~ 0
tt Sx
![Page 14: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/14.jpg)
Distributed Compressive Video Sensing April 24, 2009 14
Key frame (t - 1)
Non-key frame t
Key frame (t + 1)
yt-1 = Φxt-1
with higher MR
yt = Φxt
with lower MR
yt+1 = Φxt+1
with higher MR
GPSR reconstruction
GPSR reconstruction
Proposed Modified
GPSR reconstruction
Side information (t)
Reconstructed frame (t)
Reconstructed frame (t-1)
Side information generation
Reconstructed frame (t+1)
![Page 15: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/15.jpg)
Distributed Compressive Video Sensing April 24, 2009 15
Distributed Compressive Video Sensing itx~
xt St
α(xt, )
α(xt, St)
,21min 1
221 tttt AyF
t
22 StttF
it
it
it FWFWF ~~~
2211
itx~
α( , St) itx~
![Page 16: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/16.jpg)
Distributed Compressive Video Sensing April 24, 2009 16
Proposed Termination Criterion
• First:
• Second:
• Third:
T
Sx
SxSx
ti
t
ti
tti
t
,~,~,~
1
1
0~~ 1 it
it FF
Fi
t
it
it
TF
FF
1
1
~
~~
![Page 17: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/17.jpg)
Distributed Compressive Video Sensing April 24, 2009 17
Proposed Termination Criterion
• MR is low (MR ≤ 20%): if the First criterion with Tα = 0.9 is satisfied, the algorithm will stop
• MR is middle (20% < MR ≤ 70%): if the First criterion with Tα = 0.05 or the Second criterion is satisfied, the algorithm will stop
• MR is high (MR > 70%): if the Third criterion with TF = 0.001 is satisfied, the algorithm will stop
![Page 18: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/18.jpg)
Distributed Compressive Video Sensing April 24, 2009 18
Simulation Results
• Foreman and Coastguard CIF video sequences with 300 Y frames (352×288 = 101376 samples for each Y frame) and GOP size = 3 (Key, Non-key, Non-key, Key, …)
• The three approaches for comparison (all with default settings)GPSR, TwIST, OMP
• For OMP, block size = 32×32 suggested by V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling,” in Proc.
of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).
![Page 19: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/19.jpg)
Distributed Compressive Video Sensing April 24, 2009 19
Simulation Results
![Page 20: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/20.jpg)
Distributed Compressive Video Sensing April 24, 2009 20
Simulation Results
![Page 21: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/21.jpg)
Distributed Compressive Video Sensing April 24, 2009 21
Simulation Results
The reconstruction complexities for the Foreman sequence
![Page 22: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/22.jpg)
Distributed Compressive Video Sensing April 24, 2009 22
Simulation Results
The PSNR performance at different reconstruction complexities for the Foreman sequence
![Page 23: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/23.jpg)
Distributed Compressive Video Sensing April 24, 2009 23
Simulation Results
(a) Side information (b) Reconstructed frame
![Page 24: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/24.jpg)
Distributed Compressive Video Sensing April 24, 2009 24
Simulation Results
The reconstructed Foreman sequences (352×288 for each frame) at measurement rate (MR) = 0.3 using (a) GPSR (gradient projection for sparse reconstruction) (average PSNR = 27.68 dB) (average reconstruction time = 15.14 seconds per frame); and (b) our DCVS (average PSNR = 29.48 dB) (average reconstruction time = 3.68 seconds per frame) (This example shows the 54-th frame).
![Page 25: Li-Wei Kang and Chun-Shien Lu Institute of Information Science, Academia Sinica Taipei, Taiwan, ROC {lwkang, April 2009 2009 IEEE](https://reader036.vdocuments.net/reader036/viewer/2022062401/5a4d1b6e7f8b9ab0599b4628/html5/thumbnails/25.jpg)
Distributed Compressive Video Sensing April 24, 2009 25
Conclusions
• The proposed DCVS approach exploits the two characteristicsdistributed video coding (DVC)compressive sensing (CS)
• The proposed DCVS can outperform or be comparable with the three existing approaches for comparison, especially at lower measurement rates
• The proposed DCVS can significant outperform the three existing approaches at the same reconstruction complexity