distributed video compression for smart visual sensors
TRANSCRIPT
![Page 1: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/1.jpg)
Distributed Video Compression for Smart Visual Sensors
Nikos Deligiannis
![Page 2: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/2.jpg)
Pag.
Wireless Multimedia Sensor Networks
6/06/2013 2
• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors (Little Sister Project)
The mouse sensor©Xetal
![Page 3: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/3.jpg)
Pag.
Wireless Multimedia Sensor Networks
6/06/2013 3
• Wireless Lightweight Multimedia Applications:– Wireless visual surveillance sensors– Wearable/On-body visual sensors– In-body Sensors (Wireless capsule endoscopy)
Video Compression Requirements: Low complexity encoding High compression performance Scalability Communication error resilience
©Given Imaging
![Page 4: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/4.jpg)
Pag.
Traditional Predictive Coding
6/06/2013 4
Traditional Predictive Video Coding (H.26x, MPEG)• Redundancies exploited at the encoder
– Motion Compensated Prediction• Characteristics
– High compression performance– Complex encoder − Simple decoder– Downlink model (Broadcast Scenario)
![Page 5: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/5.jpg)
Pag.
Distributed Video Coding
6/06/2013 5
Distributed Video Coding• Redundancies exploited at the decoder
– Motion/Disparity Estimation at the decoder• Characteristics
– Simple encoder − Complex decoder– Error Resilience– Good compression performance– Uplink model (Multi-terminal Communication Scenario)– **No need for communication between the sensors**
![Page 6: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/6.jpg)
Pag.
Intra Coding
Mono-View Distributed Video Coding**High-level Architecture**
6/06/2013 6
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
![Page 7: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/7.jpg)
Pag.
Stereo / Multi-View DVC**High-level Principle**
6/06/2013 7
Keyframe
WZ frame
WZframe
Key frame
![Page 8: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/8.jpg)
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
• Current DVC Systems H.263+ or H.264/AVC Intra
• Proposed Designated System:
6/06/2013 8
![Page 9: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/9.jpg)
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
6/06/2013 9
0 20 40 60 80 10015
20
25
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
B2_Huffman_noIntraPredB2_Huffman_IntraPredB2_CAVLC_noIntraPredB2_CAVLC_IntraPredB4_Huffman_noIntraPredB4_Huffman_IntraPredB4_CAVLC_noIntraPredB4_CAVLC_IntraPredB8_Huffman_noIntraPredB8_Huffman_IntraPredB8_CAVLC_noIntraPredB8_CAVLC_IntraPred
![Page 10: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/10.jpg)
Pag.
Efficient Intra-Frame Coding**Low-Resolution Image Sensors**
6/06/2013 10
Compared to H.264/AVC Intra:Vast Reduction in Encoding Execution Time
0 10 20 30 40 50 60 7020
25
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
H.264/AVC Intra (JM18.4)B4_CAVLC_IntraPred
![Page 11: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/11.jpg)
Pag.
Intra Coding
Distributed Video Coding**High-level Architecture**
6/06/2013 11
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
![Page 12: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/12.jpg)
Pag.
SI Generation at the Decoder** Motion-Compensated Interpolation **
6/06/2013 12
![Page 13: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/13.jpg)
Pag.
Intra Coding
Endoscopic Distributed Video Coding
6/06/2013 13
DCT Q SW Enc. SW Dec. IDCTQ-1
Correlation Channel
Side Information
Hash CodingHash CodingThe EDVC system: The hash can act as SI predictor Modified OBMEC with HPS
![Page 14: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/14.jpg)
Pag.
SI Generation in EDVC** OBMEC with HPS **
6/06/2013 14
Lanczos-3 Interpolation Filter Reliability Screening MSE-optimal Motion Compensation
![Page 15: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/15.jpg)
Pag.
Compression Results** Endoscopic Sequences **
6/6/2013 15
43.3% BjøntegaardRate Savings
31
33
35
37
39
41
43
400 800 1200 1600 2000 2400
PSN
R Y
(dB
)
Total Rate Y (kbps)
Proposed DVC with HPS
TDWZ
“Endoscopy Test Video 2”, 480×320, GOP2, 30Hz
![Page 16: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/16.jpg)
Pag.
Compression Results** Endoscopic Sequences **
H.264/AVC Intra (790kbps, 37.9dB)
6/6/2013 16
Proposed EDVC (800kbps, 39.2dB)
EDVC advantages: High compression performance Low encoding complexity (~56% less than H.264/AVC Intra) Scalable coding
![Page 17: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/17.jpg)
Pag.
Compression Performance of HDVC
6/6/2013 17
31.4% BjøntegaardRate Savings
26
28
30
32
34
36
38
40
50 150 250 350 450 550
PSN
R Y
(dB
)
Total Rate Y (kbps)
DISCOVER
Proposed HDVC
Foreman Sequence, QCIF, 15Hz, GOP8
![Page 18: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/18.jpg)
Pag.
System Architecture**Codeword Formation and Side Info Refinement**
6/06/2013 18
DCT Q SW Enc. SW Dec. IDCTQ-1
Successively Refined Correlation Estimation
Side Information(OBMEC/SAD)
Buffer
DCT
Partially Decoded Frame
![Page 19: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/19.jpg)
Pag.
Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**
6/06/2013 19
Uncompressed Data Rate ~ 290 kbps
20 40 60 80 100 120 140 160 18028
30
32
34
36
38
40
42
44
Bitrate (Kbps)
PSN
R(d
B)
Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra
![Page 20: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/20.jpg)
Pag.
Proposed Distributed Video Coding**Results on Low-Resolution Sensor Data**
6/06/2013 20
Uncompressed Data Rate ~ 316 kbps
20 40 60 80 100 120 140 160 18025
30
35
40
45
Bitrate (Kbps)
PSN
R(d
B)
Proposed DVC GOP2Proposed DVC GOP4Proposed DVC GOP8H.264/AVC Intra
![Page 21: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/21.jpg)
Pag.
Architecture with Encoder Rate Control**Feedback Channel Removal**
6/06/2013 21
DCT Q SW Enc. SW Dec. IDCTQ-1
Successively Refined Correlation Estimation
Side Information
Buffer
DCT
Coarse Side Info
Rate Allocation
Mode Signal.
Entropy Dec-1
Mode Decision(Intra/Skip/WZ)
Arithmetic Entropy Enc.
![Page 22: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/22.jpg)
Pag.
Architecture with Encoder Rate Control**Performance Evaluation**
Vs. feedback-based benchmark (DISCOVER) Better Rate-Distortion Performance No Encoder Syndrome Buffering No FB channel Latency
6/6/2013 22
26
28
30
32
34
36
38
40
0 100 200 300 400 500 600
PS
NR
Y (
dB
)
Total Rate (Kbps)
DISCOVERH.264/AVC IntraProposed feedback-free DVC
26
28
30
32
34
36
38
40
0 100 200 300 400 500 600
PS
NR
Y (
dB
)
Total Rate (Kbps)
DISCOVER
H.264/AVC Intra
Proposed feedback-free DVC
Foreman QCIF GOP4 Soccer QCIF GOP4
![Page 23: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/23.jpg)
Pag.
Conclusions
Novel DVC architectures• Visual Sensor Applications• Significant gains over prior art
Novel Intra-Frame Codec Novel SI generation methods Novel Encoder-Rate Control Mechanism
6/6/2013 23
![Page 24: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/24.jpg)
Pag.
Selected PublicationsJournal Papers
1. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Progressively refined Wyner-Ziv video coding for visual sensors," ACM Transactions on Sensor Networks, Special Issue on Advancements in Distributed Smart Camera Networks, to appear, May 2014. (IF 2011: 1.808).
2. S. M. Satti, N. Deligiannis, A. Munteanu, P. Schelkens, and J. Cornelis, "Symmetric scalable multiple description scalar quantization," IEEE Transactions on Signal Processing, vol. 60, no. 7, pp. 3628-3643, July 2012 (IF 2011: 2.628).
3. N. Deligiannis, F. Verbist, A. Iossifides, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Wyner-Ziv video coding for wireless lightweight multimedia applications," EURASIP Journal on Wireless Communications and Networking, no. 106, 2012. Special Issue on Recent Advances in Mobile Lightweight Wireless Systems. (IF 2011: 0.873).
4. N. Deligiannis, J. Barbarien, M. Jacobs, A. Munteanu, A. Skodras, and P. Schelkens, " Side-information-dependent correlation channel estimation in hash-based distributed video coding," IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1934-1949, April 2012 (IF 2011: 3.042).
5. J. Slowack, J. Škorupa, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, "Distributed video coding with feedback channel constraints," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 7, pp. 1014-1026, July 2012 (IF 2011: 1.649).
6. J. Škorupa, J. Slowack, S. Mys, N. Deligiannis, J. De Cock, P. Lambert, C. Grecos, A. Munteanu, and R. Van de Walle, "Efficient low-delay distributed video coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 4, pp. 530-544, April 2012 (IF 2011: 1.649).
7. N. Deligiannis, A. Munteanu, T. Clerckx, J. Cornelis and P. Schelkens, “Overlapped block motion estimation and probabilistic compensation with application in distributed video coding,” IEEE Signal Processing Letters, vol. 16, no. 9, pp. 743-746, September 2009 (IF 2009: 1.173).
8. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "Maximum likelihood motion compensation for distributed video coding," Integrated Computer Aided Engineering, vol. 19, no. 3, pp. 215-227, 2012 (IF 2011: 3.451).
6/06/2013 24
![Page 25: Distributed Video Compression for Smart Visual Sensors](https://reader034.vdocuments.net/reader034/viewer/2022042101/62564db5431d5608833eee03/html5/thumbnails/25.jpg)
Pag.
Selected PublicationsConference Papers
1. S. Wang, X. Jiang, L. Cui, W. Dai, N. Deligiannis, P. Li, H. Xiong, S. Cheng, L. Ohno-Machado, “Genome sequence compression with distributed source coding,” in IEEE Data Compression Conference, DCC’13, Snowbird, Utah, USA, March 2013.
2. F. Verbist, N. Deligiannis, S. M. Satti, A. Munteanu, and P. Schelkens, "Iterative Wyner-Ziv decoding and successive side information refinement in feedback channel-free hash-based distributed video coding," in SPIE Optics and Photonics, Optical Engineering and Applications, Applications of Digital Image Processing XXXV, San Diego, California, USA, August 2012. Special Session on Distributed Video Coding.
3. N. Deligiannis, F. Verbist, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Joint successive correlation estimation and side information refinement in distributed video coding," in European Signal Processing Conference, EUSIPCO’12, Bucarest, Romania, August 2012. Special Session on Distributed Video Coding.
4. J. Slowack, N. Deligiannis, P. Lambert, A. Munteanu, and R. Van de Walle, " Feedback-constrained Wyner-Ziv video coding," in Picture Coding Symposium, PCS’12, Krakow, Poland, May 2012.
5. N. Deligiannis, F. Verbist, J. Barbarien, J. Slowack, R. Van de Walle, P. Schelkens, and A. Munteanu, "Distributed coding of endoscopic video," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011. Special Session on Distributed Compression.
6. J. Slowack, J. Skorupa, P. Lambert, R. Van de Walle, N. Deligiannis, and A. Munteanu, "Intra-WZ quantization mismatch in distributed video coding," in IEEE International Conference on Image Processing, ICIP’11, Brussels, Belgium, September 2011.
7. F. Verbist, N. Deligiannis, M. Jacobs, J. Barbarien, P. Schelkens, and A. Munteanu, "A statistical approach to create side information in distributed video coding," in ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC’11, Ghent, Belgium, August 2011. (This paper received the ACM/IEEE ICDSC’11 Best Paper Award).
6/06/2013 25