High performance hardware architectures for Intra BlockCopy and Palette Coding for HEVC Screen Content

Coding extension

Rishan Senanayake1, Namitha Liyanage1, Sasindu Wijeratne1, SachilleAtapattu1, Kasun Athukorala1, P.M.K. Tharaka1, Geethan

Karunaratne1, R.M.A.U. Senarath1, Ishantha Perera1, AshenEkanayake1 and Ajith Pasqual2

1Paraqum Technologies,Colombo, Sri Lanka

2Department of Electronic and Telecommunication EngineeringUniversity of Moratuwa, Sri Lanka

ASAP 2017, The 27th Annual IEEE International Conference onApplication-specific Systems, Architectures and Processors

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 1 / 23

Outline

1 Background

2 Approaches

3 Novelties

4 Intra Block Copy

5 Palette Coding

6 Results

7 Conclusion

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 2 / 23

Background

H.265, introduced in 2013 achieves around 50% compressionefficiency compared to H.264

Mainly targets camera captured content

Modern videos include text, graphical motion and animations too

Extension to HEVC =⇒ HEVC Screen Content Coding(HEVC-SCC) finalized in 2016

HEVC-SCC achieves about 50% efficiency over HEVC for syntheticcontent

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 3 / 23

Features of Synthetic Content

Large uniformly flat areas

Repeated patterns

Highly saturated areas

Use of limited colors in an area

Numerically identical blocks

HEVC-SCC introduces Intra Block Copy (IBC) and Palette Coding (PLT)in addition to conventional transformation based coding

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 4 / 23

Examples : Gaming

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 5 / 23

Examples : Console Sharing

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 6 / 23

Examples : Navigation

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 7 / 23

Examples : Desktops

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 8 / 23

Challenge

50% bit rate reduction

...... But Encoding time increases by more than 2x

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 9 / 23

Approaches

SCC test model (SCM) approach - developed for testing andevaluating of the standard

Reduce complexity through pruning and low complexity estimating

Hash based speed ups by Kuo et.al and Tsang et.al.

Search area reduction by Tsang et.al.

Optimizing K-means clustering by Saegusa et.al and Winterstein et.al.

=⇒ Novel hardware based architectures proposed for IBC and Palettecoding

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 10 / 23

Novelties

Novel Three stage architectures for IBC and palette coding

New Hash Table to reduce resource usage by 8 times

Palette sorting architecture used in palette comparisons

Alternate coding approach to code palette syntax in parallel forRate-Distortion (RD) cost estimation

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 11 / 23

Intra Block Copy

Copies the content from a previously coded similar block

Challenge =⇒ Identifying best block

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 12 / 23

Architecture

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 13 / 23

Hash Optimization

....Proposed scheme reduces table size by a factor of 8

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 14 / 23

Palette Coding

Sending of color values of each pixel using indices

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 15 / 23

Palette Coding

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 16 / 23

Results - IBC

11% coding overheadcompared to SCMencoder

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 17 / 23

Results - Palette

5% coding overheadcompared to SCMencoder

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 18 / 23

Comparison

Total = 66K LUTs Total = 51K LUTs

Prototype implementations were synthesized for Xilinx VC707 platform

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 19 / 23

Conclusion

Hardware solutions are suitable for real time encoding of 1080p 30fpsfor HEVC-SCC

Significant amount of hardware resources can be saved with a slightcoding overhead

These novel architectures can be added to existing HEVC hardwareencoders to support HEVC-SCC profile

Future work will be on easy adaption of these architectures to existingHEVC encoders

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 20 / 23

References I

SCM Software repositoryAvailable : https://hevc.hhi.fraunhofer.de/scc/Accessed April 2017

C.-W. Kuo, H.-M. Hang, and C.-L. ChienIntra block copy hash reduction for hevc screen content codingSignal and Information Processing Association Annual Summit andConference (APSIPA) 2016

S.-H. Tsang, Y.-L. Chan, and W.-C. SiuHash based fast local search for intra block copy (intrabc) mode inhevc screen content codingSignal and Information Processing Association Annual Summit andConference (APSIPA), 2015

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 21 / 23

References II

S.-H. Tsang, W. Kuang, Y.-L. Chan, and W.-C. SiuFast hevc screen content coding by skipping unnecessary checking ofintra block copy mode based on cu activity and gradient.Signal and Information Processing Association Annual Summit andConference (APSIPA) 2016

T. Saegusa and T. MaruyamaReal-time segmentation of color images based on the k-meansclustering on fpga.International Conference on Field-Programmable Technology, Dec2007

F. Winterstein, S. Bayliss, and G. A. ConstantinidesFpga-based k-means clustering using tree-based data structures.23rd International Conference on Field programmable Logic andApplications

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 22 / 23

Thank you

High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 23 / 23