Framebuffer Compression Using Dynamic Color Palettes (DCP)

Ayub A. GubranTor M. Aamodt

Motivation: framebuffer life cycleApplication 1 Application 2 Application 3

OS Graphics Libraries / Graphic API calls

CPU (Software Rendering) / GPU (Hardware Rendering)

Render Framebuffer 1 Render Framebuffer 2 Render Framebuffer 3

System Compositor (e.g., Android’s SurfaceFlinger)

Display framebuffer

Display Controller

Device Screen

@ screen refresh rate (i.e., 60 FPS)

Refresh rates vary by

application and user activity

Recompose after every

framebuffer update

GPU or CPU Composition

OS Level

App Level

HW Level

Legends

Framebuffer Compression Using Dynamic Color Palettes 2

Framebuffer Compression Using Dynamic Color Palettes 3

Motivation: The cost of accessing the framebuffer

•Every frame is consumed/processed multiple times in multiple stages.

•Energy cost: off-chip memory.

Framebuffer Compression Using Dynamic Color Palettes 4

Motivation: Mobile Applications Usage

Gaming32%

UI Apps68%

Time Spent on iOS and Android Connected Devices1

1- Flurry Analytics: It’s an App World. The Web Just Lives in It.

Framebuffer Compression Using Dynamic Color Palettes 5

Framebuffer Compression for UI Apps

•Most of the time is spent on UI apps with simple content (solid backgrounds, text, icons…).

•Simple content can be compressed well using color palettes (dictionaries).

•But how we can accurately predict the right color palette for each frame before we start rendering it.

Framebuffer Compression Using Dynamic Color Palettes 6

Observation: Inter-frame Temporal Coherence

2 187 372 557 742 927 11121297148216671852203722220%

10%20%30%40%50%60%70%80%90%

Pixels color change (new content) Per pixel change (moving content)

New web searchScrolling Loading BBC newsLoading Amazon

• Web browsing (Chrome)

Time (frames)

% o

f scr

een

size

Mostly the same content

Framebuffer Compression Using Dynamic Color Palettes 7

Dynamic Color Palettes (DCP)

Framebuffer Compression Using Dynamic Color Palettes 8

Compression Steps

Frame 0 Frame 1 Frame 2

FVC

Time

Construct FVC from current frame’s pixel

values

Construct the CCD dictionary using the FVC values

To memory

CCDFVC

To memory

FVCCCD

To memory

Framebuffer access

compressed using the CCD

FVC: Frequent Values Collector

CCD: Common Colors Dictionary

Framebuffer Compression Using Dynamic Color Palettes 9

Compression Pipeline

9

GPU

Pixel tiles

1

CCD (Compress current tile)

2b

Write a compressed tile

Write a non-compressed tile

Can compress all pixels in the current tile?

Yes No

3

FVC (constructs a CCD for the next frame)

2a

Send Tile

Buffer compressed & non-compressed tiles

of block

Write to memory

4

Associative search. Evict & replace on miss. Size 16-128

Lookup table

Framebuffer Compression Using Dynamic Color Palettes 10

DCP Variations

• Improving compression rates: DCP: A fixed size color palette. Variable DCP: Varying the color palette size every frame using the feedback

from the previous frame. Adaptive DCP: Encoding tiles based on the frequencies of their colors.

• Reducing the cost of DCP:• Lower sampling rates: Only sending every nth pixel to the FVC to construct

the palette.• Using the same palette for multiple frames.

Framebuffer Compression Using Dynamic Color Palettes

Results

AB PH CH FB HO KD MS ST TW UI HM

TR GS NS FN 3D HM

1

10

100 RAS REDDCP VDCPADCP

Com

pres

sion

Rat

e

11UI and 2D applications 3D games

RED: Locally compressing similar colors in a tile2

RAS: A more general prediction based compression3

ADCP 4.13:1VDCP 3.5:1DCP 2.86:1

RAS 2.5:1RED 2.62:1

RAS

Cost: using 64 FVC and CCDFVC: 456 bytesCCD: 264 bytes

Framebuffer Compression Using Dynamic Color Palettes 12

Limitations and Potential

• 3D games: Complex color patterns. Color palettes perform poorly.

• Multiple render targets: Need to construct a palette for every target but this is uncommon in UI apps.

• Compressing non-color render targets in 3D games: We see potential in compressing some non-color targets.

Framebuffer Compression Using Dynamic Color Palettes 13

Thank you

• Questions?

Framebuffer Compression Using Dynamic Color Palettes 14

References

1- Flurry Analytics: It’s an App World. The Web Just Lives in It.2- NVIDIA, 2015. NVIDIA Tegra X1 Whitepaper. URL: international.download.nvidia.com/pdf/tegra/Tegra-X1-whitepaper-v1.0.pdf3- RASMUSSON, J., HASSELGREN, J., AND AKENINE-MOLLER, T. 2007. Exact and error-bounded approximate color buffer compression and decompression. In SIGGRAPH/EUROGRAPHICS Conference On Graphics Hardware: Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware, vol. 4, 41–48.

Framebuffer Compression Using Dynamic Color Palettes 15

Backup slides

Framebuffer Compression Using Dynamic Color Palettes 16

Workloads

• AB: Angry Birds.• PH: Android Phone.• CH: Chrome.• FB: Facebook.• HO: Android Home Screen.• KD: Amazon Kindle. • MS: Android Messaging.• ST: Android Settings.• TW: Twitter.• TR: Temple Run 2.• GS: Gun Ship 2.• NS: Need for Speed.• FN: Fruit Ninja.

AB PH CH FB HO KD MS ST TW HM0.6

0.7

0.8

0.9

1

1.1

1.2 1:4 1:32 1:256 1:10241:4096 1:8192 1:16384

Com

pres

sion

rate

rela

tive

to 1

:1 s

ampl

ing

Lower pixel sampling at the FVC

AB PH CH FB HO KD MS ST TW HM0.5

0.6

0.7

0.8

0.9

1

2 4 8 16 32

Com

pres

sion

rate

rela

tive

to

sam

plin

g ev

ery

fram

eUsing the same palette for multiple frames

Framebuffer Compression Using Dynamic Color Palettes 19

ADCP Example

CCDC0

C1

C2

C3

C4

C5

C6

C7

000001010011100101110111

{00,01}{0,1}{φ}

{000,101}{110,010}{110,111}

{C0,Cy}{Cx,C2}

{C0, C2}{C0, C1}{C0, C0}{C0, C5}{C6, C2}{C6, C7}{C0, Cy}{Cx, C2}

010001000011011011111111

Tile colors Encoding CSB

AB PH CH FB HO KD MS ST TW HMEAN0.5

0.6

0.7

0.8

0.9

1

8 16 32 64 128

FVC

rela

tive

cove

rage

FVC Fidelity vs size