hardware acceleration on mobile, ariya hidayat & jarred nicholls

Wednesday, November 2, 11

Ariya Hidayat & Jarred NichollsSencha

Twitter: @ariyahidayat @jarrednicholls

Hardware Accelerationon Mobile


Challenges


Game vs Web


Game

Animation

Physics

Textured objects

Text

Transformation


Web Pages

Images

Text


How Browser Works

Layout engine

Network stack

GraphicsJavaScript engine

User interface


WebKit Components

WebKit Library

JavaScriptCore

WebCore

HTMLrendering

SVG

DOM CSS


Platform Abstraction

Network Unicode Clipboard

Graphics Theme Events

Thread Geolocation Timer


Graphics Abstraction

Mac Chromium Qt Gtk

CoreGraphicsSkia

QPainter

graphics stack

GraphicsContext


Components of Mobile

Touch interface

GPU

CPU

Radio

Power


Graphics Processor

Divide and conquer

Very specific purpose


Optimized for Games

Transformation

Textured triangles

Fixed geometry

Parallelism


Challenges

Predictable contents (assets) ✔Mostly text ✔Mostly images ✔Expected run-time response immediate


Primitive Drawing


Path is Everything

Path

Triangle Rectangle

PolygonEllipse


Stroke = Outline

Solid Dashed Dotted Textured


Brush = Fill

SolidNone Gradient Textured


Smooth via Antialiasing

Multiple levels of transparency

Perfect for parallelism


Path Approximation

Curves

Polygon


Gradient: ExpensiveCPU: sequential, tedious

GPU: parallel, still a lot of work


Blur Shadow: Really Posh

Involved pixel “blending”

GPU: parallel, still tedious


Transformation

Scaling

RotationPerspective


Text Rasterization


Font Atlas

Bye

Buffer


Simple to Complex

Hello!

Simple shapeSolid color

CurvesGradient brushTextured stroke

Blur shadowSerif textRotated

Make it as an image


HOW FAST?


Android Drawing

GraphicsContext

WebCore graphics

Skia

Log file


Example: BingplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff fffillRect 0,0 800x556 color ff ff ff ff


How Fast?

#include "TimeCounter.h"

bool WebViewCore::drawContent(SkCanvas* canvas, SkColor){ uint32_t timestamp = getThreadMsec();

.... painting code .... DBG_SET_LOGD("% ms", getThreadMsec() - timestamp);}

external/webkit/WebKit/android/jni/WebViewCore.cpp


Example: Google News

16:24:04.070 D/webcoreglue( 273): drawContent 11 ms16:24:04.110 D/webcoreglue( 273): drawContent 13 ms16:24:04.150 D/webcoreglue( 273): drawContent 13 ms16:24:04.190 D/webcoreglue( 273): drawContent 10 ms16:24:04.240 D/webcoreglue( 273): drawContent 10 ms16:24:04.280 D/webcoreglue( 273): drawContent 13 ms16:24:04.320 D/webcoreglue( 273): drawContent 13 ms16:24:04.360 D/webcoreglue( 273): drawContent 13 ms16:24:06.080 D/webcoreglue( 273): drawContent 12 ms16:24:06.140 D/webcoreglue( 273): drawContent 10 ms16:24:06.180 D/webcoreglue( 273): drawContent 13 ms16:24:06.230 D/webcoreglue( 273): drawContent 14 ms16:24:06.600 D/webcoreglue( 273): drawContent 26 ms16:24:06.640 D/webcoreglue( 273): drawContent 13 ms16:24:06.860 D/webcoreglue( 273): drawContent 33 ms16:24:06.890 D/webcoreglue( 273): drawContent 12 ms16:24:06.930 D/webcoreglue( 273): drawContent 13 ms16:24:06.960 D/webcoreglue( 273): drawContent 13 ms16:24:07.000 D/webcoreglue( 273): drawContent 13 ms

adb logcat -v time | grep drawContent


Backing Store


Maps

Tile


Pinch to Zoom

when you pinch...


Responsive UI

Processor

Rendering

User interaction

decoupled


Rendering vs Interaction

Web Page

Backing StoreScreen

Rendering

User interaction


Checkerboard Pattern


Progressive Rendering

Crisp but slow

Fast but blurry


Tiling System

CPU GPU

........

Texture upload


Tile Transformation

Panning = Translation Zooming = Scaling


Backing Store Support

Texture-based

Vector-based

Honeycomb and later


Y U NOposition:fixed


Demo


Layer & Compositing


Elements = Layers


Typical Animation

opacity, movement, scaling, rotation, ...


Immediate Mode

every animation tick

setInterval(function() { box.draw(x, y); x += 10;}, 20);


Scene Graph

box.drawInto(layer);setInterval(function() { layer.translate(10, 0);}, 20);

Change transformation matrix


Logical 3-D


Keep the Textures


Demo


Compositing Support

Honeycomb and later


Well-known Trick

-webkit-transform: translateZ(0)

forcing 3-D transform

Not needed for CSS animation!


Debugging in Safari

defaults write com.apple.Safari IncludeInternalDebugMenu


Compositing Indicators

Composited layer

Container layer

No texture

Overflow

Texture (number = upload)


Debugging in Chrome

about:flags


Avoid Texture Reupload

No reupload Upload

OpacityPosition

SizeAnimation

Everything else!

“Hardware accelerated CSS”


Avoid Overcapacity

GPU = Limited silicon spaceLarge layer ➔ broken into “tiles”


Prepare & Reuse

Viewport

Hide the layer offscreen


Wrap Up


Hardware Acceleration

Drawing primitives

Backing stores

Layer & compositing


THANK YOU!


QUESTIONS?

ariya @ sencha.com

ariyahidayat

ariya.ofilabs.com

jarred @ sencha.com

jarrednicholls


hardware acceleration on mobile, ariya hidayat & jarred nicholls

Technology

fast

310 fillrect 0

function

transform

debugging

pinch

laterwednesday

wednesday