lce13: android hwcomposer on kms
DESCRIPTION
Resource: LCE13 Name: Android Hwcomposer on KMS Date: 11-07-2013 Speaker: Ross Oldfield Video: https://www.youtube.com/watch?v=AuoFABKS_DkTRANSCRIPT
Group photograph at Linaro Connect in Copenhagen
Monday 29 Oct 2012
Android hwcomposer on KMS
LCE-13
Graphics WG
www.linaro.org
What?
Attempt to have a common display backend for both Linux
and Android.
Would have several advantages: Common Android display HAL implementation for multiple SoCs
Reduce the amount of work to get an SoC’s display going?
KMS ought to be a decent fit, surely? KMS has “planes” and Android has “layers”
Things get synchronised to the vertical blanking
Android can always fall back to GLES composition if KMS can’t
support a particular layer
www.linaro.org
Android SurfaceFlinger
Android displays things using SurfaceFlinger
Can composite multiple layers together using OpenGL|ES
or use hwcomposer
GLES composition will always be possible Android requires that GLES be present
Using SoC composition hardware will beLower power
Higher performance
www.linaro.org
Why?
Why would we want this?
To get development platforms up-and-running getting a platform up-and-running
just want to be able to see the UI
not having to make any special changes to go from a Linux system to
an Android one (to get the display going...)
High performance HAL suitable for a production platform Use any hardware overlay support
Reduce reliance on GPU (let it render the UI!)
www.linaro.org
hwcomposer HAL
Most of the behaviour is described in the header https://source.android.com/devices/reference/hwcomposer_8h_source.html
Broadly three distinct functions Modesetting, handling display hotplug etc.
Displaying the output of any GLES composition
Offloading layer composition to dedicated hardware
www.linaro.org
hwcomposer
Display Layers Buffers
Blending modePlane AlphaTransform (Rotation)Acquire & Release FencesSource & Dest Windows
HintsPixel FormatDimensions
www.linaro.org
Prepare and Set
prepare() chooses which layers are to be handled by Hardware ( layer type is set to HWC_OVERLAY )
GLES ( layer type is set to HWC_FRAMEBUFFER )
Potentially many calls to Prepare() per display/composition
cycle
Set() consumes layers HWC_BACKGROUND
HWC_FRAMEBUFFER_TARGET
HWC_OVERLAY
prepare()GLES
compositionset()
www.linaro.org
Layer Attributes
Blending mode Are RGBA pixels pre-multiplied alpha or not?
Plane alpha applied to the whole layer
Transform Flip or rotate (clockwise) the source layer
Source and Destination windows “scaling”
www.linaro.org
Brief Digression: Explicit Sync
Every layer has an acquire and release fence
Each array of layers has a retire fence
Acquire fence “don't show this buffer until the fence is signalled”
Release fence “the buffer for this layer is no longer being read”
Retire fence “this scene / array of layers is no longer being shown”
www.linaro.org
Fences
Used to signal read/write ownership of the buffer
GPU Display
Buffer owned by GPU
Buffer owned by Display
Buffer owned by GPU
Signal acquire fence
Signal release fence
www.linaro.org
Acquire Fences
Normally a pageflip would cause the hardware to be set up
Next buffers flipped to when the VSYNC occurs
VSYNC
Pageflip B
A B C
VSYNC
Program hardwareShow B next
Pageflip C
Program hardwareShow C next
www.linaro.org
Acquire Fences
Acquire fence must be signalled before the h/w is set up Otherwise the buffer might be read before its pixels are valid!
If registers are set asynchronously How can you signal any error status back to the pageflip?
VSYNC
Pageflip B, fence
A B C
VSYNC
Remember B
Pageflip C, fence
Fence cb()
Set h/w to B Remember C
Fence cb()
Set h/w to C
www.linaro.org
Release Fences
Signal when a buffer is no longer being read
For a display controller A buffer is no longer being read… After the next scene gets set()
… and then after the following VSYNC
All the buffers get their release fences signalled at the
same time sync timeline increments after VSYNC after a pageflip
same sync_pt set to current timeline position + 1
Each buffer has its own fence potentially lots of file handles to open and return
www.linaro.org
Release Fences
The release fence for buffer A is signalled for every scene it was in that has been replaced
even if the next scene also is going to read from A
VSYNC
Scene
Buffers
Fence to Signal
A B A C A C
VSYNC
X Y Z
Release X ARelease X B
VSYNC
Release Y ARelease Y C
www.linaro.org
Requirements
We must be able to modeset a display
handle the VSYNC event from a display
map the memory backing a GLES-composited framebuffer
“wait” until the acquire fence is signalled before actually presenting
anything to the screen
signal release and retire fences after the next pageflip completes
We would like to show YUV and RGBA layers as planes
not have to cover the screen in buffers (show the background colour)
handle all the layer attributes per-plane alpha
non and pre-multiplied alpha
www.linaro.org
How?
Using KMS gets us all the modesetting goodness
This leaves Mapping buffers
Handling sync (especially the Acquire fences)
Handling layers & planes
www.linaro.org
Buffers
Given a layer with a native_buffer_t Usage hint of GRALLOC_USAGE_HW_COMPOSER
Gralloc is responsible for ensuring the buffer’s
memory is suitable for the display controller
Want A dma_buf file descriptor
Dimensions and pitch of the image
Pixel format 4CC
Any other metadata maybe YUV luma range?
www.linaro.org
Buffers & Pixel Formats
Funny pixel formats Maybe your video decoder generates some proprietary
tiled format?
HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED
Need a platform-specific routine to generate a 4CC
This should live in gralloc It is already platform specific
Want to add gralloc extensions in gralloc_perform() return a dma_buf
Any other metadata we need
www.linaro.org
Sync – wait in user space?
Certainly possible
Would block set() until all buffers have been acquired
Good news Would work without having to add anything to KMS drivers
Bad news A bit of extra scheduling overhead to set up your screen
Pageflip is back to being serialised behind content generation
www.linaro.org
Sync – handle in the kernel
Would require some sort of ‘delayed pageflip’ Defer the actual pageflip work until after the acquire fence has
signalled
Issues with accurate error reporting we care that our pageflip will actually succeed!
Some sort of new API into KMS? Use Android fences or dmabuf fences?
Ought to be common but would impact a lot of drivers?
Testing?VSYNC
Pageflip B, fence
A B C
VSYNC
Remember B
Pageflip C, fence
Fence cb()
Set h/w to B Remember C
Fence cb()
Set h/w to C
www.linaro.org
Layers
For each new composition scene we get Lots of layers
Lots of attributes per layer Rotation, alpha etc.
Lots of release fences to create
Acquire fences to deal with
Want to deal with them all at once SurfaceFlinger always specifies the entire scene for each display
every time it changes any part of the composition scene
Should we be looking at adopting one of the ‘atomic
modeset’ patches to KMS for this? Must support scenes with no ‘fullscreen CRTC framebuffer’
Opportunity to add some of this ‘delayed pageflip’ behaviour?
www.linaro.org
Summary
What to do is strongly influenced by how we want to handle
sync
Broadly three options
Fences Pro Con
1 Userland
synchronous wait
Correct behaviour for SoC bring-up
No KMS changes
GPU composition only
More likely to ‘jank’
2 Kernel async wait Less likely to ‘jank’ than (1) GPU composition only
KMS API additions
3 Kernel async wait Uses SoC composition hardware
Lower power
Large KMS changes
(atomic pageflip?)
Thank you