kernel recipes 2017 - an introduction to the linux drm subsystem - maxime ripard
TRANSCRIPT
![Page 1: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/1.jpg)
Kernel Recipes 2017
An introduction to theLinux DRM subsystemMaxime RipardFree [email protected]
c© Copyright 2004-2017, Free Electrons.Creative Commons BY-SA 3.0 license.Corrections, suggestions, contributions and translations are welcome!
Embedded LinuxDevelopers
Free Electrons
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 1/1
![Page 2: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/2.jpg)
Maxime Ripard
I Embedded Linux engineer and trainer at Free ElectronsI Embedded Linux development: kernel and driver development, system integration,
boot time and power consumption optimization, consulting, etc.I Embedded Linux training, Linux driver development training and Android system
development training, with materials freely available under a Creative Commonslicense.
I http://free-electrons.com
I ContributionsI Co-maintainer for the sunXi SoCs from AllwinnerI Contributor to a couple of other open-source projects, Buildroot, U-Boot, Barebox
I Living in Toulouse, south west of France
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 2/1
![Page 3: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/3.jpg)
An introduction to the Linux DRM subsystem
Introduction
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 3/1
![Page 4: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/4.jpg)
A long long time ago, in a galaxy (not so) far, far away
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 4/1
![Page 5: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/5.jpg)
Framebuffers
I Display hardware was dead simple...
I .. and so was the API to drive it.
I Introducing... fbdev!I Allows for three things:
I Mode-SettingI Accessing the (only) bufferI Optional 2d acceleration: draw, copy, etc.I And access to the device registers...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 5/1
![Page 6: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/6.jpg)
That 90’s show
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 6/1
![Page 7: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/7.jpg)
Back to the future
I Two different trendsI Embedded devices starting to show up, with their low power needs ⇒ Display
engines need to accelerate more thingsI Desktop displays getting more and more fancy in order to play Quake in 4k VR ⇒
Bigger and bigger GPUs
I Led to two different outcomes:I Interface to drive GPU devices through the kernel: DRMI Hacks piling on in order to fit embedded use-cases: omapdss, pxafb
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 7/1
![Page 8: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/8.jpg)
Composition Evolved
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 8/1
![Page 9: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/9.jpg)
The droid you’re looking at
Sean Paul and Zach Reizner - Google - XDC2016
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 9/1
![Page 10: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/10.jpg)
Can I talk to the manager?
I DRM was initially introduced to deal with the GPU’s needI Initialize the card, load its firmware, etc.I Share the GPU command queue between multiple applicationsI Manage the memory (allocation, access)I But not modesetting!
I All the modesetting was in the userspace, especially in X
I Race between rendering and modesetting
I Only one graphical application that needed to remain there all the time
I (Lack of) Abstraction!
I Introduction of the Kernel Mode-Setting (KMS) to move the modesetting backinto the kernel
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 10/1
![Page 11: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/11.jpg)
Kill it with fire!
I Now, fbdev could be implemented on top of KMS...
I ... Or removed entirely
I Call for deprecation in 2012 (Hi Laurent!)
I Last fbdev driver merged in 2014
I First ARM DRM driver: exynos in 2011
I Followed: arm, armada, atmel-hclcdc, fsl-dcu, hisilicon, imx, mediatek, meson,msm, mxsfb, omapdrm, pl111, rcar-du, rockchip, shmobile, sti, stm, sun4i, tegra,tve200, etc...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 11/1
![Page 12: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/12.jpg)
Two nodes to go please
I Initially, DRM was created for devices that were both displaying and rendering(your traditionnal PC graphics card).
I On embedded devices, it’s never really been like thatI the GPU is discrete and comes from a third partyI the display engine is usually designed by the SoC vendor
I DRM and KMS APIs requiring the same level of privilege, with one master, andwere both exposed on the same device file
I Creation of render nodes
I Also useful for things like GPGPU, off-screen rendering, more flexible accesscontrol
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 12/1
![Page 13: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/13.jpg)
An introduction to the Linux DRM subsystem
DRM/KMS
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 13/1
![Page 14: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/14.jpg)
The pixels must flow
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 14/1
![Page 15: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/15.jpg)
KMS components
I PlanesI Image sourceI Associated with one (or more!) framebuffersI Holds a resized / cropped version of that framebuffer
I CRTCsI Take the planes, and does the compositionI Contains the display mode and parameters
I EncodersI Take the raw data from the CRTC and convert it to a particular format
I ConnectorsI Outputs the encoded data to an external displayI Handles hotplug eventsI Reads EDIDs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 15/1
![Page 16: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/16.jpg)
Page flipping
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 16/1
![Page 17: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/17.jpg)
KMS components
I PlanesI Image sourceI Associated with one (or more!) framebuffersI Holds a resized / cropped version of that framebuffer
I CRTCsI Take the planes, and does the compositionI Contains the display mode and parameters
I EncodersI Take the raw data from the CRTC and convert it to a particular format
I ConnectorsI Outputs the encoded data to an external displayI Handles hotplug eventsI Reads EDIDs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
![Page 18: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/18.jpg)
KMS components
I PlanesI Image sourceI Associated with one (or more!) framebuffersI Holds a resized / cropped version of that framebuffer
I CRTCsI Take the planes, and does the compositionI Contains the display mode and parameters
I EncodersI Take the raw data from the CRTC and convert it to a particular format
I ConnectorsI Outputs the encoded data to an external displayI Handles hotplug eventsI Reads EDIDs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
![Page 19: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/19.jpg)
KMS components
I PlanesI Image sourceI Associated with one (or more!) framebuffersI Holds a resized / cropped version of that framebuffer
I CRTCsI Take the planes, and does the compositionI Contains the display mode and parameters
I EncodersI Take the raw data from the CRTC and convert it to a particular format
I ConnectorsI Outputs the encoded data to an external displayI Handles hotplug eventsI Reads EDIDs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 17/1
![Page 20: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/20.jpg)
Allwinner display pipeline
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 18/1
![Page 21: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/21.jpg)
DRM vs SoC pipeline
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 19/1
![Page 22: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/22.jpg)
DRM Stack: KMS
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 20/1
![Page 23: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/23.jpg)
DRM Stack: GEM
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 21/1
![Page 24: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/24.jpg)
DRM Stack: CMA
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 22/1
![Page 25: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/25.jpg)
DRM Stack: PRIME
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 23/1
![Page 26: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/26.jpg)
DRM Stack: GPUs
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 24/1
![Page 27: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/27.jpg)
An introduction to the Linux DRM subsystem
Vendor solutions...
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 25/1
![Page 28: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/28.jpg)
Solutions
I The GPU found in most Allwinner SoCs is the Mali-400 from ARM (with avariable number of cores)
I There are two options to support that GPU:I Lima
I Reversed engineered proof-of-conceptI Triggered the reverse engineering effort of the GPUs (freedreno, etnaviv, etc.)I Development (close to?) stopped three years ago, and then resumed a couple of
monthes ago
I ARM-Provided supportI FeaturefulI Two parts: GPL kernel driver and proprietary OpenGL ES implementation
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 26/1
![Page 29: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/29.jpg)
DRM Stack: GPU
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 27/1
![Page 30: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/30.jpg)
Development
I Everything is provided by ARM on their website (if you’re lucky)
I On the userspace side, you just need to put the library they provided on yoursystem
I On the driver side, you need to create a platform glue that will deal with:I Memory mappingI InterruptsI ClocksI Reset linesI Power DomainsI Basically everything needed for the GPU to operate properly on your SoC
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 28/1
![Page 31: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/31.jpg)
X11 integration
I We need a DDX (Device Dependent X) driver
I xf86-video-modesetting is working on top of KMS and GBM (MESA-defineduser-space API to allocate buffers)
I ARM developped xf86-video-armsoc for SoC using a 3rd party GPU (Mali,PowerVR, Vivante, etc.)
I Relies on KMS for the display configuration, driver-specific ioctl for bufferallocations and vendor-provided OpenGL ES implementation
I Just have to write a small glue to use your driver allocator, and give some hints toX about what your hardware support (hw cursor, vblank, etc.)
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 29/1
![Page 32: Kernel Recipes 2017 - An introduction to the Linux DRM subsystem - Maxime Ripard](https://reader034.vdocuments.net/reader034/viewer/2022050613/5a64e0a47f8b9ac21c8b7463/html5/thumbnails/32.jpg)
Questions?
Maxime [email protected]
Slides under CC-BY-SA 3.0http://free-electrons.com/pub/conferences/2017/kr/ripard-drm
Free Electrons. Kernel, drivers, embedded Linux and Android - Development, consulting, training and support. http://free-electrons.com 30/1