createacustomembeddedlinux …...createacustomembeddedlinux...

Create a Custom Embedded Linux Distribu3on for Any Embedded Device Using the Yocto Project

Chris Hallinan Mentor Graphics

October 16, 2014

Yocto Project Developer Day – Oct 2014 1

Yocto Project Developer Day 2014 Intro to Yocto Project

It's not an embedded Linux distribu2on – It creates a custom one for you

Agenda!!  Introduc3on to the Yocto Project ! Key Concepts

"  Build System Overview & Workflow "  Exercise 1: Exploring the Host System

! Recipes In-‐Depth "  Standard Recipe Build Steps "  Exercise 2: Examining Recipes

! Building and Boo3ng an Image "  Exercise 3: Building Your First Linux Image "  Exercise 4: BooNng Your Linux Image Using QEMU

!  Layers and BSPs "  Exercise 5: CreaNng a Custom Layer "  Exercise 6-‐7: Adding a graphical boot logo and SSH server "  Exercise 8-‐9: BooNng an embedded hardware board


Yocto Project Overview!! Collec3on of tools and methods enabling

" Rapid evaluaNon of embedded Linux on many popular off-‐the-‐shelf boards

" Easy customizaNon of distribuNon characterisNcs

! Supports x86, ARM, MIPS, Power

! Based on technology from OpenEmbedded Project

! Layer architecture allows for easy re-‐use of code


oe-‐core (meta)

meta-‐yocto-‐bsp

meta-‐yocto

other layers

Yocto Project Overview!! Supports use of popular package formats including: " rpm, deb, ipk

! Releases on a 6-‐month cadence

! Latest (stable) kernel, toolchain and packages, documenta3on

! App Development Tools including Eclipse plugin, ADT, hob


Yocto Project Overview!! Governance

" Organized under the Linux FoundaNon

" Split governance model " Technical Leadership Team " Advisory Board made up of parNcipaNng organizaNons


Yocto Project Release Versions!! Major Version Releases


Name Revision Poky Release Date

Bernard 1.0 5.0 Apr 5, 2011

Edison 1.1 6.0 Oct 17, 2011

Denzil 1.2 7.0 Apr 30, 2012

Danny 1.3 8.0 Oct 24, 2012

Dylan 1.4 9.0 Apr 26, 2013

Dora 1.5 10.0 Oct 19, 2013

Daisy 1.6 11.0 Apr 24, 2014

(unnamed) 1.7 12.0 Oct 31, 2014 (planned)

Intro to OpenEmbedded!! Bitbake

" Powerful and flexible build engine

" Determines dependencies and schedules tasks

7

Metadata – a structured collecNon of "recipes" which tell BitBake what to build

Yocto is based on openembedded-core!

8

Metadata describing approximately 900 popular "core" recipes used for building boot images. Includes support for graphics, Qt, networking, kernel recipes, tools, much more.

oe-‐core (meta)


meta-‐yocto

Other layers

OK, so what is Poky?!! Poky is both a reference distribu3on and a build system

! Poky has its own git repo " git clone git://git.yoctoproject.org/poky

! Primary poky layers " oe-‐core (poky/meta)

" meta-‐yocto-‐bsp " meta-‐yocto

! Poky is the basis of YP


What is Metadata?

10

•  Metadata exists in four general categories: •  Recipes (*.bb)

– Usually describe build instrucNons for a single package •  PackageGroups (special *.bb)

– Oeen used to group packages together for a FS image

•  Classes (*.bbclass) –  Inheritance mechanism for common funcNonality

•  Configura3on(*.conf) – Drives the overall behavior of the build process

OE-CORE Breakdown"


*.bb: 892 *.conf 73 *.bbclass 161 packagegroup* 40

Recipe Basics

12

•  Most common form of metadata – The Recipe •  Provides the “list of ingredients” and “cooking instruc3ons” to build a package(s)

•  Has a common set of tasks… busybox

glibc

sysvinit

inetutils

mtdutils

Metadata

BitBake

Recipe Basics!! A recipe is a set of instruc3ons for building packages, including: " Where to obtain the upstream sources and which patches to apply o SRC_URI!

" Dependencies (on libraries or other recipes) o DEPENDS, RDEPENDS!

" ConfiguraNon/compilaNon opNons o EXTRA_OECONF, EXTRA_OEMAKE!

" Define which files go into what output packages o FILES_*!


Example Recipe – ethtool_3.15.bb"


Recipe Basics – Default Tasks*

18

•  If a cache object exists and is valid, use it

•  Locate and download source code

•  Unpack source into working directory

•  Apply any patches

•  Perform any necessary pre-‐build configura3on

•  Compile the source code

•  Installa3on of resul3ng build ar3facts

•  Create binary package(s)

do_setscene!

do_fetch!

do_unpack!

do_patch!

do_configure!

do_install!

do_compile!

*Simplified for illustration

do_package!

Note: to see the list of all possible tasks for a recipe, do this: $ bitbake -c listtasks <recipe_name>!

Quick Start Guide in a Slide!! Download Yocto Project sources:

"  $ wget http://downloads.yoctoproject.org/releases/yocto/yocto-1.6.1/poky-daisy-11.0.1.tar.bz2!

"  $ tar xf poky-daisy-11.0.1.tar.bz2!"  $ cd poky-daisy-11.0.1!"  Can also use git and checkout a known branch i.e. Daisy

o  $ git clone git://git.yoctoproject.org/poky.git!o  $ git checkout -b daisy!

!  Build one of the reference Linux distribu3ons: "  $ source oe-‐init-‐build-‐env "  Check/Edit local.conf for sanity

o  Modify MACHINE=qemux86 !"  $ bitbake core-image-base!

!  Run the image under emula3on: "  $ runqemu qemux86


Lab 1: Exploring the Host System!

! Objec3ves " Familiarize yourself with how the YP metadata sources are organized

" Learn where you can find conf files, BitBake class files, and recipe files


Log into your lab cloud-based host:!Instructor will supply URL/credentials!

Host System Layout!/scratch! |---sandbox!

Storage for your software and mods!

|---working!! Project build directory!

|---yocto! Yocto sources and caches. Do Not Modify!


Note: This is your instructor's preferred setup – others may have their own favorite working layout

Host System Layout!/scratch/yocto! |---binary-images!

Pre-built binary boot images for convenience! |---downloads (DL_DIR)!! Source cache (downloaded for each recipe)!

|---sources! Yocto source code. Do Not Modifiy! |---sstate-cache (SSTATE_DIR)!! Common binary cache!



Exercise 2: Examining Recipes!! meta/recipes-‐extended/bc/bc_1.06.bb

" Uses LIC_FILES_CHKSUM and SRC_URI checksums

" Note the DEPENDS build dependency declaraNon indicaNng that this package depends on flex to build


Exercise 2: Examining Recipes!! meta/recipes-‐mul3media/flac/flac_1.3.0.bb

" Includes custom source patches to apply to the sources

" Customizes autoconf configure opNons (EXTRA_OECONF) based on TUNE features

" Breaks up output into mulNple binary packages

26

Exercise 2: Examining Recipes!! meta/recipes-‐connec3vity/ofono/

" Splits recipe into common .inc file to share metadata between mulNple recipes

" Sets a condiNonal build DEPENDS based on a distro feature (in the .inc file)

"  Sets up an init service via do_install_append()!

" Has a _git version of the recipe


EXERCISE 3

Building a Linux Image


What board are you building?!! You must follow the correct procedure based on the board you are using

! This Lab supports " Beaglebone Black " Minnowboard Max

" WandBoard " QEMU x86

! Make sure you follow the correct set of steps for your board


Beaglebone Black Build - Overview!! Checkout sources

" Daisy branch ! Create project directory

" Edit local.conf o MACHINE = "beagleboard" o Change DL_DIR and SSTATE_DIR

! Bitbake image " Bitbake core-‐image-‐base

! Format SD card (details in Ex. 4) " 2 parNNons, boot (FAT 32 LBA 'c') and root (ext3) " Copy MLO, u-‐boot to 'boot'boot " Copy roops to 'root'


Beaglebone Black Build - Detailed!!  Sources are already downloaded for you ! Create project directory, edit local.conf and bblayers.conf

$ cd /scratch/working!$ . /scratch/yocto/sources/poky/oe-init-build-env build-beaglebone!$ vi conf/local.conf!

o Modify MACHINE = "beaglebone"!o Modify DL_DIR = "/scratch/yocto/downloads"!o Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"!

"  Nothing to change in bblayers.conf, beaglebone is supported in meta-‐yocto-‐bsp

! Bitbake your system image recipe $ bitbake core-image-base!


WandBoard Build - Overview!! Checkout sources

" Daisy branch

! Create project directory " Edit local.conf, bblayers.conf

o MACHINE = "wandboard-‐quad" in local.conf o Modify DL_DIR and SSTATE_DIR o ACCEPT_FSL_EULA = "1"

! Bitbake image " core-‐image-‐base

! Copy sdcard image and boot (details in Ex. 4) " Use SD slot on cpu module on bokom of board


WandBoard Build - Detailed!!  Sources are already downloaded for you !  Create project directory, edit local.conf and bblayers.conf

$ cd /scratch/working!$ . /scratch/yocto/sources/poky/oe-init-build-env build-wbquad!$ vi conf/local.conf!

o  Modify MACHINE = "wandboard-quad"!o  Modify DL_DIR = "/scratch/yocto/downloads"!o  Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"!o  ACCEPT_FSL_EULA = "1" (bokom of file is good locaNon for this)

$ vi conf/bblayers.conf!o  Add layers to BBLAYERS

BBLAYERS ?= " \! /scratch/yocto/sources/meta-fsl-arm \! /scratch/yocto/sources/meta-fsl-arm-extra \! /scratch/yocto/sources/meta-fsl-demos \!…!

!  Bitbake your system image recipe $ bitbake core-image-base!


Minnowboard Max Build - Overview!! Checkout sources

" Daisy branch ! Create project directory

" Edit local.conf o Change DL_DIR and SSTATE_DIR

! Bitbake image " Bitbake core-‐image-‐base

! Format SD card (details in Ex. 4) " 2 parNNons, boot (FAT 32 LBA 'c') and root (ext3) " Copy MLO, u-‐boot.img to boot parNNon " Copy roops to root parNNon


Minnowboard Max Build - Detailed!!  Sources are already downloaded for you !  Create project directory, edit local.conf and bblayers.conf

$ cd /scratch/working!$ . /scratch/yocto/sources/poky/oe-init-build-env build-minnowmx!$ vi conf/local.conf!

o  Modify MACHINE = "intel-corei7-64"!o  Modify DL_DIR = "/scratch/yocto/downloads"!o  Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"!

$ vi conf/bblayers.conf!o  Add layers to BBLAYERS

BBLAYERS ?= " \! /scratch/yocto/sources/meta-intel \!…!

!  Bitbake your system image recipe $ bitbake core-image-base!


QEMUx86 Build - Overview!! Checkout sources

" Daisy branch

! Create project directory " Edit local.conf, bblayers.conf

o MACHINE = "qemux86" in local.conf (Note: this is the default)

o Modify DL_DIR and SSTATE_DIR

! Bitbake image " core-‐image-‐minimal

! Boot using Yocto's helper script "runqemu" " Use -nographic if using SSH to your host


QEMUx86 Build - Detailed!!  Sources are already downloaded for you !  Create project directory, edit local.conf and bblayers.conf!

$ cd /scratch/working!$ . /scratch/yocto/sources/poky/oe-init-build-env build-qemux86!$ vi conf/local.conf!

o  Modify DL_DIR = "/scratch/yocto/downloads"!o  Modify SSTATE_DIR = "/scratch/yocto/sstate-cache"!

"  Bblayers.conf: OK with the defaults, no addiNonal layers necessary

!  Build your image using BitBake !$ bitbake core-image-minimal!


EXERCISE 4

Boo3ng your board


What board are you booting?!! You must follow the correct procedure based on the board you are using

! This Lab supports " Beaglebone Black " Minnowboard Max

" WandBoard " QEMU x86

! Make sure you follow the correct set of steps for your board


Beaglebone Black Boot - Detailed!!  Format blank SD Card for Beaglebone Black

$ export DISK=/dev/sdc $ umount ${DISK}1 <<<Note the addition of the '1' $ sudo dd if=/dev/zero of=${DISK} bs=512 count=20 $ sudo sfdisk --in-order --Linux --unit M ${DISK} <<-__EOF__ 1,12,0xE,* ,,,- __EOF__ $ sudo mkfs.vfat -F 16 ${DISK}1 -n boot $ sudo mkfs.ext4 ${DISK}2 -L rootfs

!  Now unplug and replug your SD Card for automount $ cd <path-to-your-binary-images> $ sudo cp -v MLO-beaglebone /media/boot/MLO $ sudo cp -v u-boot.img /media/boot/ $ sudo tar xf core-image-base-beaglebone.tar.bz2 -C /media/rootfs $ sync (flush to device, not neccesary, but illustrative) $ umount /media/rootfs /media/boot


Beaglebone Black Boot - Detailed!

! Copy sdcard image and boot " Use SD slot on cpu module on bokom of board

! For convenience, we will only replace the root file system

! On your laptop Linux, insert your microSD: "  $ scp -‐P <port> ilab01@devdayN:/scratch/working/build-‐beaglebone/tmp/deploy/images/beaglebone/core-‐image-‐base-‐beagleboard.tar.bz2 .

"  $ sudo rm -‐rf /media/roops/* "  $ sudo tar xf core-‐image-‐base-‐beaglebone.tar.bz2 -‐C /media/roops

"  $ umount /media/roops


Minnowboard Max Boot – Detailed (1)!

! Note: Due to the size of the HDD Image, we offer two op3ons for upda3ng the image on your board. Chose only one of the following: " Copy the enNre image (.hddimg) to the microSD or USB flash drive, creaNng a brand new image

" Update only the roops image (much smaller download from cloud)



! To build a brand new microSD or USB Flash drive image: " Download the .hddimg from your cloud-‐based host: o  $ scp -P <port> [email protected]:/scratch/working/build-minnowmx/tmp/deploy/images/intel-corei7-64/core-image-base-intel-corei7-64.hddimg .

" On your laptop Linux, insert your microSD or USB flash drive: $ umount /media/boot!

!$ dd if=core-image-base-intel-corei7-64.hddimg of=/dev/sd(n) <<<replace with proper dev



! To replace only the root file system image, do the following:

! On your laptop Linux, insert your usb flash drive or microSD: $ scp -P <port> [email protected]:/scratch/working/build-minnowmx/tmp/deploy/images/intel-corei7-64/core-image-base-intel-corei7-64.ext3 . !

$ cp core-image-base-intel-corei7-64.ext3 /media/boot/rootfs.img!

$ umount /media/boot!



! Boo3ng the board " Insert the new MicroSD card

Shell> connect -r!Shell> map -r!Shell> fs0:!Shell> bootx64!

! That should get you to a Linux login prompt


WandBoard Boot - Detailed!! Prepare new SD Card for Wandboard

" Insert SD card in Card Reader

$ sudo dd if=core-image-base-wandboard-quad.sdcard of=/dev/sdc


Obviously, you'll need to verify your SD card device node

WandBoard Boot - Detailed!! For convenience, we will only replace the root file system on a pre-‐populated SD Card

! On your laptop Linux, insert your microSD

! It should auto-‐mount. If it doesn't this won't work $ scp -P <port> [email protected]:/scratch/working/build-wbquad/tmp/deploy/images/wandboard-quad/core-image-base-wandboard-quad.tar.bz2 .

$ sudo rm -rf /media/rootfs/*

$ sudo tar xf core-image-base.tar.bz2 -C /media/rootfs

$ umount /media/rootfs


A Detailed Look at Layers!! Introduc3on to Layers

! Stacking Customiza3ons

! Adding Layers

! Board Support Packages


oe-‐core (poky/meta)


meta-‐yocto

UI layers

Commercial Layers

Developer Layers

Patch Layers

Layers!! The Yocto Project build system is composed of layers " A layer is a logical collecNon of recipes represenNng the core, a BSP, an applicaNon stack, patches, or other funcNonality.

! All layers have a priority and can override policy and config semngs of the layers of lower priority


Layer Hierarchy"


oe-‐core (poky/meta)


meta-‐yocto

UI layers

Commercial Layers

Developer Layers

Patch Layers

Using Layers!! Layers are added to your build by inser3ng them into the global BBLAYERS variable within your …/conf/bblayers.conf file:

BBLAYERS = “! ! ! ! ! \! /<install-path>/poky/meta ! ! ! \! /<install-path>/poky/meta-yocto ! \! /<my-meta-dir>/meta-my-custom-layer \! “!


Board Support Packages!! BSPs are layers to enable support for specific hardware planorms

! Defines machine configura3on for the board

! Adds machine-‐specific recipes and customiza3ons " Kernel config

" Graphics drivers (e.g, Xorg) " AddiNonal recipes to support hardware features


Exercise 5: Create a Custom Layer!

! When doing development with Yocto, never edit files within the Yocto/Poky source tree " Use a custom layer for modularity and maintainability

! Create custom layers to hold a custom image recipe

! Let's call this layer meta-‐ypdd ! This layer must include:

" Meta-ypdd/conf/layer.conf " Recipes directory (meta-ypdd/recipes-core/) " A meta-‐ypdd/README file (basic documentaNon for the layer, including maintainer info)


Exercise 5: Create a Custom Layer!

! Yocto includes a helper script to create a new layer

! It can also supply an example recipe or bbappends file " $ yocto-layer <new-layer>!

! For this exercise, a layer is already provided " /scratch/sandbox/meta-ypdd!


meta-ypdd/conf/layer.conf!


Must match layer name in three places

Exercise 5: Custom Layer!! Add this new layer to your project's bblayer.conf file " Note that this layer is already provided on your VM $ vi …/conf/bblayers.conf!$ BBLAYERS ?= " \! /scratch/sandbox/meta-ypdd \!! !…!

! Rebuild core-‐image-‐minimal: $ bitbake core-image-minimal!

" NoNce the new login banner line!!


Exercise 6: Creating a Custom Image Recipe!

! Derive from core-‐image-‐minimal "  Add support for a graphical boot logo (via psplash) "  Add support for SSH server (dropbear)

! Use the layer we created in Exercise 5: meta-‐ypdd

! We'll name our custom image ypdd-image!"  The recipe will be named:

meta-ypdd/recipes-core/images/ypdd-image.bb!

! Simplest way to add packages to a predefined image "  Append them to IMAGE_INSTALL within the image recipe


Exercise 6: Creating a Custom Image Recipe"


Exercise 7: Build and Boot Your Custom Image!

! We added the new layer in Exercise 5 " /scratch/sandbox/meta-ypdd!

! Now build your custom image: " $ bitbake ypdd-image

! Boot the image using instruc3ons specific to your target board " Replace the roops contents on the SD/USB

! If using QEMU target, simply do this: " $ runqemu qemux86 -nographic tmp/deploy/images/ypdd-image-qemux86.ext3!


Common Gotchas When Getting Started!

! Do not try to re-‐use the same shell environment when moving between copies of the build system

! oe-init-build-env script appends to your $PATH, so repeated use in same shell is not recommended and may lead to unpredictable results

! Do not try to share sstate-‐cache between hosts running different Linux distros even if they say it works ;)


Project Resources!! The Yocto Project is an open source project, and aims to deliver an open standard for the embedded Linux community and industry

! Development is done in the open through public mailing lists: "  openembedded-‐[email protected] "  [email protected] "  [email protected]

! And public code repositories: "  hkp://git.yoctoproject.org and "  hkp://git.openembedded.net

! Bug reports and feature requests "  hkp://bugzilla.yoctoproject.org


Tip: ack-grep!! Much faster than grep for the relevant use cases

! Designed for code search ! Searches only relevant files

" Knows about many types: C, asm, perl " By default, skips .git, .svn, etc.

" Can be taught arbitrary types

! Perfect for searching metadata " $ bback


Tip: ack-grep"


alias bback='ack-grep --type bitbake'!

TIP: VIM Syntax Highlighting!!  hrps://github.com/openembedded/bitbake/tree/master/contrib/vim

"  Install files from the above repo in ~/.vim/ "  Add "syntax on" in ~/.vimrc

$ tree ~/.vim/!/home/ilab01/.vim/!├── ftdetect!│ └── bitbake.vim!├── ftplugin!│ └── bitbake.vim!├── plugin!│ └── newbb.vim!└── syntax! └── bitbake.vim!

!  You do use VI, right? ;-‐)


TIP: VIM Syntax Highlighting"



It’s not an embedded Linux distribu2on

It creates a custom one for you

createacustomembeddedlinux …...createacustomembeddedlinux...

Documents