embed your ideas

© 1

Embedded Software Made Easy

Embed your Ideas


A connected chief hat reading you a recipe you selected by voice control and waiting you at each step with a voice signal before continuing to the next one.

Let’s take an example

2


The following content is • Under the software point-of-view. • Is simplified for a broader audience. • Opinionated by 25 years of farjump’s team

experience.

Warning

3

Embedded Software Made Easy 4

Before we get started: Hardware vs Software

Hardware



Hardware

General-PurposeSingle-Purpose



Hardware

CPUMCUASIC GPU

General-Purpose

DSP

Single-Purpose

I/O Peripheral



Hardware

CPUMCUASIC GPU

General-Purpose

DSP

Single-Purpose

I/O Peripheral

May contain a General-Purpose CPU



Hardware



Hardware

ComputingI/O



Hardware

ASIC GPU

Computing

DSP

I/O

I/O Peripheral



Hardware

CPUMCUASIC GPU

Computing

DSP

I/O

I/O Peripheral



Hardware

CPUMCUASIC GPU

Computing

DSP

The Hardware/Software Interface (HSI)

I/O

I/O Peripheral



Hardware

CPUMCUASIC GPU

Computing

DSP


I/O

We (the software people) are here

I/O Peripheral



Hardware

CPUMCUASIC GPU

Computing

DSP


I/O


I/O Peripheral

Managed by internal (embedded) software



Hardware

CPUMCUASIC GPU

Computing

DSP


I/O


I/O Peripheral

Managed by internal (embedded) softwareExternally controlled by

Computing Hardware


System-on-Chip (SoC)


T h e g e n e r a l - p u r p o s e computing hardware you deal with is in fact compound of many specialised sub-units.

CPU

I/OCoreI/O

6

Tree map of usual sub-units in a SoC

MCU

CPU

I/OCoreI/O

Tree map of usual sub-units in a MCU

Memory


System-on-Chip (SoC)


T h e g e n e r a l - p u r p o s e computing hardware you deal with is in fact compound of many specialised sub-units.

CPU

I/OCoreI/O

6

Tree map of usual sub-units in a SoC

Somehow, this is your software which specialises it. Once it is

embedded in a closed system, its purpose is fixed.

MCU

CPU

I/OCoreI/O

Tree map of usual sub-units in a MCU

Memory



Example of an ARM SoC



Example of the Intel 8021 Microcontroller



HSI

Level of Software

Abstraction

Hardware



Low-Level Software

HSI

Level of Software

Abstraction

Hardware



Low-Level Software

HSI

FirmwareBare-metal

Level of Software

AbstractionSystem Call

Operating System

Hardware



Low-Level Software

HSI

FirmwareBare-metal

Level of Software

Abstraction

API

System Call

Runtime SystemLibrary

Operating System

Hardware

OS Service



Low-Level Software

HSI

FirmwareBare-metal

Level of Software

Abstraction

API

System Call


Program

Operating System

Hardware

OS Service



Low-Level Software

HSI

FirmwareBare-metal

Level of Software

Abstraction

API

System Call


Program

Operating System

Hardware

Be very careful with that…

OS Service


Challenge #1 - How do I know what should be software and hardware?

10


We need to break down the idea.


10




10

Recipe Search Recipe ReadingPower Button




10

Recipe Search Recipe ReadingPower Button

Note that these are the user interfaces…

Is that enough…?



11

Recipe Search



11

Speech Recognition

Recipe Search

Text-to-Speech



11

Speech Recognition

Recipe Search

Text-to-Speech

Microphone



11

Speech Recognition

Speaker

Recipe Search

Text-to-Speech

Microphone



11

Speech Recognition

Speaker

Recipe Search

Text-to-Speech

Microphone

Hardware

Software


Hardware

Software


12

Recipe Reading

Speech Recognition

Speaker

Step-by-Step Processing

Text-to-Speech

Microphone


Software

Hardware


13

Power Button

Boot Usually handled by an OS driver

Power On Power Off

Power Unit

Power Events

Usually handled by a firmware and

bootloader



14

The basic general picture for now



14

Drivers

Power Button Speaker Microphone

CPU

Text-to-Speech Speech Recognition

HSI


Recipe Search Recipe Reading


Hardware


14

Drivers


CPU


HSI




Hardware

I/Os


14

Drivers


CPU


HSI




Hardware

Computing

I/Os


14

Drivers


CPU


HSI




Software

Hardware

Computing

I/Os


14

Drivers


CPU


HSI




Software

Hardware

Computing

I/Os


14

Drivers


CPU


HSI

Executed by a CPU




Software

Hardware

Computing

I/Os


14

Drivers


CPU


HSI

Managed by drivers

Executed by a CPU




Software

Hardware

Computing

I/Os


15

Drivers


CPU


HSI

But we still don’t know what it exactly looks like…



Software

Hardware

Computing

I/Os


15

Drivers


CPU


HSI

Which exact I/Os?




Software

Hardware

Computing

I/Os


15

Drivers


CPU


HSI

Which exact CPU? Which exact I/Os?




We need to scale the hardware to our precise needs.

Challenge #2 - How do I select the hardware?


We need to scale the hardware to our precise needs.


Having a « chicken and egg » problem?


Use an empirical approach by prototyping starting from the first observations & deductions. Then test & evaluate and repeat further specifying your hardware until you reach your local optimum.



Use an empirical approach by prototyping starting from the first observations & deductions. Then test & evaluate and repeat further specifying your hardware until you reach your local optimum.

When you have no idea of a starting point, start with an XXL superset of hardware, rich enough to run your idea, and create a functional prototype.



Note that:• This process can be endless. It only depends on

what you are trying to accomplish. So be as lean as possible here ;-)



Note that:• This process can be endless. It only depends on

what you are trying to accomplish. So be as lean as possible here ;-)

• This is the moment where you may start doing hardware development in parallel of your software development: once the hardware chosen, the hardware team keeps further specifying it while you can start the software development.



In any case, the overall goal here is to make compromises between:




• Your physical constraints (space, weight, power, etc.).




• Your physical constraints (space, weight, power, etc.).• Your I/O requirements, including the interfaces with the

CPU.





CPU.• Your computing requirements.






And only targeting the current stage of your product:







1. Research







1. Research

2. Development







1. Research

2. Development

3. Production



The IoT gives us a very unique chance in the embedded world: it has the same I/O interfaces and peripheral as our mainstream general-purpose personal computers.




Which means that you can connect your IoT I/O peripherals to your own workstation to start prototyping!


Functional Prototype

Your OS

Your Workstation


Hardware


Software



Which means that you can connect your IoT I/O peripherals to your own workstation to start prototyping!


Functional Prototype

Your OS

Your Workstation


Hardware


Software

Assuming your OS has the driver (otherwise

implement it?)…


Going back to our chief hat!



I/O Requirements




23

It is made of single-purpose hardware which are selected according to their characteristics and our requirements.

I/OsPower Button Speaker Microphone



23

It is made of single-purpose hardware which are selected according to their characteristics and our requirements.


A push button A speaker for human ears…

A mic for human sounds…



24






24

It gives us our first CPU requirements:






24

It gives us our first CPU requirements:• They must be somehow connected to the CPU

using their interfaces (SPI, USB, PCIe…).






24

It gives us our first CPU requirements:• They must be somehow connected to the CPU

using their interfaces (SPI, USB, PCIe…).• They have bandwidth requirements of several

kbits/s.





Computing Requirements



Select a processor according to: • The kind of software you need to execute. • Your I/Os requirements. • Your physical constraints (power

consumption, temperature, etc.).



There are two main kind of processors:



There are two main kind of processors:• Computing-oriented: fast CPU with lots of

fast memory for CPU-bound programs (e.g., image processing).



There are two main kind of processors:• Computing-oriented: fast CPU with lots of

fast memory for CPU-bound programs (e.g., image processing).

• I/O-oriented: large I/O bandwidth, optimised I/O bridge for I/O-bound programs (e.g., a web server).



• CPU-bound programs scale according to the computing power of the CPU.

• I/O-bound programs scale according to the power of the CPU required to manage these I/Os.



So what kind of software do we run?


Drivers


Recipe Search Recipe ReadingAt run-time, most of the execution time will be probably spent in the speech algorithms. Which makes this software CPU-bound.




Drivers


Recipe Search Recipe ReadingOr not? Speech recognition is complex enough to require a huge amount of computing power we can’t afford in our embedded system. So let’s make benefit from the « I » of IoT and let’s implement this unit in the cloud.




Drivers


Recipe Search Recipe ReadingSo now, most of its execution time is spent waiting for I/Os to complete: sending requests to a speech server in the cloud and driving the speakers & microphone. Our software is I/O-bound.


And don’t forget about the new hardware and

software now required:


Software

Hardware

Computing

I/Os

Drivers


CPU


HSI


Network Interface Controller

Network Stack


Our CPU should include:



Our CPU should include:• The interfaces required to connect with our I/

O peripherals, unless already present in the selected CPU/SoC/MCU.



Our CPU should include:• The interfaces required to connect with our I/

O peripherals, unless already present in the selected CPU/SoC/MCU.

• Enough computing and I/O power to run our peripherals and our software fast enough for audio processing.



Running benchmarks are necessary when you have to further refine your assumptions to get closer to the perfect hardware set.



Running benchmarks in a bare-metal execution environment is a good practice to:




• Be alone in the CPU and get rid of hardware side-effects coming from concurrent or parallel software.





• Be at the lowest software abstraction level, above the HSI, to fully understand what’s going on during the benchmark.





• Be at the lowest software abstraction level, above the HSI, to fully understand what’s going on during the benchmark.

• Do whatever you want.



To do so…:



To do so…:• Select a JTAG probe compatible with your

CPU




CPU• affordable




CPU• affordable• implementable in less than a week




CPU• affordable• implementable in less than a week• with documentation & support



Or you can use instead Alpha, a GDB server, which brings:




• the ability to run standard C (malloc, fprintf, etc.) programs bare metal with the support of GDB.





• a bare metal execution environment ready-to-use.





• a bare metal execution environment ready-to-use.

Without any single JTAG probe.



Challenge #3 - How do I choose the right software architecture?

38

Low-Level Software

FirmwareBare-metal

Level of Software

Abstraction

API

System Call


Program

Operating System

OS Service

By understanding the main roles in this diagram:


• Started by the hardware from a read-only memory (flash, etc.).

• Given the constraints of its hardware environment, it is usually reserved for very special cases like booting hardware units (CPU, GPU, etc.).


39

Firmware



40

Bare-Metal


• Started by a firmware.• No abstraction but the one performed by the firmware(s).• Lowest-level software, right next to the HSI.• Doesn’t use any OS but usually libraries provided in the CPU/

SoC/MCU SDK, from simple helper functions to full network stacks.


40

Bare-Metal


• Started by a firmware.• No abstraction but the one performed by the firmware(s).• Lowest-level software, right next to the HSI.• Doesn’t use any OS but usually libraries provided in the CPU/

SoC/MCU SDK, from simple helper functions to full network stacks.

Used when:

✓ There is not enough memory for a complete OS.

✓ High-performance computing software.


40

Bare-Metal



41

Operating System


• Started by an OS-loader.

The two sides of an operating system:

1. Manages the hardware.

2. Provides a software abstraction of it with its own standard or specific interface.


41

Operating System



42

Operating System


Like any other software, consider its features and use the one that best matches your needs.


42

Operating System

Real-Time Support

Safety Security Standard API

Standard API

Driver APIs

Tooling


• Started by the OS. • Benefits from the abstraction of its language

library, runtime, shared libraries, etc.


43

Program


Use any language you like, the one you feel confident with to implement your features…


44

Program




44

Program

C++

GoPython

C

NodeJS

Java




44

Program

C++

GoPython

C

NodeJS

JavaCorrectly managing errors is a feature…

…that some languages provide built-in!


Chief Hat Program

Our connected chief hat requires enough high-level features, such as a network stack or a power event handler, that are good candidates to start at least prototyping using Linux on a simple ARM SoC include a network controller (e.g., wifi). The main advantage of using Linux being its amazing development tooling…


45

Linux


ARM



Network Interface Controller

Network StackARM Kernel Drivers


Challenge #4 - How do I do embedded programming?



Hardware



HardwareYour Workstation




Any kind of link




Any kind of link

Called the « target »




Any kind of link

Called the « target »Called the

« remote » link




Any kind of link

Called the « target »Called the « host »Called the

« remote » link




Any kind of link


« remote » link

Cross-Toolchain




Any kind of link


« remote » link

Cross-Toolchain

Tools to be executed on your workstation to manipulate « target » software.




Any kind of link


« remote » link

Cross-Compiler





Any kind of link


« remote » link


Cross-Compiled Program





Any kind of link





Any kind of link

Not working? Do it again...


Challenge #5 - How do I make embedded development simple?

49

It’s still such a « hard » world…



50

In such « hard » world…

Can we make it easier for Bob, its

teammates, its boss?



51



Firstly…

OS



51



Cross-GDB

GDB is the recommended tool. It allows to embed, debug and test the program, in one single amazing tool.

Firstly…

OS



51



Cross-GDB


Debug Informations of

the Cross-Compiled Program

Firstly…

OS



51



Cross-GDB


Alpha



Firstly…

OS



51



Cross-GDB


Alpha



Firstly…

OS

GDB Debug Protocol



51



Cross-GDB


Alpha



Firstly…

OS

The program is dynamically debugged, so no more static logs/

prints everywhere in the code

GDB Debug Protocol



51



Cross-GDB


Alpha



Firstly…

OS

The program is dynamically debugged, so no more static logs/

prints everywhere in the code

But also the entire system, including its

OS, drivers, and other processes.

GDB Debug Protocol


And finally, how would you find mistakes before they become bugs? Agile development? Test-Driven Development? Continuous Integration?

Dedicate your time and energy to your product using our second tool, Lab, which automates the setup of a target and its interfaces. Useful even on a day-to-day basis for your tedious manual setups of the targets and to be able later on to run automatic tests from a continuous integration server.


53


Challenge #6 - How do I go in production?

Hardware

Program in Read-Only Memory



HardwareProduction Test Bench


Any kind of link





Any kind of link

In the chief hat case, the network controller could be used.





Any kind of linkAlpha






Any kind of link

Alpha






Any kind of link

Cross-GDBAlpha






Any kind of linkDebug

Informations of Program in Read-

Only Memory

Cross-GDBAlpha



Hardware test benches are expensive and are a whole project of their own.

We recommend also using Lab to make it as much software as possible:

• Connect your hardware to your test instruments.

• Connect your instruments to Lab and you are down with hardware, you can continue fully software…


Simplicity is prerequisite of reliability

Edsger W. Dijkstra Computer Science Pioneer

linkedin.com/in/guerrajulio


Have Fun & Good Luck!

github.com/farjump/raspberry-pi

[email protected]

https://linkedin.com/in/guerrajulio

https://github.com/farjump/raspberry-pi

mailto:[email protected]