agl audio advance architecture - iot.bzh€¦ · agl advanced audio architecture feb-2018 7 high...

AGL Advanced Audio Architecture

Tokyo AGL/AMM Feb/2018Fulup Ar Foll Lead Architect

[email protected]

Feb-2018AGL Advanced Audio Architecture 2

Who are we?


Initial Assumptions Remain● Friendly API to developers

● Normalized high level API (Application portability)● Independence to sound card hardware● AGL framework native service (API transparency + Security)

● Support of advanced sound card controls● Volume Rump Up/Down● Mixing / DSP● Etc.

● Capability to integrate non-alsa features● Microchip Unicens● Power On/Off● External players● Etc.

● Functional Channels Mixing● Multimedia● Navigation● Telephony● Notification● Etc.


Architecture Layering


High Level Architecture


High Level Policy Binding

● Stable and standardized API to audio features● Different underlying implementations of common API

● Easily extensible API● Expose single access point to current and future AGL audio

capabilities

● Application isolation● Applications should not control audio behaviours of others● Access control resides on policy enforcement


High Level API Concepts

● Audio roles (e.g. entertainment, warning, communications, etc.)● Audio endpoints (source and sink endpoints)

● Provide applications display name for device (e.g. UI selection)● Provide applications device URI to stream to selected endpoint● Automatically retrieve associated volume control for ALSA softvol URI● Volume and properties (numeric (e.g. balance, EQ), or string (e.g. preset))

● Audio streams (audio role assignment)● Stream state (e.g. idle/running/suspended)● Stream mute state● Sound events (audio role assignment)● Integrate sound generation with audio stream management

● Connect to a custom renderer (e.g. HMI events, startup/ending sound, etc., AVAS, ...)


High Level Audio Config


HAL (Hardware Abstraction Layer)

Normalize an AGL virtual Sound Card

STATIC alsaHalMapT alsaHalMap= { { .alsa={.control=Master_Playback_Volume

,.numid=16,.group=OUTVOL,.values=1,.minval=0,.maxval= 87 ,.step=0,.acl=RW}, .info= "Master Playback Volume" },

{ .alsa={.control=PCM_Playback_Volume,.numid=27,.group=PCMVOL,.values=2,.minval=0,.maxval= 255,.step=0,.acl=RW}, .info= "PCM Playback Volume" },

{ .alsa={.control=PCM_Playback_Switch,.numid=17,.group=SWITCH,.values=1,.minval=0,.maxval= 1 ,.step=0,.acl=RW}, .info= "Master Playback Switch" },

{ .alsa={.control=Non_Alsa_Call,.numid=12,.group=INVOL ,.values=2,.minval=0,.maxval= 31 ,.step=0,.acl=RW},.cb={callback=MyCustomFunction, handle=MyHandle}, .info= "Capture Volume" },


AlsaCore Binding

● Expose ALSA native API (controls+UCM)● .name="getinfo", .info= "List All/One Sound Cards Info" },● { .name= "getctls", .info= "Get Controls" },● { .name= "setctls", .info= "Set Controls" },● { .name= "subscribe", .info= "Subscribe to events" },● { .name= "getcardid", .info= "Get CardId from its short/long name" },● { .name= "registerHal", .info= "Register Hal CardName/ApiPrefix" },● { .name= "ucmquery", .info= "Use Case Query" },● { .name= "ucmset", .info= "Use Case Set" },● { .name= "ucmget", .info= "Use Case Get" },● { .name= "ucmreset", .info= "Use Case Reset to Default" },● { .name= "ucmclose", .info= "Use Case Close Manager" },

● callbinder('alsacore','getctls', {devid:devid, numids:17})● callbinder('alsacore','setctls', {devid:devid, numids:[{id:1,val:[50,50]},

{id:17,val:50,50}]})

Generic Exposure of Lib Alsa API through AGL Framework.


Initial Architecture Proposal

UIHTML5, QT, GTK, ...

shadow audio API (ws-client:unix/usr/share/wsock/audio)

Application Business Logic

AGL Native Audio Apps

HAL (Hardware Abstraction Binding)

AGL AAAA(Stable Public API Binding)

Alsa Low Level Binding Unicens Binding

UCM

Sound Card

Controls

Libasound(alsa-lib)

(ws-server:unix/usr/share/wsock/audio)

Lib Unicens2Lib ADSP

Most/EthAVADSP

Application Binder

shadow Player API (ws-client:unix/usr/share/wsock/payer)

Unicens, Renesas, Intel, ...

ADSP Binding

Controls & Policies (Binding)

Eve

nts

Controls Controls

Events Events

AGL security Framework

AGL PCM Hook

Legacy AppsP

ulse

Aud

io

Mus

ic P

laye

r D

aem

on

Smack+Tag

AG

L A

dvan

ce A

udio

Ag

ent

MPC Music PlayerClient Binding

Smack+Cynary

Events

Controls

LUA Scripts

ﾲ


Issue with 4A/ELL implementation

● Configuration is too complex● One binary per HAL● Asound.rc depends on the audio configuration● One ALSA config per audio role

● Limited interface for Legacy applications● No direct Pulse or Gstreamer policy plugin● Lack of automatic PCM attachment to role in AlsaHook.

● Lack of flexibility● Policy engine is hard coded with high level API● No support for dynamic devices● No support for alsa-mixer API● No support for “on boot” automatic config


Alsa Core (proposal)

● Add support for Mixer API● Better discovery of control names● Support for linear volume management


4A/HighLevel API (proposal)

● Decouple Policy engine from the audio API● As a plugin● As a binding or an external binder

● Simplify API & secure API● Reduce the number of verbs● Provide default values to parameters● Create per role vers to leverage AGL security model

● Move 4A/HL to controller model● Allow to leverage dynamic API (one verb/role)● Provide scripting capability (LUA)● Provide automatic decoupling for policy engine


4A/HAL (proposal)

● Simplify HALs● Most HAL are very similar● JSON config should be enough to support >80% of HAL● Move HAL registration out of AlsaCore

● Use Controller model for standard cases● Allow to move out from C to JSON+LUA● Leverage dynamic API to support dynamic hot plug● Leverage controller plugin for sound effect

● Specific HAL (e.g. Unicens) ● May remain as C dedicated binding


AlsaHook (proposal)

● Simplify configuration● Unique PCM supporting multiple audio role● Dynamic config to handle

● Hardware differences● Device hotplug

● Merge AlsaHook and Softvol plugins● Support of hardware DSP mixer (e.g. Gen3)


Make Pulse Audio Optionnal

● Motivation● Useful for developer and demos● Out of scope for most production cases● Pulse should be install/remove at runtime

● Native integration of pulse with 4A● Create a PulseHook plugin to support 4A policies● Minimize the number of audio roles preempted by pulse.

● Enable ALSA/Bluetooth direct support ● Bluez-alsa as reference API● Proprietary stack

● Support Gsteamer/4A native interface● Create a policy GstreamerHook plugin


Pulse Audio

● Pulse ++● Out of the box support for existing apps (e.g. chrome)● Official Linux interface for Bluetooth

● Pulse - -● Resource usage (especially CPU)● Restart each time an error occur● Conciser any audio resource is preemtable at any time


Interface 4A with other binders

● Interface with signal composer● e.g. volume adaptation to speed

● Support of Speech interface● Priority management● Interface with audio roles

● Identity binding● e.g. language preference for messages

● Video● Multi SoC


Target 4A/FF Architecture

LegendUI

Binder

API (binding)System component

Shadow API

Plugin

Business logic binder/application

UIHTML5/QT

App Logic(Code/Controller

afb-media binder/service

4A-binder/service

Policies

Audio API

Alsa Core

LUAApp

scripts HAL(s)

LUAC/C++Policy

mediaplayer

4A API & events

Media APIMyPlayer Low Level ALSA

(libasound)

OptionalExternal

Policy Engine

ALSA (Sound Card)

Non Alsae.g.Unicens

Most

My Media Player Daemon

Audio Application

Signaling

Audio Service

Platform Services

even

ts


Please visit us.

● References: http://iot.bzh/en/publications● Contact: [email protected]

ﾲﾲﾲ

JonathanLoic/Identity

Seb/XLS Jose/AppFwRonan/YoMo

Romain/Signalling

Stephane/CTO

Fred/CIJeanLucBizzDevFulup/CEO

New Coffee Machine

New Open Space

http://iot.bzh/en/publications

agl audio advance architecture - iot.bzh€¦ · agl advanced audio architecture feb-2018 7 high...

Documents