android internals

Android Development&

Internals

1

What is and isn’t Android� AndroidIS:

� Tailored for touch-based app UX� App marketplace� Custom OS distribution� Complete and coherent app development API� Java based� Fully-integrated development suite� Very well documented development platform� Growing development community

2

What is and isn’t Android� AndroidIsn’t:

� Traditional Java (SE/ME/foo or otherwise)� Traditional “application” development model� Traditional Embedded Linux system

3

History�2002:

� Sergey Brin and Larry Page started using Sidekick smartphone� Sidedick one of 1st smartphones integrating web, IM, mail, etc.� Sidedick was made by Danger inc., co-founded by Andy Rubin (CEO)� Brin/Page met Rubin at Stanf. talk he gave on Sidekick’s development� Google was default search engine on Sidekick

� 2004:� Despite cult following, Sidekick wasn’t making $� Danger inc. board decided to replace Rubin� Rubin left. Got seed $. Started Android inc. Started looking for VCs.� Goal: Open mobile hand-set platform

� 2005 – July:Got bought by Google for undisclosed sum :)

� 2007 – November:Open Handset Alliance announced along with Android

� 2008 – September:�Android 1.0 is released

4

History� 2009 - Feb.: Android 1.1

� 2009 - Apr.: Android 1.5 / Cupcake

� 2009 - Sept.: Android 1.6 / Donut

� 2009 - Oct.: Android 2.0/2.1 / Eclair

� 2010 - May: Android 2.2 / Froyo

� 2010 - Dec.: Android 2.3 / Gingerbread

� 2011 - Jan : Android 3.0 / Honeycomb - Tablet-optimized

� 2011 - May : Android 3.1

� 2011 - Nov : Android 4.0 / Ice-Cream Sandwich - merge ofGingerbread and Honeycomb

� …..

� 2015– October:Android6/Marshmallow

� 2016– August:Android7/Nougat

5

App Model� No single entry point (No main() !?!?)

� Unlike Windows or Unix API/semantics in many ways

� Processes and apps will be killed at random: developermust codeaccordingly

� Apps are isolated, very

� Behavior predicated on low-memory conditions

6

User Experience

7

Features � Application framework enabling reuse and replacement of components

� Dalvik virtual machine optimized for mobile devices

� Integrated browser based on the open source WebKit engine

� Optimized graphics powered by a custom 2D graphics library; 3Dgraphics based on the OpenGL ES 1.0specification (hardwareacceleration optional)

� SQLite for structured data storage

� Media support for common audio, video, and still image formats(MPEG4, H.264, MP3, AAC, AMR, JPG,PNG, GIF)

� GSM Telephony (hardware dependent)

� Bluetooth, EDGE, 3G, and WiFi (hardware dependent)

� Camera, GPS, compass, and accelerometer (hardware dependent)

� Rich development environment including a device emulator, tools for debugging, memory andperformance profiling, and a plugin for theEclipse IDE

8

Android Stack

9

Tools� SDK tools

� Eclipse IDE / Android Studio� Integrated in ADT bundle

� http://developer.android.com

� NDK

� Tools for ROM building � http://source.android.com

10

Tools� android - manage AVDs and SDK components� apkbuilder - creating .apk packages� dx - converting .jar to .dex� adb - debug bridge� ddms - Dalvik debug monitor� monkey - test UI randomly� traceview - view app’s execution logs� logcat - view system logs

11

Basic Debugging� Using Log

Log.d(“tag”, ”message");

� Use‘adb logcat’tovieworeclipsewindow

� UsingToastToast.makeText(this, ”message", Toast.LENGTH_SHORT).show();

12

Demo� Building first application

� Simple debug using Log and Toast

13

APK� All app contents packaged into .apk:

� Compiled Java� Resources� Data� Native code� Etc.

� .apk generated by aapttool

� .apk is what is downloaded by users to theirdevice

� 1 apk = 1 app

14

Android Application� Apps live in isolated worlds:

� Each App is assigned its own UID:� Separate permissions (ex: /home/foo vs. /home/bar)� No files shared

� Every App is its own separate Linux process:� Process startup/shutdown is automagic/on-demand� No single point of failure

� Each process has its own Dalvik VM:� No inter-app code interference� No direct cross-app calls

� Exceptions:� Can arrange for 2 apps = 1 UID (shared files)� Can arrange for 2 apps of 1 UID = 1 Process

15

Application Components� 1 App = N Components

� Apps can use components of other applications

� App processes are automatically started whenever any partis needed

� N entry points, !1, and !main()

� Components:� Activities� Services� Broadcast Receivers� Content Providers

16

Intent� Intent = asynchronous message w/ or w/o designated target

� Like a polymorphic Unix signal, but w/o required target

� Intents ”payload” held in Intentobject

� Intent Filters specified in Manifest file

17

Activity� Visual interface for a single user interaction� Activities are independent from one another� public class Foo extends Activity { ...� 1 Activity = 1 default window, usually full-screen� Visual content = hierarchy of views (Viewclass):

ex.: containers, buttons, scroll bars, etc.

� Activity.setContentView() = set root Viewclass

18

Activity� Activated through passing Intent to:

Context.startActivity()

Activity.startActivityForResult()

� Activity’s onCreate() doesn’t provide the Intent

� Use getIntent() to look at initial Intent

� Subsequent Intents sent to onNewIntent() callback

� Intent results sent to onActivityResult() callback

� Shut down:� Self: finish()� Other Activity started w/ startActivityForResult(): finishActivity()� System

19

Service� Background service� Runs indefinitely� Talk to service = bind to it� public class Foo extends Service { ...� Runs in main app process� Remote calls run on thread pool

21

Service� Activated through:

Passing Intent to Context.startService()Call to Context.bindService()

� Both generate an onCreate() callback� Context.startService() generates onStart() callback:

Takes Intent as parameter� Context.bindService() generates onBind() callback:

Use RPC thereafter� Shut down:

Self: stopSelf()Other Component: Context.stopService()System

22

Broadcast Receiver� Receive and react to broadcast announcement� Usually system event:

Ex.: low bat, new pic, timezone change

� Apps can also initiate broadcasts� public class Foo extends BroadcastReceiver { ...� 1 App can have N Broadcast Receivers� No UI, but can start Activity or send Notifications

23

Broadcast Receiver� Send Intent through:

Context.sendBroadcast()

Context.sendOrderedBroadcast()

Context.sendStickyBroadcast()

� Trigger onReceive() callback

� Active while it’s responding to broadcast message

� Can be shut down by system

24

Content Provider� Makes data available to other apps� Data can be stored in FS or SQLite� public class Foo extends ContentProvider { ...� Apps use ContentResolverobject to talk to Content Provider� All IPC is transparent when using ContentResolverobject

� Activated through request from ContentResolver� Active while it’s responding to ContentResolver request� Can be shut down by system

25

The Manifest File� Informs system about app’s components� XML format� Always called AndroidManifest.xml� Activity = <activity> ... Static� Service = <service> ... Static� Broadcast Receiver:

Static = <receiver>

Dynamic = Context.registerReceiver()

� Content Provider = <provider> ... static

26

Processes and Threads� 1st time Components need to run: System starts Linux

process

� Default: all Components of an app run in single process

thread

� Defaults can be overriden:

� Run Components in other processes

� Spawn additional threads for any process

27

Threads� Create using the regular Java Threadobject

� Android API provides thread helper classes:� Looper: for running a message loop with a thread

� Handler: for processing messages

� HandlerThread: for setting up a thread with a message loop

28

Android Binder� Android RPCs = Binder mechanism� Binder is a low-level functionality, not used as-is

� must define interface using Interface DefinitionLanguage(IDL)

� IDL fed to aidltool to generate Java interface definitions

29

Managing Resources� Usually declared on XML file

� The “Designer” area for� Images� Layouts

� Styles� Colors

� Strings� Menus� …

30

Building UI� UI is built on top of Viewobject and ViewGroupobject

� Viewclass is root for subclassed ”widgets” (text fields, buttons, etc.)

� ViewGroupclass is root for subclassed ”layouts”� Viewobject = rectangle on screen. Handled by object for rectangle:

Measurement

Layout

Drawing

Focus change

Scrolling

Key/gesture interactions

31

Views � Activity UI defined as Viewobject and ViewGroupobject hierarchy

� Viewobject = leaves� ViewGroupobject = branch

32

Layout� Declare in XML (statically) or in Java (at runtime)� Easiest to boot: XML� Coherent element nomenclature:

� <TextView> = TextViewclass

� <LinearLayout> = LinearLayoutclass

� System creates Java objects corresponding to XML layout

33

Layout Example

34

<?xml version="1.0" encoding="utf-8"?><LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"

android:layout_width="fill_parent"

android:layout_height="fill_parent"

android:orientation="vertical" >

<TextView android:id="@+id/text"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:text="Hello, I am a TextView" />

<Button android:id="@+id/button"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:text="Hello, I am a Button" /></LinearLayout>

Layouts� Nested layout elements allow creating tree� More layout elements exist:

LinearLayout

RelativeLayout

TableLayout

GridLayout

...

� Use addView(View) at runtime to insert additional Viewobject

and/orViewGroupobject

35

Load the XML Resource� On compile, a Viewresource is created based on XML� Loading resource at Activity creation:

public void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

setContentView.(R.layout.main_layout);

}

� The resource is R.layout.* where *.xml is the filenamein ’res/layout’

36

ID� Viewobjects can have unique IDs in hierarchy tree:

Specified as string in XML

Compiled as unique integer

� ’@+’ tells aapttool to create new resource integerandroid:id="@+id/my_button"

� Can then refer to Viewobject in code:Button myButton = (Button) findViewById(R.id.my_button);

� Android resource IDs can be referenced without ’+’:android:id="@android:id/empty"

37

Widgets �Widget = Viewobject w/ specific UI:

ButtonsCheckboxesText-entry fieldDate pickerClockZoom controls

� See android.widgetpackage

� You can create custom widgets

38

UI Events� To get events:

� Define and register event listener, or� Override existing one of widget’s callbacks

� Event listener:� Most common case� Viewclass contains collections of nested interfaces w/ callbacks� Must implement interface/callback and register it to Viewobject

� Generic form:On*Listenerinterface & On*() callback; View.setOn*Listener()

39

Events� Event listener:Examples:View.OnClickListener & onClick(); View.setOnClickListener()View.OnTouchListener & onTouch(); View.setOnTouchListener()View.OnKeyListener & onKey(); View.setOnKeyListener()

� Callback override:For custom widgetsActual Widgetobject callbacks:onTouchEvent()onKeyDown()onKeyUp()

40

// Create an anonymous implementation of OnClickListener

private OnClickListener mCorkyListener = new OnClickListener() {

public void onClick(View v) {

// do something when the button is clicked

}

};

protected void onCreate(Bundle savedValues) {

...

// Capture our button from layout

Button button = (Button)findViewById(R.id.corky);

// Register the onClick listener with the implementation above

button.setOnClickListener(mCorkyListener);

...

41

public class ExampleActivity extends Activity implements OnClickListener {

protected void onCreate(Bundle savedValues) {

...

Button button = (Button)findViewById(R.id.corky);

button.setOnClickListener(this);

}

// Implement the OnClickListener callback

public void onClick(View v) {

// do something when the button is clicked

}

...

}

42

Menus � Types:

� Main app menu view through the MENU key� Contextual menus

� Menus are Viewobject hierarchies too� However:

� Hierarchy is automatically created by system, not you� No need to register event listeners

� Instead, implement callbacks for:� Populating menu:

� onCreateOptionsMenu()� onCreateContextMenu()

� Handling menu selection:� onOptionsItemSelected()� onContextItemSelected()

43

Inflating a Menu Resource� Convert XML to programmable object:

@Override

public boolean onCreateOptionsMenu(Menu menu) {

MenuInflater inflater = getMenuInflater();

inflater.inflate(R.menu.game_menu, menu);

return true;

}

� Activity’s onCreateOptionsMenu() called on 1st MENU key press

� Can also use Menu.add() to add items

44

Context Menu� ”Right-click” equivalent� Activated on long-press on Viewobject

� No icons or shortcuts� Most often implemented for items in ListViewobject

� To enable: registerForContextMenu() and pass Viewobject

� For ListView:registerForContextMenu(getListView());

� To inflate: override onCreateContextMenu()� To handle: override onContextItemSelected()

45

@Overridepublic void onCreateContextMenu(ContextMenu menu, View v,

ContextMenuInfo menuInfo) {super.onCreateContextMenu(menu, v, menuInfo);MenuInflater inflater = getMenuInflater();inflater.inflate(R.menu.context_menu, menu);}

@Overridepublic boolean onContextItemSelected(MenuItem item) {AdapterContextMenuInfo info = (AdapterContextMenuInfo) item.getMenuInfo();switch (item.getItemId()) {case R.id.edit:editNote(info.id);return true;case R.id.delete:deleteNote(info.id);return true;default:return super.onContextItemSelected(item);}}

46

Creating Dialogs � Small window appearing in front of Activity� Types:

AlertDialog, can be used to construct most dialogs

ProgressDialog, progress wheel or bar

DatePickerDialog

TimePickerDialog

� Typically created programmatically in Java� Can create custom dialogs using XML

47

Showing The Dialog� Always part of Activity

� To display:� Define unique IDs for each dialog

� Call showDialog() w/ ID when appropriate

� Implement switch-case in onCreateDialog() to create dialogs

� Implement switch-case in onPrepareDialog() to modify after creaion

48

Exampleprotected Dialog onCreateDialog(int id) {

Dialog dialog;switch(id) {case DIALOG_PAUSED_ID:

// do the work to define the pause Dialogbreak;

case DIALOG_GAMEOVER_ID:// do the work to define the game over Dialogbreak;

default:dialog = null;

}return dialog;

}

49

Dismissing� Dimissing dialog:

Dialog.dismiss()

dismissDialog(int)

� Dimissing dialog (if shown) and removing all references:

removeDialog(int)

� Listening to dismissal:

DialogInterface.OnDismissListenerinterface ->onDismiss(DialogInterface)

Use Dialog.setOnDismissListener()

� Listening for cancel (BACK key or ”Cancel” selected):

DialogInterface.OnCancelListenerinterface ->onCancel(DialogInterface)

Use Dialog.setOnCancelListener()

50

AlertDialog� Use for dialogs containing any of:

Title

Text message

One, two, or three buttons

Selectable items list (checkboxes, radio buttons)

� Instantiate AlertDialog.Builderobject using current Contextobject

� Call on AlertDialog.Builder.* to populate dialog

� Retrieve AlertDialogobject using AlertDialog.Builder.create()

51

AlertDialog.Builder builder = new AlertDialog.Builder(this);

builder.setMessage("Are you sure you want to exit?")

.setCancelable(false)

.setPositiveButton("Yes", new DialogInterface.OnClickListener() {

public void onClick(DialogInterface dialog, int id) {

MyActivity.this.finish();

}

})

.setNegativeButton("No", new DialogInterface.OnClickListener() {

public void onClick(DialogInterface dialog, int id) {

dialog.cancel();

}

});

AlertDialog alert = builder.create();

52

53

Android Kernel

54

Android Stack

55

Android Kernell Significant efforts have been made by the Linux community to

move most of the Android-specific changes back into the mainline Linux kernel (3.3 and 3.5), with a hope that Android could then switch to the official kernel.

l Android is not "Linux"- no glibc- no X11- many standard configuration files are missing: no

/etc/passwd, no /etc/fstab, etc.- many standard Linux utilities are missing: no

/bin/cp, no /bin/su, etc.

56

Android Kernell Android runs on a modified (a.k.a. forked) Linux kernel

57

Kernel Providesl Hardware abstraction layer (low level)

- Well-understood driver model- Many drivers for common devices- "Free" drivers for future devices

l Process and memory management

l Simple, but secure, per-process sandboxing (permissions-based security model)

l Support for shared libraries

l Network stack

58

l Application developers and users never "see" the Linux kernel

l The adb shell command opens Linux shell (remember, limited standard utils)

l Maintains a separate "forked" git tree (https://android.googlesource.com/)

59

Binder IPCl OpenBinder-based IPC driver enables "object-oriented

operating system environment"

- Exposed via /dev/binder- Runtime info at /proc/misc/binder- Like CORBA, but much simpler- Initially developed for BeOS later used by Palm (which

acquired BeOS)

l By default, apps and services (including system services) run in separate processes, but often need to share data

l Traditional IPC leads to security challenges and adds overhead, which is amplified on a mobile device

l Most of Android infrastructure (services) is supported by Binder

60

Binder IPCl Binder is lightweight and high-performance

- Bound services are automatically reference-counted and "garbage collected" when no longer in use

- Provides automatic per-process thread pooling for services (with remote clients)

- Remote (service) method calls are synchronous (feels like just a function call, even though it’s IPC)

l Supports oneway (asynchronous) execution model as well

l Services defined/exposed via AIDL

l Implementation is at drivers/misc/binder.c with include at include/linux/binder.h

61

Ashmem (Anonymous shared memory)l A reference-counted, virtually mapped, named memory block that is

shared between processes that the kernel is allowed to free

l Similar to POSIX SHM but with different behavior and a simpler file-based API (POSIX SHM does not allow the kernel to free shared memory)

l Programs open /dev/ashmem, use mmap() on it, and then share file handles via binder

- When all processes close(fd) to the shared memory region, the kernel automatically reclaims that memory (because it is reference-counted)

62

Ashmeml Supports a number of ioctl()-s: ASHMEM_SET_NAME,

ASHMEM_GET_NAME, ASHMEM_SET_SIZE, ASHMEM_GET_SIZE, ASHMEM_SET_PROT_MASK, ASHMEM_GET_PROT_MASK, ASHMEM_PIN, ASHMEM_UNPIN, ASHMEM_GET_PIN_STATUS, ASHMEM_PURGE_ALL_CACHES

l Android uses ashmem to share resources to minimize redundancy across processes

l Kernel can discard unused shared blocks of memory when under pressure

l Implementation at mm/ashmem.c with include at include/linux/ashmem.h

l Represented in the Java layer as android.os.MemoryFile

63

Physical Memory Allocatorl Allocates physically-mapped kernel-owned memory buffers and

facilitates sharing of these buffers by user-space processes and devices (like GPU and Camera)

- Buffers can be pre-allocated to be contiguous, depending on the needs of the device

- By sharing physical memory with the userspace, the kernel can avoid copying large memory buffers where latency is an issue (GPU, camera)

- Unlike Ashmem, the use of this physical memory is not reference counted, and as such it is not meant for general-purpose sharing of data

64

l Multiple implementations:

- Pmem - originally used on MSM SoCs to manage large (1-16+ MB) physically contiguous regions of memory shared between userspace and kernel drivers for DSP, GPU, etc.

l Implementation at drivers/misc/pmem.c with include at include/linux/android_pmem.h

l Supports a number of ioctl()-s: PMEM_GET_PHYS, PMEM_MAP, PMEM_GET_SIZE, PMEM_UNMAP, PMEM_ALLOCATE, PMEM_CONNECT, PMEM_GET_TOTAL_SIZE, PMEM_CACHE_FLUSH

l ION - used on newer Android devices (like Galaxy Nexus) as a more general-purpose memory manager with a common API

- It comes with multiple memory allocators: vmalloc, kmalloc, carveout (genalloc), and CMA

- Exposed as /dev/ion it takes multiple ioctl()'s: ION_IOC_ALLOC, ION_IOC_FREE, ION_IOC_MAP, ION_IOC_SHARE, ION_IOC_IMPORT, ION_IOC_CUSTOM (API by libion)

65

Wakelockl Extended power management

l More aggressive power-manager policy than standard Linux PM

l "CPU shouldn’t consume power if no applications or services require power" - i.e. the CPU shuts down eagerly

l Applications and services that wish to continue running (after a short timeout following a user activity) are required to request "wake locks" via the app framework or native libs

l Wake locks are used by applications and services to request CPU resources

l WAKE_LOCK_SUSPEND: prevents a full system suspend

l WAKE_LOCK_IDLE: prevents entering low-power states from idle to avoid large interrupt latencies (disabled interrupts) - so the system is more responsive

66

Wakelock

68

Wakelockl Alls user-space wake-lock management must go through Java-

based PowerManagerService (even from user-space native libraries)

l Baseband processor normally does not shut down, so network traffic still raises interrupts allowing CPU to wake up

l Stats exposed via /proc/wakelocks

l Implementation at drivers/android/power.c with include at include/linux/wakelock.h

l Linux kernel 3.5 adds Android-style opportunistic suspend

69

Early Suspendl An extension to the standard Linux power management stages

l Not meant for suspending the device

l Allows the system to power-down some of its components (like screen, sensors, etc.)

l When user requested sleep state changes, kernel notifies drivers of early suspend

l User-space changes the sleep state by writing to /sys/power/request_state: EARLY_SUSPEND_LEVEL_BLANK_SCREEN, EARLY_SUSPEND_LEVEL_STOP_DRAWING, EARLY_SUSPEND_LEVEL_DISABLE_FB, EARLY_SUSPEND_LEVEL_STOP_INPUT

l The opposite of early suspend is "late resume" - the stage that resumes these devices

l Enabled by EARLYSUSPEND

l Configured in one of three modes:

l NO_USER_SPACE_SCREEN_ACCESS_CONTROL - no user-space control

l CONSOLE_EARLYSUSPEND - user-space control via the console

l FB_EARLYSUSPEND - user-space control via the sysfs

l Implementation at kernel/power/fbearlysuspend.c (or kernel/power/consoleearlysuspend.c) with include at include/linux/earlysuspend.h

l See http://lwn.net/Articles/416690/

70

Alarml Kernel support for Android’s AlarmManager

l User-space tells kernel when it would like to wake up

l Kernel schedules a time-based call-back (while holding a WakeLock) regardless of the sleep-state of the CPU

l Apps can then run (need to hold their own WakeLocks)

l Implementation at drivers/rtc/alarm.c with include at include/linux/android_alarm.h

l Exposed via /dev/alarm

l Supports a number of ioctl()-s: ANDROID_ALARM_CLEAR, ANDROID_ALARM_WAIT, ANDROID_ALARM_SET, ANDROID_ALARM_SET_AND_WAIT, ANDROID_ALARM_GET_TIME, ANDROID_ALARM_GET_TIME

l See https://lwn.net/Articles/429925/

71

Low Memory Killerl Automatically SIGKILLs eligible least-recently-used processes

when running low on memory

l More aggressive than standard OOM handling

l Implementation at drivers/staging/android/lowmemorykiller.c and security/lowmem.c

l System sets up 6 priority slots via init.rc (writes to /sys/module/lowmemorykiller/parameters/) and user-space ActivityManager then sets each app’s /proc/<pid>/oom_adj (from -16 to 15)

72

# Define the oom_adj values for the classes of processes that can be# killed by the kernel. These are used in ActivityManagerService.

setprop ro.FOREGROUND_APP_ADJ 0setprop ro.VISIBLE_APP_ADJ 1setprop ro.PERCEPTIBLE_APP_ADJ 2setprop ro.HEAVY_WEIGHT_APP_ADJ 3setprop ro.SECONDARY_SERVER_ADJ 4setprop ro.BACKUP_APP_ADJ 5setprop ro.HOME_APP_ADJ 6setprop ro.HIDDEN_APP_MIN_ADJ 7setprop ro.EMPTY_APP_ADJ 15

# Define the memory thresholds at which the above process classes will# be killed. These numbers are in pages (4k).

setprop ro.FOREGROUND_APP_MEM 2048setprop ro.VISIBLE_APP_MEM 3072setprop ro.PERCEPTIBLE_APP_MEM 4096setprop ro.HEAVY_WEIGHT_APP_MEM 4096setprop ro.SECONDARY_SERVER_MEM 6144setprop ro.BACKUP_APP_MEM 6144setprop ro.HOME_APP_MEM 6144setprop ro.HIDDEN_APP_MEM 7168setprop ro.EMPTY_APP_MEM 8192

# Write value must be consistent with the above properties.# Note that the driver only supports 6 slots, so we have combined some of# the classes into the same memory level; the associated processes of higher# classes will still be killed first.

write /sys/module/lowmemorykiller/parameters/adj 0,1,2,4,7,15

write /proc/sys/vm/overcommit_memory 1write /proc/sys/vm/min_free_order_shift 4write /sys/module/lowmemorykiller/parameters/minfree 2048,3072,4096,6144,7168,8192

# Set init its forked children's oom_adj.write /proc/1/oom_adj -16

73

l To application developers this means that low memory killer stacks processes based on the following order:

1. Foreground processes - with an Activity that just ran onResume(), or a Service bound to it or started as foreground, or executing its callback methods, or a BroadcastReceiver executing onReceive()

2. Visible processes - with an Activity that just ran onPause() but is still visible or a Service bound to a component from a visible process

3. Service processes - with a Service that has been started with Context.startService()

4. Background processes - with an Activity that just ran onStop()

5. Empty processes - with no components (kept around just for caching purposes)

l See http://developer.android.com/guide/topics/fundamentals/processes-and-threads.htmll Everything directly started from init.rc (including the system_server) has its

oom_adj set to -16l If we get to killing those, the system is toast anywayl Applications can request that they be kept persistent (in memory) by setting

<application android:persistent="true" ...> in their AndroidManifest.xml file

l ActivityManager then starts persistent apps and initializes their oom_adj to -12

74

Loggerl System-wide logging facility (from Kernel all the way to apps)

l This is what logcat command reads from

l Supports four auto-rotating log buffers managed by the kernel- /dev/log/main (64KB)

l Destination for Android appsl Most logging happens via android.util.Log

- /dev/log/system (64KB)

l Destination for Android framework’s system services and libraries

l Most logging happens via hidden android.util.Slog or directly via liblog library

75

l /dev/log/events (256KB)

- Destination for Android system diagnostic events - e.g. garbage collections, activity manager state, system watchdogs, and other low level activity

- Logging via android.util.EventLog or directly via liblog library

- Binary-encoded - can be decoded via /system/etc/event-log-tags

l /dev/log/radio (64KB)

- Destination for radio and phone-related information

l Log reading and writing is done via normal Linux file I/O

- Calls to open(), write(), and close() are extremely low-overhead on these devices

- Each read() returns exactly one log entry (up to 4KB) and can be both blocking and non-blocking

l Implementation at drivers/misc/logger.c (with a logger.h in the same directory)

l Supports a number of ioctl()-s: LOGGER_GET_LOG_BUF_SIZE, LOGGER_GET_LOG_LEN, LOGGER_GET_NEXT_ENTRY_LEN, LOGGER_FLUSH_LOG

76

Paranoid Network Securityl Restricts access to some networking features depending

on the group of the calling process

l Enabled via CONFIG_ANDROID_PARANOID_NETWORKkernel build option, which defines process group IDs that have special network access:

80

Other Kernel Changesl Timed output / Timed GPIO

- Generic GPIO allows user space to access and manipulate GPIO registers

- Timed GPIO allows changing a GPIO pin and having it restored automatically after a specified timeout

- Implementation at drivers/android/timed_output.c and drivers/android/timed_gpio.c

- Used by the vibrator by default

81

l Linux Scheduler

- Not a custom scheduler, just Android-specific configuration in init.rc

- write /proc/sys/kernel/panic_on_oops 1

- write /proc/sys/kernel/hung_task_timeout_secs 0

- write /proc/cpu/alignment 4

- write /proc/sys/kernel/sched_latency_ns 10000000

- write /proc/sys/kernel/sched_wakeup_granularity_ns2000000

- write /proc/sys/kernel/sched_compat_yield 1

- write /proc/sys/kernel/sched_child_runs_first 0

82

l Switch events - userspace support for monitoring GPIO used by vold to detect USB

l USB gadget driver for ADB (drivers/usb/gadget/android.c)

l yaffs2 flash filesystem, though this is switching to ext4

l RAM console

- Kernel’s printk goes to a RAM buffer

- A kernel panic can be viewed in the next kernel invocation via /proc/last_kmsg

l Support in FAT filesystem for FVAT_IOCTL_GET_VOLUME_ID

83

Android User Space Native Layer

84

Bionic Libraryl Custom standard C library (libc) derived from BSD optimized for

Android

l Why Bionic?- BSD licensed (business-friendly - i.e. keeps GPL out

of user-space)l Proprietary code linked to bionic can remain

proprietary

- Lean (~200KB, or about half the size of glibc)l It is loaded into every process, so it needs to be small

- Fast (custom pthread impl) - perfect for embedded use

85

Changes From BSD libcl Support for arbitrary Android system-properties via

<sys/system_properties.h>

l Support for Android Kernel Logger Driver (via liblog)

l Support for Android-specific user/group management- Enabled via getpwnam(), getgrouplist(), and,

getgrgid(), which are aware of generated UIDs of the applications (>10000) and their corresponding synthetic user/group-names (e.g. app_123)

- Basic UID/GIDs defined in /system/core/include/private/android_filesystem_config.h

86

l Support for Android-specific getservent(), getservbyport(), and getservbyname() in place of /etc/services

l No support for /etc/protocol

l "Clean" kernel headers that allow user-space to use kernel-specific declarations (e.g. ioctl’s, structure declarations, constants, etc.)

l Custom pthread implementation based on Linux futexes

- Bundled-in (i.e. -lpthread not required)

- Optimized for embedded use - strives to provide very short code paths for common operations

l Normal, recursive and error-check mutexes are supported

l No support for process-shared mutexes and condition variables (use Android’s own Binder-IPC instead) as well as read/write locks, priority-ceiling in mutexes, pthread_cancel(), and other more advanced features (not a priority for embedded use)

l Provides only 64 as opposed to 128 thread-specific storage slots required by POSIX

l No support for read/write memory barriers (restricts SMP/multi-core on certain architectures)

- Does not support all the relocations generated by other GCC ARM toolchains

87

l No support for System V IPCs - to avoid denial-of-service

- SysV has no way to automatically release a semaphore allocated in the kernel when

l a buggy or malicious process exitsl a non-buggy and non-malicious process crashes

or is explicitly killed (e.g. via low-memory-killer)

- Again, we use Android’s own Binder-IPC instead

l No support for locales (I18N done at the application/Dalvik layer via well-defined resource mechanism)

- No support for wide chars (i.e. multi-byte characters)

l time_t is 32-bit on 32-bit hardware

l Timezones are defined via TZ env-vars or via persist.sys.timezone system property

88

l NetBSD-derived DNS resolver library

- Reads from /system/etc/resolv.conf

- Uses name servers defined in net.dns1, net.dns2 system properties

l Can be process specific: net.dns1.<pid>

l Built-in linker

- Support pre-linked mapping filesl Support for x86, ARM and ARM thumb CPU instruction sets and kernel

interfaces

l Not binary compatible with any other known Linux C library (glibc, ucLibc, etc.)

- Not event fully POSIX-compliant

- No support for C++ exceptions

- Requires recompile of existing legacy code against bionic

l See ndk/docs/system/libc/OVERVIEW.html for more info (in the Android source tree)

89

User space HALl User-space C/C++ hardware abstraction layer - as shared libraries

l Communicate with Linux drivers via /dev/, /sys/, or /proc/

l Why not just use Linux drivers directly?- Separates Android platform logic from specific hardware

interfaces- Linux does not have common definitions of hardware that upper

layers depend on- User-space HAL offer standard "driver" definitions for graphics, audio,

camera, bluetooth, GPS, radio (RIL), WiFi, etc.- Makes porting easier

l OEMs implement "drivers" for specific hardware as shared libraries- libhardware

- libhardware_legacy- The code can remain proprietary since the user-space drivers link

against bionic, not the kernel (i.e. no GPL)

90

l Platform loads these HAL libs at runtime via pre-determined naming strategies

- libhardware is a simple shared library that can load device-specific shared libraries via hw_get_module(const char *id, conststruct hw_module_t **module)

l First checks under /vendor/lib/hw/ and then /system/lib/hw/ as follows:

- <*_HARDWARE_MODULE_ID>.<ro.product.board>.so- <*_HARDWARE_MODULE_ID>.<ro.board.platform>.s

o- <*_HARDWARE_MODULE_ID>.<ro.arch>.so

- <*_HARDWARE_MODULE_ID>.default.so

l For example, on a Nexus S (where TARGET_BOARD_PLATFORM=s5pc110, board=herring, and /vendor/→ /system/vendor)

- The GPS "driver" is loaded from /system/vendor/lib/hw/gps.s5pc110.so (where GPS_HARDWARE_MODULE_ID="gps")

- The Sensors "driver" is loaded from /system/lib/hw/sensors.herring.so (where SENSORS_HARDWARE_MODULE_ID="sensors")

91

l libhardware_legacy is a shared library for vibrator, wifi-module-loader, power, uevent, audio, camera, etc.

- Some board-independent hardware (like vibrator, power, wifi, etc.) is directly supported by /system/lib/libhardware_legacy.so via well-defined paths on the file system (mostly via /proc or /sys) or well-defined system properties

- Board-specific devices (like audio, camera, etc.) are supported by separate shared libraries loaded by well-defined names (e.g. /system/lib/libaudio.so, /system/lib/libcamera.so, etc.)

l Radio is a bit special, as its rild (Radio Interface Link Daemon) loads "libril" as defined by rild.libpath system property

- On Nexus S, this is /vendor/lib/libsec-ril.so

92

Native Daemonsl /system/bin/servicemanager

- The naming service for all other system_server's (i.e. framework) services

- Registers as BINDER_SET_CONTEXT_MGR on /dev/binder and starts reading from it (in a loop)

93

voldl /system/bin/vold

l Volume Daemon used for mounting/unmounting removable media (like /mnt/sdcard) on demand

l Configured via /system/etc/vold.fstab

l For example, Nexus One’s SD Card is automatically re/mounted:dev_mount sdcard /mnt/sdcard auto /devices/platform/goldfish_mmc.0

/devices/platform/msm_sdcc.2/mmc_host/mmc1l While Nexus S, has a "virtual" but fixed SDCard:

dev_mount sdcard /mnt/sdcard 3 /devices/platform/s3c-sdhci.0/mmc_host/mmc0/mmc0:0001/block/mmcblk0

94

rildl /system/bin/rild

l Acts as a bridge between platform framework services (TelephonyManager) and libril, which is OEM-specific interface to the baseband modem

l Stateful - helps handle incoming calls/messages (unsolicited requests)

l Interfaces with upper layers via a Unix socket connection

95

netdl /system/bin/netd

l Manages network connections, routing, PPP, etc.

l Enables tethering, connections over USB/Bluetooth, etc.

96

Media Serverl /system/bin/mediaserver

l Home of - AudioFlinger- MediaPlayerService- CameraService- AudioPolicyService

97

installdl /system/bin/installd

l Listens on a unix socket and performs installation/uninstallation of packages (i.e. apps)

l Used by the PackageManager

98

keystorel /system/bin/keystore

l Listens on a unix socket and provides secured storage for key-value pairs

l Keys are encoded in file names, and values are encrypted with checksums

l The encryption key is protected by a user-defined password

99

debuggerdl /system/bin/debuggerd

l Catches crashes of native processes and dumps their stack trace to /data/tombtones/

l When native processes initialize, they implicitly connect (through a unix socket) to debuggerd through a separate thread spawned by bionic

100

wpa_supplicant and dhcpdl /system/bin/wpa_supplicant

- Handles WPA authentication for WiFi networks

l /system/bin/dhcpd

- Requests (leases) IPs from DHCP servers- Handles network changes (e.g. 3G to Wifi)

101

BlueZl /system/bin/dbus-daemon

- A simple IPC (bus) framework- Provides system_server with a way to access hcid

(Bluetooth Host Controller Interface Daemon)

l /system/bin/bluetoothd

- Manages device pairings and the rest of the stack

l /system/bin/sdptool

- Used to manage individual Bluetooth profile as servicesl hfag - Hands-Free Profilel hsag - Headset Profilel opush - Object Push Profilel pbap - Phonebook-Access Profile

l etc.

102

ueventdl /system/ueventd

l Handles uevents from the Kernel and sets up correct ownership/permissions on the device file descriptors

l Applies configuration from /ueventd.rc

103

l /system/bin/racoon

- Assists with ipsec key negotiations (IKE)- Used for VPN connections

l /sbin/adbd

- End-point of Android Debug Bridge- Accepts ADB connections over USB- Possible to accept network connections as

welll Vendor-specific daemons which fasciliate interaction with the

hardware

- For example /system/vendor/bin/gpsd on Nexus S

104

Surface flingerl SurfaceFlinger is Android’s system-wide screen composer that draws into

standard Linux frame-buffer (/dev/fb0)

l Apps draw (in 2D or 3D) into "windows", which are implemented as double-buffered Surface objects backed by the surface flinger

- Front-buffer used for composition, back-buffer for drawing- Buffers are flipped after drawing

l Minimal buffer copyingl Avoids flickers and artifacts as the front-buffer is always

available for composition

l Surface flinger expects the video driver to offer:- A linear address space of mappable memory

l Video memory is mmap()'ed to process address-space for direct writing

l Enough video memory for twice the physical screen area- Otherwise, regular system memory has to be used

for buffering, and is copied on flips (slow!)- Support for RGB 565 pixel format

105

Audio flingerl AudioFlinger is Android’s system-wide audio stream routing

engine/mixer and audio input capture facility- Sits on top of device-specific libaudio.so

implementation, which usually simply bridges to ALSA

l To play audio, apps send uncompressed mono/stereo PCM streams to audio flinger (usually via MediaPlayer)

- Streams include ringtones, notifications, voice calls, touch tones, key tones, music

- Audio flinger routes these streams to various outputs (earpiece, speakers, Bluetooth)

l To capture audio, apps request access to uncompressed input path managed by the audio flinger (usually via MediaRecorder)

106

Function Librariesl Provide computation-intensive services to the rest of the

platform- This is in addition to bionic

l Many pieces borrowed from other open source projects- LibWebCore/WebKit, V8, SQLite, OpenSSL,

FreeType, etc.- Usually abstracted by Java counterparts

107

l Media Framework Libraries

- Originally based on PacketVideo’s OpenCORE platform- Switched to Stagefright with Gingerbread- Support for playback and recording of many popular

audio and video formats, as well as static image filesl MPEG4, H.264, VP8/WebM, MP3, AAC, AMR, JPG, and

PNG

- Pluggable via Khronos' OpenMAX IL (supports for codecs in both software and hardware)

l 3D libraries

- Support for OpenGL ES 1.0 and 2.0 APIs- Comes with highly optimized 3D software rasterizer -

when hardware does not offer native OpenGL supportl 2D libraries

- SGL (Skia) - the underlying 2D graphics engine

108

Dalvikl Dalvik is a custom clean-room implementation of a virtual machine,

semantically similar to a JVM but not a JVM

- Licensed under Apache 2.0 open-source license

l Provides Android app portability and consistency across various hardware (like a JVM)

l Runs Dalvik byte-code, stored in .dex files (not Java byte code)- Developers program in the Java language (i.e. .java files),

which get compiled into Java byte-code (i.e. .class files)- Build-tools compile Java’s .class files into a .dex file

before packaging (into .apk files)

l 3rd party libraries are also re-compiled into dex code

- Dalvik never sees any Java byte-code

109

Sun/Oracle java? l Why not Java SE?

- Java SE is too bloated for mobile environment- Not well-optimized for mobile (at the bytecode and

interpreter level)

l Why not Java ME?- Costs $$$ - hinders adoption- Designed by a committee - hard to imagine iOS-like

developer appeal- Apps share a single VM - not great for security

sandboxing- Apps are second-rate citizens - don’t get access to

all the hardware

110

Dalvik – optimized for embedded

l Minimal-memory footprint while providing a secure sandboxing model- Uncompressed .dex files are smaller than compressed .jar files due to

more efficient bytecodel On average Dalvik byte code is 30% smaller than JVM byte

codel Multiple classes in one .dex filel Shared constant pool (assumes 32-bit indexes)l Simpler class-loadingl Because it is uncompressed, dex code can be memory-mapped and

shared (i.e. mmap()-ed)- Each app runs in a separate instance of Dalvik

l At startup, system launches zygote, a half-baked Dalvik process, which is forked any time a new VM is needed

l Due to copy-on-write support, large sections of the heap are shared (including 1800+ preloaded classes)

l Since each VM runs in a separate process, we get great security isolation

111

l Register-based fixed-width CPU-optimized byte-code interpreter

- Standard JVM bytecode executes 8-bit stack instructions - local variables must be copied to or from the operand stack by separate instructions

l Memory speed to CPU speed is amplified on mobile CPUs -we want to minimize access to the main memory

- Dalvik uses 16-bit instruction set that works directly on local variables (managed via a 4-bit virtual register field)

l With JIT support, as of Froyo (2-5x performance improvement in CPU-bound code)

- Trace-level granularity (more optimal than whole method-level compilations)

- Fast context-switching (interpreted mode to native mode and back)

- Well-balanced from performance vs. memory overhead perspective (~ 100-200KB overhead per app)

- ~ 1:8 ratio of Dalvik to native code (mostly due to optimizations, like inlining)

112

Dalvikl Includes support for instrumentation to allow tracing and profiling of running

code

l Core libraries based on Java SE 5 (mostly from Apache Harmony), with many differences

- No support for java.applet, java.awt, java.lang.managementand javax.management (JMX), java.rmi and javax.rmi, javax.accessibiliy, javax.activity, javax.imageio, javax.naming (JNDI), javax.print, javax.security.auth.kerberos, javax.security.auth.spi, javax.security.spi, javax.security.sasl, javax.sound, javax.swing, javax.transaction, javax.xml (except for javax.xml.parsers), org.ietf, org.omg, org.w3c.dom.* (subpackages)

- But support for Android APIs (including wrappers for OpenGL, SQLite, etc.), Apache HTTP Client (org.apache.http), JSON parser (org.json), XML SAX parser (org.xml.sax), XML Pull Parser (org.xmlpull), etc.

113

Android Application Framework Layer

114

Overview• The rich set of system services wrapped in intuitive Java APIs

• Most managed by the system_server process and accessible via Binder/AIDL

• Abstraction of hardware services• Location, telephony, WiFi, Bluetooth, sensors, camera,

etc.

• Java-language bindings for the native libraries (e.g. OpenGL, SQLite)

• Core platform services (like life-cycle management)• Essential to the apps, even if most are not used

directly

Writing ClientsSomeTypeManager proxy =

(SomeTypeManager)getSystemService(NAME);

proxy.fn();

//Looks like local call but use the binder for RPC

116

117

Activity Manager Service• Manages lifecycle of applications and their components

• Sets up oom_ajd setting read by Low Memory Killer ([Android_Linux_Kernel_Low_Memory_Killer]) Android extension to the Linux kernel

• Handles application requests to startActivity(), sendBroadcast(), startService(), bindService(), etc.

• Enforces security permissions on those requests

• Maintains user task state - i.e. the back-stack

118

Package Manager Service• Along with installd responsible for installation of .apk-s on

the Android system

• Maintains internal data structures representing installed packages as well as their individual components

• Used by Activity Manager when handling intents (i.e. intent resolution is handled here)

• Provides this info on demand to other services and apps

• Very central to the platform’s security

ActivityManager am = (ActivityManager)this.getSystemService(ACTIVITY_SERVICE);

List l = am.getRunningAppProcesses();

Iterator i = l.iterator();

PackageManager pm = this.getPackageManager();

while(i.hasNext()) {

ActivityManager.RunningAppProcessInfo info =

(ActivityManager.RunningAppProcessInfo)(i.next());

try {

CharSequence c = pm.getApplicationLabel(

pm.getApplicationInfo(info.processName, PackageManager.GET_META_DATA));

Log.w("LABEL", c.toString());

}catch(Exception e) {

//Name Not FOund Exception

}

}

119

120

Power Manager Service• Controls power management

• Provides access to wake locks

PowerManager mgr =

(PowerManager)getSystemService(Context.POWER_SERVICE);

WakeLock wakeLock =

mgr.newWakeLock(PowerManager.FULL_WAKE_LOCK, ”myappWL");

wakeLock.acquire();

121

Alarm Manager Service• Manages wake-up alarms for applications

• Supports inexact wakeup frequencies - helps consolidate wake-ups into fewer slots

• Uses power manager for wake locks

alarmManager = (AlarmManager) getSystemService(ALARM_SERVICE);

alarmManager.setRepeating(AlarmManager.RTC_WAKEUP,

calendar.getTimeInMillis(), 10000, pendingIntent);

122

Notification Manager Service• Used by apps and other services to notify the user of

events that may be of interest• This is how background events "bubble up" as

notifications

• Supports persistent notification, as well as notifications that use LEDs, screen backlight, sound, and/or vibration to notify the user

Examplelong when = System.currentTimeMillis();

NotificationManager notificationManager =

(NotificationManager) context.getSystemService

(Context.NOTIFICATION_SERVICE);

Notification notification = new Notification(icon, message, when);

….

notificationManager.notify(0, notification);

123

124

Location Manager Service• Handles geographic location updates (e.g. GPS) and

distributes them to the listening applications

• Support proximity alerts (via Intents)

• Supports providers of different granularity (GPS, Network, WiFi)

125

LocationManager locationManager = (LocationManager)

getSystemService(Context.LOCATION_SERVICE);

String provider = LocationManager.GPS_PROVIDER;

// Or use LocationManager.NETWORK_PROVIDER

// now get the “current” location

Location lastKnownLocation =

locationManager.getLastKnownLocation(provider);

if (lastKnownLocation != null)

{

// do whatever you want with the Location object

// you can get: latitude, longitude, altitude

// speed, bearing

// time, accuracy

...

}

126

LocationManager locationManager = (LocationManager) getSystemService(Context.LOCATION_SERVICE);

LocationListener locationListener = new LocationListener() {public void onLocationChanged(Location location) {

// do whatever with the Location object}

public void onStatusChanged(String provider, int status,Bundle extras) {}

public void onProviderEnabled(String provider) {}public void onProviderDisabled(String provider) {}

};

// Register listener with Location Manager to receive updateslocationManager.requestLocationUpdates(

LocationManager.GPS_PROVIDER, 0, // time interval0, // distance intervallocationListener);

127

Sensor Manager Service• Provides a uniform access to the device’s sensors

• Apps request sensor notifications via this manager• Sensor manager delivers sensor updates via a

generic (timestamped) array of values (which are sensor-dependent)

• Supported sensor types: accelerometer, linear acceleration, gravity, gyroscope, light, magnetic field, orientation, pressure, proximity, rotation vector, temperature

• Actual sensor support is (obviously) hardware-dependent

128

Search Manager Service• Provides a framework for

• Device-wide (global) • App-specific search

129

Connectivity Manager Service• Monitors network connections (Wi-Fi, GPRS, UMTS, etc.)

and• Send broadcast intents when network connectivity

changes• Attempts to "fail over" to another network when

connectivity to a network is lost

• Provides an API that allows applications to query the coarse-grained or fine-grained state of the available networks

130

ConnectivityManager manager = (ConnectivityManager)

getSystemService(Context.CONNECTIVITY_SERVICE);

NetworkInfo info = manager.getActiveNetworkInfo();

if (info.isConnected())

{

// see what type of network it is

int type = info.getType();

// check if it's wifi connectivity

if (type == ConnectivityManager.TYPE_WIFI) ...

// check if it's 3G connectivity

else if (type ==

ConnectivityManager.TYPE_MOBILE) ...

}

131

Wifi Manager Service• Unlike the connectivity manager, the Wifi Manager

supports Wifi-specific operations

• Provides a list and allows management of configured networks

• Provides access to and management of the state of the currently active Wi-Fi network connection, if any

• Enables access point scans

• Broadcasts Intents on Wifi-connectivity state change events

132

Telephony Manager Service• Provides access to information about the telephony

services on the device

• Apps query Telephony Manager to determine telephony services and states, as well as to access some types of subscriber information (e.g. device id)

• Apps can also register a listener to receive notification of telephony state changes

• Handles tethering requests

133

Input Method Manager Service• Central system to the overall input method framework

(IMF) architecture• Arbitrates interaction between applications (each

has a separate client) and the current input method

• Responsible for creating and running an input method (IME) to capture the actual input and translate it into text

• Allows multiple apps to requests input focus and control over the state of IME

EditText txtName = (EditText) findViewById(R.id.txtName);

InputMethodManager inputMethodManager = (InputMethodManager) getSystemService(Context.INPUT_METHOD_SERVICE);

// only will trigger it if no physical keyboard is open

inputMethodManager.showSoftInput(txtName, InputMethodManager.SHOW_IMPLICIT);

// or hide

imm.hideSoftInputFromWindow(myEditText.getWindowToken(), 0);

134

135

UI Mode Manager Service• Provides access to the system UI mode

• Enable/disable car-mode• Enable/disable night-mode

• System uses it to implement automatic UI mode changes

• Apps use it to manually control UI modes of the device

136

Download Manager Service• Handles long-running HTTP downloads

• Clients request that a URI be downloaded to a particular destination file

• The download manager handles the download in the background, taking care of HTTP interactions and retrying downloads after failures or across connectivity changes and system reboots

137

Storage Manager Service• Handles storage-related items such as Opaque Binary

Blobs (OBBs)

• "OBBs contain a filesystem that maybe be encrypted on disk and mounted on-demand from an application. OBBs are a good way of providing large amounts of binary assets without packaging them into APKs as they may be multiple gigabytes in size. However, due to their size, they’re most likely stored in a shared storage pool accessible from all programs. The system does not guarantee the security of the OBB file itself…"

• E.g. great for GPS/Mapping applications that need support for off-line maps

138

Audio Manager Service• Provides access to volume and ringer mode control

• Allows management/querying for the state of the audio system

• Set/Get the current mode: normal, ringtone, in-call, in-communications

• Set/Get the current ringer mode: normal, silent, vibrate• Set/Get the state of audio channels: speaker, bluetooth

headset, wired headset, speakerphone• Set/Get the volume

• Get audio focus requests

• Allows playing of sound effects: clicks, key-presses, navigation, etc.

SoundPool mSoundPool = new SoundPool(4, AudioManager.STREAM_MUSIC, 0);

mSoundPoolMap = new HashMap<Integer, Integer>();

AudioManager mAudioManager = (AudioManager)mContext.

getSystemService(Context.AUDIO_SERVICE);

mSoundPoolMap.put(1, mSoundPool.load(mContext, R.raw.t1 , 1));

…..

public static void playSound(int index,float speed)

{

float streamVolume =

mAudioManager.getStreamVolume(AudioManager.STREAM_MUSIC);

streamVolume = streamVolume /

mAudioManager.getStreamMaxVolume(AudioManager.STREAM_MUSIC);

mSoundPool.play(mSoundPoolMap.get(index), streamVolume, streamVolume, 1, 0, speed);

}

139

140

Additional Services• Window Manager

• Manages windows (z-order) composed in a surface• Allows us to place custom views (windows)

• Layout Inflater Manager• Used to instantiate layout XML files into its

corresponding View object trees

• Resource Manager• Provides access to non-code resources such as

localized strings, graphics, and layout files

141

Additional Services• Lights Service: handles status lights on the device

• Throttle Service: answers queries about data transfer amounts and throttling

• Mount Service: mount/unmount removable storage

• Battery Service: reports on battery health/status

• Wallpaper Service: manages changes to the wallpaper

• Backup Agent: provides remote backup/restore capabilities

• Bluetooth Service: manages bluetooth pairings

• Headset, Dock, USB Observers: observe specific device connections

142

Android Native Development Kit

143

What it can and cannot dol Useful for:

- Porting existing body of code to Android

- Developing optimized native apps, especially for gaming

l Provides:- Tools and build files to generate native code libraries from C/C++

- Way to embed native libs into .apk

- Set of stable (forward-compatible) native libs

- Documentation, samples and tutorials

l Enables:- Calling native code from Java using JNI

- Implementing fully native apps (since 2.3)

l Doesn't allow you to:- Compile traditional Linux/Unix apps as-is

144

Getting and installing the NDKl What’s in the NDK?

- Development tools

- Stable native APIs system headers

- Documentation - IMPORTANT

- Samples

l Getting the NDK

- http://developer.android.com/sdk/ndk/index.html

l Prerequisites

- Windows, Mac or Linux

- Complete SDK

- make (GNU’s) and awk

- For Windows, Cygwin 1.7 or higher

l NDK set up:

- Make sure prerequisites are installed

- Download and install NDK

145

Implementing fully native appsl Android 2.3 and up

l Native lifecycle management

l Still runs within context of dedicated Dalvik VM

l Can use JNI to call on Java functions

l Limited API:- Activity lifecycle management- Input events and sensors- Window management- Direct access to assets

l Make sure your activity is called: “android.app.NativeActivity”

146

NDK in Action

147

NDK� Android’s Dalvik VM allows our applications written in Java to

call methods implemented in native code through the Java Native Interface (JNI)

� For example:

148

NDKl NDK is a tool-chain to build and cross-compile our native code for the device-

specific architecture- At the moment, NDK supports ARMv5TE, ARMv7-A, x86 (since

r6), and mips (since r8) ABIs- For example, we would implement native fetch method in C and

compile it into a library libfetchurl.so

l NDK offers a way to package our native code library (as lib<something>.so) into the APK file so we can distribute it with our application easily

- For example, our library would be packaged as libs/armeabi/libfetchurl.so in the APK

l NDK provides a set of native system headers that will be supported for the future releases of Android platform (libc, libm, libz, liblog, libjnigrahics, OpenGL/OpenSL ES, JNI headers, minimal C++ support headers, and Android native app APIs)

l NDK comes with extensive documentation, sample code, and examples

149

JNIl An interface that allows Java to interact with code written in another language

l Motivation for JNI

- Code reusability

l Reuse existing/legacy code with Java (mostly C/C++)

- Performance

l Native code used to be up to 20 times faster than Java, when running in interpreted mode

l Modern JIT compilers (HotSpot, Dalvik) make this a moot point

- Allow Java to tap into low level O/S, H/W routines

l JNI code is not portable!

l JNI can also be used to invoke Java code from within natively-written applications -such as those written in C/C++.

In fact, the java command-line utility is an example of one such application, that launches Java code in a Java Virtual Machine.

150

JNI Componentsl javah -JDK tool that builds C-style header files from a

given Java class that includes native methods- Adapts Java method signatures to native

function prototypes

l jni.h - C/C+ header file included with the JDK that maps Java types to their native counterparts- javah automatically includes this file in the

application header files

151

JNI Example

� Compile the code and generate the header

� javah -jni -classpath bin/classes -d jni com.marakana.jniexamples.Hello

152

Generate Header

153

C Implementation

154

Native Methods Argumentsl A C/C++ implementation of a native Java method accepts the following

function arguments:- JNIEnv *env, an interface pointer (pointer to a pointer) to a function

table, where each entry in the table is a pointer to a function that enables Java-to-C/C++ integration (e.g. type conversion)

- The second argument varies depending on whether the native method is a static method or an instance (i.e. non-static) method:

l In case of instance methods, the second argument is a jobject obj, which is a reference to the object on which the method is invoked

l In case of static methods, the second is a jclass clazz, which is a reference to the class in which the method is defined

- The remaining arguments correspond to regular Java method arguments (subject to the mapping described below)

l The native function can pass its result back to Java via the return value (or return void)

155

Primitive Type Mappingl Java primitives have well-known size and are signed by

default

l C/C++ primitives vary in size, depending on the platform (e.g. the size of the long depends on the size of the word)

l Because booleans are treated as unsigned bytes, the following definitions are also useful

158

Global and Local Referencesl Unlike primitives, arbitrary Java objects are always passed by reference

l The VM must keep track of all objects that have been passed to the native code, so that these objects are not freed by the GC (i.e. these references are considered object roots while in use)

- Consequently, the native code must have a way to tell the VM that it no longer needs references to objects

- Additionally, the GC must be able to relocate an object referred to by native code (in the case of a copying GC), which means that these references must not be dereferenced by the native code

- We will access the fields/methods on these references via JNI accessor functions

159

l All objects passed to native code and returned from JNI functions are considered local references

- Automatically "freed" after the native function call returns

l The VM creates a table of references before every function call, and frees it when the call completes

l Consequently, the native function calls are more expensive than regular method calls

- Also possible to explicitly free using: void DeleteLocalRef(JNIEnv *env, jobject localRef);

- In practice, we only really want to use DeleteLocalRef(JNIEnv *, jobject) if we know we are not going to need the reference in the rest of the function body, say before we start executing code that may take a long time. This allows the referenced memory to be freed by GC, assuming nobody else is using it.

- Only valid in the thread in which they are created

- If we end up using more than 16 local references (default) within a native function scope, we can ask the VM for more:

l jint EnsureLocalCapacity(JNIEnv *env, jintcapacity);

l jint PushLocalFrame(JNIEnv *env, jint capacity);

l jobject PopLocalFrame(JNIEnv *env, jobject result);

160

l To hold on to an object beyond a single native function call, we can convert a local reference to a global reference jobject NewGlobalRef(JNIEnv*env, jobject obj);

l Now we could save the result of NewGlobalRef(JNIEnv *, jobect) to a global/static variable and use it later, in another funciton call.

l Becomes our responsibility to explicitly delete the reference when no longer in use: void DeleteGlobalRef(JNIEnv *env, jobject globalRef);

l Also supported are the weak global references

- jweak NewWeakGlobalRef(JNIEnv *env, jobject obj);

- void DeleteWeakGlobalRef(JNIEnv *env, jweak obj);

l JNI accessor functions do not make a distinction between the different reference types

161

Using Stringsl In C, a string is a pointer to a \0-terminated character

array where each character is one byte

l In Java, an instance of java.lang.String is an immutable object which wraps a character array, which itself is an object (i.e. it knows its length so it is not \0-terminated), and each character is represented in UTF16 as two bytes

l String Operations are provided by JNIEnv:

162

Wrong code!!!

This example would not work (would likely crash the VM) since the jstring type represents strings in the Java virtual machine. This is different from the C string type (char *).

163

Correct Code

1. This returns a pointer to an array of bytes representing the string in modified UTF-8 encoding; or NULL if we ran out of memory, in which case java.lang.OutOfMemoryError would also be thrown (upon returning back to the Java runtime).

2. Calling ReleaseStringUTFChars indicates that the native method no longer needs the UTF-8 string returned by GetStringUTFChars, thus the memory taken by the UTF-8 string can be freed. Failure to do this would result in a memory leak, which couldultimately lead to memory exhaustion.

164

Convert C string to java#include <stdio.h>

#include "com_marakana_jniexamples_GetName.h"

JNIEXPORT jstring JNICALLJava_com_marakana_jniexamples_ReturnName_GetName(JNIEnv *env,jclass class) {

char buf[20];

fgets(buf, sizeof(buf), stdin);

return (*env)->NewStringUTF(env, buf);

}

165

Print Each Character#include <stdio.h>

#include "com_marakana_jniexamples_HelloName.h"

JNIEXPORT void JNICALL Java_com_marakana_jniexamples_HelloName_sayHelloName(JNIEnv*env, jclass clazz, jstring name) {

char buf[4];

jint i;

jsize len = (*env)->GetStringUTFLength(env, name);

fputs("Hello ", stdout);

for (i = 0; i < len; i++) {

(*env)->GetStringUTFRegion(env, name, i, 1, buf);

putc(buf[0], stdout); /* assumes ASCII */

}

putc('\n', stdout);

}

166

Arraysl As with Strings, Java arrays are different than C arrays;

the former are objects (which know their length) whereas

the latter are just pointers to memory addresses- So, int[] in Java is not the same as jint[] in C/C++- Instead, int[] in Java, becomes jintArray in C/C++

l JNI provides functions for Creating arrays:

169

Example

170

Copy C Array to java

171

Direct Memory Buffersl To avoid any chance of memory copying, we could also use

direct memory buffers (java.nio.ByteBuffer):

jobject NewDirectByteBuffer(JNIEnv* env, void* address, jlongcapacity);

void* GetDirectBufferAddress(JNIEnv* env, jobject buf);

jlong GetDirectBufferCapacity(JNIEnv* env, jobject buf);

173

Reflectionl JNI supports getting a reference to a Java class in native code:

jclass FindClass(JNIEnv *env, const char *name);

l The name argument is a fully-qualified classname (with / as the package separators)

/* load the java.lang.String class */

(*env)->FindClass(env, "java/lang/String");

174

l For arrays, we use [Lclassname; format

/* load the java.lang.String[] class */(*env)->FindClass(env, "[Ljava/lang/String;");

l Given a reference to a class, we can find out its super class, as well as whether it is assignable from another class:

jclass GetSuperclass(JNIEnv *env, jclass clazz);

jboolean IsAssignableFrom(JNIEnv *env, jclass clazz1, jclassclazz2);

l Given a reference to an object, we can find out its class, as well as whether it is an instance of another class:

jclass GetObjectClass(JNIEnv *env, jobject obj);

jboolean IsInstanceOf(JNIEnv *env, jobject obj, jclass clazz);

175

Once we have a class, JNI offers functions to lookup, get/set fields and invoke methods of that class - based on the following field and method signatures:

178

Registering Native Methods on load

JNI supports JNI_OnLoad function, which if exported by the library, will automatically be called when the library is loaded by System.loadLibrary(String)

jint JNI_OnLoad(JavaVM *vm, void *reserved);

When used with JNI’s RegisterNatives, we can pre-bind native methods as soon as our library is first loaded, and consequently, we no longer need to use javah:

Notice that we JNI function that will be bound to the native Java method is declared as static. It won’t even be exported to the symbol table. It does not need to be because RegisterNatives will bind it via a function pointer to the Java method.

JNI supports JNI_OnUnload, but since this is only invoked when the class-loader that loaded the library is GC’ed, it’s not commonly used

179

Exceptionsl JNI permits us to throw exceptions from native code

l Assuming we have a Throwable object:

jint Throw(JNIEnv *env, jthrowable obj);

l Assuming we have a Throwable class, and that it has a constructor that takes a string message:

jint ThrowNew(JNIEnv *env, jclass clazz, const char *message);

180

To get to the message of a throwable object we would do something like

181

Using NDK with C

182

Using NDK with C++

183

Using the NDK1.Place native code under <project>/jni/...

2.Create <project>/jni/Android.mk to describe native code to NDK

3.Optional: create <project>/jni/Application.mk for describing which natives sources are required by app

4.Build native code:• cd <project>• <ndk>/ndk-build

5.Compile app with SDK. Native code will be shared lib in .apk file.

184

Debugging with Eclipsel As of ADT r20, NDK also adds a plugin to ADT that enables debugging NDK projects

in Eclipse and requires:

- CDT (v7.0.2 or 8.0.2) for Eclipse from http://download.eclipse.org/tools/cdt/releases/indigo

- ADT (r20) with SDK/NDK plugins for Eclipse from https://dl-ssl.google.com/android/eclipse/ with SDK/NDK paths configured

- Before an Eclipse project app can be debugged, it must have:

l native support (project properties → Android Tools → Add native support) enabled

l NDK_DEBUG=1 (project properties → C/C++ Build → Build Command → ndk-build NDK_DEBUG=1) set

- To debug an Eclipse project app:

l Set the break point(s) as desired

l Select project → Run → Debug As → Android Native Application

185

NDK Stable API� The header files for NDK stable APIs are available at

� /path/to/ndk/platforms/<android-platform>/<arch-name>/usr/include.

186

Android Specific Log Supportl Include <android/log.h> to access various

functionality that can be used to send log messages to the kernel (i.e. logcat buffers) from our native code

l Requires that our code be linked to /system/lib/liblog.so with

LOCAL_LDLIBS += -llog

in our Android.mk file

187

Zlib Compression Libraryl Include <zlib.h> and <zconf.h> to access ZLib

compression library- See http://www.zlib.net/manual.html

for more info on ZLib

l Requires that our code be linked to /system/lib/libz.so with LOCAL_LDLIBS += -lzin our Android.mk file

188

Open GL ES 1.0l Include <GLES/gl.h> and <GLES/glext.h> to access OpenGL

ES 1.x rendering calls from native code- The "1.x" here refers to both versions 1.0 and 1.1

l Using 1.1 requires OpenGL-capable GPUl Using 1.0 is universally supported since Android

includes software renderer for GPU-less devicesl Requires that we include <uses-feature> tag in

our manifest file to indicate the actual OpenGL version that we expect

l Requires that our code be linked to /system/lib/libGLESv1_CM.so with LOCAL_LDLIBS += -lGLESv1_CM.so in our Android.mk file

l Since API 4 (Android 1.6)

189

Open GL ES 2.0l Include <GLES2/gl2.h> and <GLES2/gl2ext.h> to access

OpenGL ES 2.0 rendering calls from native code

l Enables the use of vertex and fragment shaders via the GLSL language

l Since not all devices support OpenGL 2.0, we should include <uses-feature> tag in our manifest file to indicate this requirement

l Requires that our code be linked to /system/lib/libGLESv2.so with LOCAL_LDLIBS += -lGLESv2.so in our Android.mk file


190

The jnigraphics Libraryl Include <android/bitmap.h> to reliably access the

pixel buffers of Java bitmap objects from native code

l Requires that our code be linked to /system/lib/libjnigraphics.so with LOCAL_LDLIBS += -ljnigraphics in our Android.mk file


191

Open SL-ES native audio library

l Include <SLES/OpenSLES.h> and <SLES/OpenSLES_Platform.h> to perform audio input and output from native code

l Based on Khronos Group OpenSL ES™ 1.0.1

l Requires that our code be linked to /system/lib/libOpenSLES.so with LOCAL_LDLIBS += -lOpenSLES in our Android.mkfile


192

Android Binder IPC With AIDL

193

Native Access

194

Client - Server

195

Using Binder

196

Why IPC?• Each Android application runs in a separate process

• Android application-framework services also run in a separate process called system_server

• Often we wish to make use of services offered by other applications, or we simply wish to expose our applications' capabilities to 3rd party apps

• Even Android’s Intent-based IPC-like mechanism (used for starting activities and services, as well as delivering events), is internally based on Binder

• By design (for security reasons), processes cannot directly access each other’s data

• To cross the process boundaries, we need support of a inter-process communication transport mechanism, which handles passing of data from one process (caller) to another (callee)

• Caller’s data is marshaled into tokens that IPC understands, copied to callee’sprocess, and finally unmarshaled into what callee expects

• Callee’s response is also marshaled, copied to caller’s process where it is unmarshaled into what caller expects

• Marshaling/unmarshaling is automatically provided by the IPC mechanism

197

What is Binder

198

• Binder provides a lightweight remote procedure call (RPC) mechanism designed for high performance when performing in-process and cross-process calls (IPC)

• Binder-capable services are described in Android Interface Definition Language (AIDL), not unlike other IDL languages

• Since Binder is provided as a Linux driver, the services can be written in both C/C++ as well as Java

• Most Android services are written in Java• All caller calls go through Binder’s transact() method, which

automatically marshals the arguments and return values via Parcelobjects

What is Binder

199

• Parcel is a generic buffer of data (decomposed into primitives) that also maintains some meta-data about its contents - such as object references to ensure object identity across processes

• Caller calls to transact() are by default synchronous - i.e. provide the same semantics as a local method call

• On callee side, the Binder framework maintains a pool of transaction threads, which are used to handle the incoming IPC requests (unless the call is local, in which case the same thread is used)

• Callee methods can be marked as oneway, in which case caller calls do not block (i.e. calls return immediately)

• Callee' mutable state needs to be thread-safe - since callee’s can accept concurrent requests from multiple callers

• The Binder system also supports recursion across processes - i.e. behaves the same as recursion semantics when calling methods on local objects

What is Binder

200

AIDL• Android Interface Definition Language is a Android-specific language

for defining Binder-based service interfaces

• AIDL follows Java-like interface syntax and allows us to declare our "business" methods

• Each Binder-based service is defined in a separate .aidl file, typically named IFooService.aidl, and saved in the src/ directory

201

l The aidl build tool (part of Android SDK) is used to extract a real Java interface (along with a Stub providing Android’s android.os.IBinder) from each .aidl file and place it into our gen/ directory

l Eclipse ADT automatically calls aidl for each .aidl file that it finds in our src/ directory

202

AIDL Typesl null

l boolean, boolean[], byte, byte[], char[], int, int[], long, long[], float, float[], double, double[]

l java.lang.CharSequence, java.lang.String

l java.io.FileDescriptor- Gets transferred as a dup of the original file

descriptor - while the fd is different, it points to the same underlying stream and position

l java.io.Serializable (not efficient)

l java.util.List (of supported types, including generic definitions)- Internally, Binder always uses java.util.ArrayList

as the concrete implementation

203

AIDL Typesl java.util.Map (of supported types)

l Internally, Binder always uses java.util.HashMap as the concrete implementation

l java.lang.Object[] (supporting objects of the same type defined here, but also primitive wrappers)

l android.util.SparseArray, android.util.SparseBooleanArray

l android.os.Bundle

l android.os.IBinder, android.os.IInterface- The contents of these objects are not actually transferred -

instead a special token serving as a self-reference is written- Reading these objects from a parcel returns a handle to the

original object that was written

205

AIDLl AIDL-defined methods can take zero or more parameters, and must

return a value or void- All non-primitive parameters require a directional tag

indicating which way the data goes: one of: in, out, or inout

l Direction for primitives is always in (can be omitted)l The direction tag tells binder when to marshal

the data, so its use has direct consequences on performance

l All .aidl comments are copied over to the generated Java interface (except for comments before the import and package statements).

l Static fields are not supported in .aidl files

206

Limitations• Binder supports a maximum of 15 binder threads per process

• Binder limits its transactional buffer to 1Mb per process across all concurrent transactions (i.e. Parcels)

• If arguments/return values are too large to fit into this buffer, TransactionTooLargeException is thrown

• Because this buffer is shared across all transactions in a given process, many moderately sized transactions could also exhaust its limit

• When this exception is thrown, we don’t know whether we failed to send the request, or failed to receive the response

• Keep transaction data small or use Ashmem

207

Async IPC with Binderl Binder allows for the asynchronous communication

between the client and its service via the onewaydeclaration on the AIDL interface

l Of course, we still care about the result, so generally asynccalls are used with call-backs - typically through listeners

l When clients provide a reference to themselves as call-back listeners, then the roles reverse at the time the listeners are called: clients' listeners become the services, and services become the clients to those listeners

Native Services

208

Writing Native Services• Use /dev/binder character device file

• Very complex ioctl’s and data structures

• Not recommended (kernel driver)

• Use libbinder• Hide most of the complexity• C++ implementation

• /frameworks/native/lib/binder

209

Simple Exampleclass SampNativeService : public BBinder

{

private:

enum {OP_ADD, OP_SUB};

int add(int a,int b){return a+b;}

int sub(int a,int b){return a-b;}

public:

static int Instance();

virtual status_t onTransact(uint32_t, const Parcel&, Parcel*, uint32_t);

};

210

int SampNativeService::Instance(){

int ret = defaultServiceManager()->addService(String16("samp.native.service"),

new SampNativeService());return ret;

}

status_t SampNativeService::onTransact(uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags){

switch(code){case OP_ADD:

{pid_t pid = data.readInt32();int num1 = data.readInt32();int num2 = data.readInt32();int res = add(num1,num2);reply->writeInt32(res);return NO_ERROR;

} .....

211

Android.mkLOCAL_PATH:=$(call my-dir)

include $(CLEAR_VARS)

LOCAL_SRC_FILES:=SampNativeService.cpp

LOCAL_SHARED_LIBRARIES:=libutils libbinder liblog

LOCAL_MODULE_TAGS:=optional

LOCAL_MODULE:=libNativeService

LOCAL_PRELINK_MODULE:=false

include $(BUILD_SHARED_LIBRARY)

212

Server Applicationint main(int arg, char** argv)

{

printf("server begin\n");

sp<ProcessState> proc(ProcessState::self());

sp<IServiceManager> sm = defaultServiceManager();

printf("server - ServiceManager: %p\n", sm.get());

int ret = SampNativeService::Instance();

printf("server - SampNativeService::Instance return %d\n", ret);

ProcessState::self()->startThreadPool();

IPCThreadState::self()->joinThreadPool();

}

213

Client Libraryclass SampClient

{

public:

enum {OP_ADD, OP_SUB};

int add(int a,int b);

int sub(int a,int b);

private:

static sp<IBinder> binder;

static void getService();

};

214

void SampClient::getService(){

sp<IServiceManager> sm = defaultServiceManager();binder = sm->getService(String16("samp.native.service"));

}int SampClient::add(int a,int b){

getService();Parcel data, reply;data.writeInt32(getpid());data.writeInt32(a);data.writeInt32(b);

binder->transact(OP_ADD, data, &reply);int r = reply.readInt32();return r;

}

215

Test Applicationint main(int argc, char** argv)

{

SampClient client;

int ret = client.add(20,12);

printf("add return: %d\n", ret);

return 0;

}

216

class IHelloWorldService : public IInterface {

public:

DECLARE_META_INTERFACE(HelloWorldService);

virtual void print(const char * message) = 0;

virtual void sayHi(const char * message, char * response) = 0;

};

218

class BnHelloWorldService : public BnInterface<IHelloWorldService> {

public:

static void instantiate();

virtual status_t onTransact(uint32_t code,

const Parcel &data,

Parcel *reply,

uint32_t flags);

void print(const char * message);

void sayHi(const char * message, char * response);

};

219

void BnHelloWorldService::instantiate() {

android::defaultServiceManager()->addService(

IHelloWorldService::descriptor, new BnHelloWorldService());

}

status_t BnHelloWorldService::onTransact(uint32_t code, const Parcel &data, Parcel *reply,uint32_t flags) {

switch(code) {

case PRINT: {

CHECK_INTERFACE(IHelloWorldService, data, reply);

const char * message = data.readCString();

print(message);

return android::NO_ERROR;

}

case SAY_HI: { ….. }

220

class BpHelloWorldService : public BpInterface<IHelloWorldService> {

public:

BpHelloWorldService(const sp<IBinder>& impl) : BpInterface<IHelloWorldService>(impl) {

}

void print(const char * message) {

Parcel data, reply;

data.writeInterfaceToken(IHelloWorldService::getInterfaceDescriptor());

data.writeCString(message);

remote()->transact(PRINT, data, &reply, IBinder::FLAG_ONEWAY);

}

void sayHi(const char * message, char * response) {

Parcel data, reply;

data.writeInterfaceToken(IHelloWorldService::getInterfaceDescriptor());

data.writeCString(message);

remote()->transact(SAY_HI, data, &reply, 0);

strcpy(response, reply.readCString());

}

};

IMPLEMENT_META_INTERFACE(HelloWorldService, SERVICE_NAME);

221

Server Applicationint main(int argc, char *argv[]) {

BnHelloWorldService::instantiate();

android::ProcessState::self()->startThreadPool();

android::IPCThreadState::self()->joinThreadPool();

return 0 ;

}

222

Client Applicationint main(int argc, char *argv[]) {

android::sp<android::IServiceManager> sm = android::defaultServiceManager();

android::sp<android::IBinder> binder= sm->getService(android::String16(SERVICE_NAME));

if (binder == 0) {

return -1;

}

android::sp<IHelloWorldService> shw = android::interface_cast<IHelloWorldService>(binder);

shw->print("Hello, world.");

char response[128];

shw->sayHi("Hi, this is hello-world client", response);

printf("%s\n", response);

return 0;

}

223

System Services

224

Creating a System service� Creating a shared library for the common interface

� Declare aidl

� Create a manager

� Create a system application to host the service� Add service implementation� Declare the service to run as system

� Declare the service app as persistent

� Use the library to create a clients

225

The Common Library- framework

- Android.mk - include $(call all-subdir-makefiles)

- sampservice

- Android.mk

- com.android.sampservice.xml

- com

- android

- sampservice- ISampService.aidl- SampManager.java

226

ISampService.aidlpackage com.android.sampservice;

interface ISampService {

int add(int a,int b);

int sub(int a,int b);

}

227

SampManagerpublic class SampManager {

private static final String REMOTE_SERVICE_NAME = ISampService.class.getName();

private final ISampService service;

public static SampManager getInstance() {

return new SampManager();

}

private SampManager() {

this.service = ISampService.Stub.asInterface(

ServiceManager.getService(REMOTE_SERVICE_NAME));

if (this.service == null) {

throw new IllegalStateException("Failed to find ISampService " + REMOTE_SERVICE_NAME);

}

228

public int add(int a,int b)

{

try

{

return service.add(a,b);

}catch(Exception ec){ … }

return 0;

}

public int sub(int a,int b)

{

try

{

return service.sub(a,b);

}catch(Exception ec){ … }

return 0;

}

229

com.android.sampservice.xml

<?xml version="1.0" encoding="utf-8"?>

<permissions>

<library name="com.android.sampservice"

file="/system/framework/com.android.sampservice.jar"/>

</permissions>

230

Android.mkLOCAL_PATH := $(call my-dir)

# Build the libraryinclude $(CLEAR_VARS)LOCAL_MODULE_TAGS := optionalLOCAL_MODULE := com.android.sampserviceLOCAL_SRC_FILES := $(call all-java-files-under,.)LOCAL_SRC_FILES += com/android/sampservice/ISampService.aidlinclude $(BUILD_JAVA_LIBRARY)

# Copy com.android.sampservice.xml to /system/etc/permissions/include $(CLEAR_VARS)LOCAL_MODULE_TAGS := optionalLOCAL_MODULE := com.android.sampservice.xmlLOCAL_MODULE_CLASS := ETCLOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/permissionsLOCAL_SRC_FILES := $(LOCAL_MODULE)include $(BUILD_PREBUILT)

231

The Service Application- app- Android.mk - include $(call all-subdir-makefiles)- SampServiceApp- Android.mk- AndroidManifest.xml- res- values- strings.xml

- src- com- android- sampappservice- ISampServiceImpl.java- SampServiceApp.java

232

package com.android.sampappservice;

import android.os.Binder;import android.os.IBinder;import android.os.RemoteException;import com.android.sampservice.*;

class ISampServiceImpl extends ISampService.Stub {public int add(int a,int b) {return a+b;

}public int sub(int a,int b) {return a-b;

}}

233

package com.android.sampappservice;

import android.app.Application;

import android.os.ServiceManager;

import com.android.sampservice.ISampService;

public class SampServiceApp extends Application {

private static final String REMOTE_SERVICE_NAME = ISampService.class.getName();

private ISampServiceImpl serviceImpl;

public void onCreate() {

super.onCreate();

this.serviceImpl = new ISampServiceImpl();

ServiceManager.addService(REMOTE_SERVICE_NAME, this.serviceImpl);

}

public void onTerminate() {

super.onTerminate();

}

}

234

AndroidManifest.xml<?xml version="1.0" encoding="utf-8"?>

<manifest xmlns:android="http://schemas.android.com/apk/res/android"

package="com.android.sampappservice"

android:sharedUserId="android.uid.system">

<application android:name=".SampServiceApp" android:persistent="true">

<uses-library android:name="com.android.sampservice" />

</application>

</manifest>

235

Android.mkLOCAL_PATH:= $(call my-dir)include $(CLEAR_VARS)LOCAL_MODULE_TAGS := optionalLOCAL_SRC_FILES := $(call all-java-files-under,src)LOCAL_REQUIRED_MODULES := com.android.sampserviceLOCAL_JAVA_LIBRARIES := com.android.sampservice \

core \framework

LOCAL_PACKAGE_NAME := SampServiceAppLOCAL_SDK_VERSION := currentLOCAL_PROGUARD_ENABLED := disabledLOCAL_CERTIFICATE := platforminclude $(BUILD_PACKAGE)

236

User Application� First, we need to update core.mk, build the rom and

check that the service is running

� The library for the client (classes-full-debug.jar) is generated in:

out/target/common/obj/JAVA_LIBRARIES/

com.android.sampservice_intermediates/

� Project->properties->java build path->libraries-> add library-> user library -> user libraries -> new -> set name -> add jar

237

User ApplicationSampManager man=SampManager.getInstance();

int res = man.add(10, 20);

Add to the application tag (AndroidManifest.xml)

<uses-library android:name="com.android.sampservice" android:required="true" />

238

239

Android Security

240

Foundations of Android Securityl Application Isolation and Permission-Control

- Can we control what applica]ons are able to do?- Can a misbehaving app affect the rest of the

system?

l Application "Provenance"- Can we trust the author of an app?- Can we trust our apps to be tamper-resistant?

l Data Encryption- Is our data safe if our device is

hacked/lost/stolen?

l Device Access Control- Can we protect our device against unauthorized

use?

241

Security Architecture

242

Android Application Isolationl By default, each app runs in a separate process with a

distinct user/group ID (fixed for the lifetime of the app)- Possible for multiple apps to share UID and process- Based on decades-old, well-understood UNIX

security model (processes and file-system permissions)

l Application-framework services also run in a separate

process (system_server)

l Linux kernel is the sole mechanism of app sandboxing

l Dalvik VM is not a security boundary- Coding in Java or C/C++ code – no difference - Enables use of JNI (unlike JavaME!)

l Same rules apply to system apps

243

Default Android Permissions Policyl No app can do anything to adversely affect – Other apps

- The system itself

- The user of the device

l So, by default, apps cannot:- Read/write files outside their own directory

- Install/uninstall/modify other apps

- Use other apps' private components

- Access network

- Access user's data (contacts, SMS, email)

- Use cost-sensitive APIs (make phone calls, send SMS, NFC)

- Keep device awake, automatically start on boot, etc.

244

Escaping The Sandbox

l Actually, apps can talk to other apps via- Intents- IPC (a.k.a. Binder) - ContentProviders

l Otherwise, to escape our sandbox, we need to use permissions

- Some permissions are only available to system apps

245

Built-in Android Permissions� ACCESS_FINE_LOCATION, ACCESS_NETWORK_STATE,

� ACCESS_WIFI_STATE, ACCOUNT_MANAGER,

� BLUETOOTH, BRICK, CALL_PHONE, CAMERA,

� CHANGE_WIFI_STATE, DELETE_PACKAGES,

� INSTALL_PACKAGES, INTERNET, MANAGE_ACCOUNTS,

� MASTER_CLEAR, READ_CONTACTS, READ_LOGS,

� READ_SMS, RECEIVE_SMS, RECORD_AUDIO,

� SEND_SMS, VIBRATE, WAKE_LOCK, WRITE_CONTACTS,

� WRITE_SETTINGS, WRITE_SMS, ... http://developer.android.com/reference/android/Manifest.permission.html

246

Example

� For example, an app that vibrates your phone any time you get in close vicinity to a friend would need to use at least the following permissions:

� • App's AndroidManifest.xml:

247

Logical Permission Enforcement

248

Permission Enforcement Example

249

Kernel Permission Enforcement

250

Application Provenancel Can we trust the developer of an applica]on we are

about to install? (mostly, no)

l Can we trust that our apps are resistant to tampering once installed? (mostly, yes)

l To get onto Android Market, a developer just needs to register with Google and pay $25 with a valid credit card

- A mild deterrent against authors of malicious apps

l Apps can also be side-loaded

251

Application Provenance (Signing)� All apps (.apk files) must be digitally signed prior to

installation on a device (and uploading to Android Market)

l The embedded certificate can be self-signed (no CA needed!) and valid for 25+ years

l App signing on Android is used to:- Ensure the authenticity of the author on updates- Establish trust relationship among apps signed with

the same key (share permissions, UID, process)- Make app contents tamper-resistant (moot point)

l An app can be signed with multiple keys

252

Application Provenance (Signing)

l Lost/expired key? No way to update the app(s)

l Stolen key? No way to revoke

l How do we trust the author on the first install?- Is this the real author, or an imposter? Can I

check the cert?- Has this app been vetted?

- Go by the number of installs? l Follow the sheep?

253

Application Provenance (Signing)l The result?

- Android.Rootcager - Android.Pjapps

- Android.Bgserv

l All took advantage of weak trust relationship - Take an existing (popular) app- Inject malicious code (e.g. a trojan)

- Re-package and re-sign with a new key/cert- Upload to market (or distribute via web)- Wait for the "sheep" to come (not really our

fault)

254

Safeguarding Apps' Datal Apps' files are private by default

- Owned by distinct apps' UIDs

l Exceptions- Apps can create files that are

l MODE_WORLD_READABLE l MODE_WORLD_WRITABLE

- Other apps (signed with the same key) can run with the same UID – thereby gexng access to shared files

- /mnt/sdcard is world-readable and world-writable (with WRITE_TO_EXTERNAL_STORAGE)

255

Data Encryption

256

Data Encryptionl Privacy and integrity of data can be achieved using encryption

l Data being transported off device is usually encrypted via TLS/SSL, which Android supports

- At the native level - via OpenSSL(/system/lib/libssl.so)

- At the Java level - using Java Cryptography Extension (JCE), which on Android is implicitly provided by BouncyCastle provider

- For example, for HTTPS with client-side authentication, we could use HttpsURLConnection:

257

To Encrypt/Decrypt byte array

Discretionary Access Control (DAC) � Typical form of access control in Linux.

� Access to data is entirely at the discretion of the owner/creator of the data.

� Some processes (e.g. uid 0) can override and some objects (e.g. sockets) are unchecked.

� Based on user & group identity.

� Limited granularity, coarse-grained privilege.

258

Android & DAC � Restrict use of system facilities by apps.

� e.g. bluetooth, network, storage access

� requires kernel modifications, “special” group IDs

� Isolate apps from each other. � unique user and group ID per installed app � assigned to app processes and files

� Hardcoded, scattered “policy”.

259

SELinux� Mandatory Access Control (MAC) for Linux.

� Enforces a system-wide security policy.

� Over all processes, objects, and operations. � Based on security labels.

� Can confine flawed and malicious applications. � Even ones that run as “root” / uid 0.

� Can prevent privilege escalation.

260

SELinux in Android� Confine privileged daemons.

� Protect from misuse.

� Limit the damage that can be done via them.

� Sandbox and isolate apps. � Strongly separate apps from one another. � Prevent privilege escalation by apps.

� Provide centralized, analyzable policy.

261

What can't SELinux protect against? � Kernel vulnerabilities, in general.

� Although it may block exploitation of specific vulnerabilities.

� Other kernel hardening measures (e.g. grsecurity) can be used in combination with SELinux.

� Anything allowed by the security policy. � Good policy is important. � Application architecture matters.

� Decomposition, least privilege

262

263

Android Startup

264

Android Startupl On power-up, CPU is uninitialized - wait for stable power

l Execute Boot ROM (hardwired into CPU)

- Locate the first-stage boot loader - Load the first-stage boot loader into internal RAM - Jump to first-stage boot loader’s memory location to

execute it

l First-stage boot loader runs

- Detect and initialize external RAM - Locate the second-stage boot loader - Load the second-stage boot loader into external

RAM - Jump to the second-stage boot loader’s memory

location to execute it

265

Android Startupl Second-stage boot loader runs

- Setup file systems (typically on Flash media)

- Optionally setup display, network, additional memory, and other devices

- Enable additional CPU features

- Enable low-level memory protection

- Optionally load security protections (e.g. ROM validation code)

- Locate Linux Kernel

- Load Linux Kernel into RAM

- Place Linux Kernel boot parameters into memory so that kernel knows what to run upon startup

- Jump to Linux Kernel memory address to run it

266

Kernel Initializationl Initialize and start the kernel

- Scheduler - Memory zones

- Buddy system allocator - IDT

- SoftIRQs - Date and Time - Slab allocator

- …

l Create process 1 (/init) and run it

267

Android Init

268

Init Processl The granddaddy of all other processes on the system (PID=1)

l Custom initialization script (with its own language)

- Different than more traditional /etc/inittab and SysV-init-levels initialization options

l When started by the kernel, init parses and executes commands from two files:- /init.rc - provides generic initialization instructions (see

system/core/rootdir/init.rc)- /init.<board-name>.rc - provides machine-specific initialization

instructions - sometimes overriding /init.rcl /init.goldfish.rc for the emulatorl /init.trout.rc for HTC’s ADP1l /init.herring.rc for Samsung’s Nexus S (see

device/samsung/crespo/init.herring.rc)

269

Init Processl The init program never exists (if it were, the kernel would panic), so it continues to

monitor started services

l The init's language consists of four broad classes of statements (see system/core/init/readme.txt)

- Actions - named sequences of commands queued to be executed on a unique trigger

on <trigger><command><command><command>

l Triggers - named events that trigger actions- early-init, init, early-fs, fs, post-fs, early-boot, boot - built-

in stages of init- <name>=<value> - fires when a system property is set to

<name>=<value>- device-added-<path> - fires when a device node at <path> is added- device-removed-<path> - fires when a device node at <path> is

removed- service-exited-<name> - fires when a service by <name> exists

270

Commands� chdir <directory> - change working directory

� chmod <octal-mode> <path> - change file access permissions� chown <owner> <group> <path> - change file user and group ownership

� chroot <directory> - change process root directory� class_start <serviceclass> - start all non-running services of the specified class

� class_stop <serviceclass> - stop all running services of the specified class

� domainname <name> - set the domain name.� exec <path> [ <argument> ]* - fork and execute <path> <argument> ...

blocks init until exec returns� export <name> <value> - set globally visible environment variable <name> =<value>

� hostname <name> - set the host name

� ifup <interface> - bring the network interface <interface> online� import <filename> - parse and process <filename> init configuration file

� insmod <path> - install the kernel module at <path>

271

Commandsl loglevel <level> - initialize the logger to <level>

l mkdir <path> [mode] [owner] [group] - create a directory at <path> and optionally change its default permissions (755) and user (root) / group (root) ownership

l mount <type> <device> <dir> [ <mountoption> ]* - attempt to mount named <device> at <dir> with the optional `<mountoption>`s

l setprop <name> <value> - set system property <name>=<value>

l setrlimit <resource> <cur> <max> - set the rlimit for a <resource>

l start <service> - start named <service> if it is not already running

l stop <service> - stop name <service> if it is currently running

l symlink <target> <path> - symbolically link <path> to <target>

l sysclktz <mins_west_of_gmt> - set the system clock base (0 for GMT)

l trigger <event> - trigger an event - i.e. call one action from another

l write <path> <string> [ <string> ]* - write arbitrary <string>`s to file by specified `<path>

272

Services� Services - persistent daemon programs to be launched

(optionally) restarted if they exit

� service <name> <pathname> [ <argument> ]*

� <option>

� <option>

� ...

273

OptionsModifiers to services, modifying how/when they are run re/launched

l critical - a device-critical service - devices goes into recovery if the service exits more than 4 times in 4 minutes

l disabled - not started by default - i.e. it has to be started explicitly by name

l setenv <name> <value> - set the environment variable <name>=<value> in the service

l socket <name> <dgram|stream|seqpacket> <perm> [ <user> [ <group> ] ] -create a unix domain socket named /dev/socket/<name> and pass its fd to the service

l user <username> - change service’s effective user ID to <username>

l group <groupname> [ <groupname> ]* - change service’s effective group ID to <groupname> (additional groups are supplemented via setgroups())

l oneshot - ignore service exist (default is to auto-restart)

l class <name> - set the class name of the service (defaults to default) so that all services of a particular class can be started/stopped together

l onrestart - execute a command on restart

274

Init.rcl Starts ueventd

l Initializes the system clock and logger

l Sets up global environment

l Sets up the file system (mount points and symbolic links)

l Configures kernel timeouts and scheduler

l Configures process groups

l Mounts the file systems

l Creates a basic directory structure on /data and applies permissions

l Applies permissions on /cache

l Applies permissions on certain /proc points

l Initializes local network (i.e. localhost)

l Configures the parameters for the low memory killer [Android_Linux_Kernel_Low_Memory_Killer]

l Applies permissions for system_server and daemons

l Defines TCP buffer sizes for various networks

l Configures and (optionally) loads various daemons (i.e. services): ueventd, console, adbd, servicemanager, vold, netd, debuggerd, rild, zygote (which in turn starts system_server), mediaserver, bootanimation (one time), and various Bluetooth daemons (like dbus-daemon, bluetoothd, etc.), installd, racoon, mtpd, keystore

Enabling SELinux in Android: init� init / ueventd

� load policy, set enforcing mode, set context

� label sockets, devices, runtime files

� init.rc� setcon, restorecon commands � seclabel option

275

276

Init.<device>.rcl Sets up product info

l Initializes device-driver-specific info, file system structures, and permissions: battery, wifi, phone, uart_switch, GPS, radio, bluetooth, NFC, lights

l Initializes and (re)mounts file systems

l Loads additional device-specific daemons

277

Zygote Startup� Zygote starts from /init.rc

� This translates to frameworks/base/cmds/app_process/app_main.cpp:main()

278

Zygote Startupl The command app_process then launches

frameworks/base/core/java/com/android/internal/os/ZygoteInit.java:main()in a Dalvik VM via frameworks/base/core/jni/AndroidRuntime.cpp:start()

l ZygoteInit.main() then

- Registers for zygote socket

- Pre-loads classes defined in frameworks/base/preloaded-classes(1800+)

- Pre-loads resources preloaded_drawables and preloaded_color_state_lists from frameworks/base/core/res/res/values/arrays.xml

- Runs garbage collector (to clean the memory as much as possible) - Forks itself to start system_server

- Starts listening for requests to fork itself for other apps

279

System Server Startupl When Zygote forks itself to launch the system_server process

(in ZygoteInit.java:startSystemServer()), it executes frameworks/base/services/java/com/android/server/SystemServer:java.main()

l The SystemServer:java.main() method loads android_servers JNI lib from frameworks/base/services/jni and invokes init1() native method

l Before init1() runs, the JNI loader first runs frameworks/base/services/jni/onload.cpp:JNI_OnLoad(), which registers native services - to be used as JNI counterparts to Java-based service manager loaded later

l Now frameworks/base/services/jni/com_android_server_SystemServer.cpp:init1()is invoked, which simply wraps a call to frameworks/base/cmds/system_server/library/system_init.cpp:system_init()

280

l The system_init.cpp:system_init() function

- First starts native services (some optionally): l frameworks/base/services/surfaceflinger/

SurfaceFlinger.cpp

l frameworks/base/services/sensorservice/ SensorService.cpp

l frameworks/base/services/audioflinger/ AudioFlinger.cpp

l frameworks/base/media/libmediaplayerservice/MediaPlayerService.cpp

l frameworks/base/camera/libcameraservice/CameraService.cpp

l frameworks/base/services/audioflinger/AudioPolicyService.cpp

l Then goes back to frameworks/base/services/java/com/android/server/ SystemServer.java:init2(), again via frameworks/base/core/jni/AndroidRuntime.cpp:start() JNI call

281

l The SystemServer.java:init2() method then starts Java service managers in a separate thread (ServerThread), readies them, and registers each one with frameworks/base/core/java/android/os/ServiceManager:addService()(which in turn delegates to to ServiceManagerNative.java, which effectively talks to servicemanager daemon previously started by init)

- frameworks/base/services/java/com/android/server/PowerManagerService.java

- frameworks/base/services/java/com/android/server/am/ActivityManagerService.java

- frameworks/base/services/java/com/android/server/TelephonyRegistry.java

- frameworks/base/services/java/com/android/server/PackageManagerService.java

- frameworks/base/services/java/com/android/server/BatteryService.java

- frameworks/base/services/java/com/android/server/VibratorService.java

- etc

282

l

frameworks/base/services/java/com/android/server/ am/ActivityManagerService.java:finishBooting()sets sys.boot_completed=1 and sends out

- a broadcast intent with android.intent.action.PRE_BOOT_COMPLETEDaction (to give apps a chance to reach to boot upgrades)

- an activity intent with android.intent.category.HOME category to launch the Home (or Launcher) application

- a broadcast intent with android.intent.action.BOOT_COMPLETEDaction, which launches applications subscribed to this intent (while using android.permission.RECEIVE_BOOT_COMPLETED)

android internals

Software