Chameleon: Multi-Persona Binary Compatibility for Mobile Devices

Jeremy Andrus, Alexander Van’t Hof, Naser AlDuaij, Christoffer Dall, Nicolas Viennot, and Jason Nieh{jeremya, alexvh, alduaij, cdall, nviennot, nieh}@cs.columbia.edu

Department of Computer ScienceColumbia University

Technical Report CUCS-011-13April 2013

AbstractMobile devices are vertically integrated systems that arepowerful, useful platforms, but unfortunately limit userchoice and lock users and developers into a particularmobile ecosystem, such as iOS or Android. We presentChameleon, a multi-persona binary compatibility archi-tecture that allows mobile device users to run applica-tions built for different mobile ecosystems together onthe same smartphone or tablet. Chameleon enhances thedomestic operating system of a device with personas tomimic the application binary interface of a foreign op-erating system to run unmodified foreign binary appli-cations. To accomplish this without reimplementing theentire foreign operating system from scratch, Chameleonprovides four key mechanisms. First, a multi-personabinary interface is used that can load and execute bothdomestic and foreign applications that use different setsof system calls. Second, compile-time code adaptationmakes it simple to reuse existing unmodified foreign ker-nel code in the domestic kernel. Third, API interposi-tion and passport system calls make it possible to reuseforeign user code together with domestic kernel facili-ties to support foreign kernel functionality in user space.Fourth, schizophrenic processes allow foreign applica-tions to use domestic libraries to access proprietary soft-ware and hardware interfaces on the device. We havebuilt a Chameleon prototype and demonstrate that it im-poses only modest performance overhead and can runiOS applications from the Apple App Store together withAndroid applications from Google Play on a Nexus 7tablet running the latest version of Android.

1 IntroductionMobile devices such as tablets and smartphones arechanging the way that computing platforms are de-signed, from the separation of hardware and softwareconcerns in the traditional PC world, to vertically inte-grated platforms. Hardware components are integrated

together in compact devices using non-standard inter-faces. Software is customized for the hardware, oftenusing proprietary libraries to interface with specializedhardware. Applications are tightly integrated with par-ticular libraries and frameworks, and often only availableon particular hardware devices.

These design decisions and the maturity of the mo-bile market can limit user choice and stifle innovation.Users who want to run iOS gaming applications on theirsmartphones are stuck with the smaller screen sizes ofthose devices. Users who prefer the larger selection ofhardware form factors available for Android are stuckwith the poorer quality and selection of Android gamesavailable compared to the well populated Apple AppStore [15]. Android users cannot access the rich multi-media content available in Apple iTunes, and iOS userscannot easily access Flash-based Web content. Somecompanies release cross-platform variants of their soft-ware, but this requires developers to master many dif-ferent graphical, system, and library APIs, and createsa massive support and maintenance burden on the com-pany. Many developers who lack such resources chooseone platform over another, limiting user choice. Com-panies or researchers that want to build innovative newdevices or mobile software platforms are limited in thefunctionality they can provide because they lack accessto the huge application base of existing platforms. Newplatforms without an enormous pool of user applicationsface the difficult, if not impossible, task of end useradoption, creating huge barriers to entry into the mobiledevice market.

In the traditional PC world, users can use virtual ma-chines (VMs) such as VMWare Workstation [29] or Par-allels Desktop [25] to run applications intended for oneplatform on a different platform. While VMs are use-ful for desktop and server computers, using them forsmartphones and tablets is problematic for at least tworeasons. First, mobile devices are more resource con-strained, and running an entire additional operating sys-tem (OS) and user space environment in a VM just to

1

run an application imposes high overhead. High over-head and slow system responsiveness are much less ac-ceptable on a smartphone than on a desktop computerbecause smartphones are often used for just a few min-utes or even seconds at a time. Second, mobile devicesare tightly integrated hardware platforms that incorpo-rate a plethora of devices using non-standard interfaces,such as GPUs. VMs provide no effective mechanism toenable applications to directly leverage these hardwaredevice features, severely limiting performance and mak-ing existing VM-based approaches unusable on smart-phones and tablets.

To give users greater freedom of choice, make it pos-sible for developers to write once and run on multipleplatforms, reduce barriers to entry in the mobile devicemarket, and help spur innovation by making huge ap-plication markets available on alternative platforms, wehave created Chameleon. Chameleon is a binary compat-ibility architecture that allows users to run applicationswritten and compiled for different mobile ecosystems si-multaneously on the same smartphone or tablet. We in-troduce the notion of foreign binaries, those developedfor another OS, the foreign OS, and domestic binaries,those developed for the given device’s OS, the domesticOS. Chameleon defines a persona as an execution modeassigned to each thread on the system, identifying thethread as executing either a foreign or domestic binary.Chameleon supports multiple personas by extending thekernel’s application binary interface (ABI) to be awareof both foreign and domestic threads.

Chameleon supports running foreign binaries by en-hancing the domestic OS with four key mechanismsto leverage existing unmodified application frameworksand kernel code. First, Chameleon provides a multi-persona binary interface. For each persona, Chameleonprovides a binary loader to interpret the actual contentsof the application binary and associated libraries, a sys-tem call interface, and an asynchronous signal deliverymechanism that works with each persona. Chameleoncan then load and execute foreign applications writ-ten for a different mobile ecosystem that uses a com-pletely different kernel ABI. Second, Chameleon intro-duces duct tape, a novel compile-time code adaptationlayer, that allows unmodified foreign kernel code to bedirectly compiled into the domestic kernel to provide for-eign binaries with kernel services not otherwise presentin the domestic kernel. Third, Chameleon introducesAPI interposition that allows foreign kernel services tobe implemented in user space leveraging existing do-mestic kernel services via passport system calls to ex-port the familiar foreign API to the foreign binary. Us-ing these mechanisms, Chameleon can then execute for-eign applications that require kernel-level services notavailable in the domestic OS. Fourth, Chameleon cre-

ates schizophrenic processes that can allow foreign ap-plications to use domestic libraries to access propri-etary software and hardware interfaces on the device.A schizophrenic process is a process that executes codeusing multiple binary personas. Because it is commonfor user space libraries on mobile devices to directly useor manage custom hardware resources, those librariescannot be used on a different mobile device with dif-ferent hardware. However, mobile devices typicallyhave similar functionality implemented using differentlibraries and hardware. Using schizophrenic processes,Chameleon replaces the foreign library used by a foreignapplication with the existing domestic library used bydomestic applications to provide fast and efficient directaccess to underlying closed hardware, such as GPUs.

Using these mechanisms, we have built a Chameleonprototype for use on Android devices that can run un-modified iOS applications along with Android applica-tions on the same device. Our approach leverages exist-ing software infrastructure as much as possible, includ-ing using existing application frameworks across bothiOS and Android ecosystems with no modification. Wedemonstrate the effectiveness of our prototype by run-ning various iOS applications from the Apple App Storetogether with Android applications from Google Play ona Nexus 7 tablet running the latest version of Android.Users can interact with iOS applications using multi-touch input, and iOS applications can leverage GPUhardware to display smooth accelerated graphics. Usingboth system-level and application-level benchmarks, wedemonstrate that Chameleon imposes only modest per-formance overhead and can yield faster performance oniOS applications than their Android counterparts on An-droid hardware because of the greater efficiencies of run-ning native iOS code instead of Java as used by Android.

2 Overview of Android and iOS

To understand how Chameleon runs iOS applications onAndroid, we first provide a brief overview of the opera-tion of Android and iOS. We limit our discussion to cen-tral components providing application startup, graphics,and input on both systems. A complete explanation ofboth systems is beyond the scope of this paper.

Figure 1 shows an overview of these two systems.Android is built on the Linux kernel and runs on ARMCPUs. The Android framework consists of a number ofsystem services and libraries used to provide applicationservices, graphics, input, and more. For example, Sys-temServer starts Launcher, the homescreen applicationon Android, and SurfaceFlinger, the rendering enginewhich uses the GPU to compose all the display surfaces

2

iOS Kernel

IOKitAndroid Linux

Kernel

Surfa

ceFl

inge

r

Laun

cher

Andr

oid

App

Syst

emSe

rver

Sprin

gBoa

rd

notif

yd

iOS

App

laun

chd

confi

gd

VFS Layer

Sockets

SecurityG

raph

ics

Inpu

t

Mac

h IP

C

Inpu

t

Gra

phics

Mach BSD

Figure 1: Android and iOS architectures

for different applications and display the final composedsurface to the screen.

Each Android application is compiled into Dalvik’sDex bytecode format. Tapping the icon on the home-screen launches a separate Dalvik VM instance to run theapplication. When a user interacts with Android appli-cations input events are delivered from the Linux devicedriver through the Android framework to the application.The application displays content by obtaining windowmemory in the form of a display surface from Surface-Flinger and draws directly into the window memory. Anapplication can attach an OpenGL context to the windowmemory and use the OpenGL ES framework to renderhigh-end, animated graphics in the window memory us-ing the GPU.

iOS runs on ARM CPUs like Android, but its softwareecosystem is substantially different. iOS is built on theXNU kernel [6], a hybrid combination of a monolithicBSD kernel and a Mach microkernel running in a singlekernel address space. XNU leverages the high perfor-mance BSD socket and VFS subsystems, but also bene-fits from the IPC mechanisms provided by Mach IPC.iOS makes extensive use of both the BSD and MachXNU services. The iOS user space framework consistsof a number of user space daemons. launchd is re-sponsible for booting the system, and starting, stopping,and maintaining services and applications. launchdstarts Mach IPC services such as configd, the systemconfiguration daemon, notifyd, the asynchronous no-tification server, and mediaserverd, the audio/videoserver. launchd also starts the SpringBoard applica-tion, which displays the iOS homescreen from whichusers can launch applications, handles and routes userinput to applications, and uses the GPU to compose dis-play surfaces for applications onto the screen. Spring-board is analogous to an amalgamation of Surface-Flinger, Launcher, and SystemServer in Android.

User-Level iOS Services

Chameleon Kernel

Surfa

ceFl

inge

r

Laun

cher

Andr

oid

App

Syst

emSe

rver launchd notifyd

configd

iOS App

iOS App

ChameleonW

ChameleonW eventpump

eventpump

Figure 2: System Integration Overview

iOS applications are written in Objective-C and arecompiled and run as native binaries in an extendedMach-O [3] format. This is in stark contrast to the Javaarchives used by Android applications, which are notloaded as native binaries, but interpreted by the DalvikVM. On iOS, binaries are loaded directly by an OS levelMach-O loader, which reads the application binary, loadsits text and data segments and jumps to the applicationentry point. Dynamically linked libraries are loaded bydyld, a user space binary, which is called from theMach-O loader. Examples of frequently used libraries iniOS applications include UIKit, the user interface frame-work, QuartzCore and OpenGL ES, the core graphicsframeworks, and WebKit, the Web browser engine.

3 Operational Model and SystemIntegration

Chameleon provides a familiar user experience whenrunning iOS applications on Android. Applications arelaunched from the Android home screen, just like anyother Android application, and users can switch seam-lessly between domestic Android applications and for-eign iOS applications. Chameleon accomplishes thiswithout running the iOS kernel or the main SpringBoardapplication, but instead overlays a file system (FS) hier-archy on the existing Android FS, allowing iOS appli-cations to access familiar iOS paths, and provides back-ground user-level services required by iOS applications.The background user-level services are not visible tousers, but establish key OS level functionality in userspace necessary to run iOS applications. Since we fo-cus on running iOS applications on Android, we use theterms foreign and iOS, and domestic and Android, inter-changeably to help describe Chameleon. Figure 2 showsan overview of the integration of iOS functionality into

3

our Android-based Chameleon prototype.Chameleon overlays the iOS FS hierarchy onto the ex-

isting Android FS, allowing iOS to access files usingexisting iOS-specific FS paths, while at the same timemaintaining the domestic Android FS hierarchy, allow-ing domestic applications and services to function un-interrupted. Chameleon supports the use of personas toisolate and switch between foreign and domestic viewsof the FS using familiar techniques such as chroot andFS unioning [30], but because there are no naming con-flicts between iOS and Android, our prototype imple-mentation avoids the overhead of such approaches andsimply creates the iOS FS entries inside the Android FS.Chameleon sets up subdirectories for running iOS appli-cations, including /Applications, /Documents, /Library,and /System/Library.

iOS applications running on Chameleon need accessto a number of application framework components in-cluding libraries and user-level services. Instead of re-implementing these components, a task which wouldrequire substantial engineering and reverse-engineeringefforts, we simply copy the existing binaries fromiOS and run them on the domestic system, leveragingChameleon’s binary compatibility architecture. How-ever, not all libraries can simply be copied from the iOSdevice due to optimizations on the iOS device discussedin Section 6. Therefore, we combined binaries froman iOS device with binaries from the Xcode SDK, Ap-ple’s development environment. Background user-levelservices such as launchd, configd, and notifydwere copied from the iOS device, where core frameworklibraries were copied from the Xcode SDK.

Chameleon fully supports Apple’s I/O Kit kernel-leveldriver support framework as part of the Chameleon ker-nel, described in more detail in Section 4. I/O Kit worksseamlessly with existing Android device drivers to ac-cess the domestic device hardware resources. Servicessuch as launchd require functional I/O Kit and kernel-level Mach IPC services, also supported by Chameleon,to start and function properly.

However, launchd assumes it is the first applicationon the system and therefore has a PID of 1. It furtherrelies on iOS-specific OS services to be notified of avail-able applications in the file system. Chameleon lever-ages PID virtualization [24] in the form of namespacesto allow launchd to run with a perceived PID of 1,despite it not being the first process on the system. Wecomplement Chameleon’s seamless integration of PIDnamespaces with imported foreign Mach IPC function-ality for fully compliant IPC mechanisms.

Since the Android application environment differssubstantially from iOS, Chameleon provides some user-level integration mechanisms to ensure a familiar userexperience. For example, in iOS, where launchd de-

tects new applications and ensures that they appear inSpringBoard, Chameleon introduces chameleond, whichleverages Linux’s existing inotify framework to detectnew iOS applications, extract the application icon fromthe foreign application package, and notify the AndroidLauncher of the new application. Launcher then placesthe iOS icon on the Android homescreen, as shown inFigure 6a.

Because the application launch procedure and the bi-nary format of iOS and Android applications are com-pletely different, iOS applications cannot simply bestarted by the Android Launcher. To provide seamlesssystem integration, and to keep changes to Android coreuser space components to a minimum, Chameleon intro-duces a proxy service, ChameleonW, which integratesthe execution of an application, from tapping an iconin the Android Launcher, to running a foreign binaryand integrating with background user-level iOS services.ChameleonW, is a standard Android application that theAndroid Launcher can start directly. It receives eventsand can be managed by Launcher like any other An-droid application, providing an integrated user experi-ence when running iOS applications. For example, theiOS application shows up under Android’s recent activ-ity list, and can be selected or terminated like a normalAndroid application. Our approach is also compatiblewith Android’s security model, but a detailed discussionis beyond the scope of this paper. Chameleon launchesa separate copy of ChameleonW for each iOS applica-tion running on the system. ChameleonW adapts to therespective iOS application it is running, providing thefamiliar application icon, and facilitates starting iOS bi-nary applications by issuing Java Native Interface (JNI)calls to call the unmodified iOS launchctl command,which in turn requests that the launchd service runthe iOS application, replicating the application launchmechanism on iOS.

To further provide seamless integration between thetwo systems, the iOS application must be notified aboutAndroid system events, such as the application goinginto the background, the display rotating, or simply toreceive input when the application is active. BecauseChameleonW is a standard Android application, it willautomatically be notified of such events by the Androidframework, but it must forward this info to the iOS ap-plication. We accomplish this last step by leveragingChameleon’s API interposition to hook into the iOS ap-plication startup procedure at a known point and launcha per-application thread, the eventpump. The eventpumpruns in the context of the iOS application and receivesevents from the Android ChameleonW application overa standard BSD socket, a mechanism ubiquitously avail-able on both Android and iOS. ChameleonW can thensend events via eventpump to the iOS application to

4

direct its execution, such as touch input, display rotation,accelerometer, proximity, and volume change events. Infact, ChameleonW will even use eventpump to notifythe iOS application that it is headed into the background,so that the application can save any necessary state andprepare for imminent termination, just as it would whenrunning on an iPhone or iPad.

4 Architecture

To support its operational model, Chameleon needs to beable to run unmodified iOS binaries on Android, includ-ing iOS applications, frameworks, and services. This ischallenging because iOS binaries are built to run on iOS,not Linux or Android. iOS provides a different systemcall interface from Linux, and iOS binaries make exten-sive use of OS services not available on Linux, such asMach IPC [5].

At a high-level, the solution to providing binary com-patibility is straightforward. The interaction between ap-plications and an OS is defined by the kernel applicationbinary interface (ABI). The ABI consists of the binaryloader which interprets the physical contents of the ap-plication binary and associated libraries, asynchronoussignal delivery, and the system call interface. To run iOSbinaries, we just need to implement the iOS ABI in An-droid. However, there are two key challenges. First, iOSis a complex, closed-source system and simply reimple-menting the system call interface from scratch would bea tedious, difficult, and error-prone process. Second,the behavior of some system calls is not well-defined.For example, the ioctl system call passes in a device-specific request code and a pointer to memory, and itsbehavior is device-specific. Without any understandingof how ioctls are used, simply implementing the systemcall itself is of little benefit to applications.

To address these problems, Chameleon provides amulti-persona binary compatibility architecture that pro-vides two key features. First, it provides mechanismsthat make it simple to use existing kernel and user codefrom foreign sources and incorporate them in Android.Although iOS is closed source, it is built on the XNUkernel, which is open source, and there are a variety ofopen source projects based on XNU. Chameleon makesit possible to import the code from XNU and relatedprojects to into Android to implement substantial por-tions of the iOS ABI without a substantial reimplemen-tation effort. Second, Chameleon introduces a funda-mentally new capability, schizophrenic processes. Thismechanism supports iOS applications that use closediOS libraries which issue device-specific system calls toaccess iOS proprietary hardware. By replacing thoselibraries with Android libraries, which provide access

iOS Application

Mach-OLoader

XNU ABI Compatibility

Standard Linux ABI

iOS Libraries

ChameleonLibraries libschizo

Android Libraries

iOSMach IPCServices

configdlaunchd

notifyd

IOKitDriver Bridge

XNU SourceIOKit

Mach IPC

Duct Tape

Chameleon Linux Kernel

Figure 3: Overview of Chameleon architecture

to Android proprietary hardware, unmodified iOS ap-plications can access Android hardware. This mecha-nism is based on the fact that closed libraries and closedhardware must rely on open interfaces for applicationsto use them. Rather than operating on opaque systemcalls, Chameleon operates at a higher level to leveragethe same open interfaces used by applications to providethe required functionality.

Figure 3 provides an overview of the Chameleon ar-chitecture, which can be thought of in terms of four com-ponents. First, Chameleon provides XNU ABI compat-ibility by implementing a Mach–O loader for the Linuxkernel, the XNU system call interface, and supportingsignal delivery. For most system calls, a simple wrapperis implemented which maps arguments from XNU struc-tures to Linux structures, then calls the correspondingLinux system call or reuses existing Linux kernel APIs.Second, Chameleon provides a duct tape layer to importforeign kernel code to support system calls that requirea core kernel subsystem not in Linux. Third, Chameleonprovides an API interposition layer implemented as iOSlibraries to support system calls that can be more easilyimplemented in user space by using existing foreign usercode together with special Linux system calls. Finally,Chameleon provides schizophrenic processes using thelibschizo iOS library to support applications that useclosed iOS libraries which issue device-specific systemcalls such as ioctl. We describe these components infurther detail in the following sections.

4.1 Kernel ABIBefore an application is executed, its binary formatmust be properly loaded and interpreted by the kernel.iOS uses the Mach–O format, not supported in Linux.Chameleon provides a Mach–O loader for the Linux ker-

5

nel which is registered as a standard binary format han-dler within the kernel, allowing the kernel to automat-ically choose to use the Mach–O loader when an iOSbinary is executed in user space. When the Chameleonloader launches a new process, it tags the process witha persona that is used by the kernel in all subsequentinteractions with user space. Personas are tracked on aper thread basis, namely per task struct in Linux,enabling an application process with multiple threads tosupport multiple personalities.

An application’s primary interface to the kernel isthrough system calls. Each kernel may have a differ-ent system call entry and exit path, and a different set ofsystem calls available. Chameleon uses the persona ofa thread to select among different kernel ABIs. It main-tains one or more system call dispatch tables for eachpersona, switches among them based on the persona ofthe calling thread and the system call number, and man-ages the entry and exit path differences through persona-tagged support functions. For example, the system callABI on iOS is vastly different from that of Linux in thatthere are multiple categories of system calls, each withits own calling convention and error reporting mecha-nism. Chameleon is aware of the foreign kernel’s callingconventions, and uses the persona of the calling threadto decide how to handle a system call upon entry to thekernel. Chameleon further translates the foreign callingconvention to an internal Linux calling convention, mak-ing it possible to, for example, call existing Linux systemcall implementation functions. Similarly, Chameleonalso examines a thread’s persona on its return path touser space, and reports the exit status of the system callas per the thread’s persona.

Because the iOS kernel is based on a POSIX-compliant BSD, most of the BSD system calls overlapwith functionality already provided by the Linux ker-nel. For most system calls, a simple wrapper is imple-mented which maps arguments from XNU structures toLinux structures when necessary and then simply callsthe Linux implementation. For BSD system calls thathave no corresponding Linux system calls, but for whichsimilar Linux functionality exists, the wrapper reuses ex-isting Linux kernel functions to implement the respectivesystem calls. For example, Chameleon implements theposix spawn system call, which is a flexible methodof starting a thread or new application, by leveraging thelinux clone and exec system calls.

Chameleon uses the persona of a thread for proper sig-nal delivery. While most OSes can deliver asynchronoussignals to an application, the semantics and implemen-tation can vary widely between OSes. When a threadreturns to user space from the kernel, Chameleon usesthe thread’s persona to account for these semantic differ-ences in signal delivery, including the user space signal

delivery entry point and function parameters passed tothe signal handler. Processes or threads of different per-sonas can send and receive signals amongst one another.

4.2 Duct Tape

Some iOS system calls require a core subsystem thatdoes not exist in the Linux kernel. A prime example ofthis is the Mach IPC mechanism used extensively by iOSapplications. The Mach IPC interface is a rich and com-plicated API providing interprocess communication andmemory sharing. Implementing such a subsystem fromscratch in the Linux kernel would be a daunting task.

To address this problem, Chameleon introduces ducttape, a novel compile–time code adaptation layer thatallows source code from a foreign kernel to be directlycompiled into the domestic kernel. The duct tape layertranslates foreign kernel APIs such as synchronization,memory allocation, process control, and list manage-ment, into domestic kernel APIs. The resulting mod-ule or subsystem is a first–class member of the domestickernel and can be accessed by both foreign and domes-tic applications. We refer to the process of compilingforeign kernel code in a domestic kernel as cross–kernelcompilation.

Chameleon successfully uses duct tape for three dif-ferent subsystems from the XNU kernel [6] into An-droid’s Linux kernel: BSD pthread support, Mach IPC,and Apple’s I/O Kit device driver framework, the lat-ter is discussed in Section 5.1. BSD pthread support iniOS differs substantially from Linux in how functional-ity is separated between the pthread library and the ker-nel. The iOS user space pthread library makes extensiveuse of kernel–level support for mutexes, semaphores,and condition variables. This support is found in thebsd/kern/pthread support.c file in the opensource XNU code provided by Apple. Chameleon usedduct tape to directly compile this file without modifica-tion.

The Mach IPC subsystem is significantly more com-plicated than BSD pthread support. It involves many dif-ferent header files and a large collection of C files foundin the osfmk/ipc/ and osfmk/kern/ XNU sourcedirectories. Chameleon uses duct tape to directly com-pile the majority of Mach IPC as a part of Linux, butsome parts were designed to work in a significantly dif-ferent kernel environment, and therefore had to be reim-plemented. In particular, the XNU kernel stack defaultsto 16KB, twice that of the Linux kernel, and the XNUMach IPC code takes direct advantage of deep call stacksto access queuing structures recursively, which wouldnot work in Linux. Therefore we reimplemented thequeuing structures used in the XNU source to better fitwith the Linux kernel environment.

6

4.3 API Interposition

Some iOS system calls require a core subsystem thatdoes not exist in the Linux kernel and cannot be eas-ily implemented using duct tape. For example, the XNUkernel provides the BSD kqueue and kevent notificationmechanism. The implementation of this mechanism isdistributed throughout the XNU kernel source so thatit can notify user space of myriad kernel events. Themechanism is built on a single system call entry point,but data semantics to register for and receive notificationof events are highly subsystem-specific. Adding supportfor this mechanism in Linux, would involve changes toalmost all core subsystems of the kernel, and is there-fore not an attractive approach. To address this problem,Chameleon uses API interposition to interpose betweenthe foreign binary and the kernel, allowing a user spacelibrary to implement the intended notification mecha-nism on top of existing Linux functionality. Library in-terposition is as simple as configuring an environmentvariable to load a library, which overrides symbolic ref-erences. As it turns out, existing open source user levelimplementations of some iOS services are available andcan be directly used with API interposition.

Because we are implementing foreign OS servicesin user space based on domestic kernel functionality,Chameleon needs access to a variety of domestic systemcalls, not available from a foreign persona. For example,we implement FS notification support using Linux’s in-otify functionality, but there are not inotify system callsin iOS. To address this issue, we introduce passport sys-tem calls. A passport system call is a new foreign systemcall in Chameleon, which allows threads with a foreignpersonality to indirectly call domestic system calls.

Chameleon successfully combines API interpositionand passport system calls to provide two important iOSkernel subsystems: (1) The kqueue and kevent kernelnotification mechanism, and (2) pthread workqueues, aset of APIs introduced by Apple that extend the pthreadAPI. Chameleon implements kqueue and kevent sys-tem calls in the libkqueue library, based on the opensource XDispatch project [22]. A number of bug fixesand semantic changes to the original XDispatch were in-corporated into libkqueue to fully support iOS binaries.

The pthread workqueue API is a feature in the iOSkernel that extends basic pthread support with a num-ber of features to support prioritized workqueue threads.Pthread workqueues are used heavily by Apple’s GrandCentral Dispatch [4] libraries, which are linked into al-most all applications and system services running inuser space. Implementing pthread workqueue sup-port in Linux would involve radical changes to exist-ing process management components due to the se-mantics of the API, and other attempts to implement

such functionality in existing kernels are not yet com-plete [16]. Chameleon implements pthread workqueuesupport in user space using existing Linux pthread sup-port. Chameleon pthread workqueue support libraryis called pthread workqueue and is also based onXDispatch.

4.4 Schizophrenic Processes

Applications on mobile devices often make use of closedand proprietary hardware and software stacks. For ex-ample, the OpenGL ES libraries on both Android andiOS directly communicate with the graphics hardwarethrough proprietary software and hardware interfaces us-ing device-specific ioctls, or opaque IPC messages.

Chameleon cannot simply implement kernel-levelsupport for foreign closed libraries because the seman-tics of ioctls or opaque IPC messages used by the closedlibraries are unknown. Further, the semantics are likelyto be closely tied to the underlying foreign hardwarewhich is not present on the domestic device. Becauseof the tight vertical integration across hardware and soft-ware on mobile devices, library developers often discardcumbersome abstractions present on desktop PCs in fa-vor of direct communication with hardware.

Chameleon solves the problem of direct access toproprietary hardware through the novel concept ofschizophrenic processes. A schizophrenic process is aprocess which executes code using multiple personas.Endowing processes with the ability to execute code us-ing multiple personas allows foreign applications to usedomestic libraries to interact with the domestic hard-ware. For example, iOS applications in Chameleon canload and execute code from the Android OpenGL ESlibrary and thereby directly interact with the hardwareon the underlying Android device. Chameleon leveragesthe fact that while the implementation of proprietary li-braries and their interface to kernel drivers is closed, theAPI on the application side is well-known, and is typi-cally similar across platforms such as iOS and Android.

Note that schizophrenic processes differ from bothAPI interposition and passport system calls. API inter-position allows the execution of new foreign user spacecode in a foreign binary. Passport system calls allow theissuing of specifically-identified domestic system callsfrom a foreign binary. In contrast, schizophrenic pro-cesses allow the execution of domestic user space bina-ries within a foreign binary, including the issuing of alldomestic system calls.

Chameleon encapsulates schizophrenic process sup-port in a library called libschizo. This library iscompiled as a foreign binary library, so it can link di-rectly with the foreign binary that wishes to use domesticservices. It provides a switching API comprising three

7

key components. First, it provides the ability to inter-pret and load domestic binaries, which involves the useof a complete domestic dynamic loader within libschizo.For example, Chameleon incorporates a cross-compiled,library version of the Android ELF loader.

Second, libschizo provides the ability to switch thethread local storage (TLS) pointer from one persona toanother so that each persona used by a thread can main-tain its own state. The contents of the TLS vary from sys-tem to system, and initializing domestic libraries wouldwipe out any state already initialized by the foreign ap-plication. For example, the errno variable is stored in aparticular location in the TLS which may differ betweenthe foreign and domestic systems.

Third, libschizo provides the ability to switch from theforeign persona to the domestic persona and back againusing a passport system call. This passport system callallows the foreign calling thread to tell the domestic ker-nel to switch personas, and using the same system callwithin the domestic persona to switch back to the for-eign persona.

Chameleon replaces a foreign library with domesticfunctionality by generating a surrogate library, which isused to direct calls to foreign libraries from the applica-tion into domestic libraries. Surrogate libraries are cre-ated by scanning the foreign library being replaced forentry points and generating a wrapper function, called asurrogate function, for each possible entry point. Thesurrogate library is dynamically loaded instead of theoriginal foreign library thereby intercepting calls madeto the foreign library.

When a foreign application calls a surrogate function,the surrogate uses the libschizo switching API to enact aschizophrenic function call as follows:

1. Upon first invocation, the surrogate function useslibschizo’s ability to load and interpret domesticbinaries to load the appropriate domestic libraryand locate the required entry point. The libschizoswitching API stores a pointer to the entry point ina locally scoped static variable for efficient reuse.

2. The arguments to the function call are stored on thestack.

3. The TLS pointer is changed to point to the domesticTLS instead of the foreign TLS.

4. A passport system call is executed that switches thecalling thread’s persona from foreign to domestic.

5. The arguments to the function call are restored fromthe stack.

6. The domestic function call is invoked through thesymbol stored in step 1.

7. Upon return from the function, the return value issaved on the stack.

8. A domestic system call is executed that switchesthe calling thread’s persona from domestic back toforeign.

9. The TLS pointer is changed to point to the foreignTLS instead of the domestic TLS, and any TLS val-ues such as errno are copied into the proper for-eign TLS location.

10. The function return value is restored from the stack,and control is returned to the calling function.

Because Chameleon maintains kernel binary personason a per thread basis, a foreign application can exe-cute using multiple personas simultaneously from mul-tiple threads, and each thread executing schizophreni-cally within a foreign application is free to use all ofthe facilities of the domestic OS including spawningadditional threads which will inherit the current per-sona. Chameleon leverages a schizophrenic OpenGLES library, discussed in detail in Section 5.3, to pro-vide fully accelerated graphics to foreign iOS applica-tions. Chameleon also uses schizophrenic function callsto a custom integration library to enable iOS applicationsto perform actions normally performed by SpringBoardsuch as displaying alerts and opening URLs.

5 iOS Subsystems on AndroidTo describe more clearly how the binary compatibil-ity mechanisms work, we give a few examples of howChameleon uses them together to provide binary com-patibility in the context of key iOS subsystems. We dis-cuss devices, input, and graphics. A discussion of allsubsystems is beyond the scope of this paper.

5.1 Devices

Chameleon uses duct tape to make underlying hardwaredevices available via I/O Kit to iOS applications and li-braries in the same manner as I/O Kit is used on iOS.I/O Kit is Apple’s open source driver framework basedon NeXTSTEP’s DriverKit. It is written primarily in arestricted subset of C++, and is accessed via Mach IPC.To directly compile the I/O Kit framework, Chameleonadded a basic C++ runtime to the Linux kernel. The run-time support was based on Android’s Bionic libc, andkernel Makefile support was added such that the compi-lation of C++ files from within the kernel required noth-ing more than assigning the object name to the obj-yMakefile variable. Chameleon used duct tape and its

8

Linux kernel C++ runtime to directly compile the major-ity of the I/O Kit code found in the XNU iokit sourcedirectory without modification. 1 In fact, we initiallycompiled Chameleon with the I/O Kit framework foundin XNU v1699.24.8, but later directly applied sourcecode patches to upgrade to the I/O Kit framework foundin XNU v2050.18.24.

Chameleon makes devices available via both theLinux device driver framework and I/O Kit. Usinga small hook in the Linux device add function,Chameleon creates an I/O Kit registry entry for everyregistered Linux device. For each device, Chameleonprovides a I/O Kit driver class that interfaces with thecorresponding Linux device driver. This allows iOS ap-plications to use I/O Kit to query the I/O Kit registryto locate devices or properties, as well as access the de-vices.

For example, iOS applications expect to inter-act with the device framebuffer through a C++class named AppleM2CLCD which derives from theIOMobileFramebuffer C++ class interface. Us-ing the C++ runtime support added to the Linux ker-nel, the Chameleon prototype added a single C++ filein the Linux display driver’s source tree that defines aclass named AppleM2CLCD. This C++ class acts as athin wrapper around the Linux device driver’s function-ality. The class is instantiated and registered with I/O Kitthrough a small interface function called on Linux ker-nel boot. The duct taped I/O Kit code will then matchthe C++ driver with the Linux device node (previouslyadded from the Linux device add function). Afterthe driver class instance is matched to the device classinstance, iOS user space can query and use the deviceas a standard iOS device. We believe that a similar pro-cess can be done for most devices found on a tablet orsmartphone.

5.2 Input

No user-facing application would be complete withoutinput from both the user and devices such as the ac-celerometer. In iOS, every application monitors a MachIPC port for incoming low–level event notifications andpasses these events up the user space stack through ges-ture recognizers and event handlers. The events sent tothis port include mouse, button, accelerometer, proxim-ity and touch screen events.

Chameleon creates a new thread in each iOS applica-tion to act as a bridge between the Android input sys-

1Portions of the I/O Kit codebasesuch as IODMAController.cpp andIOInterruptController.cpp were not necessary as theyare primarily used by I/O Kit drivers communicating directly withhardware.

Mach IPC

I/O Kit

OpenGL ES CoreGraphics

GPU

QuartzCore / CoreAnimation

UIKit

CPU

IOCoreSurface

iOS Kernel

iOSUser Space

IOMobileFramebufferGPU Driver

Hardware

IOSurface

Figure 4: Overview of iOS Graphics Components

tem and the Mach IPC port expecting input events. Thisthread, the eventpump, simply listens for events from theAndroid ChameleonW wrapper application via a BSDsocket and pumps those events into the iOS applicationvia Mach IPC. In the future, this intermediary threadcould be avoided with a minimal Linux Mach IPC wrap-per ABI.

5.3 Graphics

Chameleon uses a combination of kernel ABI emula-tion, duct taped driver interfaces, API interposition, andschizophrenic processes to provide binary compatibilityfor 2D and 3D graphics. To understand how this works,we first provide an overview of the iOS graphics sub-system, as shown in Figure 4. User space libraries suchas UIKit and CoreAnimation render user content, suchas buttons, text, and images, using CoreGraphics andOpenGL ES system libraries. These system librariescommunicate directly to the iOS kernel via Mach IPC,and use I/O Kit drivers to allocate and share graphicsmemory, control hardware facilities such as frame ratesand subsystem power, and perform more complex ren-dering tasks such as those required for 3D graphics. 2Dgraphics in iOS uses the CoreGraphics or QuartzCoreAPI to draw into graphics memory allocated by IOSur-face and the IOCoreSurface I/O Kit driver, and then dis-play the results using the IOMobileFramebuffer driver.3D graphics uses the standard OpenGL ES API [21, 23]as well as some Apple-specific extensions and a customObjective-C interface to the iOS native windowing sys-tem.

The iOS graphics subsystem presents a particularlyinteresting challenge to any mobile binary compatibil-ity system: highly optimized user space libraries are

9

Linux Display Driver

IOMobileFramebuffer wrapper

ChameleonLinux Kernel

QuartzCore, CoreGraphics...

Linux GPU driver

LinuxCoreSurface

AndroidOpenGLES,EGL, gralloc

Linux gralloc driver

IOSurface

iOSApp

OpenGLES

Figure 5: Chameleon Graphics Compatibility Architec-ture

tightly integrated with mobile hardware. Libraries suchas QuartzCore and OpenGL ES, call into a set of pro-prietary, closed source, helper libraries which, in turn,use closed source opaque calls via Mach IPC to closedsource kernel drivers which control black-box pieces ofhardware. The Mach IPC calls to the kernel drivers areessentially used as device-specific system calls. Unlikethe modern desktop OS, there are no well-defined inter-faces to graphics acceleration hardware, such as the Di-rect Rendering Infrastructure used by the X Window sys-tem. Neither implementing kernel–level emulation codenor duct taping a piece of non–existent GPU driver codewill solve this problem.

Chameleon enables 2D and 3D graphics in iOS appli-cations through a novel combination of I/O Kit Linuxdriver wrappers, a schizophrenic OpenGL ES library,and API interposition. Figure 5 shows an overview ofhow the iOS graphics subsystem is mapped to Android.Chameleon uses all of the original unmodified iOS li-braries from Figure 4, except for OpenGL ES, and re-places iOS kernel components from Figure 4 with Linuxkernel components.

For 2D graphics, Chameleon supports the iOS Core-Graphics or QuartzCore API using the original unmod-ified CoreGraphics and QuartzCore/CoreAnimation li-braries, but it interposes on the IOSurface library to al-locate the graphics memory needed using the unmodi-fied Android gralloc library and the underlying Linuxgralloc driver. Chameleon uses API interposition andschizophrenic calls to the Android gralloc library to tiethe allocated memory to Android GraphicBuffer objectswhich are designed to be shared between graphics hard-ware via OpenGL ES and the CPU via direct pixel ma-nipulation. Instead of using the iOS IOMobileFrame-buffer driver to display the results, Chameleon uses an

IOMobileFramebuffer I/O Kit interface wrapped aroundthe existing Linux display driver to display the results.To provide the zero-copy memory semantics expectedby the IOSurface library, Chameleon provides Linux-CoreSurface, a reverse-engineered IOCoreSurface I/OKit driver. However, 2D graphics support is insufficientfor iOS applications to work because 3D graphics is usedextensively by iOS drawing and composing functions.

For 3D graphics, Chameleon supports the iOSOpenGL ES API by replacing the entire iOS OpenGLES library using schizophrenic processes to call the An-droid graphics libraries. The iOS OpenGL ES API con-sists of two parts, the standard OpenGL ES API [21, 23]and some Apple-specific EAGL extensions. Chameleonprovides an iOS replacement OpenGL ES library witha schizophrenic surrogate entry point for every exportedsymbol in both of these categories.

For the standard OpenGL ES API entry points,Chameleon provides a set of surrogate functionsconforming to the schizophrenic switching API inChameleon’s iOS replacement OpenGL ES librarywhich then call the Android OpenGL ES library. Be-cause each of these entry points has a well-definedfunction prototype, the process of creating surrogatefunctions is automated by a script that analyzes ex-ported symbols in the iOS OpenGL ES Mach-O library,searches through a directory of Android ELF shared ob-jects for a matching export, and automatically generatesa surrogate function.

For the Apple-specific EAGL extensions, these do notexist on Android. These extensions are used to con-trol window memory and graphics context. The scriptused to analyze exported symbols in the iOS OpenGLES Mach-O library will generate a C function stub forthe EAGL exported symbols which do not match any ex-port in any Android ELF library. These then need to beimplemented in some fashion. However, the EAGL ex-tensions are Apple’s replacement for the Native PlatformGraphics Interface Layer (EGL) standard, and this is im-plemented in an Android EGL library. Chameleon usesschizophrenic calls to the Android EGL functions to im-plement the EAGL extensions. Android’s EGL library,used in combination with Android’s SurfaceFlinger ser-vice, manages windows memory into which iOS appli-cations render. Allocating this memory via the stan-dard Android SurfaceFlinger service allows Chameleonto manage the iOS display in the same manner all An-droid application displays are managed.

The Chameleon graphics subsystem allows many iOSapplications to render without issue and supports thenecessary 3D graphics drawing and composing functionsused by iOS. However, because our initial prototype wasdesigned to run system services, such as the windowcomposer, in the same process as the application itself,

10

and because the OpenGL ES standard only allows a sin-gle rendering thread per process, our prototype is lim-ited to a single graphics context. This means advanced3D games, or any application that explicitly creates anOpenGL ES context will not work properly. Many ap-plications, however, exclusively use the QuartzCore 2Ddrawing APIs that utilize the system OpenGL ES con-text. These applications render without issue.

6 Experimental Results

We have implemented a Chameleon prototype for run-ning iOS and Android applications on an Android de-vice and present some experimental results to measureits performance. We compared three different Androidsystem configurations to measure the performance ofChameleon: (1) Linux binary and Android applicationsrunning on vanilla Android, (2) Linux binary and An-droid applications running on Chameleon, and (3) iOSbinary applications running on Chameleon. For our ex-periments, we used a Nexus 7 tablet with a 1.3 GHzquad-core NVIDIA Tegra 3 CPU, 1 GB RAM, 16 GB offlash storage, and 7” LED-backlit IPS LCD display with1280x800 screen resolution at 216ppi, running Android4.2, also known as Jelly Bean, the latest version of An-droid. We also ran iOS binary applications on a jailbro-ken iPad mini with a 1 GHz dual-core A5 CPU, 512 MBRAM, 16 GB of flash storage, and 7.9” LED-backlit IPSLCD display with 1024x768 screen resolution at 163ppi,running iOS 6.1.2, the latest version of iOS. Since theiPad Mini was released at roughly the same time frameas the Nexus 7 and has a similar form factor, it providesa useful point of comparison even though it costs over50% more than the Nexus 7.

We used both micro-benchmarks and real applicationsto evaluate the performance of Chameleon. To mea-sure the latency of common low-level OS operations, weused micro-benchmarks from lmbench 3.0 and com-piled two versions, a ELF Linux binary and a Mach–O iOS binary, using the standard Linux GCC 4.4.1 andXcode 4.2.1 compilers, respectively. We used four cate-gories of lmbench tests: basic operations, system callsand signals, process creation, and local communicationand file operations. To measure real application perfor-mance, we used comparable iOS and Android PassMarkapplications available from the Apple App Store [26]and Google Play [27], respectively. PassMark conductsa wide range of resource intensive throughout tests toevaluate CPU, memory, disk, and graphics performance.For example, it measures how many graphics operationsit can do per second. We used PassMark because it is awidely used, commercially-supported application avail-able on both iOS and Android, and provides a conser-

vative measure of application performance. We normal-ize all results using the vanilla Android performance asthe baseline to compare across systems. This is usefulto measure Chameleon performance overhead, but alsoprovides some key observations regarding the character-istics of Android and iOS applications.

6.1 Obtaining iOS Applications

For our experiments, we downloaded iOS applicationsfrom the Apple App Store. In the future, we envisionthat developers and application distributors would be in-centivized to provide alternative distribution methods.For example, Google Play might be incentivized to takeadvantage of Chameleon to make a greater number andhigher quality of applications available for Android de-vices. However, using the App Store required a few moresteps to install the applications on an Android devicebecause of various security measures used by the AppStore.

App Store applications, unlike iOS system applica-tions such as Stocks, are encrypted and can only be de-crypted using the key stored on an iOS device. We use ascript that can download and decrypt applications usingany jailbroken iOS device. To illustrate this point, weused an old iPhone 3GS updated with iOS 5.0.1 for thispurpose. The decryption process we based our methodon is widely used and documented [28].

Figure 6 shows screenshots of the Nexus 7 tablet withvarious iOS applications that we installed and ran on thedevice. The homescreen shows iOS and Android appli-cations side by side that users can execute in a seamlessmanner. The iOS Stocks application receives live marketdata over WiFi; quotes are from March 27, 2013. TheiOS Constitution application by Clint Bagwell Consult-ing is only available on the Apple App Store and pro-vides a superior user experience to those available onGoogle Play [9].

6.2 Microbenchmark Measurements

Figure 7 shows the results of running lmbench mi-crobenchmarks on the four different system configura-tions. Vanilla Android performance is normalized to onein all cases, so those results are not explicitly shown.These measurements are latencies, so smaller numbersare better. Measurements are shown at two scales to pro-vide a clear comparison despite the wide range of results.

First, Figure 7 shows basic CPU operation measure-ments for integer multiply, integer divide, double preci-sion floating point add, double precision floating pointmultiply, and double precision bogomflop tests. Noneof these tests exercise Chameleon functionality, but they

11

(a) Android homescreen with iOS applications (b) iOS Stocks application (c) Constitution for iPhone and iPod Touch

Figure 6: Chameleon running iOS applications

provide a comparison that reflect differences in the An-droid and iOS hardware and compilers used. The basicCPU operation measurements were essentially the samefor all three system configurations using the Android de-vice, except for the integer divide test, which showedthat the Linux compiler generated more optimized codethan the iOS compiler. In all cases, the measurementsfor the iOS device were worse than the Android device,confirming that the iPad mini’s CPU is not as fast as theNexus 7’s CPU for basic math operations.

Second, Figure 7 shows system call and signal han-dler measurements including null system call, read,write, open/close, and signal handler tests. Thenull system call measurement shows the overhead in-curred by Chameleon on a system call that does nowork, providing a conservative measure of the cost ofChameleon. The overhead is 8.5% over vanilla Androidin running the same Linux binary, which is due to theneed to determine the persona of the calling thread androute the call to the corresponding system call dispatchtable. The overhead is 40% for running the iOS binaryover vanilla Android running the Linux binary, whichshows the additional cost of using the iOS persona andtranslating the system call into the corresponding Linuxsystem call. These overheads end up in the noise for sys-tem calls that actually perform some function, as shownby the measurements for the other system calls. Run-ning the iOS binary on the Nexus 7 using Chameleonis much faster on these system call measurements thanrunning the same binary on the iPad mini, illustrating a

benefit of using Chameleon to leverage the different per-formance characteristics of Android hardware instead ofbeing limited to only iOS hardware.

The signal handler measurement shows the overheadincurred by Chameleon in delivering a signal to the sameprocess that generated the signal. This is a conservativemeasurement because no work is done by the process asa result of signal delivery. The overhead is small, 3%over vanilla Android in running the same Linux binary,which is due to the added cost of determining the per-sona of the target thread. The overhead is 25% for run-ning the iOS binary over vanilla Android running theLinux binary, which shows the additional cost of us-ing the iOS persona. This involves some translation ofthe signal information and using a larger signal deliverystructure with more information, as expected by iOS bi-naries. Running the iOS binary on the iPad mini takes175% longer than running the same binary on the Nexus7 using Chameleon for the signal handler test.

Third, Figure 7 shows five sets of process cre-ation measurements, fork+exit, fork+exec, andfork+sh tests. The fork+exit measurementshows that Chameleon incurs negligible overhead ver-sus vanilla Android in running a Linux binary despitethe fact that it must do some extra work in the form ofMach IPC initialization. However, Chameleon takes al-most 14 times longer to run the iOS binary version ofthe test compared to the Linux binary. The absolute dif-ference in time is roughly 3.5 ms, with the Linux bi-nary taking 245 µs and the iOSbinary taking 3.75 ms.

12

0  

1  

2  

3  0  

10  20  30  40  50  60  70  

int  mul  

int  div  

double  add  

double  mul  

null  syscall  

read  

write  

open/close  

signal  

fork+exit  

exec  android  

exec  ios  

sh  -­‐c  android  

sh  -­‐c  ios  

bogomflops  

pipe  

AF_UNIX  

select  10  fd  

select  100  fd  

select  250  fd  

create  0K  file  

delete  0K  file  

create  10K  file  

delete  10K  file  

Chameleon  Android  Chameleon  iOS  iOS  

Figure 7: Microbenchmark latency measurements normalized to vanilla Android; lower is better performance.

There are two reasons for this difference. First, the pro-cess running the iOS binary consumes significantly morememory than the Linux binary because the iOS dynamiclinker, dyld, maps 90 MB of extra memory from 115different libraries, irrespective of whether those librariesare used by the binary. fork must then duplicate thepage table entries corresponding to all of that extra mem-ory, incurring almost 1 ms of extra overhead. Second, aniOS process does a lot more work in user space when itforks because iOS libraries use pthread atfork toregister callbacks that are called before and after fork.Similarly, for each library, dyld registers a callback thatis called on exit, resulting in the execution of 115 han-dlers on exit. These user space callbacks account for2.5 ms of extra overhead. Note that the fork+exitmeasurement on the iPad mini is significantly faster thanusing Chameleon on the Android device due to a sharedlibrary cache optimization that is not yet supported inthe Chameleon prototype implementation. To save timeon library loading, iOS’s dyld stores common librariespre-linked on disk in a shared cache in lieu of storingthe libraries separately. iOS treats the shared cache in aspecial way and optimizes how it is handled.

The fork+exec measurement is done in severalunique variations on Chameleon made possible by us-ing schizophrenic processes. The test spawns a childprocess which executes a simple hello world program,but we compile two versions of the program, a Linux bi-nary and an iOS binary. The test itself is also compiledas both a Linux binary and an iOS binary. On a vanillaAndroid system, the only way to run the test is to run aLinux binary that spawns a child to run a Linux binary.Similarly, on an iOS system, the only way to run the testis to run an iOS binary that spawns a child to run an iOSbinary. Using Chameleon, the test can be run four differ-ent ways, a Linux binary can spawn a child to run eithera Linux or an iOS binary, and an iOS binary can spawn

a child to run either a Linux or an iOS binary.

Figure 7 shows all fork+exec measurements usingChameleon. exec android tests spawn a child to runa Linux binary. Chameleon incurs negligible overheadversus vanilla Android when the test program is a Linuxbinary that spawns a child to run a Linux binary. The ac-tual time it takes to run this test is roughly 590 µs, a littlemore than twice the time it takes to run the fork+exitmeasurement, reflecting the fact that executing the helloworld program is more expensive than simply exiting.Chameleon takes 4.8 times longer to run the test whenthe test program is an iOS binary that spawns a child torun a Linux binary. The extra overhead is due to the costof an iOS binary calling fork, as discussed previously inthe case of the fork+exitmeasurement. Interestingly,the fork+exec measurement is 3.42 ms, which is lessthan the fork+exit measurement. This is because thechild process replaces its iOS binary with the hello worldLinux binary, and this is less expensive than having theoriginal iOS binary exit because of all the exit handlersthat it has to execute.

exec ios tests spawn a child to run an iOS binary.This is not possible on vanilla Android, so there is novanilla Android performance that we can normalize allresults against as a baseline. To compare the differentfork+exec measurements, we instead normalize theperformance against the vanilla Android system runninga Linux binary that spawns a child running a Linux bi-nary. This comparison is intentionally unfair and skewsthe results against the tests that spawn a child running themore heavyweight iOS binary. Nevertheless, using thiscomparison, Figure 7 shows that spawning a child to runan iOS binary is much more expensive. This is becausedyld loads 90 MB of extra libraries when it starts theiOS binary. Unlike fork in which copy-on-write can beused to limit this cost to the duplicating page tables, thecost for exec involves not just creating the page tables,

13

but also mapping in the actual libraries themselves. Thisis very expensive because the Chameleon prototype im-plementation does not use the shared cache optimization,but uses non-prelinked libraries and dyld walks thefilesystem to load all the libraries on every exec. Theextra overhead for starting with an iOS binary insteadof a Linux binary is due to the cost of the iOS binarycalling fork, as discussed previously in the case of thefork+exit measurement. Running the fork+exectest on the iPad mini is faster than using Chameleon onthe Android device because of its use of the shared cacheoptimization, which avoids the need to walk the filesys-tem to load all the libraries.

Similar to the fork+exec measurement, thefork+sh -c measurement is also done in sev-eral variations on Chameleon made possible by us-ing schizophrenic processes. sh -c android testslaunch a shell to run a Linux binary. Chameleon againincurs negligible overhead versus vanilla Android whenthe test program is a Linux binary, but takes 110% longerwhen the test program is an iOS binary. The extra over-head is due to the cost of an iOS binary calling fork,as discussed previously in the case of the fork+execmeasurement, but because the sh -c android mea-surement takes longer, 6.8 ms using the iOS binary,the relative overhead is less than in the case of execandroid.

sh -c ios tests launch a shell to run an iOS bi-nary. This is not possible on vanilla Android, so weagain use the same sh -c android baseline for com-parison, skewing the results against the sh -c iostests. Nevertheless, using this comparison, Figure 7shows that spawning a child to run an iOS binary ismuch more expensive for the same reasons as for thefork+exec measurement. Because the sh -c iostest takes longer, the relative overhead is less than in thecase of exec ios.

Fourth, Figure 7 shows local communication andfilesystem measurements including pipe, AF UNIX,select on 10 to 250 file descriptors, and creating anddeleting 0 KB and 10 KB files. The measurements werequite similar for all three system configurations using theAndroid device. However, the measurements for the iOSdevice were significantly worse than the Android devicein a number of cases. Perhaps the worst offender was theselect test, whose overhead increased linearly withthe number of file descriptors to more than 10 times thecost of running the test on vanilla Android, and simplyfailed to complete for 250 file descriptors. In contrast,the same iOS binary runs using Chameleon on Androidwith performance that was the same as running a Linuxbinary on vanilla Android across the wide range in thenumber of file descriptors used for the measurements.

6.3 Application Measurements

Figure 8 shows the results of running the iOS and An-droid PassMark benchmark applications [26, 27] on thefour different system configurations. Vanilla Androidperformance is normalized to one in all cases, so thoseresults are not explicitly shown. These measurements arein operations per second, so larger numbers are better.

Figure 8 shows CPU operation measurements for inte-ger, floating point, find primes, random string sort, dataencryption, and data compression tests. Because Pass-Mark is a full graphical application, running the applica-tion on Chameleon necessarily exercises its functional-ity, but because these tests are primarily CPU intensive,the resulting overhead for using Chameleon is negligi-ble. This is evident by comparing the results for run-ning the Android PassMark application on vanilla An-droid versus Chameleon. However, unlike the basic CPUoperation measurements obtained using lmbench, thePassMark measurements show that Chameleon deliverssignificantly faster performance when running the iOSPassMark application on Android. This is because theAndroid version is written in Java and is interpretedthrough the Dalvik VM while the iOS version is writ-ten in Objective-C and is compiled and run as a nativebinary. Because the Android device is faster than theiOS device, Chameleon outperforms iOS in running theCPU tests for the same iOS PassMark application.

Figure 8 shows storage operation measurements forwrite and read tests. There is negligible overhead causedby Chameleon for these tests as evident by comparingthe results for running the Android PassMark applica-tion on vanilla Android versus Chameleon. However,Chameleon does have somewhat slower performance onthe storage write test when running the iOS version ofthe application. This is not due to any performance over-head caused by Chameleon, but the modest difference ismore likely caused by small differences in the native li-braries used by the respective applications. Chameleonhas much better storage read performance when runningthe iOS application than vanilla Android. In compari-son with the iOS device, Chameleon has better storageread performance but worse storage write performance.Because storage performance can depend heavily on theOS, these results may reflect differences in both the hard-ware and the OS.

Figure 8 shows memory operation measurements forwrite and read tests. There is negligible overhead causedby Chameleon for these tests as evident by comparingthe results for running the Android PassMark applica-tion on vanilla Android versus Chameleon. Chameleondelivers significantly faster performance when runningthe iOS PassMark application on Android. This is againbecause Chameleon can run the iOS application natively

14

0  

1  

2  

3  

4  0  

5  

10  

15  

20  

Integer   Floa2ng  Point  

Find  Primes   Random  String  Sort  

Data  Encryp2on  

Data  Compression  

Storage  Write  

Storage  Read   Memory  Write  

Memory  Read  

Solid  Vectors   Transparent  Vectors  

Complex  Vectors  

Image  Rendering  

Image  Filters  

Chameleon  Android  

Chameleon  iOS  

iOS  

Figure 8: Application throughput measurements normalized to vanilla Android; higher is better performance.

while Android runs the application through the DalvikVM. Chameleon outperforms iOS in running the mem-ory tests for the same iOS PassMark application, againreflecting the benefit of using faster Android hardware.

Figure 8 shows graphics measurements for a varietyof graphics operations, including solid vectors, trans-parent vectors, complex vectors, image rendering, andimage filters. There is negligible overhead caused byChameleon for these tests as evident by comparingthe results for running the Android PassMark appli-cation on vanilla Android versus Chameleon. How-ever, Chameleon has slower performance when run-ning the iOS PassMark application on Android. Bothversions use the same native Android OpenGL ESlibrary, which is why the Android application per-forms well since the GPU-intensive operations thatdominate this test are performed natively. However,Chameleon incurs additional overhead due to a subopti-mal mapping in our unoptimized prototype between theIOMobileFramebuffer interface and the Linux de-vice driver. Despite this cost, Chameleon still signifi-cantly outperforms iOS in running the same iOS Pass-Mark application across all graphics operations becauseof the faster GPU hardware in the Android device. Forexample, the solid vector test runs almost twice as fastusing Chameleon versus iOS. Note that PassMark’s 3Dgraphics measurements are not included in the results be-cause the Chameleon prototype implementation does notyet have complete support for advanced 3D rendering.

7 Related WorkVarious previous approaches have explored binary com-patibility in the context of desktop computers, but littlework has been done for mobile devices. Wine [1] runsWindows applications on x86 computers running Linux,including widely-used Microsoft Office productivity ap-plications. It achieves this by attempting to reimplementthe entire foreign user space library API, such as Win32,using native APIs. This approach is tedious and over-whelmingly complex, and Wine has been under devel-

opment for a long time, but continues to chase Windowsas every new release contains new APIs that need to beimplemented. Darling [11] takes a similar approach totry to run Mac OS X applications on Linux, though itremains a work in progress unable to run anything otherthan very simple applications. In contrast, Chameleonprovides binary personality support in the OS to lever-age existing unmodified application libraries and frame-works and avoid rewriting huge amounts of user spacecode.

Wabi [19] was a product from Sun Microsystems forrunning Windows applications on Solaris. It supportedapplications developed for Windows 3.1, but did not sup-port later versions of Windows and was discontinued.Unlike Wine, it required Windows 3.1. It replaced low-level Windows API libraries with versions that translatedfrom Windows to Solaris calls. Wabi ran on top of So-laris and provided all of its functionality outside of theOS, limiting its ability to support applications that re-quire kernel-level services not available in Solaris. Incontrast, Chameleon is not limited to API interpositionand provides binary personality support in the OS to sup-port foreign kernel services.

Several BSD variants maintain a binary compatibil-ity layer for other OSes [12, 13, 17]. The BSD approachreimplements foreign system calls in the OS, using a dif-ferent system call dispatch table for each OS to glue thecalls to the BSD kernel. It works for foreign OSes thatare close enough to BSD such as Linux, but attemptsto extend this approach to supporting Mac OS X appli-cations only provide limited support for command linetools, not Mac OS X GUI applications [14]. In contrast,Chameleon provides duct tape and interposition layersto simplify adding unmodified foreign kernel code to anOS, and introduces schizophrenic processes that can runmixtures of foreign and domestic binaries. Unlike previ-ous approaches, this unique mechanism provides a solu-tion for supporting graphical applications in mobile de-vices where hardware acceleration is often exposed di-rectly to user space and libraries such as OpenGL calldevice and platform specific functions.

15

Other partial solutions to binary compatibility havealso been explored. For example, Shinichiro Hamaji’sMach-O loader for Linux [18] can load and run somedesktop Apple OS X command line binaries in Linux.This project supports only command line binaries us-ing the Linux “misc” binary format and dynamicallyoverwriting C entry points to system calls. In contrast,Chameleon provides a complete environment for foreignbinaries including graphics libraries and device access.

Virtual machines have perhaps become the most pop-ular way to run foreign applications on desktop com-puters. However, they require purchasing and runninga full guest OS instance, using much more memoryand disk space, and incur additional performance over-head. Various approaches [7, 10, 20] have attempted tobring VMs to mobile devices, but these approaches can-not run unmodified OSes and incur even higher over-head than their desktop counterparts; memory overheadis a key problem on memory constrained smartphones.Lightweight mobile virtualization [2, 8] claims lowervirtualization overhead than other mobile counterparts,but does not support running different OS instances andtherefore cannot run foreign applications at all. UnlikeChameleon, none of these previous virtualization ap-proaches can run foreign iOS applications on Android.

8 ConclusionsChameleon is the first system to allow users to run un-modified mobile applications written for one OS on an-other OS. It provides novel binary compatibility mecha-nisms including (1) a multi-persona binary interface toallow foreign binaries to run directly on top of a do-mestic OS kernel, (2) a static code compilation transla-tion layer, duct tape, to allow foreign open-source ker-nel code to be directly compiled as a part of the do-mestic kernel without modification, (3) API interpositionwith passport system calls to implement OS level ser-vices in user space to support foreign binaries, and (4)schizophrenic processes, which take advantage of bothforeign and domestic OS services at the same time. Wehave built a Chameleon prototype and demonstrate thatit imposes only modest performance overhead and suc-cessfully runs widely used iOS applications seamlesslywith Android applications.

9 AcknowledgmentsYan Zou helped with running benchmarks to obtainmany of the measurements in this paper. This workwas supported in part by a Facebook Graduate Fellow-ship and NSF grants CNS-1162447, CNS-1018355, andCNS-0905246.

References[1] AMSTADT, B., AND JOHNSON, M. K. Wine.

Linux Journal (Aug. 1994).

[2] ANDRUS, J., DALL, C., VAN’T HOF, A.,LAADAN, O., AND NIEH, J. Cells: A Virtual Mo-bile Smartphone Architecture. In Proceedings ofthe 23rd ACM Symposium on Operating SystemsPrinciples (Cascais, Portugal, Oct. 2011), pp. 173–187.

[3] APPLE, INC. OS X ABI Mach-O File FormatReference. https://developer.apple.com/library/mac/#documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html, Feb. 2009. Accessed: 3/20/2013.

[4] APPLE, INC. Grand Central Dispatch (GCD)Reference. http://developer.apple.com/library/ios/#documentation/Performance/Reference/GCD_libdispatch_Ref/Reference/reference.html, Nov. 2011. Accessed:3/28/2013.

[5] APPLE, INC. Porting UNIX/Linux Applicationsto OS X. https://developer.apple.com/library/mac/#documentation/Porting/Conceptual/PortingUnix/background/background.html, June2012. Accessed: 3/27/2013.

[6] APPLE, INC. Source Browser. http://www.opensource.apple.com/source/xnu/xnu-2050.18.24/, Aug.2012. Accessed: 3/21/2013.

[7] BARR, K., BUNGALE, P., DEASY, S., GYURIS,V., HUNG, P., NEWELL, C., TUCH, H., ANDZOPPIS, B. The VMware Mobile VirtualizationPlatform: Is That a Hypervisor in Your Pocket?ACM SIGOPS Operating Systems Review (Dec.2010), 124–135.

[8] CELLROX. Cellrox ThinVisor Technology. http://www.cellrox.com/how-it-works/,Feb. 2013. Accessed: 4/5/2013.

[9] CLINT BAGWELL CONSULTING. Constitution foriPhone. https://itunes.apple.com/ca/app/constitution-for-iphone-ipod/id288657710, Mar. 2013. Accessed: 3/20/2013.

16

[10] DALL, C., AND NIEH, J. KVM for ARM. In Pro-ceedings of the Ottawa Linux Symposium (Ottawa,Canada, June 2010).

[11] DOLEEL, L. The Darling Project. http://darling.dolezel.info/en/Darling,Aug. 2012. Accessed: 4/5/2013.

[12] DREYFUS, E. Linux Compatibility on BSD for thePPC Platform. http://onlamp.com/lpt/a/833, May 2001. Accessed: 5/11/2012.

[13] DREYFUS, E. IRIX Binary Compatibility, Part1. http://onlamp.com/lpt/a/2623, Aug.2002. Accessed: 5/11/2012.

[14] DREYFUS, E. Mac OS X binary compatibilityon NetBSD: challenges and implementation. InProceedings of the 2004 EuroBSDCon (Karlsruhe,Germany, Oct. 2004).

[15] FARADAY, OWEN. Android is a deso-late wasteland when it comes to games.http://www.wired.co.uk/news/archive/2012-10/31/android-games,Oct. 2012. Accessed: 3/21/2013.

[16] FREEBSD. Grand Central Dispatch (GCD) onFreeBSD. https://wiki.freebsd.org/GCD, May 2011. Accessed: 3/21/2013.

[17] FREEBSD DOCUMENTATION PROJECT. Linuxbinary compatibility. In FreeBSD Handbook, B. N.Handy, R. Murphey, and J. Mock, Eds. 2000,ch. 11.

[18] HAMAJI, S. Mach-O Loader for Linux.https://github.com/shinh/maloader,Mar. 2011. Accessed: 3/15/2013.

[19] HOHENSEE, P., MYSZEWSKI, M., AND REESE,D. Wabi CPU emulation. In Hot Chips 8 (1996).

[20] HWANG, J., SUH, S., HEO, S., PARK, C., RYU,J., PARK, S., AND KIM, C. Xen on ARM: Sys-tem Virtualization using Xen Hypervisor for ARM-based Secure Mobile Phones. In Proceedings of the5th Consumer Communications and Network Con-ference (Las Vegas, NV, Jan. 2008).

[21] KHRONOS GROUP. OpenGL ES – The Stan-dard for Embedded Accelerated 3D Graphics.http://www.khronos.org/opengles/,Jan. 2013. Accessed: 3/22/2013.

[22] MLBA TEAM. XDispatch - Overview.http://opensource.mlba-team.de/xdispatch/docs/current/index.html,Jan. 2013. Accessed: 3/27/2013.

[23] MUNSHI, A., AND LEECH, J. OpenGLES Common Profile Specification Ver-sion 2.0.25 (Full Specification). http://www.khronos.org/registry/gles/specs/2.0/es_full_spec_2.0.25.pdf,Nov. 2010. Accessed: 4/8/2013.

[24] OSMAN, S., SUBHRAVETI, D., SU, G., ANDNIEH, J. The Design and Implementation of Zap:a System for Migrating Computing Environments.In Proceedings of the 5th Symposium on OperatingSystems Design and Implementation (Boston, MA,Dec. 2002).

[25] PARALLELS IP HOLDINGS GMBH. ParallelsDesktop. http://www.parallels.com/products/desktop/. Accessed: 3/22/2013.

[26] PASSMARK SOFTWARE, INC. Per-formancetestmobile for iphone, ipodtouch, and ipad on the itunes app store.https://itunes.apple.com/us/app/performancetest-mobile/id494438360?ls=1&mt=8, June 2012.Accessed: 3/14/2013.

[27] PASSMARK SOFTWARE, INC. Passmark perfor-mancetest – android apps on google play. https://play.google.com/store/apps/details?id=com.passmark.pt_mobile,Jan. 2013. Accessed: 3/14/2013.

[28] TUNG, C. K. CK’s IT blog: How To DecryptiPhone IPA file. http://tungchingkai.blogspot.com/2009/02/how-to-decrypt-iphone-ipa-file.html, Feb. 2009. Accessed: 3/14/2013.

[29] VMWARE, INC. VMware Workstation.http://www.vmware.com/products/workstation/. Accessed: 3/22/2013.

[30] WRIGHT, C. P., DAVE, J., GUPTA, P., KRISH-NAN, H., QUIGLEY, D. P., ZADOK, E., ANDZUBAIR, M. N. Versatility and Unix Semanticsin Namespace Unification. ACM Transactions onStorage (Feb. 2006), 74–105.

17