arxiv:1510.01419v1 [cs.ni] 6 oct 2015

13
Haystack: In Situ Mobile Traffic Analysis in User Space Abbas Razaghpanah Stony Brook University Narseo Vallina-Rodriguez ICSI Srikanth Sundaresan ICSI Christian Kreibich ICSI / Lastline Phillipa Gill Stony Brook University Mark Allman ICSI Vern Paxson ICSI / UC Berkeley Abstract Despite our growing reliance on mobile phones for a wide range of daily tasks, we remain largely in the dark about the operation and performance of our devices, in- cluding how (or whether) they protect the information we entrust to them, and with whom they share it. The ab- sence of easy, device-local access to the traffic of our mo- bile phones presents a fundamental impediment to im- proving this state of affairs. To develop detailed visibil- ity, we devise Haystack, a system for unobtrusive and comprehensive monitoring of network communications on mobile phones, entirely from user-space. Haystack correlates disparate contextual information such as app identifiers and radio state with specific traffic flows des- tined to remote services, even if encrypted. Haystack fa- cilitates user-friendly, large-scale deployment of mobile traffic measurements and services to illuminate mobile app performance, privacy and security. We discuss the design of Haystack and demonstrate its feasibility with an implementation that provides 26–55 Mbps through- put with less than 5% CPU overhead. Our system and results highlight the potential for client-side traffic anal- ysis to help understand the mobile ecosystem at scale. 1 Introduction Mobile phones now offer users capabilities that rival those of general purpose computers. However, despite the myriad applications they support, mobile systems remain notoriously opaque. Mobile operating systems tightly control access to system resources and traffic. While this helps prevent compromised or malicious apps from wreaking havoc, these security measures pose hur- dles for understanding the performance and behavior of mobile apps. The opacity of mobile systems to monitoring presents two serious drawbacks. First, it makes it very hard for researchers to understand how mobile systems perform in the wild, requiring them to rely upon instrumenting individual phones to run experiments, or privileged ac- cess to ISPs to obtain large-scale traffic traces. Second, it opens the door for potentially unchecked exfiltration of personal data by applications, especially since we rely upon these devices for an increasing number of tasks and entrust them with increasingly sensitive data. Gaining insight into the internal activities of mobile phones currently proves difficult even for experts. Var- ious techniques have provided insight into the general contours of phone operation—from capturing packets on rooted phones [13] and within ISPs [15, 29] to redirect- ing traffic through instrumented VPNs [26] to static and dynamic analysis [11, 23] of apps. However, none of the currently available methodologies provides a com- prehensive view of mobile platform operation and net- work interaction both at scale and in the wild. In this paper we propose Haystack, a handset-, traffic-, and user-centric platform for Android devices that pro- vides high-fidelity information about mobile phone op- eration and network traffic. Our solution combines three key approaches. First, Haystack runs completely in user- space without requiring root permissions, capturing user traffic locally on the device. To do so, Haystack takes ad- vantage of Android’s VPN permission 1 to route transmit- ted packets through a process running in user-space. This process implements a simplified layer-3/4 network stack, conducting traffic analysis while relaying flow contents to the destination via regular sockets. Second, by run- ning locally on the phone Haystack can directly observe crucial app context (e.g., determining the identity of the particular app generating the traffic), device status (e.g., location, screen, and radio state), user-related informa- tion (e.g., accounts, messages, recent calls, and contact lists), and the network traffic associated with user activ- ities. Third, to identify privacy leaks more effectively, Haystack optionally intercepts encrypted traffic via a lo- 1 The recent iOS 9 release provides similar VPN capabilities. arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

Upload: others

Post on 24-Apr-2022

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

Haystack: In Situ Mobile Traffic Analysis in User Space

Abbas RazaghpanahStony Brook University

Narseo Vallina-RodriguezICSI

Srikanth SundaresanICSI

Christian KreibichICSI / Lastline

Phillipa GillStony Brook University

Mark AllmanICSI

Vern PaxsonICSI / UC Berkeley

Abstract

Despite our growing reliance on mobile phones for awide range of daily tasks, we remain largely in the darkabout the operation and performance of our devices, in-cluding how (or whether) they protect the informationwe entrust to them, and with whom they share it. The ab-sence of easy, device-local access to the traffic of our mo-bile phones presents a fundamental impediment to im-proving this state of affairs. To develop detailed visibil-ity, we devise Haystack, a system for unobtrusive andcomprehensive monitoring of network communicationson mobile phones, entirely from user-space. Haystackcorrelates disparate contextual information such as appidentifiers and radio state with specific traffic flows des-tined to remote services, even if encrypted. Haystack fa-cilitates user-friendly, large-scale deployment of mobiletraffic measurements and services to illuminate mobileapp performance, privacy and security. We discuss thedesign of Haystack and demonstrate its feasibility withan implementation that provides 26–55 Mbps through-put with less than 5% CPU overhead. Our system andresults highlight the potential for client-side traffic anal-ysis to help understand the mobile ecosystem at scale.

1 Introduction

Mobile phones now offer users capabilities that rivalthose of general purpose computers. However, despitethe myriad applications they support, mobile systemsremain notoriously opaque. Mobile operating systemstightly control access to system resources and traffic.While this helps prevent compromised or malicious appsfrom wreaking havoc, these security measures pose hur-dles for understanding the performance and behavior ofmobile apps.

The opacity of mobile systems to monitoring presentstwo serious drawbacks. First, it makes it very hard forresearchers to understand how mobile systems perform

in the wild, requiring them to rely upon instrumentingindividual phones to run experiments, or privileged ac-cess to ISPs to obtain large-scale traffic traces. Second,it opens the door for potentially unchecked exfiltrationof personal data by applications, especially since we relyupon these devices for an increasing number of tasks andentrust them with increasingly sensitive data.

Gaining insight into the internal activities of mobilephones currently proves difficult even for experts. Var-ious techniques have provided insight into the generalcontours of phone operation—from capturing packets onrooted phones [13] and within ISPs [15, 29] to redirect-ing traffic through instrumented VPNs [26] to static anddynamic analysis [11, 23] of apps. However, none ofthe currently available methodologies provides a com-prehensive view of mobile platform operation and net-work interaction both at scale and in the wild.

In this paper we propose Haystack, a handset-, traffic-,and user-centric platform for Android devices that pro-vides high-fidelity information about mobile phone op-eration and network traffic. Our solution combines threekey approaches. First, Haystack runs completely in user-space without requiring root permissions, capturing usertraffic locally on the device. To do so, Haystack takes ad-vantage of Android’s VPN permission1 to route transmit-ted packets through a process running in user-space. Thisprocess implements a simplified layer-3/4 network stack,conducting traffic analysis while relaying flow contentsto the destination via regular sockets. Second, by run-ning locally on the phone Haystack can directly observecrucial app context (e.g., determining the identity of theparticular app generating the traffic), device status (e.g.,location, screen, and radio state), user-related informa-tion (e.g., accounts, messages, recent calls, and contactlists), and the network traffic associated with user activ-ities. Third, to identify privacy leaks more effectively,Haystack optionally intercepts encrypted traffic via a lo-

1The recent iOS 9 release provides similar VPN capabilities.

arX

iv:1

510.

0141

9v1

[cs

.NI]

6 O

ct 2

015

Page 2: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

cal TLS proxy. The resulting combination of networktraffic, app activity, and detailed contextual metadata al-lows Haystack to provide unprecedented visibility forcharacterizing mobile traffic and performance, assessingapp security, and detecting privacy leaks.

Haystack’s contributions are three-fold. First, we in-troduce an architecture that enables fully device-local,context-aware traffic inspection on commodity mobiledevices using a standard app distributable via the usualapp stores. Second, we demonstrate that the approachcan deliver sufficient performance, with a low resourcefootprint, to operate in an ongoing fashion with minimalimpact on user activity. Third, we demonstrate imme-diate benefits from Haystack’s capabilities by perform-ing a privacy leakage analysis, identifying the types andrecipients of sensitive information that apps leak. Thecombination of ease of installation through a standardmarketplace, low performance overhead, and immediatebenefit for users gives Haystack the potential to scale tothousands of users, something that has been historicallydifficult to achieve.

Our results counter the assumption that client-sidetraffic analysis would be prohibitively resource inten-sive on mobile devices [22]. We demonstrate that user-space packet forwarding is feasible on Android with3–9% power overhead and less than 5% CPU overhead.We stress-test Haystack using network-intensive applica-tions and find that it achieves throughput well above thatrequired for modern mobile applications (26–55 Mbps).To demonstrate Haystack’s utility to users, we implementa detector for privacy-sensitive information exfiltrated byinstalled apps and report on the detected leaks.

We have made the Haystack app available on GooglePlay, currently available on request, but soon pub-licly [18]. We also plan to open-source the Haystackcodebase and release the data that the deployments col-lect to foster adoption of the platform by the commu-nity.2

We motivate the pressing need for Haystack in § 2 andpresent its design in § 3. We evaluate Haystack exten-sively both in lab settings (§ 4) and “in the wild” amongsta small group of volunteers (§ 5). We discuss potentialapplications of Haystack (§ 6) before summarizing ourfindings (§ 7).

2 Background and Related Work

Previous studies have investigated various techniques forunderstanding performance and privacy risks of mobileapps. The inability to measure traffic in user-space onthe device means that these techniques often grapple with

2We note that UC Berkeley’s IRB considered our current data col-lection process as not-human-subjects research.

issues of scalability (e.g., if they require custom OSes, ormanual interaction with apps) and correlating networklayer observations with device context (e.g., which appsare running). We divide these previous approaches intothree categories: passive trace analysis, static analysis,and dynamic analysis. The first category involves passiveanalysis of app network traffic, whereas the latter twofocus on active, targeted investigation on a per app basis.Passive trace analysis. In the absence of instrumenta-tion on user devices, some studies have investigated thebehavior of mobile apps by observing their network traf-fic, either at a large ISP [15, 29] or by enticing users toleverage a proxy deployed by the researchers [22, 26].These approaches have the advantage of potential scale(e.g., ∼3M users in a large ISP) but trade off device con-text such as knowing which application generated thetraffic. AntMonitor [22] works to alleviate this issue bypairing a remote VPN proxy with client-side software,forwarding user traffic to a remote VPN server responsi-ble for packet capture. With Haystack we demonstratethat with careful system design it is actually possibleto perform VPN proxying on client devices without ex-hausting system resources (e.g., power, CPU).Dynamic analysis which exercises applications’ func-tionality and monitors app activity (e.g., leakage ofprivacy-sensitive data) has been a very popular approach.The researcher has control over the device running theapp and can monitor device context (e.g., location, cur-rent apps, etc.). Dynamic analysis either monitors net-work traffic (e.g., by using a proxy [21] or placing theapp in a VM [31]) or employs instrumented OSes to trackthe flow of sensitive information [11, 17]. A key chal-lenge of this approach is scale, in terms of the number ofapps tested, and the representativeness and coverage ofapp behaviors. Manual analysis (e.g., [11, 17, 21]) canprovide more representative usage of the app but remainslimited in terms of the number of apps that can undergotesting. In contrast, fuzzer tools like Monkey [2] auto-mate the testing process to increase scale, but often misscommon functionality (e.g., passing log-in screens).Static analysis involves analysis of the app code(e.g., decompiling binaries and producing control flowgraphs [4,10]) and properties (e.g., third-party library useand requested permissions [8, 23]). More classic staticanalysis techniques such as symbolic execution, madefeasible by constraining analysis to execution paths withpotential data leakage, have also been investigated in thisspace [30]. While these approaches provide good scale,analyzing over 1,000 apps in most cases, they do not nec-essarily reflect code paths that would be executed in prac-tice.

Related to the above techniques for characterizing in-formation leakage, there has been active investigation ofSSL on mobile and using device IDs to obtain access to

2

Page 3: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

raw_packet Java Sockets

TLS Proxy

TLS interception

TLS stream

Intelligence Service !

Off-path traffic Analysis

flowudpflowtcp

decrypted flow

Internet

Aho-Corasick Parsers

Location Contacts…

SSLSockets

Forwarder

Non-encrypted TrafficEncrypted TrafficOff-path channels

tun Default Gateway

Apps

Figure 1: Haystack architecture design, highlighting the differentsystem components and the data forwarding process for an outgoingpacket created by a local application. The solid lines represent the ac-tual forwarding path for traffic generated by mobile apps even if en-crypted, whereas the dashed lines represent the off-line path used forprivacy and performance analysis.

users’ analytics profiles [8]. Fahl et al. showed that mostmobile apps do not encrypt the data they send to onlineservers [12] and Georgiev et al. reveal critical vulnera-bilities in how mobile devices validate SSL [14].Our approach. Haystack aims to avoid the pitfalls ofexisting approaches by enabling traffic analysis on mo-bile devices locally and in user space. This allows forwider deployment than approaches that require users toinstall custom OSes (e.g., [11]) and allows us to observeinformation leakage as triggered by real user behavior,as opposed to potential leakage identified via static anal-ysis. Finally, by performing monitoring on the device,Haystack can correlate traffic data with application con-text, for example in order to provide more accurate iden-tification of applications that leak privacy-sensitive data.

3 The Haystack System

Our design for Haystack reflects some basic require-ments. First, we aim to enable low-overhead and high-throughput real-time traffic analysis, keeping the sys-tem’s operation as transparent to the user as possible.Second, we desire an architecture that runs in user-space without requiring additional customizations on thephone, such as rooting the device. Third, we aim to sup-port in-depth traffic analysis across the protocol stack, al-lowing us to add protocol analyzers over time as needed.

In the remainder of this section we present our imple-mentation of Haystack in form of an Android app. Oursolution consists of three main components, as shown inFigure 1: the Forwarder, which handles packet forward-ing and performs flow reconstruction, the TLS proxy,which intercepts TLS traffic, and the Intelligence Ser-vice, which handles payload and traffic analysis off-path,but in near real-time. The Haystack codebase (exclud-

idle cycles = 0;while true do

/* Start tun operations */;read cyclestun = 0;while read cyclestun < max readtun do

read packet of length len from tun;if len == 0 then

break; /* Skip to NIO operations */

elseprocess tun packet;idle cycles = 0;read cyclestun ++;

endend

/* Start NIO operations */;read cyclesnio = 0;while read cyclesnio < max readnio do

check for NIO event;if no NIO event then

idle cycles ++;if idle cycles == max idle cycles then

sleep for idle sleep milliseconds;break; /* Skip to tun operations */

elsebreak; /* Skip to tun operations */

endelse

process NIO event;idle cycles = 0;read cyclesnio ++;

endend

endAlgorithm 1: Haystack’s traffic-forwarding logic. The process at-tempts to balance reading from tun and event-based Java sockets(NIO). With both interfaces idle, the system sleeps to reduce powerusage.

ing external libraries we leverage for TLS interception,IP/TCP/UDP packet handling, and geolocation) amountsto 15,000 lines of code, excluding XML GUI layouts.

We begin with a description of Android’s VPN API,as it informs Haystack’s overall design. We then discussHaystack’s components in turn, and conclude with a dis-cussion of basic limitations of the platform.

3.1 User-space VPNs in Android

The Android security model makes it possible for appsto inspect network traffic using one of two methods. Thefirst applies to rooted phones: in this mode, any app canrun commands in privileged mode. For example, an appcan run tcpdump on the default network interface to ac-cess all traffic. The second requires a special AndroidAPI. Since 2011 (version 4.0), Android has offered aVPN API that enables developers to create a virtual tuninterface and direct all network traffic to a user-spaceprocess. This technique is meant for use by VPN clientapps to forward arriving packets over a secure connectionto a remote VPN server [3]. To enable this functionality,

3

Page 4: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

Name Meaning Effect

max readtun Maximum number ofpackets to read from tun

during one iteration.

Prevents NIOoperations fromstarving.

max readnio Maximum number offlows to read from NIOduring one iteration.

Prevents tun

operations fromstarving.

max idle cycles Number of idle tun andNIO iterations before go-ing to sleep.

Polls for a shortwhile after a pe-riod of activity andprevents aggressivesleeping.

idle sleep Duration of sleep in mil-liseconds.

Suspends pollingwhen there is noactivity.

Table 1: The parameters we use to control forwarding effectively.

apps need to request the BIND VPN SERVICE permission.At execution time, users get a warning that an app maymonitor their traffic and need to allow the app to use thepermission. Once enabled, the app can access networktraffic in user space and without root privileges.

We note that this feature offers a way around An-droid’s sand-boxing model. Any application granted thispermission gets access to all network traffic. Users haveto explicitly enable this access by checking an “I trustthis application” box and Android reminds the user ofVPN presence via its notification bar. Users unfamiliarwith VPN operation might not realize the privacy and se-curity implications of these notifications.

Haystack’s use of the VPN API Haystack leveragesthe VPN interface in order to intercept and analyze traf-fic. Instead of relaying packets to a remote VPN server,Haystack processes the traffic locally and forwards it onto its intended destination. To this end, Haystack uses theForwarder module to capture and relay traffic betweenthe tun interface and the default system gateway. Akey challenge for Haystack is the fact that non-privilegedapps may only use regular Java sockets (as opposed toraw sockets) to communicate with remote entities. Thismeans that the Forwarder module has to bridge packet-level communication on the tun side and flow-level in-teraction with the outside network as we describe next.

3.2 The Forwarder

The Forwarder acts as the intermediary between tun andthe network, and makes it possible for Haystack to trans-parently forward traffic. We now describe the forwardingmechanism and steps we took to implement it efficiently.

Maintaining flow state The Forwarder receives rawIP packets from tun. Ideally, it would just forward theraw packet onto the network. As we have noted pre-viously, doing so would require root privileges. TheForwarder therefore extracts the payload from the rawpacket and, if appropriate, sends it to its intended desti-nation through a regular Java socket. As a consequence,the Forwarder must parse the packet to obtain meaning-ful information about the flow and maintain state (i.e.,a transmission control block, updated by the observedIP and TCP/UDP headers) for tun packets destined to agiven Java socket. This allows the Forwarder to updateand recreate this state before forwarding packets froma given Java socket back to the tun interface. In otherwords, the Forwarder performs the role of a layer 3/layer4 translator, including a simplified TCP stack.

Handling UDP and TCP headers The nature of staterequired by the Forwarder differs markedly for UDP andTCP. A simple flow-to-socket mapping suffices for UDPbecause of its connectionless nature. Header reconstruc-tion remains easy, even more so in IPv4, as IPv4 doesnot enforce UDP checksums. (We can assume that thetun interface operates without loss, as it is a local de-vice). TCP state, however, proves more complex: weneed to track the TCP state machine and maintain se-quence and acknowledgment numbers based on the flow.This is complicated by the fact that we deal with Javasockets, which only expose the notion of streams ratherthan packets coming from the network, and we do nothave any view into the TCP headers, including the flagsand window information.

We therefore use a basic TCP header for our communi-cation with tun. When we read a SYN packet from tun,we create a new Java socket, connect to the target, createadditional state (the acknowledgment number), store thisstate, and send a SYN/ACK to tun when the OS estab-lishes the socket. Since we deal with a stream/datagramsocket on the network side, we attempt to read fixed-sized packets from the network (at a maximum of 1450bytes); we cannot recreate the exact packet flow thatwe get from the network onto tun. Since we cannotsee TCP flags on the socket, we do not know when theserver sends us a packet with a PSH flag. This can proveproblematic because the client can wait an indeterminateamount of time to get the data unless the TCP stack getsa packet with the PSH flag set. To avoid this scenario, weset the PSH flag for all packets we write to tun. In future,we plan to test better strategies such as setting the PSHflag for the last packet we read from the buffer beforeit gets empty. Likewise, the Forwarder handles connec-tion shutdown when it sees an explicit FIN requests fromtun, or it infers the end of a connection on the socketside when read or write operation on the socket fails, by

4

Page 5: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

exchanging FIN/ACK messages with tun.

Overhead of keeping state Our approach requires theForwarder to create a socket for each TCP/UDP flow andmaintain connection state for it. Because Haystack is aregular non-privileged app, it must obey a hard limit onthe number of file descriptors (including sockets) it cankeep active at any given time. Exhausting the availablefile descriptors will prevent Haystack from handling newconnections, which in turn disrupts connectivity for ap-plications. Therefore we have to close sockets and flushstate periodically. For TCP connections, this is straight-forward, as we know (or can infer) when the connectionends. Given UDP’s connectionless nature we set a time-out of 15 seconds of idle time for its flows, after whichHaystack closes the socket. Longer UDP timeouts can beproblematic when applications generate frequent multi-cast traffic (we discuss this further in § 5), but reducethe overhead of creating/purging state. We are explor-ing techniques that adapt to the nature of the flows; inparticular, we plan to multiplex certain kinds of traffic,such as DNS, over a single UDP socket. Even with in-telligent handling, the limit on file descriptors remainsa limitation in theory—Haystack will be depleted of filedescriptors if an application (or set of applications) opena large number of connections concurrently. As we willsee in § 5, we manage to avoid this problem in practice.

Implementing the packet forwarding mechanism ef-ficiently The Forwarder must balance application per-formance with conserving battery power on the device.This task is made challenging by the fact that the tun in-terface does not expose an event-based API, thus requir-ing us to implement a polling scheme which checks thetun and Java sockets (implemented using non-blockingNIO sockets [25]) for data and forwarding the data fromone interface to the other.

Algorithm 1 gives pseudocode for the polling proce-dure. It reads up to max readtun packets from tun orup to max readnio sockets from the NIO (socket) inter-face before switching to the other interface. This pre-vents either operation from starving. Each read oper-ation in tun potentially becomes a write operation forsockets, and vice versa. Pure TCP ACKs from tun forma special case. We discard these since the Java sockethandles ACKs transparently. tun writes complete veryquickly and socket writes do not block, so we performthem immediately upon reading a packet from the cur-rent interface. If no packets or flows are read for a to-tal of max idle cycles consecutive iterations of the mainwhile loop, the Forwarder will sleep for idle sleep mil-liseconds. This reduces power usage when no data trans-fer occurs, at the cost of slightly increased traffic latency.

We note that tun reads yield packets, whereas NIOsocket reads happen on streams. Therefore, each NIOread event potentially yields multiple packets, whereasreading from the tun interface returns a single packet.This makes the Forwarder more efficient for downstreamtraffic because the overhead of polling is amortized overpotentially multiple packets. While this could impacttun operations if the buffers grow large, we have not ob-served this in practice. We note that even if this situationarises we can easily resolve it by setting a volume limiton individual NIO read operations.

Haystack suspends polling during periods of inactivityto conserve battery and minimize CPU overhead. How-ever, this resource conservation implies a latency trade-off, as packets that arrive during this idle time must waitfor the polling to resume before being forwarded. Weconsider the trade-off between resource conservation andperformance in depth in § 4.2. We summarize the pa-rameters that manage the poll and sleep cycles, as wellas their impact on performance, in Table 1. We note thatmax readtun and max readnio do not have any impact onperformance over a wide range of values. However, aswe show in § 4, max idle cycles and idle sleep have asignificant impact on overhead and performance.

Other considerations The VPN service (and thereforethe Forwarder) gets disrupted when users roam betweendifferent networks (e.g., moving between 3G and WiFi,or between different WiFi networks) or when a networkblack-out occurs. Haystack identifies such events andtries to reconnect seamlessly. Similarly, phone calls dis-able the data network interfaces, thus stopping the VPNservice. While currently this disables Haystack, we areworking on using Android APIs to identify when thecalls complete so we can transparently restart the VPN.The Forwarder also provides IPv6 support, except for ex-tended headers. We have not noticed any performanceissues on IPv6 flows due to this limitation. The VPNinterface does not intercept DNS traffic by default; how-ever, we configure a default DNS server so that we canalso intercept DNS queries.

3.3 TLS Proxy

A key impediment for inspecting Web traffic is the in-creasing use of encryption technology, particularly TLS.Haystack addresses this challenge by employing a trans-parent man-in-the-middle (MITM) proxy for TLS traf-fic [7]. At install time Haystack prompts the user to in-stall a self-signed Haystack CA certificate in the user CAcertificate store (which they may accept or decline). Wecustomize the message shown to users at this time, toexplain why Haystack intercepts encrypted traffic.

5

Page 6: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

Identifying and initiating proxied TLS flows WhileHaystack runs, the Forwarder monitors for TCP streamsbeginning with a TLS “Client Hello” message andforwards these flows, along with flow-tuple meta-information the proxy requires in order to connect to theserver (e.g., IP address, port, SNI), to the TLS proxy.

The proxy uses this information to connect to the re-mote host and reports back to the Forwarder whetherthe connection was successful or not. If successful, theproxy fetches the TLS certificate from the remote hostand generates a new certificate and key-pair signed withthe private key of the Haystack CA certificate. It thenuses this certificate to establish a TLS connection be-tween the proxy and client application. Upon successfulestablishment of the client and server connections, theproxy decrypts incoming traffic from each of the parties,re-encrypts it with the appropriate key, and relays it tothe other party. It also sends the decrypted contents tothe Intelligence Service in the process.

Exceptions in TLS interception Our TLS proxy willnot be able to intercept the flow if the client applicationuses TLS extensions that are not supported by Haystack,bundles its own trust store, or implements certificate pin-ning. The same occurs if the server expects to see certainTLS extensions not supported by Haystack in “ClientHello” or if it performs certificate-based client authen-tication. To allow such applications to bypass the proxy,we add apps with failed TLS handshakes to a whitelistwhich bypasses the TLS proxy for a period of five min-utes. Experiences with our pilot study indicate that appsrecover gracefully from TLS failures. After five minuteswe remove the app from the whitelist to account for tran-sient failures in the handshake process. While we cannotdecrypt all or parts of the traffic of these applications, wecan still record which apps take these security measuresand potentially communicate more securely.

Security considerations Since using one shared cer-tificate and key-pair combination across all installs ex-poses users to man-in-the-middle attacks, Haystack gen-erates a unique certificate and key-pair for each new in-stallation of the app. Additionally, Haystack saves theprivate key to its private storage to prevent other applica-tions from accessing it. While these precautions still per-mit malicious applications with root access to retrieve thekey, such apps can already tap into the user’s encryptedtraffic without using Haystack’s CA certificate (e.g. bysurreptitiously injecting their own CA certificate into thesystem’s trust store).

3.4 Intelligence ServiceNetwork flows pass from the Forwarder and TLS proxy(after decryption) to the Intelligence Service (IS). IS aug-ments these flows with contextual information, and per-forms analysis of app and protocol performance.

Enabling off-path traffic analysis The operations thatthe IS performs could negatively affect the user experi-ence if done in-path: tracking flow and packet statistics,and parsing protocol content can consume valuable CPUcycles. We therefore implement IS as an off-path com-ponent that performs those actions outside of the for-warding path. Unfortunately, low-latency communica-tion between Android services can prove tricky to re-alize, especially in multi-threaded systems. In our im-plementation we use Java’s thread-safe queues for com-munication. In traffic-intensive scenarios these blockingqueues can become a bottleneck as a result of continuousreads and writes between the VPN Service, TLS Proxy,and Intelligence Service [19]. According to third-partybenchmarks, alternative concurrent queue implementa-tions provide similar performance [9]. We evaluate theoverhead of running the IS in § 4.5 and show that theoverhead does not affect even resource-heavy applica-tions. However, we are currently exploring using othertechniques such as sockets to reduce this overhead andimprove performance even further.

Application and entity mapping The IS receives TCPand UDP data from the Forwarder and the TLS proxy. Itmaps these streams to applications using a two-step pro-cess: it extracts the PID of the process that generated theflow from the system proc directory, and then maps thisPID to an app name using Android’s Package ManagerAPI. Compared to network-based studies which rely onthe HTTP User-Agent or destination IP address, this al-lows us to obtain highly accurate flow-to-app mappings.Since reading the PID and mapping applications requiresfile-system access for each new flow, we cache recentlyread results to minimize overhead.

DNS analysis IS conducts protocol analysis of the re-ceived traffic. Due to lack of space we here focus onone such protocol, DNS. IS tracks DNS transactions toextract CNAME records, which allows us to map flowsto target domains rather than just IP addresses. Thisgives us the ability to distinguish apps sending data totheir own backend as opposed to third-party ad/analyticsservices, even if both reside in the same cloud serviceprovider [6].

Detecting privacy-sensitive information leakage Asan example of protocol-independent app-layer traffic

6

Page 7: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

analysis, we have implemented a private-informationleakage detector. To this end, IS first obtains a list of po-tentially sensitive data from the device, such as hardwareand network information, OS fingerprints, historical ge-ographic location, as well as contact and account details.Using the Aho-Corasick algorithm [1] it then scans eachflow for the presence of strings in the list. We groupthe possible content patterns into sets based on their type(e.g., contacts, accounts) and frequency with which theywill update (e.g., device location will change frequently,whereas the IMEI will not), and generate state machinesfor each set. This allows us to regenerate state machinesfor only those sets whose information has changed. Dur-ing scanning, we take into account elementary encod-ings, such as Base64. We hasten to point out that ourgoal here is not to construct a flawless leak detector op-erating purely at the network level; rather, we use thisscenario as an example for meaningful content analysis.

3.5 Limitations

Haystack currently has a number of limitations, some ofwhich are fundamental and result from the limitationsof user-space apps in Android. The most serious limi-tation to Android apps running in user-space is the stan-dard UNIX file descriptor limit. For normal apps thisdoes not present a serious limitation, but for a VPN appthat opens a new socket for every flow it can become aproblem unless one handles socket resources prudently.Haystack aggressively closes sockets as soon as it fin-ishes handling TCP teardowns and UDP flow timeouts.While this works well for normal operations, we haveseen that some apps send high-rate Bonjour and multicasttraffic when connected to WiFi networks. This quicklyexhausts the file descriptor quota. We currently sidestepthis problem by simply ignoring such traffic. In futurewe plan to multiplex sockets in this scenario.

Another limitation stems from the ability to createonly TCP and UDP sockets via Android’s APIs, thus ex-cluding protocols such as ICMP. On Android, only oneapplication can use the VPN interface at a given time,and therefore Haystack will not run if another app is ac-tively using the interface. Finally, some vendor-lockedphones prevent apps from writing to the trusted rootstore. In such cases, we cannot intercept TLS traffic.

4 Performance Evaluation

Haystack aims to enable real-time traffic analysis in userspace on mobile phones without negatively affecting per-formance (e.g., latency, throughput), while managingsystem resources (e.g., CPU, battery life) efficiently. Weevaluate to what extent we achieve this goal in practice.

4.1 Testbed and Measurement Apparatus

To evaluate Haystack in a controlled setting, we set upa testbed with a Nexus 5 phone and a Raspberry Pi con-nected to a dedicated wireless access point. The Nexus5 is connected over a 5 GHz 802.11n link, while thePi is connected over a gigabit Ethernet link. We min-imize background traffic on the phone by only includ-ing the minimal set of pre-installed apps and not sign-ing into Google on the phone. We measure the latencyof Haystack using simple UDP and TCP echo packetswith inter-packet time varying randomly between 5 and25 ms. For non-TLS throughput tests, we use a custom-built speed-test that opens 3 parallel connections to aserver for 15 seconds in order to saturate the link. We testuplink and downlink separately. For profiling TLS estab-lishment latency and downlink speed-test, we cannot useour speedtest, as it does not employ a TLS session. In-stead, we download a 1B and 20M object, respectively,over HTTPS from an Apache 2 web server with a self-signed x.509 certificate. We repeat each test 25 times.

While our testbed allows us to explore the designspace in terms of many parameters, Android’s VPN secu-rity model precludes full automation of our experiments.While we are able to write an app to generate traffic au-tomatically, user interaction is required to enable/disablethe tun interface. Thus we focus on the impact ofmax idle cycles and idle sleep and fix max readtun andmax readnio to 100 which favors downlink traffic.

4.2 CPU and Power Overhead

We investigate the impact of max idle cycles andidle sleep on resource usage, in particular CPU load.CPU usage impacts interactivity of foreground apps, andalso serves as a proxy for battery usage. We then usea power meter to directly analyze Haystack’s impact onbattery life. We focus on Haystack’s performance bothin the idle state and with heavy usage, streaming a high-resolution video. Optimizing Haystack’s CPU overheadduring idle periods is critical to maximizing battery life,as phones remain idle the majority of time [5].

CPU load during idle periods We measure the baseCPU usage of the Nexus 5 to be 2% in the absence ofHaystack, when the system is idle: with its screen off,with normal background activity from installed apps, andpush notifications turned on. Figure 2 shows the impactof max idle cycles and idle sleep on CPU usage whenenabling Haystack. We find that idle sleep has the mostsignificant effect on CPU load; this does not surprise be-cause it determines how long the app sleeps and thereforedoes not consume CPU. With idle sleep of 1 ms CPUload varies between 45% and 55% for different values of

7

Page 8: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

0

10

20

30

40

50

0 50 100 150idle_sleep (ms)

CP

U (

%)

max_idle_cycles

1

10

100

200

Figure 2: Haystack’s CPU overhead. The horizontal line indicates theaggregated average CPU load of all apps running on the backgroundfor reference.

10

20

0 50 100 150Execution time (s)

CP

U (

%)

Process HayStack Youtube

Figure 3: YouTube and Haystack CPU load while playing a 3-minutehigh-definition (1080p) video.

max idle cycles with the Forwarder polling the interfacesat a high frequency. CPU usage drops sharply as we in-crease idle sleep, to 10.5% and 4.6% with idle sleep at10 ms and 25 ms, respectively. In contrast to idle sleep,max idle cycles shows little influence on CPU overhead,particularly at idle sleep values greater than 10 ms. Thisis because max idle cycles is measured in loop cycles (cf.Alg. 1) which take a small fraction of 1 ms each. Foridle sleep of 100 ms and max idle cycles of 10 or 100the overhead of Haystack is negligible, with the CPU us-age close to the base CPU usage (horizontal line in Fig-ure 2). We consider an idle sleep value of 100 ideal foroperating during idle periods.

CPU load during interactive periods We next pro-file Haystack’s overhead under heavy load. To do sowe run Haystack and stream a 1080p YouTube video.This stresses packet forwarding, CPU usage, and theTLS Proxy, since YouTube delivers the video over TLS.Figure 3 shows the result comparing CPU usage of theYouTube process and the Haystack process over time.Aside from a short period at the start of the videoHaystack’s CPU usage remains lower than that of theYouTube application. Crucially, we do not observe de-lay, rebuffering events, or noticeable change in resolu-tion during the video replay, suggesting that Haystack’sperformance can keep up with demanding applications.

Power consumption We use the Monsoon PowerMonitor [24] to directly measure the power consumed

Test Case Mean Power(mW) / stddev IncreaseIdle 1,089.6 / 125.9 +3.1%Idle (Haystack) 1,123.8 / 150.4YouTube 1,755.3 / 35.5 +9.1%YouTube (Haystack) 1,914.4 / 16.1

Table 2: Power consumption of Haystack when max idle cycles is 100and idle sleep is 1 ms. The percentage indicates the increase whenrunning Haystack.

packet1

tproctbuff

packet2packet1

idle_cycles

tproc

Timeidle_sleep (ms)

cpu active

cpu inactive

idle_sleep (ms)

outgoing packet

incoming packet

tproc

tbuff Packet Buffering Time

Packet Processing Time

Figure 4: Latency added by idle sleep and max idle cycles on packetsarriving during periods of activity, or inactivity.

by Haystack. This experiment required a device with re-movable battery, so we use a BLU Studio X Plus phonerunning Android 5.0.2 instead of the Nexus 5. We set thevoltage to 3.8V and measure power consumption whenthe phone is idle (with and without the screen on), andwhen streaming a 33-second high-definition YouTubevideo. We repeat each test ten times to account for vari-ation in network conditions.

Table 2 summarizes the results of our power measure-ments with max idle cycles at 100 and idle sleep at 1 ms.This configuration represents the worst-case (cf. Fig-ure 2) as Haystack sleeps for only 1 ms before polling theinterfaces again. Unfortunately, due to hardware limita-tions, we could not measure Haystack’s power consump-tion with the screen off. During idle periods with thescreen active, the overhead of Haystack proves marginal,with only 3% increase in power consumption. The over-head increases to 9% while streaming a YouTube video.

4.3 Latency Overhead

Haystack suspends polling during periods of inactivity toconserve battery and minimize CPU overhead. However,suspending polling also potentially increases latency forpackets if they arrive (at tun or at the socket) when theloop is suspended, as illustrated in Figure 4. In the fig-ure, the packet that arrives during the first idle sleep pe-riod endures the remainder of the idle period (tbu f f ), inaddition to the forwarding time (tproc), which includeslooking up relevant header state and translating betweenthe layer 3 tun interface and layer 4 NIO sockets. How-ever, the packet that arrives when polling is active doesnot experience the idle period overhead.

8

Page 9: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

0

50

100

150

0 50 100 150 200idle_sleep (ms)

UD

P R

TT

(m

s)

max_idle_cycles 1 10 100 200

(a) UDP latency

0

100

200

300

400

0 50 100 150 200idle_sleep (ms)

TC

P C

onne

ctio

n T

ime

(ms)

max_idle_cycles 1 10 100 200

(b) TCP connection time

10

20

30

40

50

Downlink Downlinkwith Intel. Service

and TLS Proxy

Uplink Uplinkwith Intel Serviceand TLS Proxy

Thr

ough

put (

Mbp

s) idle_sleep

10

100

(c) Effect of Intelligence Service and TLSProxy on non-TLS traffic.

Figure 5: Haystack performance for different idle sleep and max idle cycles configurations. For the throughput evaluation, we fix max idle cyclesto 100 cycles, also showing the impact of enabling the Intelligence Service. The throughput for this link is 73 and 83 Mbps uplink and downlink,respectively.

We now analyze the latency incurred by packets whenrunning Haystack. Specifically we focus on the impact ofmax idle cycles and idle sleep and the trade-off betweenlatency and CPU overhead. Figure 5(a) shows the resultof our experiments for UDP. When max idle cycles =1, latency closely follows idle sleep. This is becauseHaystack aggressive sleeping renders it more likely forpackets to arrive when the system is idle, and thereforeto have to wait up to idle sleep milliseconds before beingprocessed. With max idle cycles = 100 and idle sleep =100 ms, we see about 60 ms delay. Reducing idle sleepto 10 ms while keeping max idle cycles at 100 reduceslatency to as low as 3.4 ms.

We see similar patterns for TCP connections. In Fig-ure 5(b) we plot connection establishment times for theTCP echo client and server. As expected, high values ofmax idle cycles and coupled with low idle sleep resultsin quicker connection establishment. It is worth notingthat when the RTT of the link drops below the time ittakes to reach max idle cycles, Haystack processes allpackets in the TCP handshake without the Forwarder go-ing into idle state.

Our results suggest that it is possible to adapt Haystackdynamically to strike a balance between performanceand power. The phone state gives us a straightforwardway of determining whether we should be in “perfor-mance” mode or “low-power” mode: if the screen is on,it is likely that the user is interacting with applications,and we can switch to performance mode, while if thescreen is off, it is likely that the only traffic is generatedby background apps, which are more tolerant to delay.There are of course exceptions such as streaming musicapps, which we will need to consider; however, in gen-eral, Haystack can adapt automatically to the needs ofthe device and the user.

Finally, we consider the latency incurred by a packetduring processing and forwarding (tproc). To get a senseof how the latter affect performance, we evaluate tprocwhile running our speed-test app. Table 3 shows the re-sults of our speed-test for TCP and UDP connections.

Downlink Uplink New FlowTCP tproc (µs) 141.6±0.5 275.6±3.5 5,647.8±998.5UDP tproc (µs) 76.6±2.2 230.8±4.8 2,980.8±224.9

Table 3: Mean processing time (tproc) and SEM for Haystack’s for-warding operations for TCP and UDP flows. The first packet of a newflow requires a higher processing time.

Processing times for established flows are 141 µs forTCP and 76 µs for UDP, indicating that the packet for-warding should not be a bottleneck for Haystack’s per-formance. The processing times for new connections arelarger, especially for TCP, because of the overhead of ini-tiating state for the connection.

4.4 Throughput of Haystack

We now investigate the throughput of Haystack. We ini-tially consider only non-TLS traffic to understand themaximum throughput the system can achieve. We usethe our speed-test app to measure the throughput for TCPand UDP flows with idle sleep = {10 ms,100 ms} andmax idle cycles = 100. This setting provides us with agood compromise between CPU usage and latency.

Figure 5(c) shows the maximum throughput achievedby Haystack’s Forwarder. Haystack can provide up to17.2 and 54.9 Mbps uplink and downlink throughput, re-spectively. As expected, when idle sleep increases thethroughput decreases, as more packets arrive with theForwarder in idle state, thus incurring tbu f f er (cf. Fig-ure 4). Haystack also has a bias towards downstreamtraffic, which stems from two factors. First, as we dis-cussed in § 3.2, the NIO read operation reads socketstreams which may potentially contain multiple packetswhereas the tun interface reads only a single packet.Second, the operations required for upstream packetstproc are more computationally expensive (see Table 3).We plan to investigate how we can adapt the max readnioand max readtun parameters to achieve more balancedthroughput in future work. We note that the performance

9

Page 10: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

we report is still in excess of what is required for modernmobile apps.

4.5 Intelligence Service Overhead

Although the IS operates off-path, it is CPU intensiveand can inflict significant overhead on traffic. Figure 5(c)shows the IS’s impact on throughput. In the worst case,it still provides 23.3 Mbps downstream and 10.5 Mbpsupstream throughput, enough to satisfy traffic-intensiveapps like video-streaming. Even when stress-testing thesystem with our speed-test, the maximum queueing timeendured by packets before being processed by the ISdoes not exceed 650 ms. The maximum time for string-parsing with Aho-Corasick is 167 ms. This worst-casescenario arises when the queues contain a backlog ofat least 1,000 packets. Even under such circumstances,the total processing time remains low enough to providefeedback to users about their traffic in less than a second,e.g., exfiltrated private information.

We believe that there remains significant potential forimproving the overhead imposed by the IS. In particular,we plan to investigate better means of communicatingbetween the Forwarder and the IS (e.g., via sockets orIPC) to make it more efficient than the thread-safe queuebetween the two systems we currently employ.

4.6 TLS Performance in Haystack

Figure 6 summarizes the overhead of the TLS proxy fordifferent configurations. We first consider the baselineoverhead of Haystack without the TLS proxy enabled onTLS connection establishment times (Figure 6(a)). Withan idle sleep of 10 ms the overhead on TLS connectionestablishment is 218 ms. Increasing idle sleep to 100 mshas a large effect, doubling the TLS establishment time(2.14x). Using the TLS proxy increases establishmenttime by a further 1.4x with idle sleep at 10 ms, and 1.08xwith idle sleep at 100 ms.

The overhead of the TLS proxy on throughput turnsout to be milder. (Figure 6(b)). Compared to not runningHaystack at all, the overhead of the TLS proxy is 26–29%for idle sleep = 10 ms and 100 ms, respectively. De-spite this decrease in throughput, overall throughput withthe TLS proxy is still 26Mbps, which as we demonstratein § 4.2 is sufficient to play a 1080p YouTube video with-out impacting performance. which is sufficient for evenresource intensive mobile apps (e.g., video streaming).The impact of idle sleep is less extreme on throughputas subsequent packets bring the Forwarder out of the idlestate, thus avoiding tbu f f er for the bulk of the transfer.The fact that the TLS proxy reassembles the streams forthe Intelligence Service also helps reduce the overhead.

100

200

300

400

500

HaystackDisabled

Haystackwith Intel. Service

and TLS Proxydisabled

Haystackwith Intel. Service

and TLS Proxyenabled

TLS

Est

ablis

hmen

tT

ime

(ms)

idle_sleep 10 100 n/a

(a) Haystack’s impact on TLS session establishment time.

10

20

30

40

HaystackDisabled

Haystackwith Intel. Service

and TLS Proxydisabled

Haystackwith Intel. Service

and TLS Proxyenabled

TLS

Dow

nlin

kT

hrou

ghpu

t (M

bps)

idle_sleep 10 100 n/a

(b) Haystack’s impact on TLS download speeds.

Figure 6: Session establishment and throughput for TLS withHaystack.

4.7 The Haystack App

Thus far we have explored the design space and trade offsbetween resource usage and performance of Haystackwhile varying max idle cycles and idle sleep. Our ob-servations point to the possibility of combining these pa-rameters to adapt to different usage scenarios.

We consider two scenarios: 1) when the user is inter-acting with the device, or there is latency sensitive back-ground traffic (e.g., streaming audio), and 2) when thephone is idle with minimal network activity (e.g., pushnotifications). In the first case, minimizing latency andguaranteeing the best user experience is critical, whereasin the second case, traffic is delay tolerant [5]. We definethem as Foreground and Background modes respectively.Based on our results in the preceding sections we deter-mine that idle sleep =10 ms and max idle cycles =100gives the best trade off of performance and resource us-age for Foreground usage and idle sleep =100 ms andmax idle cycles =100 gives the best trade off duringBackground usage. Table 4 summarizes the overheadsand performance for each of these settings.

The Haystack app, currently available for alpha test-ing on Google Play [18], adapts to each scenario auto-matically for the user. In both cases, Haystack deliversenough throughput at low latency overhead to be able todrive high-performance applications such as HD videostreaming without noticeable artifacts such as rebuffer-ing. Finally, we put a lot of effort on providing a gooduser experience, even for non tech-savvy users. Figure 7

10

Page 11: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

(a) (b)

Figure 7: Haystack GUI screenshots.

shows our current GUI design.

5 Haystack in the Wild

We now investigate the performance of Haystack on real-world user traffic. We instrumented Haystack to log per-formance statistics and had 9 volunteers run the modifiedapp for an aggregate period of 5 days. Table 5 lists detailsabout this deployment.

File-descriptor management Per § 3.5, Haystackmust carefully manage its file descriptor budget whenhandling network sockets. We observed that some ap-plications generate a large number of UDP broadcast andmulticast packets (e.g., mDNS/Bonjour) in a short periodof time when connected over WiFi. Currently, Haystacksimply ignores them, because otherwise Haystack willconsume all available file descriptors. As we explain in§ 3.5, we are investigating the possibility of multiplex-ing UDP flows and adaptive UDP timeouts to releaseconnection state early. The number of file descriptorsconsumed while browsing the Web with a UDP time-out value of 15 seconds remains less than about 100, farbelow the process limit of 1024. Web browsing tendsto open more sockets than most other OS activities andapps, especially since peer-to-peer applications (e.g., Bit-Torrent) that open many connections remain rarely usedin the mobile context.

System stability In § 3.4 we described Haystack’s useof thread-safe queues for transmitting messages betweencomponents. We measure the size of these queues underreal-world load scenarios by sampling queue size statis-tics at 5-second intervals. On average the size for eachprocessing queue (for both TCP and UDP) remains be-low 1,000 messages approximately 63% of the time, well

Users Duration LeakyApps Flows TLS

FlowsTLS

Errors9 5 days 85 24.3K 15.3K 1.4K

Table 5: Summary of user study.

App Leak Destination TLS InstallsOokla IMEI crashlytics.com

Yes 50-100MSpeedtest speedtest.netPodcast serial# 192.99.4.19

No 1-5MAddict addictpodcast.com

Skype IMEI ad-x.co.uk No 500M-1Bskype.com YesTicket-master IMEI liv-tm-androidprod.

122.2o7.net No 1-5M

VenomROM/HUB

IMEI,serial# ruhdrom.net No n/a

VisualVoicemail IMEI t-mobile.com No 10-50M

Table 6: Sample of popular Apps leaking unique device identifiersexcluding Google services.

within the processing capacity of Haystack (recall § 4.5).The largest queue size we observe is 20,286 objects, dur-ing one of our speed test runs.

TLS flow disruption TLS interception succeeds incase of standard TLS usage, but as we mentioned ear-lier ( § 3.3) customized implementations can circumventthis approach. Some applications (e.g., MEGA [16] andFirefox) include their own private CA bundles insteadof relying on Android’s internal CA store to verify TLScertificates. Other applications leverage TLS extensionsHaystack does not yet support, while others use certifi-cate pinning (e.g., banking/financial applications) whichpreclude interception of their traffic. We see TLS excep-tions for less than 10% of TLS flows in our user study.

Detecting leakage of privacy-sensitive informationFinally, we study Haystack’s ability to uncover exfiltra-tion of private information. We find privacy-sensitive in-formation exfiltration ubiquitous: we saw about 20,000uploads of such items, associated with 85 unique appli-cations and services. While most of these instances re-lated to general device information such as device modeland OS build ID, we also observed cases of seeminglyinnocuous information such as ROM build ID and serialnumber. Such information can potentially aid the uniqueidentification of a handset in the same way as the IMEIcould, yet unlike an IMEI, retrieving it does not requirespecial permissions on part of the application. A largefraction of apps, even alternative ROM vendors such asVenom [27] leak this sensitive data not only for their owndata collection purposes, but also to online trackers andad-services (Ookla’s Speed Test, Skype, and Ticketmas-ter). We list examples in Table 6. Finally, 56% percent

11

Page 12: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

Mode idle sleep max idle cycles Mean UDPRTT (ms)

Mean TCPConnectionTime (ms)

Mean TLSEstablishment

Time (ms)

Max. Throughput[Up/Down] (Mbps)

MeanCPU(%)

Foreground 10 100 5.4 24.8 313.1 17.2/54.9 11.2Background 100 100 60.8 65.3 505.3 16.7/48.2 2.7

Table 4: Summary of Haystack’s performance for each operational mode in our test-bed (tested in a 5 GHz WiFi link with 3 ms RTT).

of the leaks we detect were transmitted over TLS con-nections that were successfully intercepted by Haystack.This shows the importance of TLS interception in ourquest to understand the mobile ecosystem; analytic ser-vices such as Google Analytics and Flurry upload largeamounts of user metadata over TLS without using cer-tificate pinning, thereby becoming easy targets for TLSinterception. This is especially a problem when the sys-tem’s CA certificate store has been compromised; andprevious studies show that this is a possibility [28] [20].

6 Future Haystack Applications

Haystack’s vantage point between applications and thenetwork gives it a unique opportunity to observe, charac-terize, and interpose on user traffic. We now outline ad-ditional applications that leverage this capability and candrive future mobile device research. Crucially, we canleverage the fact that Haystack runs locally on the deviceto directly inform users about ongoing activity and en-able them to control Haystack’s influence on the traffic.

Characterizing mobile traffic and systems Haystackcan combine its view into user traffic with contextual in-formation collected on the phone, such as the networktype, device location, and radio state to help us study apptraffic both vertically (i.e., across network layers) andhorizontally (i.e., across mobile providers). We can thenequip users with detailed insights about how apps con-sume their data plan, how they affect battery life, and atwhich time and locations their performance deteriorates.We can furthermore characterize protocol performancesuch as DNS lookup and TCP connection latencies, mon-itor content replica selection, and track adoption of noveltechnologies like HTTP/2 and QUIC.

Assessing and enhancing app security In addition tothe TLS usage profiling we have already described, wecan leverage this ability to notify users in real-time ofongoing TLS interception [20]. In addition, Haystacklets us bring to bear a wide range of misuse and anomalydetection methods well-established in the network secu-rity community, including behavioral indicators of localcompromise, censorship-induced blocking of content,and communication with known-bad data sinks/servers.

App-specific traffic management While we have notexplored content-rewriting in this paper, Haystack’s in-path location puts it in a natural location to conduct it,possibly redirecting or blocking traffic. Paired with theability to link apps to flows, Haystack can subject eachapp’s network traffic to individual forwarding policies,including firewall-style dropping, rate-limiting, or rout-ing into separate VPNs to avoid censorship or improveperformance.

Characterizing user privacy As our prototypical pri-vacy leak analysis in § 5 has demonstrated, Haystack’sability to analyze both non-encrypted and encryptedflows (with some exceptions), combined with access touser data on the phone, lets us characterize how apps ex-filtrate user data, and with whom. Larger deploymentwould allow us to deepen our view into the mobile ad-vertising and tracking ecosystem, enabling longitudinaltracking of data harvested across services and the appsthat employ them.

7 Summary

We have presented the design, implementation, and ex-tensive evaluation of Haystack, a user-space app for An-droid that enables fully device-local, context-aware traf-fic inspection on commodity mobile devices. We havedemonstrated that Haystack can deliver sufficient per-formance with modest resource overhead and minimalimpact on user activity. Using a small deployment with9 volunteers we also demonstrated Haystack’s ability toprovide meaningful insights to users about how appshandle their private information. Haystack achieves anarchitectural sweet-spot that makes it easy to install onregular user phones (thus enabling large-scale deploy-ment) while enabling in-depth visibility into device ac-tivity and traffic (thus providing installation incentivesto the user). We have released Haystack in the An-droid Play Store for beta testing—currently availableon-request—and plan to make it publicly available verysoon. Haystack need not be restricted to Android; neweriOS releases also offer the same primitives that enableHaystack on Android. We will also release the codebaseand datasets that Haystack collects to the public to helpadvance mobile systems research.

12

Page 13: arXiv:1510.01419v1 [cs.NI] 6 Oct 2015

References[1] AHO, A., AND CORASICK, M. Efficient string matching: an aid

to bibliographic search. Communications of the ACM (1975).

[2] Android developers: UI/application exerciser monkey. http:

//developer.android.com/tools/help/monkey.html.

[3] ANDROID DEVELOPER’S DOCUMENTATION. Vpn service.http://developer.android.com/reference/android/

net/VpnService.html.

[4] AU, K. W. Y., ZHOU, Y. F., HUANG, Z., AND LIE, D.PScout: Analyzing Android permission specification. In ACMCCS (2012).

[5] AUCINAS, A., VALLINA-RODRIGUEZ, N., GRUNENBERGER,Y., ERRAMILLI, V., PAPAGIANNAKI, K., CROWCROFT, J.,AND WETHERALL, D. Staying online while mobile: The hid-den costs. In ACM CoNEXT (2013).

[6] BERMUDEZ, I. N., MELLIA, M., MUNAFO, M. M., KERALA-PURA, R., AND NUCCI, A. DNS to the rescue: discerning con-tent and services in a tangled web. In ACM IMC (2012).

[7] BONEH, D., INGUVA, S., AND BAKER, I. SSL Man in the Mid-dle Proxy. https://crypto.stanford.edu/ssl-mitm/.

[8] CHEN, T., ULLAH, I., KAAFAR, M. A., AND BORELI, R. In-formation leakage through mobile analytics services. In ACMHotMobile (2014).

[9] CHRISTIAN FELDE. Busy wait and queue performance.http://blog.cfelde.com/2011/11/busy-wait-and-

queue-performance/.

[10] EGELE, M., KRUEGEL, C., KIRDA, E., AND VIGNA, G. PiOS:Detecting privacy leaks in iOS applications. In NDSS (2011).

[11] ENCK, W., GILBERT, P., CHUN, B., COX, L., JUNG, J., MC-DANIEL, P., AND SHETH, A. TaintDroid: An Information-FlowTracking System for Realtime Privacy Monitoring on Smart-phones. In USENIX OSDI (2010).

[12] FAHL, S., HARBACH, M., MUDERS, T., SMITH, M.,BAUMGARTNER, L., AND FREISLEBEN, B. Why Eve and Mal-lory love Android: An analysis of Android SSL (in) security. InACM CCS (2012).

[13] FALAKI, H., LYMBEROPOULOS, D., MAHAJAN, R., KAN-DULA, S., AND ESTRIN, D. A first look at traffic on smart-phones. In ACM IMC (2010).

[14] GEORGIEV, M., IYENGAR, S., JANA, S., ANUBHAI, R.,BONEH, D., AND SHMATIKOV, V. The most dangerous codein the world: Validating ssl certificates in non-browser software.In ACM CCS (2012).

[15] GILL, P., ERRAMILLI, V., CHAINTREAU, A., KRISHNA-MURTHY, B., PAPAGIANNAKI, K., AND RODRIGUEZ, P. Followthe money: Understanding economics of online aggregation andadvertising. In ACM IMC (2013).

[16] GOOGLE PLAY. Mega. https://play.google.com/store/apps/details?id=nz.mega.android&hl=en.

[17] HORNYACK, P., HAN, S., JUNG, J., SCHECHTER, S., ANDWETHERALL, D. These aren’t the droids you’re looking for:Retrofitting android to protect data from imperious applications.In ACM CCS (2011).

[18] ICSI - GOOGLE PLAY. Haystack alpha tester. https:

//play.google.com/apps/testing/edu.berkeley.

icsi.haystack.

[19] JAVA 7 DOCUMENTATION. Blockingqueue. http:

//docs.oracle.com/javase/7/docs/api/java/util/

concurrent/BlockingQueue.html.

[20] JOHNS, S. Gogo Inflight Internet is intentionally issuing fakeSSL certificates. http://www.neowin.net/news/gogo-

inflight-internet-is-intentionally-issuing-fake-

ssl-certificates.

[21] KRISHNAMURTHY, B., AND WILLS, C. Privacy leakage in mo-bile online social networks. In ACM WOSN (2010).

[22] LE, A., VARMARKEN, J., LANGHOFF, S., SHUBA, A., GJOKA,M., AND MARKOPOLOU, A. AntMonitor: A System for Moni-toring from Mobile Devices. In ACM C2B1D (2015).

[23] LEONTIADIS, I., EFSTRATIOU, C., PICONE, M., AND MAS-COLO, C. Don’t kill my ads!: balancing privacy in an ad-supported mobile application market. In ACM HotMobile (2012).

[24] Monsoon Power Monitor. http://www.msoon.com/

LabEquipment/PowerMonitor/.

[25] ORACLE. New i/o apis. http://docs.oracle.com/javase/1.5.0/docs/guide/nio/index.html.

[26] RAO, A., SHERRY, J., LEGOUT, A., KRISHNAMURTHY, A.,DABBOUS, W., AND CHOFFNES, D. Meddle: Middleboxes forIncreased Transparency and Control of Mobile Traffic. In ACMCoNEXT Student Workshop (2012).

[27] TEAM VENOM. Venom ROMs. http://venomroms.com/.

[28] VALLINA-RODRIGUEZ, N., AMANN, J., KREIBICH, C.,WEAVER, N., AND PAXSON, V. A Tangled Mass: The AndroidRoot Certificate Stores. In Proceedings of the ACM CoNEXT(2014).

[29] VALLINA-RODRIGUEZ, N., SHAH, J., FINAMORE, A.,GRUNENBERGER, Y., PAPAGIANNAKI, K., HADDADI, H., ANDCROWCROFT, J. Breaking for commercials: characterizing mo-bile advertising. In ACM IMC (2012).

[30] YANG, Z., YANG, M., ZHANG, Y., GU, G., NING, P., ANDWANG, X. AppIntent: analyzing sensitive data transmission inandroid for privacy leakage detection. In ACM CCS (2013).

[31] ZHANG, Y., YANG, M., XU, B., YANG, Z., GU, G., NING,P., WANG, X., AND ZANG, B. Vetting Undesirable Behaviorsin Android Apps with Permission Use Analysis. In ACM CCS(2013).

13