Cohesive approach to Android Application Performance testing

AuthorsPankaj Saini

Sundaresan Krishnaswami

The power of Android Apps

Typical Performance Issues

Abuse of power in a shared world

Battery usage- Services running in the background

- Poll for emails instead of Push- Too many notifications (Like too many Ads)

- Request and lock a service, only never to release- Location- Wakelocks

- Your app running in the foreground- Automatically download in the background without

user intervention- Contd…

Memory Usage–App too slow to use–Crashing with Out of memory exceptions

CPU Usage-Full wake-locks instead of partial-Apps do not release wake-locks-Idle CPU usage

Why bother about performance?

• A Smartphone is Smart only till its battery lasts.

• Numerous apps compete within the limited resources like memory and CPU.

• Optimal use of resources along with smooth end user experience.

Performance Testing Today

• The proliferation of apps, combination of devices, networks, endpoint servers, inter-app communication and user behavior has made performance testing extremely complex.

• What is the best approach?

The Cohesive Approach

• Understand App Usecase• Glean through Design Document• Come up with Techniques• Identify the tools• Put them together• Fit them into your QA cycle

Techniques

• Battery Usage• Network Usage • Server Latch Time• Log analysis• GPS Usage• Wake-locks• Memory leaks• CPU Usage

Usecase vs Techniques Matrix

CricinfoCricinfo Ola CabsOla Cabs FlipkartFlipkart Flipkart E-readerFlipkart E-reader TouchdownTouchdown

Battery UsageBattery Usage

Network UsageNetwork Usage

Server Latch TimeServer Latch Time

Log AnalysisLog Analysis

Wake-locks Wake-locks

Memory LeaksMemory Leaks

GPS UsageGPS Usage

CPU usageCPU usage

Tools – Battery Usage

Recommended tools –

1.Little Eye

2.Rational Test workbench

Poor man’s tool –

1.Power usage under Android Settings

2.eStar battery monitor

Battery Usage - What to test?

• Perform a scenario• Allow the application to idle• Keep a time limit based on application

usecase– Ex: On an email app, allow the device to idle for

an hour

• Observe the battery usage

Tools – Network Usage

Recommended tool – Fiddler

What to test and how?-Setup device to route network calls via fiddler

-Contd…

• Observe the network calls made• Ask questions like– Is the call relevant?– Does the app make too many calls within a

timeframe?– Does the app make networks calls in idle

scenarios?– Should a scenario run on WiFi instead of Data

network for optimum battery usage?

Tools – Server Latch Time• Observe the ‘Server latch time’ using Fiddler

• Ask questions like– Is the device connected to the server for a longer

period– Can the server performance be improved

Tools – Log Analysis

Recommended Tools – ADB Monitor

Why?

-Repeated logs consume memory

-Does the application wake up on every broadcast and log it too?

-Example scenario above

-Collect logs and analyze with Notepad ++

Tools – GPS Usage

• Look for GPS battery usage under Settings -> Location

Tools – Wake Locks

What is a Wake Lock?-Background services may require CPU or Screen to be on to complete tasks

Who requires it?

- Mostly PIM applications

How does it impact performance?

-Too many wake locks-Wake locks are not released post task completion

Recommended Tool to monitor Wake locks-Wakelock detector-Little Eye (discontinued)

What to test?-Observe the number of Wake locks in a time period-Question if the behavior is valid

Tools – Memory Leaks

Comprehensive Slideshow on Memory leaks by Zachary D. Blair http://www.slideshare.net/zblair/identifying-memory-leaks-in-android-applications

Recommended Tools – ADB Monitor, Eclipse Memory Analyzer tool

Also look for rogue processes under ‘Process Stats’ available under Developer Options

Easy Memory Leaks Testing*1) Install the app on the phone.2) Note lower memory usage of the app in DDMS.3) Run the application. Go through all possible screens.4) Note the higher memory usage.6) Using the Back key (not the Home key), close the app.7) Press "Cause GC" as many times as necessary in the DDMS tool to bring the memory usage down. 8) The memory usage should stabilize at a lower value than at step (4).

*Credit: Marius Marinescu, Senior Software Developer, IBM

Tools – CPU Usage

Recommended Tools –

1.Trepn Profiler - Snapdragon

2.Android ADB Monitor

What to test?

-Does the app show up in the stack trace once you tap the back button

-Is the usage too high compared to an app of similar nature

Summary

• Understand App Usecase• Glean through Design Document• Come up with Techniques• Identify the tools• Put them together• Fit them into your QA cycle