MobilePundits

www.mobilepundits.com

Introduction: You’ve developed a Mobile App...Now What?

By the time you finish reading this, the mobile app landscape will have changed. Along with the

development, testing mobile apps is quiet more challenging than testing based on desktop or web

applications.

Mobile app testing is an approach towards solving the emerging technological implications of

Mobile Apps. iOS and Android have been the most successful operating systems so far. It is

quite challenging to test and verify each and every application before releasing it for public use.

In today’s business environment, testing seems to be a cost-effective solution to avoid any

compromise on quality.

QA Challenges in Mobile Application Testing

Variation in Device

Mobile app testing becomes difficult because of many operating systems and myriad devices.

Compatibility is a major issue here because a mobile application can be deployed across multiple

devices. They differ in:

• Mobile operating systems like iOS, Android, BlackBerry, Windows, Symbian, etc.

• Different versions of these operating systems such as iOS 4.x, 5.x, BB 4.x, 5.x, 6.x, Android

2.0, 2.x, 3.x, 4.x, Windows 7, Windows 8.

Mobile Testing Tool Availability

The tools used for desktop and web-based applications do not work for mobile application.

Hence, a complex scripting technique and new tool development is required for mobile

application testing.

Industry Standards

Testing of mobile application must be performed according to certain industry standards so that

the applications can be accepted globally by people using different mobile operating systems.

This will help in making the application popular.

Necessity of Skilled QA specialist in Automation Testing

Every Quality team needs skilled quality assurance specialist not only to evaluate the automation

tools that are present in the market for application testing but also to quickly recognize any bugs

in the user interface as UI is very critical in the mobile world. A well-examined and fully

functional mobile application can be easily be rejected by end users because of bad interface

even though if it is working perfectly.

Cloud Computing

This technology offers a seamless web-based environment for mobile on simulators where the

QA team can deploy, automate and test any mobile application. This approach of cloud

computing has really helped in reducing the overall cost of a project under development.

Listed below are some of the most common problems that are faced by the testers in an

environment of cloud computing:

•High-cost

•A Subscription Model

•Lock-in

•The Automation is time-consuming and image based

•Automation cannot be done outside the framework

Mobile Testing Challenges for Native Applications

For many, “mobile application” means a native or a hybrid application. These are some

commonly downloaded software applications from the respective app stores. The unique

experience that they offer to the users improves the capabilities of the operating system and

device for which they are developed. The downloaded is often monitored by the store, with

mechanisms in place to charge consumers.

Though native applications offer rich user experience to the users – and plausible money

spinning for the developers. Some complexity is added to the lives of those who test it. Testing is

meant to make sure that the application can be successfully downloaded and executed on the

device. To meet the challenges of the testing procedure, backward compatibility with each older

device you are expected to support must also be checked.

Mobile Testing Challenges for Web Applications

Like any other website on internet, a mobile web app is also viewed by people around the world.

Even when you are targeting people from one particular landscape or a network, it will help you

understand dynamics of the world.

Mobile Application Testing Strategy

The Test strategy should make sure that all the quality and performance guidelines are met. A

few pointers in this area:

Selection of the devices

Analyze the market and choose the devices that are widely used. (This decision mostly relies on

the clients. The client or the app builders consider the popularity factor of a certain devices as

well as the marketing needs for the application to decide what handsets to use for testing.)

Emulators

The use of emulators is extremely useful in the initial stages of development, as they allow quick

and efficient checking of the app. Emulator is a system that runs software from one environment

to another environment without changing the software itself. It duplicates the features and work

on real system.

Types of Mobile Emulators

Device Emulator- provided by device manufacturers

Browser Emulator- simulates mobile browser environments.

Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for

Windows phones and Google Android phones

Some free and easy to use mobile device emulators are:

iPhone Tester – All you need to do with this is – enter the URL in search box and you can see

the real time preview of how it appears on an iPhone.

Mobile Phone Emulator – Used to test handsets like iPhone, blackberry, HTC, Samsung etc.

MobiReady– With this, not only can we test the web app, we can also check the code.

Responsivepx– It checks the responses of the web pages, appearances and functionality of the

websites.

Screenfly– It is a customizable tool and used to test websites under different categories.

Mobile cloud Computing Services

The testing experts can efficiently make use of the environment of mobile cloud computing for

deploying and testing the application. Using this technique, we can keep both the task and the

data on the internet instead of keeping it on an individual device. This is what brings on-demand

access in vogue. Lowering down the complexity of project implementation, organizations take

the advantage of services of cloud computing so as to reduce the development and testing cost

and boost the return on investment of a mobile software application solution.

Real Times Devices with Real Networks

It is quite natural that testing can be done against real networks. Real devices can be used at real

locations for performing testing. This is something that the QA team just cannot avoid. Testing

of real devices on real networks should always be an option whenever needed. This thing came

into picture because every application will be used at different locations by end users who might

use it from a remote area with inconsistent network signal strength.

Avoid Manual Work with Automation Tools

Automate whenever and wherever possible! Remote and Emulators, real time devices and

solutions which support playback functionality and scripts help in saving and allow execution of

more test cases with higher degree of consistency.

Any testing team must automate the task of mobile software applications so as to avoid any

manual task. This will not only save cost but time as well. There are many tools available in the

market for performing automation tools.

Types of Testing for a Mobile Application

Usability Testing

This is the type of testing that includes visibility of text in the selected language, screen

navigation, verification of online and offline functionality, feedback from interaction with the

system, which means that the download app should be prompt with messages.

Compatibility Testing

Compatibility testing means validation of mobile application for myriad devices keeping in mind

all the major operating systems, screen size and resolution as per the demand of users,

proofreading if the integration server changes, editing of the mobile app isolation with other

software apps on the device.

Interface Testing

Following are the things that integration testing covers:

•Screens Resolution

•Buttons

•Text Inputs

•Navigation flow like Facebook, Twitter, Reviews, Bookmarks, etc.

Service Testing

Validation of mobile software application not to behave as a server is included in this type of

testing. It even covers checking of a service which can be used offline or takes too long or when

a service goes down and responds back with malformed responses.

Performance Testing

Performance testing is something where you will need to verify the server connection of WIFI

from 2G/3G or vice versa, size of the shared images used by the software code is in accordance

with the desired response time, optimization of code with the CPU cycle, consumption of

battery, memory leaks, certain resources like camera, GPS, etc., freed.

Operational Testing

This type of testing checks for the back-up of all the important information in the application,

save and recovery plan if the battery goes down, if the data of mobile app is lost in the case of

up-gradation from respective app store, accessibility of the app when user gets an alarm,

message, call, reminder, etc. and power usage while accessing the application.

Low Level Resource Testing

Overuse of memory and not releasing is something that comes under low level resource testing.

It also checks if temporary files of the applications are not cleaned, local database is growing too

large. Low level testing will also see the garbage generated by the mobile app.

Security Testing

Security testing will include the encryption and decryption techniques used for communication

of sensitive data, checking of multi-user support without any interference of the data between

them, and checking of accessibility to files that are saved in the app by any unintended users and

detect areas where tested application does not receive any nasty content.

Testing Recommendation

For the types of testing mentioned it is a good idea to use some combination of real device and

emulator testing, as recommended in the table below:

Enterprise applications must meet requirements for maintenance, administration and security.

They “are more complex in functionality and architecture” and therefore it is important to test

enterprise application on performance, security, and synchronization in addition to the standard

functionality testing.

Test Automation Tools & Frameworks Testing mobile applications is traditionally done by manual execution of test cases and visual

verification of the results. However, it is very time consuming. Using automation tools and

testing frameworks yields quantifiable benefits and is recommended.

Automated testing offers fast, repeatable and comprehensive test execution, including overnight

or over weekend test runs. However the investment in tools, test script development and data

collection is required and is and is not amenable to fast changing requirements.

The development platform for mobile application which is a Personal Computer is different from

the target platform which is a handheld device. It is impossible to test the various possible states

a GUI can have. Because of these unlimited testing scenarios mobile GUI testing is more

difficult compared to functional testing of desktop application.“Manual GUI testing is very error

prone and hardly reproducible, and causes very high effort”.

Android Testing Framework The Android testing framework provides powerful tools for testing mobile applications. The

following diagram summarizes the testing framework of Android:

The Android SDK (Software Development Kit) consists of tools for developing and testing

android based mobile application. The Android SDK tools are available as Plug-in in Eclipse

with ADT, and “in command-line form for use with other IDES”. The SDK“tools get

information from the project of the application under test and use this information to

automatically create the build files, manifest file, and directory structure for the test package”.

The SDK also provides monkeyRunner, an API testing devices with Python programs, and

UI/Application Exerciser Monkey, a command-line tool for stress-testing UIs by sending

pseudo-random events to a device.

Sikuli

Sikuli Test is a GUI testing framework that enables automation testing tasks.“It allows testers to

write visual scripts to automate tests, to refer to GUI objects by their visual representation

directly, and to provide robustness to changes in spatial arrangements of GUI components”. The

script uses action statements to simulate the interactions and assertion statements to visually

verify the outcomes of the

interactions.

Test scripts under Sikuli Test are written to test traditional desktop GUI applications on

Windows and Mac OSX, as well as mobile applications in an Android emulator and iOs

simulators.

JaBUTi/ME: White Box Testing

White box testing is a technique based on the internal structure of a given implementation, from

which the test requirements are derived. In general, “white box testing criteria use a

representation known as Control Flow Graph (CFG) to abstract the structure of the program or of

part of the program, as a procedure or method”.

Server based testing JaBUTi/ME

JaBUTi (Java Bytecode Understanding and Testing Tool) is a complete tool suite for

understanding and testing Java programs and Java-based components. JaBUTi differs “from

other testing tools because it performs the static and analysis directly on Java Bytecode not on

the Java source code”.

Mobile Test: Black Box Testing

An automatic black box testing tool for mobile devices introduced in this Chapter is Mobile Test.

With Mobile Test building maintainable and reusable test cases for testing system level and

application level software on various mobile devices is possible. Mobile Test observes the input

and output information of an application. “From the input perspective, a mobile application

receives two kinds of inputs“. The first input is form the user GUI such as keyboard events,

touch events. The second input is environmental context events.

Mobile Test Architecture

Mobile Test architecture

The architecture of Mobile Test subdivides the system environment into layers to reduce the

complexity the system. Each layer provides services to upper layers with the support of lower

layers. In this way, the test control layer can be separated from the characteristics of the

underlying devices.

List of available testing tools

The table below gives a list of available testing frameworks with the platforms they can be tested

on.

Available mobile automation testing tools and the platforms they run on

Fledge: BlackBerry device simulator that enables mobile application testing on a Personal

Computer. Tests, various connectivity and state changes can be simulated.

MonkeyTalk (formerly known as FoneMonkey): offers the possibilities to save a script, load,

read and modify it. FoneMonkey is designed to support developers and quality control tests.

FoneMonkey automates testing on iOS simulators, Android emulators or real devices.

QTP(QuickTest Professional): is an automated testing tool provided by HP/Mercury

Interactive. QTP uses VB scripting language to build its flows. It provides automated and

regression testing and also generates test scripts that can be executed on local or remote mobile

devices.

CTS (Compatibility test Suit): runs on Personal Computer and manages test execution.

Individual test cases are executed on attached mobile devices or on an emulator.

UISpec: Behavior Driven Development framework for the iPhone that provides a full automated

testing solution that drives the actual iPhone UI. It is modeled after the very popular RSpec for

Ruby.

Hermes: black box automation tool for testing J2ME applications. Hermes supports

application independence, and open interfaces for extensibility.

Eggplant: black box automation tool that runs on Mac OSX and Linux. It can test applications

on a vast range of other platforms because it connects to and controls them using VNC (Virtual

Network Computing).

Robotium: black box automation tool for Android Mobile Application. Robotium is a UI testing

tool. The Framework provides API’s to test various kinds of Widgets/UI present in mobile

applications developed with android SDK.

Conclusion

Application development for mobile devices is evolving. The strategies presented in this thesis

discuss the testing of mobile device applications and how important it is to plan a test strategy

that is mobile-specific. Unique challenges like device challenges and software challenges of

mobile devices need to be considered because traditional testing does not cover all characteristic

important for mobile application. The use of automation test tools and test methods should be

conducted for a successful testing result.

The advantage of testing these apps before the deployment in a live environment cannot be

stressed out. An app with bugs and errors will not only hurt the productivity but reputation as

well.

About MobilePundits

MobilePundits partners with global enterprises to drive their innovation-led growth. That's why Deloitte awarded MobilePundits

among the top 100 most innovative companies. As a leading provider of next-generation consulting, technology and outsourcing

solutions, MobilePundits helps clients across the world. Visit www.mobilepundits.com and see how MobilePundits, offers its best

outsource and development services.

For more information, contact marketing@mobilepundits.com