The Best of Mobile Application Testing

• Introduction

• Challenges faced in Mobile App Testing

• Mobile App Testing Strategy

• Mobile Testing Challenges for Native

Applications

• Mobile Testing Challenges for Web

Applications

• Devices are the Biggest Challenge for

Mobile Testing

• Right Activities at a Glance

• Types of Testing for a Mobile App

• Conclusion

• About Us

2

Introduction

2010 has been a great year for mobile world. There were many new handset manufacturers

and mobile app developers in the market. Many of them have done exceptionally well and

changed the definition of Smartphones completely. More and more handheld users are

increasing for accessing business applications. There are expectations for better performance

with time from the devices. Performance is no longer an option, it has become mandatory for

businesses in order to ensure the performance of an app on all the major mobile platforms.

Along with the development, testing of mobile applications is quite challenging. Mobile app

testing is way more challenging than that of apps of standalone or web applications. iOS and

Android have been the most successful operating systems so far.

It is quite challenging to test and verify each and every application before releasing it for

public use. In today’s business environment, testing seems to be a cost-effective solution to

avoid any compromise on quality.

3

Challenges Faced in Mobile App Testing

• Variation in Device

• Industry Standards

• Requirement for Skilled QA Experts in

Automation Testing

• Cloud Computing

4

Variation In Device

• Mobile app testing becomes difficult because of many operating systems

and myriad devices. Compatibility is a major issue here because a mobile

application can be deployed across multiple devices. They differ in:

5

• Mobile operating systems like

iOS, Android, BlackBerry,

Windows, Symbian, etc.

• Different versions of these

operating systems such as iOS

4.x, 5.x, BB 4.x, 5.x, 6.x, Android

2.0, 2.x, 3.x, 4.x, Windows 7,

Windows 8.

• Mobile manufacturers like Apple, Samsung, HTC, Nokia, BlackBerry, Sony, etc.

• The type of keypad : QWERTY, Touchpad, Virtual Keypad or Hard Keypad.

• Moreover, the Quality Assurance team does not guarantee if the device will work

successful or not on any given device. If it works 100% on one particular device, it

is not necessary that will work efficiently on another device also even if that

device belongs to the same product family because difference comes in the

screen size, screen resolution, CPU, Processor, Memory, OS Optimization, and

hardware.

6

Industry Standards

Testing of mobile application must be performed according to certain

industry standards so that the applications can be accepted globally by

people using different mobile operating systems. This will help in making

the application popular.

7

Requirement for Skilled QA Experts in Automation Testing

Every Quality team needs skilled quality

assurance specialist not only to evaluate the

automation tools that are present in the market

for application testing but also to quickly

recognize any bugs in the user interface as UI is

very critical in the mobile world. A well-

examined and fully functional mobile

application can be easily be rejected by end

users because of bad interface even though if it

is working perfectly.

8

9

Cloud Computing

This technology offers a seamless web-

based environment for mobile on

simulators where the QA team can

deploy, automate and test any mobile

application. This approach of cloud

computing has really helped in reducing

the overall cost of a project under

development.

10

Listed below are some of the most common problems that are faced

by the testers in an environment of cloud computing:

11

•High-cost

•A Subscription Model

•Lock-in

•The Automation is time-consuming and image based

•Automation cannot be done outside the framework

Mobile Application Testing Strategy

• Device Emulator

• Mobile cloud Computing Services

• Real Times Devices with Real Networks

• Avoid Manual Work with Automation

Tools

12

Think

• Research

• Strategy

• Plan

Create

• Design

• Development

• Testing

• Release

Rate

• Marketing

• Online Promotion

• Reputation

Perform

• Measure Usage

• User Interaction

• Improvements

• Evolution

13

Plan Your Strategy!

Device Emulator

Quality Assurance experts can execute almost all of the testing in a

modern well-equipped testing environment with the help of

available device emulators with various options to choose from,

such as the capability to bypass the network, working on live

environment with the help of modems and usage of an effective

scripting language. This solution is highly cost-effective.

14

15

Mobile cloud Computing Services

Mobile cloud Computing Services

The testing experts can efficiently make use of the environment of mobile cloud

computing for deploying and testing the application. Using this technique, we can

keep both the task and the data on the internet instead of keeping it on an

individual device. This is what brings on-demand access in vogue. Lowering down

the complexity of project implementation, organizations take the advantage of

services of cloud computing so as to reduce the development and testing cost and

boost the return on investment of a mobile software application solution. 16

17

There are several advantages of using Cloud Computing Services:

• Globalize your workforce at very low costs

• Get more work done(more outputs) with less people in less amount of time

• No need to spend extra bucks on hardware, software or any licensing fees

• Keep an eye on the projects more effectively and efficiently

• Since you require less task force for completing your work, you will have to spend

less on giving personal training to people

• Change of direction without any serious “financial” or “people” at stake

18

• Growth without any pressure of buying new licensed versions of the software

• Increase in volume output or productivity with lesser people. In short,

achievement of economies of scale

• Recording of device logs to help troubleshoot any issue

• Web-based user friendly interface

• Testing of incoming calls and text messages without the need of any extra

mobile plan

• Testing can be performed on more number of devices simultaneously

19

Real Times Devices with Real Networks

It is quite natural that testing can be done against real networks. Real

devices can be used at real locations for performing testing. This is

something that the QA team just cannot avoid. Testing of real devices on

real networks should always be an option whenever needed. This thing

came into picture because every application will be used at different

locations by end users who might use it from a remote area with

inconsistent network signal strength.

20

Remote real devices are capable of recording a test for

successive replay, which is quite helpful in regression testing.

This has reduced the travelling cost for testing for foreign

networks but demands additional hardware. As the number

of devices and mobile applications are increasing with time, it

has become tougher to purchase each and everything for

testing.

21

22

Many organizations have put a foot forward and provide

paid services to other companies. You can now actually

“rent” testing time on a resource shared by others and

managed by you. Just open an account and purchase some

testing time with a given device when and wherever you

need it.

23

Avoid Manual Work with Automation Tools

Automate whenever and wherever

possible! Remote and Emulators,

real time devices and solutions

which support playback

functionality and scripts help in

saving and allow execution of more

test cases with higher degree of

consistency.

24

25

Any testing team must automate

the task of mobile software

applications so as to avoid any

manual task. This will not only

save cost but time as well. There

are many tools available in the

market for performing automation

tools.

26

Mobile Testing Challenges for Native Applications

For many, “mobile application” means a

native or a hybrid application. These are

some commonly downloaded software

applications from the respective app stores.

The unique experience that they offer to

the users improves the capabilities of the

operating system and device for which they

are developed. The downloaded is often

monitored by the store, with mechanisms in

place to charge consumers.

27

This simple mechanism has made native apps quite a hit in the recent

times. Apart from usage by normal consumers, they also enable

businesses to deliver high productivity to increasing workforce.

28

Though native applications offer rich user experience to the users – and

plausible money spinning for the developers. Some complexity is added

to the lives of those who test it. Testing is meant to make sure that the

application can be successfully downloaded and executed on the device.

This is a common perplexity that if an application has been tested in one

device and network, it will work seamlessly on others as well. But this isn’t

true at all. This is what forces developers and testers to examine the

software application on different environments and device models across

all the operating systems.

For successfully completing the task, analysis of hardware is a must

because native apps rely completely on the hardware and OS. To meet the

challenges of the testing procedure, backward compatibility with each

older device you are expected to support must also be checked.

29

Mobile Testing Challenges for Web Applications

Like any other website on internet, a

mobile web app is also viewed by

people around the world. Even when

you are targeting people from one

particular landscape or a network, it

will help you understand dynamics

of the world.

30

Devices are the Biggest Challenge for Mobile Testing

The much obvious milestone to be crossed in the field of mobile testing!

There must be millions of devices around the world on which your

application will run and you cannot afford to fail to perform on any of

them. Almost all of them must be covered when examining your mobile

application. If you reduce the number of devices, there is a chance that

the particular software program might not run on that device, locking out

a number of potential customers.

31

For handling this challenge of mobile testing, there are things you must

take care off: Exclusive examination using real devices, test exclusively

with emulated devices or a combination of both of them. It is not like that

you are limited to one choice, but it totally depends on the application for

which you are writing the code.

32

Right Activities At A Glance

Testing

Requirement Analysis

Guidelines

Desirables

Ten Step Methodology

Field Testing

Regression Testing

33

Initialization & Benchmarking

Network & connectivity

Testing

Application & Multimedia

Value Added Services

Field Testing

Types of Testing for a Mobile Application

• Usability Testing

• Compatibility Testing

• Interface Testing

• Services Testing

• Low Level Resource Testing

• Performance Testing

• Operational Testing

• Security Testing

34

Usability Testing

This is the type of testing that

includes visibility of text in the

selected language, screen

navigation, verification of online

and offline functionality,

feedback from interaction with

the system, which means that

the download app should be

prompt with messages.

35

Compatibility Testing

Compatibility testing means validation of

mobile application for myriad devices

keeping in mind all the major operating

systems, screen size and resolution as per

the demand of users, proofreading if the

integration server changes, editing of the

mobile app isolation with other software

apps on the device.

36

Interface Testing

Following are the things that integration testing covers:

• Screens Resolution

• Buttons

• Text Inputs

• Navigation flow like Facebook, Twitter, Reviews, Bookmarks, etc.

37

Service Testing

Validation of mobile software application not to behave as a server is

included in this type of testing. It even covers checking of a service which

can be used offline or takes too long or when a service goes down and

responds back with malformed responses.

38

Low Level Resource Testing

Overuse of memory and not releasing is something that comes under low

level resource testing. It also checks if temporary files of the applications

are not cleaned, local database is growing too large. Low level testing will

also see the garbage generated by the mobile app.

39

Performance Testing

Performance testing is something where

you will need to verify the server

connection of WIFI from 2G/3G or vice

versa, size of the shared images used by

the software code is in accordance with

the desired response time, optimization

of code with the CPU cycle,

consumption of battery, memory leaks,

certain resources like camera, GPS, etc.,

freed.

40

Operational Testing

This type of testing checks for the

back-up of all the important

information in the application, save

and recovery plan if the battery goes

down, if the data of mobile app is

lost in the case of up-gradation from

respective app store, accessibility of

the app when user gets an alarm,

message, call, reminder, etc. and

power usage while accessing the

application.

41

Security Testing

42

Security Testing

Security testing will include the encryption

and decryption techniques used for

communication of sensitive data, checking of

multi-user support without any interference

of the data between them, and checking of

accessibility to files that are saved in the app

by any unintended users and detect areas

where tested application does not receive

any nasty content.

43

Conclusion

44

A substantial challenge and risk involved in the testing of a mobile application can

impact a lot on the production of mobile apps. Such risks and challenges involved in

adopting myriad testing procedures have been outlined above.

The many tools available present in the market for testing of software mobile

applications will not only help in testing but also offer a platform which enables

measurement of security and compatibility compliance. This is what helps in

application certification where required.

The advantage of testing these apps before the deployment in a live environment

cannot be stressed out. An app with bugs and errors will not only hurt the productivity

but reputation as well.

About Us

45

MobilePundits is an ISO 9001:2008 certified by Bureau Veritas & UKAS,

assuring our clients with the highest level of satisfaction. We assure

minimum mobile application defects with our mobile app testing services.

For more than 8 years we have successfully offered comprehensive

outsourcing of mobile testing/Wireless Testing/Mobile Application Testing

Services. Highest degree of software quality assurance for iPhone/iPad,

BlackBerry and Android App Testing is our forte.

About Us

46