The Best of Mobile Application Testing
• Introduction
• Challenges faced in Mobile Ap
p Testing
• Mobile App Testing Strategy
• Mobile Testing Challenges for
Native Applications
• Mobile Testing Challenges for
Web Applications
• Devices are the Biggest Challe
nge for Mobile Testing
• Right Activities at a Glance
• Types of Testing for a Mobile
App
• Conclusion
• About Us
2
Introduction
2010 has been a great year for mobile world. There were many new handset manufacturers
and mobile app developers in the market. Many of them have done exceptionally well and
changed the definition of Smartphones completely. More and more handheld users are
increasing for accessing business applications. There are expectations for better performance
with time from the devices. Performance is no longer an option, it has become mandatory for
businesses in order to ensure the performance of an app on all the major mobile platforms.
Along with the development, testing of mobile applications is quite challenging. Mobile app
testing is way more challenging than that of apps of standalone or web applications. iOS and
Android have been the most successful operating systems so far.
It is quite challenging to test and verify each and every application before releasing it for
public use. In today’s business environment, testing seems to be a cost-effective solution to
avoid any compromise on quality.
3
Challenges Faced in Mobile App Testing
• Variation in Device
• Industry Standards
• Requirement for Skilled QA Experts in
Automation Testing
• Cloud Computing
4
Variation In Device
• Mobile app testing becomes difficult because of many operating systems
and myriad devices. Compatibility is a major issue here because a mobile
application can be deployed across multiple devices. They differ in:
5
• Mobile operating systems like
iOS, Android, BlackBerry,
Windows, Symbian, etc.
• Different versions of these
operating systems such as iOS
4.x, 5.x, BB 4.x, 5.x, 6.x, Android
2.0, 2.x, 3.x, 4.x, Windows 7,
Windows 8.
• Mobile manufacturers like Apple, Samsung, HTC, Nokia, BlackBerry, Sony,
etc.
• The type of keypad : QWERTY, Touchpad, Virtual Keypad or Hard Keypad.
• Moreover, the Quality Assurance team does not guarantee if the device will work
successful or not on any given device. If it works 100% on one particular device, it
is not necessary that will work efficiently on another device also even if that
device belongs to the same product family because difference comes in the screen
size, screen resolution, CPU, Processor, Memory, OS Optimization, and hardware.
6
Industry Standards
Testing of mobile application must be performed according to certain
industry standards so that the applications can be accepted globally by
people using different mobile operating systems. This will help in making
the application popular.
7
Requirement for Skilled QA Experts in Automation Testing
Every Quality team needs skilled quality
assurance specialist not only to evaluate the
automation tools that are present in the market
for application testing but also to quickly
recognize any bugs in the user interface as UI is
very critical in the mobile world. A well-
examined and fully functional mobile application
can be easily be rejected by end users because
of bad interface even though if it is working
perfectly. 8
9
Cloud Computing
This technology offers a seamless
web-based environment for mobile
on simulators where the QA team
can deploy, automate and test any
mobile application. This approach of
cloud computing has really helped
in reducing the overall cost of a
project under development.
10
Listed below are some of the most common problems that are
faced by the testers in an environment of cloud computing:
11
• High-cost
• A Subscription Model
• Lock-in
• The Automation is time-consuming and image based
• Automation cannot be done outside the framework
Mobile Application Testing Strategy
• Device Emulator
• Mobile cloud Computing Services
• Real Times Devices with Real Networks
• Avoid Manual Work with Automation T
ools
12
Think •Research •Strategy•Plan
Create•Design•Development•Testing•Release
Rate •Marketing•Online Promotion•Reputation
Perform •Measure Usage•User Interaction•Improvements•Evolution
13
Plan Your Strategy!
Device Emulator
Quality Assurance experts can execute almost all of the testing
in a modern well-equipped testing environment with the help of
available device emulators with various options to choose from,
such as the capability to bypass the network, working on live
environment with the help of modems and usage of an effective
scripting language. This solution is highly cost-effective.
14
15
Mobile cloud Computing Services
Mobile cloud Computing Services
The testing experts can efficiently make use of the environment of mobile cloud
computing for deploying and testing the application. Using this technique, we can
keep both the task and the data on the internet instead of keeping it on an
individual device. This is what brings on-demand access in vogue. Lowering down
the complexity of project implementation, organizations take the advantage of
services of cloud computing so as to reduce the development and testing cost
and boost the return on investment of a mobile software application solution.16
17
There are several advantages of using Cloud Computing Services:
• Globalize your workforce at very low costs
• Get more work done(more outputs) with less people in less amount of time
• No need to spend extra bucks on hardware, software or any licensing fees
• Keep an eye on the projects more effectively and efficiently
• Since you require less task force for completing your work, you will have to spend
less on giving personal training to people
• Change of direction without any serious “financial” or “people” at stake
18
• Growth without any pressure of buying new licensed versions of the software
• Increase in volume output or productivity with lesser people. In short,
achievement of economies of scale
• Recording of device logs to help troubleshoot any issue
• Web-based user friendly interface
• Testing of incoming calls and text messages without the need of any extra
mobile plan
• Testing can be performed on more number of devices simultaneously
19
Real Times Devices with Real Networks
It is quite natural that testing can be done against real networks. Real
devices can be used at real locations for performing testing. This is
something that the QA team just cannot avoid. Testing of real devices on
real networks should always be an option whenever needed. This thing
came into picture because every application will be used at different
locations by end users who might use it from a remote area with
inconsistent network signal strength.
20
Remote real devices are capable of recording a test for
successive replay, which is quite helpful in regression
testing. This has reduced the travelling cost for testing for
foreign networks but demands additional hardware. As the
number of devices and mobile applications are increasing
with time, it has become tougher to purchase each and
everything for testing. 21
22
Many organizations have put a foot forward and provide
paid services to other companies. You can now actually
“rent” testing time on a resource shared by others and
managed by you. Just open an account and purchase some
testing time with a given device when and wherever you
need it.
23
Avoid Manual Work with Automation Tools
Automate whenever and
wherever possible! Remote and
Emulators, real time devices and
solutions which support playback
functionality and scripts help in
saving and allow execution of
more test cases with higher
degree of consistency.
24
25
Any testing team must automate
the task of mobile software
applications so as to avoid any
manual task. This will not only
save cost but time as well. There
are many tools available in the
market for performing automation
tools.
26
Mobile Testing Challenges for Native Applications
For many, “mobile application” means a
native or a hybrid application. These are
some commonly downloaded software
applications from the respective app stores.
The unique experience that they offer to the
users improves the capabilities of the
operating system and device for which they
are developed. The downloaded is often
monitored by the store, with mechanisms in
place to charge consumers. 27
This simple mechanism has made native apps quite a hit in the recent times.
Apart from usage by normal consumers, they also enable businesses to deliver
high productivity to increasing workforce.
28
Though native applications offer rich user experience to the users – and
plausible money spinning for the developers. Some complexity is added
to the lives of those who test it. Testing is meant to make sure that the
application can be successfully downloaded and executed on the device.
This is a common perplexity that if an application has been tested in one
device and network, it will work seamlessly on others as well. But this isn’t
true at all. This is what forces developers and testers to examine the software
application on different environments and device models across all the
operating systems.
For successfully completing the task, analysis of hardware is a must because
native apps rely completely on the hardware and OS. To meet the challenges
of the testing procedure, backward compatibility with each older device you
are expected to support must also be checked.
29
Mobile Testing Challenges for Web Applications
Like any other website on internet,
a mobile web app is also viewed by
people around the world. Even
when you are targeting people
from one particular landscape or a
network, it will help you
understand dynamics of the world. 30
Devices are the Biggest Challenge for Mobile Testing
The much obvious milestone to be crossed in the field of mobile testing!
There must be millions of devices around the world on which your
application will run and you cannot afford to fail to perform on any of
them. Almost all of them must be covered when examining your mobile
application. If you reduce the number of devices, there is a chance that the
particular software program might not run on that device, locking out a
number of potential customers.31
For handling this challenge of mobile testing, there are things you must take
care off: Exclusive examination using real devices, test exclusively with
emulated devices or a combination of both of them. It is not like that you are
limited to one choice, but it totally depends on the application for which you
are writing the code. 32
Right Activities At A Glance
Testing
33
Initialization & Benchmarking
Network & connectivity
Testing
Application & Multimedia
Value Added Services Field Testing
Types of Testing for a Mobile Application
• Usability Testing
• Compatibility Testing
• Interface Testing
• Services Testing
• Low Level Resource Testi
ng
• Performance Testing
• Operational Testing
• Security Testing34
Usability Testing
This is the type of testing that
includes visibility of text in the
selected language, screen
navigation, verification of online
and offline functionality, feedback
from interaction with the system,
which means that the download
app should be prompt with
messages.
35
Compatibility Testing
Compatibility testing means validation of
mobile application for myriad devices
keeping in mind all the major operating
systems, screen size and resolution as per
the demand of users, proofreading if the
integration server changes, editing of the
mobile app isolation with other software
apps on the device.
36
Interface Testing
Following are the things that integration testing covers:
• Screens Resolution
• Buttons
• Text Inputs
• Navigation flow like Facebook, Twitter, Reviews, Bookmarks, etc.
37
Service Testing
Validation of mobile software application not to behave as a server is
included in this type of testing. It even covers checking of a service
which can be used offline or takes too long or when a service goes down
and responds back with malformed responses.
38
Low Level Resource Testing
Overuse of memory and not releasing is something that comes under low
level resource testing. It also checks if temporary files of the applications
are not cleaned, local database is growing too large. Low level testing will
also see the garbage generated by the mobile app.
39
Performance Testing
Performance testing is something where
you will need to verify the server
connection of WIFI from 2G/3G or vice
versa, size of the shared images used by
the software code is in accordance with
the desired response time, optimization
of code with the CPU cycle, consumption
of battery, memory leaks, certain
resources like camera, GPS, etc., freed.
40
Operational Testing
This type of testing checks for the back-
up of all the important information in
the application, save and recovery plan
if the battery goes down, if the data of
mobile app is lost in the case of up-
gradation from respective app store,
accessibility of the app when user gets
an alarm, message, call, reminder, etc.
and power usage while accessing the
application.
41
Security Testing
42
Security Testing
Security testing will include the encryption
and decryption techniques used for
communication of sensitive data, checking of
multi-user support without any interference
of the data between them, and checking of
accessibility to files that are saved in the app
by any unintended users and detect areas
where tested application does not receive
any nasty content. 43
Conclusion
44
A substantial challenge and risk involved in the testing of a mobile application can impact a lot
on the production of mobile apps. Such risks and challenges involved in adopting myriad
testing procedures have been outlined above.
The many tools available present in the market for testing of software mobile applications will
not only help in testing but also offer a platform which enables measurement of security and
compatibility compliance. This is what helps in application certification where required.
The advantage of testing these apps before the deployment in a live environment cannot be
stressed out. An app with bugs and errors will not only hurt the productivity but reputation
as well.
About Us
45
MobilePundits is an ISO 9001:2008 certified by Bureau Veritas & UKAS, assuring
our clients with the highest level of satisfaction. We assure minimum mobile
application defects with our mobile app testing services. For more than 8 years
we have successfully offered comprehensive outsourcing of mobile
testing/Wireless Testing/Mobile Application Testing Services. Highest degree of
software quality assurance for iPhone/iPad, BlackBerry and Android App Testing
is our forte.
About Us
46
