moet - mobile end-to-end test at selenium conf 2011
DESCRIPTION
Selenium Conference Track B Presentation - MOET : MObile End-to-end Test This session presents an experimental approach that uses mobile simulators - BlackBerry™ and Android™ to run automated end-to-end tests using the Python™ programming language. We’ll walk through the testing process - from the basics of interacting with both device simulators, writing reusable tests on both these devices and executing these tests in PyUnit. Sample code is included and available from https://github.com/eing/moetTRANSCRIPT
![Page 1: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/1.jpg)
![Page 2: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/2.jpg)
Session outline
Introduction
Simulator basics
Mobile end-to-end testing (Moet)
Building your mobile tests
Demo
Advantages and limitations
Q & A
![Page 3: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/3.jpg)
What are we solving for
Diverse mobile platforms
Low cost solution
End-to-end mobile tests
Leverage black box testers
![Page 4: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/4.jpg)
Simulator Basics
BlackBerry TM
Starting simulatorfledge.exe
/app=jvm.dll
/session=<model>
/app-param=
JvmAlxConfigFile:<model>.xml
/handheld=<model>
Communicating with simulatorfledgecontroller.exe /session=<model>
![Page 5: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/5.jpg)
Simulator commands
Actions Steps
Start 9630 Tour simulator
fledge.exe /app=jvm.dll
/session=9630 /handheld=9630
/app-param=JvmAlxConfigFile:9630.xml
Installapplication
1. Copy app.jar, app.jad, app.cod to Javaloader directory
2. JavaLoader.exe –u load app.jad
3. Delete app.jar, app.jad, app.cod
Save screenshot as test.png in $TEST_OUTPUT
1. JavaLoader.exe –u screenshot test.png
2. mv test.png $TEST_OUTPUT
![Page 6: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/6.jpg)
bblib.py
Actions Steps bblib.py
Start 9630 Tour simulator
fledge.exe /app=jvm.dll
/session=9630 /handheld=9630
/app-param=JvmAlxConfigFile:9630.xml
fledgeStart()
Installapplication
1. Copy app.jar, app.jad, app.cod to Javaloader directory
2. JavaLoader.exe –u load app.jad
3. Delete app.jar, app.jad, app.cod
install()
Save screenshot as test.png in $TEST_OUTPUT
1. JavaLoader.exe –u screenshot test.png
2. mv test.png $TEST_OUTPUT
screenshot(‘test’)
![Page 7: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/7.jpg)
Simulator commands
Action Steps
Enter 'Hello World'
StringInjection(Hello)
KeyPress(SPACE)
KeyRelease(SPACE)
StringInjection(World)
Touch screen at (10, 100)
TouchScreenPress(10, 100, 0)
TouchScreenClick()
TouchScreenUnclick()
TouchScreenUnpress(0)
Thumbwheel up twice
ThumbWheelRoll(-1)
ThumbWheelRoll(-1)
![Page 8: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/8.jpg)
bblib.py
Action Steps bblib.py
Enter 'Hello World'
StringInjection(Hello)
KeyPress(SPACE)
KeyRelease(SPACE)
StringInjection(World)
enter(‘Hello World')
Touch screen at (10, 100)
TouchScreenPress(10, 100, 0)
TouchScreenClick()
TouchScreenUnclick()
TouchScreenUnpress(0)
touch(10, 100)
Thumbwheel up twice
ThumbWheelRoll(-1)
ThumbWheelRoll(-1)
thumbwheel ('up', 2)
![Page 9: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/9.jpg)
Simulator Basics
Android TM
Create AVD$ANDROID_HOME/tools/android
Starting emulatoremulator –avd <avd>
Communicating with emulatoradb shell
![Page 10: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/10.jpg)
Simulator command
Action Steps
Enter 'Hello World'
adb shell
"sendevent /dev/input/event0 1 42 1; sendevent /dev/input/event0 1 42 0; sendevent /dev/input/event0 1 35 1; sendevent /dev/input/event0 1 35 0; sendevent /dev/input/event0 1 18 1; sendevent /dev/input/event0 1 18 0; sendevent /dev/input/event0 1 38 1; sendevent /dev/input/event0 1 38 0; sendevent /dev/input/event0 1 38 1; sendevent /dev/input/event0 1 38 0; sendevent /dev/input/event0 1 24 1; sendevent /dev/input/event0 1 24 0;
… "
![Page 11: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/11.jpg)
androidlib.py
Action Steps androidlib.py
Enter 'Hello World'
adb shell
"sendevent /dev/input/event0 1 42 1; sendevent /dev/input/event0 1 42 0; sendevent /dev/input/event0 1 35 1; sendevent /dev/input/event0 1 35 0; sendevent /dev/input/event0 1 18 1; sendevent /dev/input/event0 1 18 0; sendevent /dev/input/event0 1 38 1; sendevent /dev/input/event0 1 38 0; sendevent /dev/input/event0 1 38 1; sendevent /dev/input/event0 1 38 0; sendevent /dev/input/event0 1 24 1; sendevent /dev/input/event0 1 24 0;
… "
enter(‘Hello World’)
![Page 12: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/12.jpg)
MOET
MObile End-to-End Test
Simulator librariesandroidlib.py
bblib.py
Image processing libraryimagelib.py
Testing utilities librarytestlib.py
logger.py
![Page 13: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/13.jpg)
Moet Framework
Android app library
androidlib.py
BlackBerry app library
bblib.py
Device Independent Tests
Simulator libraries
Mobile Application Interface
Runtime binding
![Page 14: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/14.jpg)
![Page 15: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/15.jpg)
Test Automation Overview
1. Define application interfaceThis interface is device-agnostic.
2. Implement the interfaceImplement the interface in your supported devices e.g. Android.
Utilize python mobile libraries e.g. androidlib.py.
3. Write your testsTests are device independent and reusable on all supported devices.
4. Run
![Page 16: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/16.jpg)
Step 1 : Define app interface
class AppInterface:
""" Application interface for all devices to implement """
def add(self, contact):
"""Add contact """
def find(self, contact):
""" Find contact"""
def delete(self, contact):
"""Delete contact"""
![Page 17: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/17.jpg)
Test Automation Overview
1. Define application interfaceThis interface is device-agnostic.
2. Implement the interfaceImplement the interface in your supported devices.
Utilize moet libraries.
3. Write your testsTests are device independent and reusable on all supported devices.
4. Run
![Page 18: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/18.jpg)
Step 2 (Pearl) :Implement the interface
def add(self, contact):
""" Add contact """
# click add contact
enter()
# enter name
enter(contact.getFirstname()
thumbwheel('down', 1)
…
# save
menu()
enter()
![Page 19: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/19.jpg)
Step 2 (Android) :Implement the interface
def add(self, contact):""" Add contact """
# click add contactmenu()scroll(‘up’)scroll(‘right’)enter()
# enter nameenter(contact.getFirstname())scroll('down')… # savemenu()scroll(‘down’)enter()
![Page 20: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/20.jpg)
Step 2 (recap) : Implement the interface
def PearlImpl(appbase.AppInterface):
def add(self, contact):
""" Add contact """
enter() enter(contact.getFirstname()
thumbwheel('down', 1)
…
menu()
enter()
def AndroidImpl(appbase.AppInterface):
def add(self, contact):
""" Add contact """
menu()
scroll(‘up’)
scroll(‘right’)
enter()
enter(contact.getFirstname())
scroll(‘down’)
…
menu()
scroll(‘down’)
enter()
![Page 21: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/21.jpg)
Test Automation Overview
1. Define application interfaceThis interface is device-agnostic.
2. Implement the interfaceImplement the interface in your supported devices e.g. Android.
Utilize python mobile libraries e.g. androidlib.py.
3. Write your testsTests are device independent and reusable on all supported devices.
4. Run
![Page 22: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/22.jpg)
Step 3 : Writing tests
class AddContactTest(unittest.TestCase):
device = testenv.getDeviceClass()
def addContactWithOnlyFirstnameTest(self):
self.contact.setFirstname(firstname)
self.device.add(self.contact)
def addContactWithOnlyLastnameTest(self):
self.contact.setLastname(lastname)
self.device.add(self.contact)
![Page 23: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/23.jpg)
Step 3 : Runtime binding
def getDeviceClass(self):""" Returns the device to test """
mobileDevice = self.getMobileDevice()
if mobileDevice == 'pearl':import pearldeviceClass = pearl.PearlImpl()
elif mobileDevice == ‘android':import androiddeviceClass = android.AndroidImpl()
return deviceClass
![Page 24: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/24.jpg)
More device-independent tests
Additional tests are easy to write
def addContactWithEmailTest(self):
def addContactWithAddressesTest(self):
def addContactWithAllDetailsTest(self):
def addContactWithLongDetailsTest(self):
def addContactAddressWithStateZip(self):
def addContactAddressWithCityStateZip(self):
def addContactAddressWithNoDataNegativeTest(self):
![Page 25: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/25.jpg)
Step 4 : Run
Basic run command
python <test.py>
Python test frameworks
unittest
PyUnit
python-nose
![Page 26: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/26.jpg)
Test Verification
Server hosted apps API assertions
Database assertions
Image assertionsself.assertTrue(
imagelib.compare(
self.device, testname, '100%x90%‘, tolerance))# Crop settings examples
# 100%x80%+10%+20% (crop size + offset)
# 320x90+0+0
# +0+90
![Page 27: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/27.jpg)
Test Logging
LogsAddressTest.log : 2010-06-10 15:19:46,773 - testCreateAddressMethod - INFO -
[Address] 200 Villa St Mountain View CA 94040 BUSINESS ADDRESS
Initializationself.log = self.device.initLogger(self._testMethodName,
self.__class__.__name__)
Usageself.log.info('Starting test: ' + self._testMethodName)self.log.debug(self.contact)self.log.error(‘Missing image to compare’)
![Page 28: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/28.jpg)
Demo
Simulators Android
BlackBerry Pearl
Moet
Test automation Address book app
○ Add contact
○ Find contact
○ Delete contact
![Page 29: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/29.jpg)
Advantages
Low cost and ease of use
Reusable end-to-end tests
No device sharing/scheduling
Bigger device pool
Reduce manual testing time
Run on developer machines
Debugging capabilities
![Page 30: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/30.jpg)
Limitations
Requires ethernet or internet connectivity
Does not simulate network performance
Does not support hardware controls testing
Dependent on simulator reliability
Limited peer-to-peer applications testing
![Page 31: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/31.jpg)
Resources
MOET http://github.com/eing/moet/
Android
Emulator http://developer.android.com/guide/developing/tools/emulator.html
ADB http://android-dls.com/wiki/index.php?title=ADB
Forum http://developer.android.com/resources/community-groups.html
BlackBerry
Downloads http://na.blackberry.com/eng/developers/javaappdev/javadevenv.jsp
Fledge Controller http://docs.blackberry.com/en/developers/deliverables/6338/Testing_apps_using_the_BBSmrtphnSmltr_607559_11.jsp
Forum http://supportforums.blackberry.com/
![Page 32: Moet - Mobile End-to-End Test at Selenium Conf 2011](https://reader033.vdocuments.net/reader033/viewer/2022042700/55946ddc1a28abea718b45de/html5/thumbnails/32.jpg)
Q & A