andevcon: building automated android app tests
TRANSCRIPT
© 2015 SOASTA. All rights reserved. July 31, 2015 1
© 2015 SOASTA. All rights reserved. July 31, 2015 2
o Please connect to the shared Google Drive: http://soasta.io/AnDevConFiles
o Open the Setup document and do the steps
© 2015 SOASTA. All rights reserved. July 31, 2015 3
o Why Automation?
o Test Automation Products
o Getting Started
o Creating Test Scripts from Code
o Creating Test Scripts by Recording
o Running Automated Test Scripts • On local devices, in cloud devices
o Setting Up Continuous Integration
© 2015 SOASTA. All rights reserved. July 31, 2015 4
o Improved testing efficiency
o Consistent and repeatable testing process
o Improved regression testing coverage
o More tests can be run in less time
o Run 24x7
o Humans free to perform advanced manual tests
o Easy to reproduce defects
© 2015 SOASTA. All rights reserved. July 31, 2015 5
o Survey Question 1: • What kind of testing are you doing now? • Doing some automated testing with manual • Doing mostly automated testing with manual • Doing only automated testing
o Survey Question 2: • How are you automating your tests? • Developers write tests as code • QA team records tests from manual scripts
o Survey Question 3: • Are you satisfied with your test automation process? • Yes. Why are you here?
© 2015 SOASTA. All rights reserved. July 31, 2015 6
o Applause
o Ranorex
o Appium/Selenium
o Robotium
o Perfecto Mobile
o Device Anywhere
o SOASTA TouchTest
o AWS Remote Device Cloud
o MobileLabs
o mAutomate
o Experitest (formerly SeeTest)
o SmartBear Test Complete
o NTT AppKitBox Remote Test Kit
o MonkeyTalk
© 2015 SOASTA. All rights reserved. July 31, 2015 7
o Is your QA team good at manual testing or writing test code in a high-level language (Java, JavaScript, etc.)?
o Do you want one tool for Android and iOS?
o Does your company prefer open source (free) or commercial products?
o Do you want tools that install locally or are SaaS/cloud-based?
o Is your team located together or multi-site?
© 2015 SOASTA. All rights reserved. July 31, 2015 8
Product Company Where License
Appium OpenSource (SauceLabs)
Desktop or Server (Mac, Win, Linux)
OpenSource
TouchTest SOASTA Cloud Commercial
TestComplete SmartBear Windows Commercial
Ranorex Ranorex Windows Commercial
DeviceAnywhere Keynote Windows Commercial
© 2015 SOASTA. All rights reserved. July 31, 2015 9
Service Support AWS Remote Device Cloud New – Android, FireOS
Hundreds of devicesAppium, Calabash, UIAutomator
SauceLabs Android simulators, devices (beta) Appium, manual
NTTR AppKit Android, iOS Hundreds of devices Any automation, manual
Keynote Device Anywhere Android, iOS Hundreds of devices Keynote, Selenium
MobileLabs Android, iOS Your own devices
© 2015 SOASTA. All rights reserved. July 31, 2015 10
o Record and Playback • Easiest approach to start mobile test automation • Script parameterization
o Code-based
© 2015 SOASTA. All rights reserved. July 31, 2015 24
o How many releases of Android [GooglePlay]
o How many device types – most popular
o Emulators • One per system; cloud hosted e.g. SauceLabs • Not the same as running on hardware
o Real devices • Real experience • Costly, hard to get
© 2015 SOASTA. All rights reserved. July 31, 2015 25
o
© 2015 SOASTA. All rights reserved. July 31, 2015 26
© 2015 SOASTA. All rights reserved. July 31, 2015 27
o Real User Monitoring – collect user metrics
o mPulse
o Crittercism
o Flurry
o …
© 2015 SOASTA. All rights reserved. July 31, 2015 28
o Desktop: direct connect devices to your desktop computer
o Lab: set up devices in a lab with automation
o Device Cloud • On-premise device cloud: your own devices set up for access by anyone at
your company. Examples: Mobile Labs, Device Anywhere • Public cloud: publicly hosted devices accessible on a pay-as-you-go model.
Examples: Device Anywhere, NTT Remote Test Kit, SauceLabs, AWS Device Farm
• Private cloud: Reserved devices hosted by third-party. Examples: NTT Remote Test Kit, Device Anywhere
© 2015 SOASTA. All rights reserved. July 31, 2015 29
o Record and Playback
o Add screenshot validation
o Add object validation
o Run on a new device
o Adjust for new devices
o Add data input from seed data
o Validate seed data
© 2015 SOASTA. All rights reserved. July 31, 2015 30
o Download from Google drive http://soasta.io/AnDevConFiles
o Install onto Android device
© 2015 SOASTA. All rights reserved. July 31, 2015 31
o Log into http://soasta.com/touchtest/sign-up
© 2015 SOASTA. All rights reserved. July 31, 2015 32
o Demo
© 2015 SOASTA. All rights reserved. July 31, 2015 33
o Create a Composition to hold the Clip (test script)
o Can easily duplicate a clip for multiple devices
o Hit Play button
o Watch test proceed, look for failures • Debugging
o Other capture test data: CPU, Memory, Battery, Bytes I/O
© 2015 SOASTA. All rights reserved. July 31, 2015 34
o Remote device can be anywhere • On someone’s desk • In a lab • In a remote device cloud
o Using NTTR Remote Test Kit • Get a device • Install an app • TouchTest Agent • App to be tested
• Launch TouchTest Agent • Use API for automation
© 2015 SOASTA. All rights reserved. July 31, 2015 35
o Scommand: command line interface to TouchTest server • Start a test job • Get test results • More…
o Also available as a Jenkins Plug-in • Click Manage Jenkins • Click Manage Plug-Ins • Click Available • Look for SOASTA CloudTest • Install
© 2015 SOASTA. All rights reserved. July 31, 2015 36
o Emma