ios and android acceptance testing with calabash - xcake dublin
TRANSCRIPT
What is Calabash?
What is Calabash?• Automated acceptance testing of mobile apps
What is Calabash?• Automated acceptance testing of mobile apps
• Cross-platform (Android & iOS)
What is Calabash?• Automated acceptance testing of mobile apps
• Cross-platform (Android & iOS)
• Open source - Company called Xamarin backing it
What is Calabash?• Automated acceptance testing of mobile apps
• Cross-platform (Android & iOS)
• Open source - Company called Xamarin backing it
• https://github.com/calabash/calabash-ios
What is Calabash?• Automated acceptance testing of mobile apps
• Cross-platform (Android & iOS)
• Open source - Company called Xamarin backing it
• https://github.com/calabash/calabash-ios
• https://github.com/calabash/calabash-android
Others
• Frank, KIF, Zucchini, UIAutomation, iCuke, …
Why would I do it?
Why would I do it?
Why would I do it?
Other Testing Methods
• Unit Tests
• Integration Tests
• Acceptance Tests
Unit Tests• Tests smallest unit of functionality
public void TestPhoneValidator(){ string goodPhone = "(123) 555-1212"; string badPhone = "555 12"
PhoneValidator validator = new PhoneValidator();
Assert.IsTrue(validator.IsValid(goodPhone)); Assert.IsFalse(validator.IsValid(badPhone));}
• Mocks and Stubs to simulate behaviour of other objects
Integration Tests
• Verify that combined modules function correctly
Acceptance Tests
• Performing tests on the full system (e.g. using your web page via a web browser)
• Described in plain English
How does it work?
• http://blog.lesspainful.com/2012/03/07/Calabash-iOS/
Install
Install• ruby -v -> recommended version 2.0+
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
• cd path-to-my-ios-project
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
• cd path-to-my-ios-project
• calabash-ios setup
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
• cd path-to-my-ios-project
• calabash-ios setup
• calabash-ios gen
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
• cd path-to-my-ios-project
• calabash-ios setup
• calabash-ios gen
• in Xcode, build -cal scheme
Install• ruby -v -> recommended version 2.0+
• gem install calabash-cucumber
• cd path-to-my-ios-project
• calabash-ios setup
• calabash-ios gen
• in Xcode, build -cal scheme
• cucumber
If all goes well:
• LPSimpleExample[11298:13703] HTTPServer: Started HTTP server on port 37265
Build Directory• export APP_BUNDLE_PATH=/Users/rolandgr/
<…>/Build/Products/Debug-iphonesimulator/<…>-cal.app
Gherkin
• https://github.com/calabash/calabash-ios/wiki/01-Getting-started-guide
• Given …
• When …
• Then …
• And …
Useful Stuff
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
• cucumber NO_STOP=1
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
• cucumber NO_STOP=1
• cucumber -t @current
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
• cucumber NO_STOP=1
• cucumber -t @current
• cucumber -t @~excludedtag
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
• cucumber NO_STOP=1
• cucumber -t @current
• cucumber -t @~excludedtag
• calabash-ios console
Useful Stuff • export DEVICE_TARGET="iPhone 5s (7.1 Simulator)"
• cucumber NO_STOP=1
• cucumber -t @current
• cucumber -t @~excludedtag
• calabash-ios console
• > start_test_server_in_background
Query Syntax• query(“button")
=> [
{"class"=>"UIRoundedRectButton", "frame"=>{"y"=>287, "width"=>72, "x"=>100, "height"=>37}, "UIType"=>"UIControl", "description"=>"<UIRoundedRectButton: 0x7d463d0; frame = (100 287; 72 37); opaque = NO; autoresize = RM+BM; layer = <CALayer: 0x7d46ae0>>"},
{"class"=>"UIRoundedRectButton", "frame"=>{"y"=>215, "width"=>73, "x"=>109, "height"=>37}, "UIType"=>"UIControl", "description"=>"<UIRoundedRectButton: 0x7d3a760; frame = (109 215; 73 37); opaque = NO; autoresize = RM+BM; layer = <CALayer: 0x7d3a8a0>>”}
]
• query("button index:0 label”)
"find the first button, and then inside of that find all labels”.
• query("button index:0 label", :text)
• touch("tabBarButton index:1")
accessibilityLabel accessibilityIdentifier
Custom Steps - Ruby API
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
• keyboard
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
• keyboard
• scroll
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
• keyboard
• scroll
• rotate
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
• keyboard
• scroll
• rotate
• backdoor -> call method in iOS app (implement in app delegate)
Custom Steps - Ruby API• https://github.com/calabash/calabash-ios/wiki/03.5-Calabash-
iOS-Ruby-API
• query, check for existence of elements, wait for elements to exist
• keyboard
• scroll
• rotate
• backdoor -> call method in iOS app (implement in app delegate)
• macro -> call other step definitions
http://www.slideshare.net/mobiletestsummit/ios-automated-testing-with-calabash-tips-and-tricks?qid=0afecec1-fe64-4168-bb69-a548bd01b832&v=default&b=&from_search=8
Tips and Tricks
Tips and Tricks• Resetting app data between scenarios
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
And I wait to see "send button"
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
And I wait to see "send button"
• Wait before you interact with any element
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
And I wait to see "send button"
• Wait before you interact with any element
• Wait after every action to finish
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
And I wait to see "send button"
• Wait before you interact with any element
• Wait after every action to finish
• Escape quotes…
Tips and Tricks• Resetting app data between scenarios
• Avoid sleep - use wait instead:
And I wait to see "send button"
• Wait before you interact with any element
• Wait after every action to finish
• Escape quotes…
• https://groups.google.com/forum/#!forum/calabash-ios
Optional: Compare Images
• gem install imagemagick-binaries
• check you have compare: $ which compare
Optional: Compare Images
• gem install imagemagick-binaries
• check you have compare: $ which compare
Continuous Integration
Code Coverage
Other Info• UIKonf talk ‘Unit Testing on iOS’ by Tim
Brückmann https://speakerdeck.com/tibr/unit-testing-on-ios
• http://calaba.sh
• https://github.com/calabash/calabash-ios
• http://developer.xamarin.com/guides/testcloud/calabash/introduction-to-calabash/
Summary
• Automated Testing - investment that pays off
• Together with unit testing etc: make changes with confidence (still, manual QA required)
• Cross-platform mature toolset
• Continuous integration / code coverage
