sensible test automation
TRANSCRIPT
T7 Session 10/27/2016 11:30:00 AM
Sensible Test Automation Presented by:
Jerry Penner
Waterloo Hydrogeologic
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starcanada.techwell.com/
Jerry Penner Waterloo Hydrogeologic Jerry Penner is a passionate tester who loves helping ridiculously smart people build better software. He has been testing desktop software, websites, web apps, mobile apps, and APIs for startups and large companies for more than ten years. Jerry's favorite testing task is helping developers and testers find important bugs faster. He regularly consumes articles, blogs, books, and opinions on testing, psychology, philosophy, and business, applying what he learns to become a better tester. Jerry has assembled two test teams from the ground up, implemented good practices at two more companies, and is currently helping the fine folks at Waterloo Hydrogeologic.
Sensible Test AutomationAutomated Book Page Turner:
1. Turn handle on toy cash register, drawer opens.2. Drawer pushes golf ball into a funnel, down a ramp.3. Golf ball pulls a string that releases the school bus down a ramp.4. Bus hits a rubber ball on a platform, ball drops into a red funnel.5. Ball lands on a mousetrap on the orange box and sets it off.6. Mousetrap pulls a nail from the yellow stick and (7) releases a weight.8. Weight pulls a cardboard "cork" from an orange tube which 9) drops a ball into a cup.10. Cup tilts a metal scale and raises a wire.11. Wire releases a ball down a red ramp.12. Ball falls into a pink paper basket.13. Basket pulls a string to turn the book page!
Jerry Penner
Test Specialist
Waterloo Hydrogeologic
Audience Participation Time!
What is Automation?
What is Sensible?
Automation Criteria (W5H1)●Who? Toolsmith? Manual tester? Developer?
●What? API? GUI? DB? Installation? Compliance?
●Where? Source / target environments?
●When? When will they have time? When does it run?
●Why? Money? Time? Cool factor? Compliance?
●How? What tools and skills do we need?
Automation Tool Name-Dropping
Interviewer: “Do you have any experience with these tools?”
Tool Selection MatrixDesktop Automation Test Tools Comparison
Company Name Company A Company B Company C
Tool Name CoolTool CoolFreeTool CoolCloudTool
URL example.com/CT www.example.com/CFT www.example.com/CCT
Cost for 2 testers with 2 real & 6 virtual environments, 2 concurrently running
$5,000 USD $0 $9,900 USD
Pros 24 hour priority support
Record/playback with Python scripting
Uses C# & VB.NET as its scripting languages
Cons Cannot be run on an unattended machine.
OS Windows Cross-platform (Java) Windows
Tester Evaluation Playback does not have step-through option to debug the script.
Price Score 3 5 2
Rec/play & script editing 5 5 5
Pros Score 3
Cons Score 3
Ease of Use Score 4
Tester Evaluation Score
Total Score 8 20 7
Tool Value Calculators
Tester Salary Saved Via Automation
Hours Saved/Tester/2 Wk Sprint Running Automation 4
Tester Hourly Salary $ 35.00
Subtotal $ 140.00
Subtotal Annual $ 3,640.00
Subtotal Annual X 2 Testers $ 7,280.00
Job Productivity Gains
Additional Story Points Available Via Automation
Story Points Saved/Tester/Sprint Via Automation 4
Average Investigative Point Count/Bug 2
Potential New Bugs Found/Tester 2
# Testers Running Automation 3
Total Potential New Bugs Found/Sprint 6
Conventional (Nearly) Free Tools
● Imacros (the Click This Website story)
● Selenium
● Sikuli (the Flaky WIFI story)
● DOS/Batch Scripting (the Whitelabel Check story)
● AOSL Scripting Javascript/Python/Ruby/Watir/VBScript/
- Perfect for single-use tasks (the 1 Million Names story)
Conventional (Nearly) Free Tools
● Smartbear products (SoapUI/LoadUI, the Mobile API Stress Testing story)
● SQL Queries (the Performance Test story)
● Xenu Link Sleuth (the Backdoor Email story)
Unconventional Tools & Pairings
“Any tool can be the right tool” - Red Green
Unconventional Use
Excel:- Concatenate() function and Increment feature are great for building URLs- Can connect to a SQL database and pull data for inputs to tests, or for reports- Test plans/cases/scripts- Pivot Tables are great for condensing test run data to reports- Macros for doing repetitive stuff within Excel- Blink testing
Unconventional Use
Notepad++:- Use regexes to find/replace (searching logs, converting Prod data to Staging)- Writing batch & scripting routines- Stripping formatting from copy/pasted data- Daily activity log- Monitoring log files- Creating corrupted files
Unconventional Tool Pairings
● Python & Selenium IDE: (the Image Checker story)● Notepad++ & Excel: (Sort in one, regex find & replace in the other)● Python & HTML & Javascript: (the Metrics Dashboard story)
Online Tools
Online Converters & Generators:- Lorem Ipsum- Richard Ishida's Code Converter- Credit Card Generator- RSS feeds- W3C Markup Validation Service
Mobile Tools
● Monkey● Elevator / microwave oven● 2.4GHz phone / headphones● Aluminum foil on the AP antenna● Keynote - Mobile device warehouse● MobTest - Real People testing● AWS & LoadUI - Load testing outside your walls (the VOIP Phone System story)
Hack Tools
Stuff you might be ashamed of, but it works (the Screen Saver Interrupter story)
Unconventional Tools
Audience Participation!
What have YOU MacGyvered?
Resources
Credit Card Generator: www.getcreditcardnumbers.com/DOS Batch Scripting: en.wikibooks.org/wiki/Windows_Batch_ScriptingFake User Data: names.igopaygo.comiMacros: addons.mozilla.org/en-US/firefox/addon/imacros-for-firefox/Keynote: www.keynote.comLorem Ipsum: www.lipsum.com/Mobtest: www.mobtest.comMonkey: developer.android.com/studio/test/monkey.htmlNotepad++: notepad-plus-plus.orgPython: www.python.orgRegex Cheat Sheet: www.regular-expressions.info/Richard Ishida's Code Converer: http://r12a.github.io/apps/conversion/Selenium: www.seleniumhq.org/Sikuli: www.sikulix.comSoapUI: www.soapui.orgW3C Markup Validation Service: validator.w3.orgXenu: home.snafu.de/tilman/xenulink.html
https://ca.linkedin.com/in/jerrypenner