![Page 1: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/1.jpg)
Selenium Web Testing
Demonstration of a Custom Framework
2012-05-17Aaron Shaver
![Page 2: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/2.jpg)
● Application under test: electronic discovery (pre-trial phase in a lawsuit)
![Page 3: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/3.jpg)
● High level view of the framework and test suite● All test logic and assertions in the test_ directories● Categorized, e.g. can run "nosetests tests_smoke"
![Page 4: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/4.jpg)
● Subfolders contain individual tests● Test dirs are Python packages via the init files, can be imported● nose (test runner) discovers tests based on "test" regular expression
![Page 5: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/5.jpg)
● setup.cfg: nose, WebDriver, and application-specific settings● Change timeouts for WebDriverWait, application task waits● Adjust server URLs, usernames, passwords
![Page 6: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/6.jpg)
● util.py -- utilities module● Product-neutral code: generic functions and classes● Could be reused to test other web apps
![Page 7: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/7.jpg)
● shift.py (our product is called Shift)● Module for all application-specific code● Pulls configuration settings from setup.cfg
![Page 8: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/8.jpg)
● data.py -- another application-specific module● Keeps document count assertions outside of the _test.py files● Easier to maintain: doc counts change sometimes as the app
improves or degrades, e.g. file formats become supported that weren't earlier
![Page 9: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/9.jpg)
● log.txt -- optional test results file● Version-controlled via TortoiseHg (Mercurial) + Bitbucket, so I can
see past test runs● Useful if results exceed cmd.exe buffer size● nose needs special syntax to pipe output of console to the log
![Page 10: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/10.jpg)
● Example of test run outside of log.txt (outputs to cmd.exe window)● Tests slow by usual unit testing standards? App performs many
operations on uploads (recursively extracts, indexes metadata properties, deduplicates, generates reports, etc.)
● Entire test suite takes ~6 hours, easily fitting into an overnight run, so I haven't bothered to setup Selenium Grid yet
![Page 11: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/11.jpg)
● 90% of tests in suite use standard setup routine● App under test needs to ingest data before doing anything useful● All it needs: project name, data source, total docs assertion
![Page 12: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/12.jpg)
● Example of a simple test which does not use the "standard intake workflow"
● Note the product "action" functions: user_login, add_proj● clean_up deletes the project and closes the browser, but only if the
test was successful● Failed tests leave the browser window open, in case it's a new bug
![Page 13: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/13.jpg)
● Built my framework before hearing about PageObjects● Plan was to make "atomic" app actions ("save a search", etc.)● Not as fine-grained as PageObjects● Still a robust layer of abstraction for when UI changes● Ended up clumping together app actions; even less fine-grained than
my own plan
![Page 14: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/14.jpg)
● Developers were nice to me: many UI elements generic across app● No AJAX yet (fingers crossed) -- timing issues from Webdriver itself
![Page 15: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/15.jpg)
● UI is very good / simple / elegant... but not always● Example: reuse of list_item_detail class for both <a> and <div>● Have to ensure what we're clicking is a link● Could do a CSS selector, but would be more fragile
![Page 16: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/16.jpg)
● Potential difficulty: can't automate Java or Silverlight uploaders● Thankfully, administrator backend allows data intake● Uses standard HTML form
![Page 17: Selenium Web Testing - Meetupfiles.meetup.com/2891252/Selenium Web Testing... · Selenium Web Testing Demonstration of a Custom Framework Aaron Shaver 2012-05-17 ... WebDriver, and](https://reader033.vdocuments.net/reader033/viewer/2022052723/5f0d57d57e708231d439e109/html5/thumbnails/17.jpg)
● Questions about my automation?● Suggestions for improvement?