agile automation-practices by shadi mousa galil software
Post on 18-Dec-2014
361 Views
Preview:
DESCRIPTION
TRANSCRIPT
1 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Implementing Agile Automation
Shadi Mousa
QA Automation Director - Galil Software
1
2 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Automation in Agile – what’s different?
So, where to begin?
Developing Automation in an Agile team
The Automation team & responsibilities in Agile
Agile Automation in large scale organizations
The ATDD/BDD approach
Case Studies
Agenda
3 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Agile requires “fast” automation development and execution; within
sprint time frames.
Collaboration in Agile teams ensures the developed products are
testable by automation tools.
Stories to automate are “simple” and clear – it is easier to prioritize
automation tasks.
Automation milestones should be aligned with product release
milestones.
Automation code refactoring may be needed several times;
especially with immature products and when automating fast.
ROI of automation in Agile is clearer and more measurable.
Automation in Agile – What’s different?
4 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Assess current available automation
Used tools
Developed code and available auto tests
Required automation skills to support current automation
Select appropriate tools and framework
Define automation targets
Automating procedures - continuous environments; development, integration and deployment
Automating installation and configuration scripts
Automating test scripts - regression & progression
Build an automation resources strategy
Who will develop automation code?
Who will develop script level?
Who will lead the automation roadmap, integration, E2E …?
Consult with Galil Software automation experts
So, where to begin?
5 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Developing automation in Agile
6 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Option 1 – Self contained team
Tester skills should include the capability to create the required stories in the automation tool format
Product developers are responsible for implementing the code layer of the automation tool
All required tools and automation framework are the team’s responsibility
Option 2 – Dedicated Automation resources
The larger and more complex the product becomes, the more the need to establish a dedicated team
Automation must include automation experts, including skills with the selected tool
The Automation team & responsibilities in Agile
Role Responsible
Option 1 Option 2
Scru
m T
eam
Test planner Manual Tester Manual Tester
Automation developer Developer Auto Expert Automation executer Manual tester Manual Tester/ Auto Expert
Automation Maintenance Developer Auto Expert …
Au
tom
atio
n
Team
Automation Road Map responsible Scrum Master/Product Owner Automation TL
Infrastructure & Tools ??? Auto Experts Integration Between Multiple Teams ??? Automation TL
7 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Automation in large scale organizations
A Core Automation Team is required
Centralized automation development and maintenance
Responsible for integration and E2E automation
Dynamic automation resource assignment
8 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
ATDD – Acceptance Test-Driven Development
BDD – Behavior-Driven Development
The main idea behind ATDD and BDD, in the aspect of automation,
is that the test automation is built before coding, and delivering the
new code depends on the success of the automation
Human language as a base for automation scripts:
User story
“As a [role] I want [feature] so that [benefit]”
Acceptance criteria
Given [initial context], when [event occurs], then [ensures outcome]
Tools that implement ATDD / BDD: JBehave, Cucumber, Robot …
The ATDD/BDD Approach
9 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Product – Operations Management & Optimizer Service
Operations Admin portal – Web Client
Operations Optimization Server – ASP .NET server
Database - MSSQL
Team and Method
Method: Scrum
Team: 5 Developers, 2 QA, 1 Team Leader (Scrum Master)
Team Responsibility
CR, bug fixes, features and improvement requests within a production environment
Sprint Description:
Sprint duration – 3 weeks
Content - contains sub tasks of CR, feature development, design and testing tasks…
Case Study 1
10 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Scrum Master requested automation:
“I want to have a strong regression testing suite that will verify the
correctness of complicated features; this will be used during the sprint
and also in the future when working on other sprints/features.
In addition, the automation should run in the integration environment
before deployment to customer.”
Selected feature to automate:
New optimization option added and activated via a button from the
Administrator Portal; the data entered by the administrator is optimized
upon request.
Automation “Starting point”:
The customer worked with an outdated and unmaintained automation.
The decision was to start from scratch.
Case Study 1 – Cont.
11 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Selected tool – Robot Framework with Selenium2Library
Resources:
The project started with 1 QA engineer who was part of the team
QA engineer had the relevant programming skills
Task assignment as part of the sprint tasks
Case Study 1 – Cont.
Defined Script:
Admin enters “Operation” data via Admin portal web application
Activate the new optimization option on the added data
Verify the entered data in the appropriate table in the DB
Verify the converted data in the targeted table is equal to the new table
Run on different sets of data and expected results
12 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Case Study 1 – Cont. Robot Script example *** Test Cases *** TestCase 1 - Login to Admin Portal Open Browser ${AdminPortalUrl} ${BROWSER} Maximize Browser Window ${username} DataReader.getValue ${Credentials_fileName} AdminPortalUser ${password} DataReader.getValue ${Credentials_fileName} AdminPortalPassword Login Portal ${username} ${password} TestCase 2 - Data Sets Import Verification ${DataFileList} ${length} DataReader.getFileList : FOR ${INDEX} IN RANGE 0 ${length} \ Click Link ${Import_Operation Data_link} \ ${XSL_fileName} DataReader.getFileName ${DataFileList} ${INDEX} \ Load Operation Data File ${XSL_fileName} \ Press Button ${Optimize_Vacation_button} \ Press Button ${Accept_Data Change_button} \ ${DB_connection} Connect to Database ${DB_host} ${DB_user} ${DB_password} \ ${isEqual} Compare DB Table to XSL ${DB_connection} ${Vacation_Opt_table} ${XSL_fileName} \ Assert True ${isEqual} Table Data is not equal between ${DB_host}:${Vacation_Opt_table} and ${XSL_fileName} \ Close Browser [Teardown] Close Browser
13 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Results
Auto script was performed successfully, but in the following sprint.
Various data sets were supplied, and as test data
Script was added to the regression suite (first to be added)
Best Practices and Conclusions
Automation should have dedicated resources, and not rely solely on QA engineers’ “best efforts”; especially in order to align automation deliveries with sprint timeframes
On the other hand, “best efforts” solved wasted time in QA engineer resources
Selecting the right tools is crucial; especially if “start point” of no tools in place
QA documentation was added directly to test scripts
The outcome of the scrum team automation must be shared with other teams
Next steps
Request for dedicated resources; by increasing the QA testers with programming capabilities and they perform manual AND automation testing.
In parallel, create and develop automation keywords for previous developed features
Define and execute a work plan for integration environment
Case Study 1 – Cont.
14 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Product – ITIL System
Phone rep client – Windows desktop application (Winforms)
ITIL Servers – farm of ITIL servers
Team and Method
Method: Scrum
Team: 4 Developers (1 remote – near shore), 3 QA (offshore), 1 Team
Leader (Scrum Master)
Team Responsibility
CR, bug fixes, features, and future versions of the client
Sprint Description
Sprint duration – 2-3 weeks
Content - contains sub tasks of CR, feature development, design, unit tests
and testing tasks…
Case Study 2
15 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Scrum Master requested automation:
“I want to have a strong regression testing suite that will verify the correctness of complicated features.”
“Automation will be added to the nightly build environment.”
“I also need automation for the client as an independent component without the need to set up the entire system.”
Selected feature to automate:
New field was added to the client; a value entered in the field is sent to the server
Server may reply with different results according to submitted value
Connection timeout should be handled
Automation “Starting point”:
Client – No automation available
Servers – Available APIs
Case Study 2 – Cont.
16 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Selected tool – Coded UI (TFS)
Resources:
QA testers in the scrum team had no development skills
Development engineers had no capacity to handle automation
External automation expert was assigned to the scrum team
Case Study 2 – Cont.
Defined Script:
Phone rep enters a value in the
client and submits
Rerun over list of optional values
and expected server replies
Simulate reply timeout
17 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Results
Task took longer than expected:
Desktop application was not ready for automation; UI controls missed IDs –
task for development was added to the sprint to reorganize and fix the UI
controls IDs
Auto script was performed successfully, yet completed in the following
sprint
Script was added to the regression suite
Isolating desktop application was performed after writing the script
Case Study 2 – Cont.
Isolating desktop application was
performed by simulating server
connection and responses on local
port of the automation client
18 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Best Practices and Conclusions:
Team Leader feedback:
Adding the automation resource to the scrum team and participating in the design made the application better designed and implemented
Automation should have dedicated resources, with the relevant skills
A supportive team for automation is required, since in such a large system, self contained automation resources cannot perform all required automation tasks
There is a need for automating utility / configuration scripts such as upgrading environments…
Next steps:
To add continuous tasks for developers to make the application testable by exposing the appropriate control IDs, recognized by automation tool
Define and execute a work plan for continuous integration environment
Case Study 2 – Cont.
19 Confidential and propriety Galil Software, Ltd. 2014 1-Jun-14
Thank You
Shadi Mousa
QA Automation Director - Galil Software
top related