10 Mantras for Test Automation Success

Success doesn’t happen, you gotta make it

Single biggest mistake you can make is to start a test automation exercise without expected outcomes In my earlier post, we discussed in detail about why you need software test automation and some points on why automation may not be effective. 15 years back, software testing was perceived to be an activity to monitor the quality of code and not connected to business outcomes. About a decade back, testing picked up steam. Based on my 22+ years in industry, and having worked on about 150+ software testing engagements, out of which 115+ are test automation exercises, I have discovered that about 50% of the projects fail and the other half succeed. Many unsuccessful projects have one thing in common – They try to automate a manual test case, which they are currently comfortable in. Test automation needs much more than just automating a manual test case. Here are the 10 Mantras for test automation success based on my experience and my conversations within the industry -

1. Process: Many SME’s today focus on getting their product out to market as early as possible and tend to skip the most critical part, which is process. Process will help us to achieve the desired outcomes in a measurable / tangible way. Testing is no different and process is an essential part of testing.

Ensure that you have one in place. Specify every part of the product / project and define coding standards for both development and testing for easy maintainability. Please note that product under has to be test automation friendly. The technology you chose to develop and the development methodology has a direct impact on the kind of tool you will need to use / buy for test automation.

2. Proof of Concept:

Tools are an essential part of test automation. Testing the feasibility as early as possible is essential. It is not only to test automation feasibility but also to ensure that right tool is chosen based on current technology stack / future roadmap. Chose the right requirement that covers all part of technology including its integration with 3rd party product and create a quick test suite to ensure that it is working before embarking on the complete exercise.

3. Test Automation Resources: Remember, test automation is not = to manual testing. Test automation resources are maybe = software developers. Manual testers can over a period of time evolve and learn test automation skills but the notion of just converting a manual tester to automation tester in a time bound automation exercise can be disastrous. Evaluate and hire the right test automation resource / expert / company.

4. Requirements:

a. Product Requirement Document b. Software Requirement Specification

Key to having your test requirements right is by ensuring that you have derived it out of up to date project / product requirement documents. Once you have all the test requirements in place, it would be good to have the entire stake holders on a single page by bring them together for a meeting to discuss core functionality, criticality, priority and severity to business. Based on the outcome, you will now have clear task cut out to start your activity. Remember, trying to automate every part of test requirement is a waste of time and may not be feasible.

5. Design: Design tests not only for today but also for tomorrow. Maintenance of test assets over a period of time can take a toll and many organization spend much time and resource maintaining it or re doing test automation. We all know that tests work on dependencies. Modularizing it properly and making each and every component as much independent as possible leads to better maintainability.

6. Test Framework:

Test Framework can help organization improve test automation productivity. Building functions / modules / resuable components / object repository / test data at a framework level would greatly help organizations in using reusable components thus reducing time to market. Frameworks today can help organizations in conducting effective risk based testing, continious integration testing and thus build a robust develop, test and build platform.

7. Test Scenarios:

I have seen organizations creating test cases out of test scenarios. I beg to disagree with this approach. Business evolves and the scenarios may change which has a huge impact on test cases and maintenance if they follow this method. Test scenarios are normally the business scenarios that the end customer business manager decides and we should develop a framework that provides him an ability to change based on his business needs. Scenarios should be a collection if independent test cases put together. Designing effective test cases helps improve test automation usability.

8. Defect Management: Defects are an outcome of the tests you conduct. By just throwing a report to the developer will not help in the success. Ensuring that you reproduce the steps leading to defects and providing developers time to fix and the ability to retest the impacted test cases using test automation is important. Integrating the test framework with effective defect management system is prescribed as organizations will have better hold of defect slippages, defect reoccurrence and defect trends.

9. Test Management: Implementing a test management tools greatly helps organizations to implement and improve test process. It also helps them in maintaining dependability matrix, one to many requirements and test cases relationships thus providing better control over the test automation exercise. Integration with requirements management tools, defect management tools and build systems provides organizations with ability to manage their test assets properly and pin point to the areas that matter the most. A requirement change impacts only few requirements / test cases. Effective test management tool provides you the ability to point to you the effected test cases that may need change thus reducing the time to maintain and execute tests. Test management tools (SeleniumALM) today provide you an ability to help deploy tests at customer environment

/ cloud and test on multiple browsers / devices and gives you and also the customer better control over your testing activities.

10. Reporting:

A decision cannot be taken without proper reports on hand. It is important that the automation tests provides you with adequate information to take a GO / NOGO decisions. It is also important that you choose the right reports to take decision. Too many reports lead to confusion and delays. Reports such as Test Coverage, Defect Trends (with priority and severity information) are some that are important to consider.

Test Automation success is in your hands and failure is not due to products and people. Having the right process and taking the right decision leads to success. Test automation success / ROI is achieved over a period of time and not overnight.