THE BUYER’S GUIDE FOR TEST MANAGEMENT SOFTWARE

IN THIS GUIDE YOU’LL LEARN how to select and integrate the right test management solution for your team:

• Why test management software?

• How to buy: some general considerations

• The 5 must-have features

• Tips for evaluating & integrating your new solution

• Where to get additional help

BUT IT’S NOT ENOUGH.

That’s because trends like agility, mobile and cloud computing have QA teams bursting at the seams. As a software development leader, you are under the gun to deliver higher-quality software at an accelerating cadence – and still find time to provide solid evidence of testing at any given moment.

Test case management systems are – and have been – a clear value driver in development organizations. But many of these systems do not support today’s unique testing requirements.

Potential cost of software defects to U.S. economy: $60B per year

35% of senior IT executives can’t deliver the quality they need to within their allocated budget

1 Source: Capgemini’s World Quality Report 2014-15, Sixth Edition: https://www.capgemini.com/thought-leadership/world-quality-report-2014-15 2 Source: http://www.codeguru.com/blog/category/programming/the-cost-of-bugs.html 3 Source: Capgemini’s World Quality Report 2014-15, Sixth Edition: https://www.capgemini.com/thought-leadership/world-quality-report-2014-15

QA & TESTING BUDGETS CONTINUE TO RISEFROM 18% OF THE ENTIRE CORPORATE IT BUDGET IN 2012 TO 26% IN 20141

$60B PER YEAR2

35%CAN’T DELIVER3

The lines between QA and dev have blurred. Quality is now an ‘everyone problem,’ and the inefficiency of managing this cross-functional process is off the charts.

WHY TEST MANAGEMENT SOFTWARE?

• Testing modern software requires access to 30 (or so) dependent apps

• 30% of scheduled testing time is spent on setup & configuration tasks

• About ½ of testing teams are still using spreadsheets to manage this

• The majority of time spent on manual testing is still the largest portion of a typical project**

• As a result, software is going out the door only 50% tested

Thus, teams are trying to manage more complexity and at the same time trying to move faster than ever before. They need software that can help them excel in this challenging environment. Unfortunately, most test management tools were created to support more traditional testing methods. They have not caught up with the unique demands of the modern agile tester.

Test management software in the new generation empowers teams to work faster and closer than ever before, with an easy-to-use process for being more effective with less documentation. It provides a platform for transparency, tight collaboration and value-driven techniques such as exploratory testing to support tester’s role as an integrated arm of the business.

ACCORDING TO A RECENT STUDY*

50%OF ALL SOFTWAREGOES UNTESTED

*Source: http://blog.parasoft.com/service-virtualization-infographic

** Source: Gartner - http://cdn.soasta.com/wp/wp-content/uploads/2013/08/Gartner-MQ.pdf

SOME GENERAL CONSIDERATIONS

NOT ALL TEST MANAGEMENT SOFTWARE IS CREATED EQUAL

• Efficiency. Does the tool accelerate the everyday repetitive tasks related to test planning and documentation? Can your testers integrate with and work within their current tool set, or will the burden of independent tool sets chip into productivity? The testing tool must integrate with other ALMs and development tools. Finally, does the tool help you drive greater test coverage and insights – without hiring more resources?

• Flexibility. How easily does the software help you support the various testing techniques employed in your organization? Can it support organizations transitioning from waterfall to agile? Can you support testing on the wide array of platforms we see available today – desktop, tablet or mobile.

• Hidden costs. The biggest mistake organizations make is to stick with an ineffective legacy suite because it’s ‘too costly to exit.’ These platforms often come with huge hidden costs beyond licensing: implementation, customized integrations, web hosting access, server maintenance, and soft costs such as productivity drain and low user adoption. A true multi-tenant, cloud-hosted environment can grow on-demand as needed, and can reduce the cost of software maintenance and upgrades.

• Usability. Is it easy to onboard new users and get them up to speed on the system? Is there a good balance between executive control/oversight and what the testing team needs to be successful? Tools that are built by testers for testers usually yield the best individual ROI.

• Support services. Is the vendor committed to providing the highest level of support possible throughout the lifetime of your contract? Perpetual or maintenance licensing models usually don’t incentivize the vendor to provide high levels of innovation and incorporation of feedback, as your payment has already been collected. Subscription models ensure that the vendor earns your continued business and allows flexibility in moving tool sets. This includes partnering with you to make sure you get true value from the software during onboarding and beyond, with 24/7/365 help, online support, etc.

• Scalability. Can the tool (REALLY) fulfill the needs of your team at your current size, but also scale up to perform well as you grow? You can avoid a big overhead mess by choosing a vendor that believes in a sustainable, democratized product roadmap through a common codebase. Also, make sure the vendor is committed to continuing development rather than creating shelfware that fits today’s needs but won’t fit in the future. These are good questions to ask:

• What exciting new features are on the vendor’s roadmap and what’s the value to your company?

• Are they leading the market through new innovation, or is the product static?

• What investments have they made in product development? A large dev team is a good indicator of solid commitment.

5 ABSOLUTELY, POSITIVELY MUST-HAVES

1. Visual Proof of Testing (including mobile): One of the most powerful tools in the tester’s tool kit is a recorder that can clearly capture everything s/he tested without manual effort. For managers, being able to see a centralized repository of these results can provide unparalleled visibility into actual testing coverage. Clear, irrefutable evidence of who did what, when they did it, and how they did it allows your team to pinpoint exactly what went wrong, without having to try to constantly recreate the issue. While manual screenshots and video evidence are popular, they’re not foolproof, and may leave gaps in documentation that cause larger issues down the road.

2. Customized integrations: The faster the pace of development, the more important this is. How easily and flexibly can the software be configured to support the latest versions of external, third-party defect management tools and automation frameworks? Are these integrations customizable and extensible via API?

3. Uninhibited workflow: The tool should fit your team’s process rather than retrofitting the process for the tool. Are there features that empower testers rather than hinder them? Some examples include functional tabs that provide insight into requirements on test case creation screens

• The ability to easily build traceability between user stories and test cases, import and export bugs

• Session manager to centralize recordings of the manual testing performed.

4. Visibility & collaboration. Seamless, drill-down visibility to all testing from a centralized platform. Especially for teams located around the world, they need one place to collaborate and track testing progress and updates efficiently – including all requirements and defects.

5. Robust analytics. It’s critical for testing software to have reporting that provides actionable insights on issue areas in the testing process. Analytics can help your team better measure and predict quality within their application to drive process improvement and greater quality. They should also give leaders and executive sponsors a high-level view of application health and team effectiveness.

TIPS FOR EVALUATING & INTEGRATING YOUR NEW SOLUTION

• Try to invest as much time as possible in getting real data

• Look at the trial as an investment; take time up front to “move-in” to the system

• Appropriately connect the trial to the production instances of your defect trackers; this ensures compatibility with the customizations in production

• Work closely with the software provider during the trial to understand the level of service and support you will receive if you become a customer

NO SHORTCUTS DURING THE TRIAL

• Provide cross-team clarity as to what business needs the system will solve

• Encourage the dev and testing teams to work together to define the testing approach moving forward

• Use the software to enable conversations and visibility about requirements, plans, priorities and metrics

INTEGRATE QA + DEV

EFFECTIVE INTEGRATION BETWEEN TESTING AND DEVELOPMENT PROVIDES GREATER ALIGNMENT AND VELOCITY IN AGILE TEAMS. FOLLOW THE STEPS BELOW TO ENSURE YOU ARE MAXIMIZING YOUR AGILE POTENTIAL.

WHERE TO GET MORE HELP

• Mobile Testing: A New Animal Needs New Strategy: http://www.qasymphony.com/mobile-testing-a-new-animal-needs-new-strategy/

• Wikipedia’s comparison of test management tools: https://en.wikipedia.org/wiki/Test_management_tools

• 15 Best Test Management Tools from Software Testing Help: http://www.softwaretestinghelp.com/15-best-test-management-tools-for-software-testers/

• QASymphony Resource Center: http://www.qasymphony.com/community.html

ADDITIONAL RESOURCES

QASymphony helps companies create better software by being the only provider of truly enterprise-level agile testing tools. With QASymphony’s solutions, businesses have the visibility and control needed to ensure application quality in fast-paced development environments. Companies like Adobe, Barclays, BetterCloud, Cardlytics, Salesforce Marketing Cloud, AirWatch and Vonage trust QASymphony to improve their teams’ communication, productivity, and collaboration. QASymphony was named a Cool Vendor in Application Development by Gartner in 2015 and is headquartered in Atlanta, GA.

Learn more at http://www.qasymphony.com/platform.html

ABOUT US

FREE TRIAL at QASymphony.com

844-798-4386

http://pi.qasymphony.com/gartner-cool-vendor-2015-report?cid=GARTCOOLVEND:HOMEPAGE:6-23-15

Gartner names QASymphony “Cool Vendor in Application Development 2015”