express functional testing approach- doing more with less

Express functional testing approach-doing more with less

Anuj Magazine

[email protected]://anujmagazine.blogspot.com

Twitter: anujmagazine

mailto:[email protected]

http://anujmagazine.blogspot.com/

© 2012 Citrix | Confidential – Do Not Distribute2

Have you…• Ever been asked to test a project

with no or very less or not to your satisfaction requirements specifications ?

• Ever been involved in testing a project which had no test cases or a very little time to create them ?

• Ever been involved in testing a project in which features/implementation changes dynamically ?


Citrix Globalization Services


Group’s key challenge


Traditional G11N Functional testing process

G11N Testing Cycle


Situations where Traditional process didn’t scale

Start-up projects Maintenance projects

Typical Shortcomings

• Less Base Language test cases or No Base Language test cases at all

• The “Rapid product changes” syndrome

• Difficulty in seamlessly keeping-up with Product team’s milestones

• Difficulty in dealing with situations when there are “no second chance” of finding bugs

• Regression testing not scientific enough


The basis of Express functional testing approach


What kind of change we were looking to see ?

• Something that can help us deal with the rapid product changes• Something that can help us find bugs soon and early• Something that can help us make Regression testing more scientific• Something that can help us deal with reducing dependency on EN(base

language) test cases• Something that can help us seamlessly merge with product team’s

milestones


We had

• Less documentation

• Lesser base test cases

• Less time at hand

• Less chances of timely bugs

• Less preparedness to deal with chances

We wanted to

• Find more bugs

• Become more responsive to project changes

• Enhances our chances of meeting internal milestones.

• Not increase human resources infinitely

In short, we wanted to do more with less


The basis of Express functional testing approach

Test caseTest case

© 2012 Citrix | Confidential – Do Not Distribute

The essence of Express functional testing

11

SBTM: Session Based Test Management


The Three Core ideas

• Basic source of tests-

a code check-in• Refined approach to testing-

express test ideas generation• Automating functional tests –

reuse the existing code

12

Explaining the basic source of tests- a code check-in

© 2012 Citrix | Confidential – Do Not Distribute Citrix Confidential - Do Not Distribute

AnalysisVerification

Track Daily check-ins

Assess the functional impact

Generate test ideas at runtime

Note the build, results

Report bug if any issue

Perform regression tests

Results are updated in the

Tracker

Elements of the Express functional testing


Components of a check-in

15


“check-in Summary”

Actual Source Code Changes…

No.of Lines in the Code that were affected by the change


check-in types

17

• check-ins related to Feature Changes

• check-ins related to Bug Fixes

• check-ins related to Document Changes

• check-ins related to Changes in Automation script


Check-in types

check-in for feature change

Using check-in Summary and the other details in the

source code we could derive ideas for tests


Some challenges

20

• Evolving the skill level of the team.

• Enhance the Developer skills and mix it appropriately with tester mindset to create an irresistible engineering combination.

• Build rapport/communication with the development team.

• Building the buy-in for the change in the approach.

• Imbibing the necessary Sense of urgency within the team.

Express test ideas generation/testing


AnalysisVerification

Track Daily check-ins

Assess the G11n impact

Generate test ideas at runtime

Note the build, results

Report bug if any issue

Perform regression tests

Results are updated in the

Tracker

Elements of the Express functional testing


Challenges in absence of test cases

• Building accountability in testing.

• Establishing the focus into specific areas.

• Knowing how much is tested and how much more needs to be tested.

• Designing the tests and running them almost at the same time.


Session based testing

• What is a session?

“A session is an uninterrupted block of reviewable, chartered test effort”• Typical sessions last for 90 minutes


Session

Bugs Investigation/Reporting

Test Design/Execution Session Setup

Test design and execution means scanning the product and looking for problems

Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem

Anything else testers do that makes the first two tasks possible, including configuring equipment, reading manuals, or writing a session report


Session Report

Automating tests- Reusing the existing code


How the Automation Process Evolved….• Understanding Architecture• Execution on EN Setup

• Scheduler• Automatic analysis of test results• Auto e-Mail test Result

ANALYSIS

SUPPORT FOR NON-EN PLATFORMS

ENABLE UNICODE SUPPORT

OPTIMIZATION

• Execute Auto BVT on JA, FR, DE, ES, SC, RU languages

• Input non-ASCII characters as Input Data • (Multi Language Data)


• Make the scripts UTF enabled

• # -*- coding: utf-8 -*-

Citrix Confidential - Do Not Distribute

Enable the scripts to input non-ascii characters


Multi Language data“ß証吞暴予禄 дюçœ€äöñé가앜刓訞一么@!23”

Results summary


More Functional bugs per release

32

If further colors are needed, please contact Creative Services

No. of EN issues

No. of I18N issues

No. of L10N issues

No. of Doc issues

0

20

40

60

80

100

120

Project K Project N


Early, Timely detection/fixing of Functional bugs

33

If further colors are needed, please contact Creative Services

Itr 1 Itr 2 Itr 3 UI Itr RC Itr0

20

40

60

80

100

120

Project K Project N


“It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change”

The crux of Express functional testing approach


Images Sources:• http://www.cleapublicrelations.com

• http://www.c3-ssi.com

• http://www.linkbuildr.com

• http://www.torwars.com

• http://www.branchpastor.com

• http://blog.workinsports.com

• http://www.121box.com

• http://www.march-hare.com

• http://www.larter.ca

References

Other Credits:

• Collègues: Rubina Gonsalves and Shivaraj Shet

• http://www.satisfice.com/sbtm/

• http://satisfice.com/articles/sbtm.pdf

http://www.satisfice.com/sbtm/







http://satisfice.com/articles/sbtm.pdf

http://satisfice.com/articles/sbtm.pdf

Work better. Live better.

Anuj Magazine

[email protected]

http://anujmagazine.blogspot.com

Twitter: anujmagazine

mailto:[email protected]

http://anujmagazine.blogspot.com/

express functional testing approach- doing more with less

Documents

citrix confidential

citrix globalization

functional tests

test management

absence of test cases

base language test cases

early testing

project changes