SW Testing -part of a better Process

CERN-BE-BI-SW Training DayM.Andersen

Testing late vs. early

Why is software late & buggy?

1. Rushed timetables ->Hope is lost 2. Lousy design -> Hope is there 3. Inadequate testing -> Hope is there

Jack Ganslle – Embedded SW Guru

Unlike HW which still can break under load, a tested SW stays always perfect!

How do we test?

cout<<“value=”<val<<endl;

System.out.println(“value=”+val);

Main(){MyNewClass a = new MyNewClass();a.doMethod(); a.doMethod();cout<< a.data()<<endl;

}

When to test?• When left to the end of the

project, which is running late... testing is the last thing on your mind.

• Final functional testing only covers 20% of the code!

• Only 20% is new code writing so you won’t escape, 80% is bugfixing and testing anyway!

• XP: Write software tests software, before or at least at the same time with coding features Output

Input

Test Driven Development 1• Writing tests clarifies the

purpose of the code.• Goes in hand with specs• Reduces time in rework• Less time spent debugging• Code simplification

1. You only do what is needed to test

2. You think in small modules to be tested

Test Driven Development 23. Makes code less

coupled- thus reusableImproves quality and

reduces bugs by:4. Making you think in

terms of purpose and specs.

5. Simple≠Complexity↔Bugs

3. Maintainable

http://agilepainrelief.com/notesfromatooluser/2008/10/advantages-of-tdd.html

Does it work?

• "measuring a over 20 projects: if you have a large number of unit tests your code will be an order of magnitude (x10) less complex.“

• Controlled study results:1. “..quality increased linearly with the number of

programmer tests...”2. “..test-first students on average wrote more

tests and, in turn, students who wrote more tests tended to be more productive...”

http://agilepainrelief.com/notesfromatooluser/2008/11/misconceptions-with-test-driven-development.htmlhttp://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf

Types of SW test

Unit Test

Functional Test

Regression Test

Acceptance Test

Integration Test

Load Test

Good to be aware which one we are doing

Acceptance (Functional) Testing

Requirements documentFeature A

descriptionFeature B

descriptionFeature C

description

Acceptance Test documentTest Feature A:

test case atest case b

Test Feature Btest case dtest case etest case f

Test Feature Ctest case h

Example Functional specifications example

In some companies done by QA section and often outsourced

Unit Testing

• Individual components, modules, classes not the final application.

• Independent of context• Makes you think and

design modular code• Requires extra code• Easy in Java & Eclipse

using JUnit

JUnit Step 1Step 2

JUnit

Step 4

Step 3

assertTrue(d4.hashCode()==d5.hashCode()

);

Testing GUI in Java

Challenge: How to automate test for a GUI?• You need human to press buttons..• You need human to read out fields, graphs..

UISpec4J framework

• .jar library to put in a class pass.• Uses JUnit• Offers a large set of wrappers for Java Swing

graphical components• Allows users to write Unit and Full system

tests for Panels or whole GUI Apps

UISpec4J mechanism

JFrame

Window

JPanel

JMenuBar

Panel

MenuBar

getMenuBar()

getPanel()

JButton

JLabel

TextBoxgetText()

getButton(“Set”)

Button

click()

getSubMenu(“Open”)

MenuItem

getTextBox()

JMenu

UISpec4J Code

Integration Testing with Bamboo

• Build server runs on cs-ccr-builds (BE-CO)• Java projects are rebuild with their

dependencies• Executes Java JUnit tests at 2:00 at night• Provides a detailed status report from the test

runs.• Sends an email with report on success or fail.• http://builds.cern.ch/browse/LHCFB-OFSUTES

TS

Example: OFSU test on Bamboo

C++ testing

• BE-CO Fesa team started using Google test framework

• Profound experience. Direct quote: “ ..The problem with testing is that you discover how bad your code is!...”

• You see bugs in code you thought was OK• You see bad design & architecture (tight coupled

vs. Loose coupled code, complex dependencies...)

Google Test

C++ Testable code

1000+ lines of code inside RT actionSmall loosely coupled objects invoked from RT action

Execute() Execute()

Obj A

Obj B

Obj C

References

• http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf

• http://www.uispec4j.org/getting-started• http://www.vogella.de/articles/JUnit/article.html• http://www.softwaretestinghelp.com/types-of-software-

testing/

• http://searchsoftwarequality.techtarget.com/resources/Software-Testing-Best-Practices

• http://www.robthefiddler.com/java/using-uispec4j/• http://java.sun.com/developer/Books/javaprogramming

/ant/ant_chap04.pdf