c / c++ testing - metropolia ammattikorkeakoulu · features of c/c++ testing environments ......

C / C++ testing

Grzegorz Filo

Cracow University of Technology

Institute of Applied Informatics

COMPUTER LAWS/MOTTOES

THE ERROR-FREE SOFTWARE DOES NOT EXIST

SOME PROGRAMS ARE JUST NOT TESTED ENOUGH

2 / 54

Presentation plan

Unit testing - why

Issues related to testing in C / C++

Features of C / C++ testing environments

UnitTest++ overview

Testing a simple C program (CodeLite IDE)

Testing a simple C program (Code::Blocks IDE)

3 / 54

Unit testing – why ?

Testing of individual fragments of program code

Carried out at the early stage of the project

Allows the software developer to reduce cost

Requirements Design Coding Unit testing Integration Acceptance

tests

Deployment Maintenance

Defe

ct

fix c

ost

Project phase 4 / 54

Unit testing in C/C++

Platform dependency

Compilers

Memory management

Environments

5 / 54

Testing in C/C++

platform dependency

Java slogan

„Write once – run anywhere”

Hardware-independent to a large extent

C/C++ characteristics

Lower-level language than Java

The need to rely on an unportable platform-specific

extentions

6 / 54

Testing in C/C++

Compiler issues

Java

One compiler

The same byte code on each platform

C / C++

Many different compilers – poor compatibility

Compilation to the machine code of the specific platform

Compiler switches – turning on / off compiler features

7 / 54

Testing in C/C++

Memory management

Java

Garbage collector – automatic memory release system

C / C++

No automatic memory management available

Possible memory leakages

8 / 54

Testing in C/C++

Testing environments

Java

A few consistent, well-developed, well-working testing

environments

C/C++

hard to write portable C/C++ code

limitations of C/C++ (no reflection feature)

many testing environments, neither universal nor portable

9 / 54

Model of C/C++ testing environments

xUnit architecture

Test runner program + result formatter

Test case

Test fixture

Test suite

Assertion

10 / 54

Features of C/C++ testing environments

Test fixture (test context)

fixed state of the software used as a baseline for running tests

actions performed in order to obtain the fixed state

commonly used in the unit testing frameworks

set-up phase – create expected state for the test

tear-down phase – return to the original state

Test fixtures may be grouped to form of the group feature

Generators – allow to generate tests semi-automatically

11 / 54

Features of C++ testing environments 1/2

Mocks

Allow the programmer to create temporary objects (simulate objects)

Simulated objects imitate the real objects behavior, but in the more controlled way

When simulated objects are useful ?

Real object which supplies non-deterministic results (i.e. temperature value from a sensor)

Real object’s state is difficult to create (i.e. error imitation)

Real object is slow (i.e. database or network operations)

Real object can’t be created because it’s not defined yet

12 / 54

Features of C++ testing environments 2/2

Exceptions

Check if exception of defined type is thrown properly

Macros

Check if macros are defined properly

Templates

Allow to conduct tests of templates (generic classes)

Grouping tests

Allow to create groups of tests and run them at once

13 / 54

Example testing frameworks:

C language, 1/2

NAME xUnit Fixtures Group fixt. Generators License

EmbeddedUnit Yes Yes MIT

RCUNIT Yes Yes Yes MIT

SeaTest Yes Yes MIT

Unity Yes Yes MIT

Parasoft C/C++test Yes Yes Yes Yes Proprietary

QA Systems Cantata Yes Yes Yes Proprietary

STRIDE Yes Yes Yes Proprietary

TPT Yes Yes Yes Yes Proprietary

VectorCAST/C Yes Yes Yes Proprietary

According to http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C

number of listed C-language testing frameworks: 45

14 / 54

http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C

Example testing frameworks:

C language, 2/2

NAME xUnit Fixtures Group fixt. Generators License

libcbdd Yes Yes Yes Apache 2.0

Cmockery Yes Apache 2.0

CTest Yes Yes Yes Apache 2.0

lcut Yes Yes Yes Apache 2.0

FCTX Yes BSD

LibU Yes BSD

AceUnit Yes Yes BSD

TF unit test Yes Yes GNU GPL

Test Dept. Yes GPL

Opmock Yes Yes Yes Yes GPLv3

API Sanity Checker Yes Yes Yes Yes LGPL

Check Yes Yes Yes LGPL

CUnit Yes LGPL

CUnitWin32 Yes LGPL

Cutter Yes LGPL 15 / 54

Example testing frameworks: C/C++ 1/2

xUnit Fixtures Group fix. Generators Mocks Exceptions Macros Templates Grouping

Boost Test

Library Yes Yes Yes Yes Option Yes Option Yes Suites

QA Systems

Cantata Yes Yes Yes Yes Yes Yes Yes Yes

CATCH Yes Yes Yes Yes Yes Yes Yes

CATCH-VC6 Yes Yes Yes Yes Yes Yes Yes

cfix Yes Yes Yes Yes

Cput Yes Yes Yes Yes Yes Yes Suites

CppUnit Yes Yes Yes Yes Yes Suites

CPUnit Yes Yes Yes Yes Yes Yes Yes

CUTE Yes Yes Yes Yes Suites

Embunit Yes

Google Test Yes Yes Yes Yes Yes Yes

Hestia Yes Yes Yes Yes Yes Yes Yes Suites

According to http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B

number of listed C++/C testing frameworks: 64

16 / 54

http://en.wikipedia.org/wiki/List_of_unit_testing_frameworksC.2B.2B

Example testing frameworks: C/C++ 2/2

xUnit Fixtures Group fix. Generators Mocks Exceptions Macros Templates Grouping

liblittletest Yes Yes Yes Yes Yes Yes Yes

libunittest Yes Yes Yes Yes Yes Yes Yes

mock++/mockcpp Yes Yes Yes Yes Yes Yes Suites

mockpp Yes Yes Yes Yes Yes Yes Yes Suites

NullUnit Yes Yes Yes Yes Yes Yes Suites

OAKUT Yes Yes Yes Yes Yes XML

Parasoft C/C++test Yes Yes Yes Yes Yes Yes Yes Yes Yes

STRIDE Yes Yes Yes Yes Yes Yes Yes Yes

TEST-DOG Yes Yes Yes Yes Yes Yes Suites

Testwell CTA++ Yes Yes Yes Yes Yes

UnitTest++ Part. Yes Yes Yes Yes Yes Suites

upp11 Yes Yes Yes Yes Yes Yes

UquoniTest Yes Yes Yes Yes Yes Yes Yes Yes

VectorCAST/C++ Yes Yes Yes Yes Yes Yes Yes Yes

xUnit++ Yes Yes Yes Yes Yes Yes Yes Yes 17 / 54


MAKARIEV’S LAWS:

1. WRITING A SOURCE CODE IS A DELIGHT

2. TESTING THE CODE IS A NIGHTMARE

18 / 54

Software Development Environment

Codelite http://codelite.org/

Open source, free IDE

GPLv2 license

Cross-platform

(Windows, Linux, OSX)

The most recent version: 5.4

January 2014

19 / 54

http://codelite.org/

http://codelite.org/

Codelite IDE main features

Built-in support for compilers: gcc/g++, clang, VC++

Support for C++11 standard

Syntax error info / correction

Integrated debugger

Wide range of code refactoring tools

Code completion / navigation

Integrated GUI development RAD tool (wxCrafter)

20 / 54

Contents of CodeLite package

Editor

version 5.4

C/C++ compiler for Windows

MinGW TDM-GCC 4.8.1 (gcc, g++)

GUI support library

wxWidgets 3.0

Testing environment

UnitTest++ 1.3

21 / 54

UnitTest++ overview Unit testing framework for C/C++

Free under the terms of the MIT/X Consortium license

Simple to use and portable

Multiplatform (Windows, Linux, OS X)

Main features minimal use of advanced or platform-dependent elements

automatic registering of tests

good crash handling

uses fixtures

advanced condition-checking functionality

small size, can be used i.e. for embedded environments

22 / 54

UnitTest++ installation process

Designed for cooperation with various compilers

Distributed as the source code

Included project files for SDK

Visual Studio project (VC++ compiler)

CodeLite project (gcc/g++ compiler)

Alternative: command line compile/build using any compiler

Important directories (after compilation)

\UnitTest++\src\ location of header files (*.h)

\UnitTest++\debug\ location of compiled libraries (*.a, *.o)

23 / 54

Project example 1

Simple implementation of linked list in C

CodeLite console-type project

Pure C code (gcc compiler)

Main objectives

Implementation of linked list for storing char sequences (texts)

Each node contains text buffer of a constant size (for code simplicity)

Functionalities

add a node,

get text from selected node,

display texts from all nodes,

clear the list

24 / 54

Project example 1

Creating a new C project in CodeLite

Create or open a workspace

Create new project

25 / 54

Project example 1

Defining structures and functions

Single node definition

struct Node, alias TNode, pointer PNode

Linked list structure definition

struct MyList, alias TMyList, pointer PMyList

Functions linked to the TMyList using pointers (keep in mind the difference: structure in C / C++)

bool add(PMyList list, const char* text);

char* get(PMyList list, int n);

void display(PMyList list);

bool clear(PMyList list);

Function which creates and returns a linked list variable: TMyList createList();

26 / 54

Project example 1

Creating list and performing some operations

Create a linked list variable

TMyList LIST = createList();

Display contents of the list

LIST.display(&LIST);

Add a node to the list

LIST.add(&LIST, "Line number 1");

Get text from the node

LIST.get(&LIST, 0);

Clear the list

LIST.clear(&LIST); 27 / 54


Conclusion from the Pierce’s law:

IF THE FIRST COMPILATION OF THE SOURCE

CODE HAS BEEN SUCCESSFULL

THEN THE PROGRAM CERTAINLY WILL NOT

WORK PROPERLY

28 / 54

Ready-to-use project template

Three default compilers

Creating a UnitTest++ project in Codelite

29 / 54

Using the UnitTest++ - practical issues

UnitTest++ is always a C++ project

However, can be used directly to test C code, no OOP knowledge is needed

You must include UnitTest++ header file

#include <UnitTest++.h>

Path to the compiled library

is set in CodeLite, in other

IDE’s must be added

manually

Only one instruction in main()

return UnitTest::RunAllTests();

30 / 54

Integration of UnitTest++

with Codelite

PluginsUnitTest++ menu option

Create new test… – semi-automatic

test generator

Run Project as (…) – run all tests, see

results on the bottom panel

31 / 54

How to create a test using UnitTest++ 1/4

Test-defining commands

Universal test macro

TEST(name){ // instructions, checks … }

Test macro which uses fixtures

TEST_FIXTURE(fixture_structure, name){ // instructions, checks … }

Test names must be unique except defining tests inside of

suites

32 / 54


Suite-defining command

Definition of suite

SUITE(name){ TEST(test_name){} TEST(test_different_name){} }

Allows for grouping tests

In different suites can be defined tests with identical names

SUITE( initialTests ){ TEST( test1 ){ //… } } SUITE( advancedTests ){ TEST( test1 ){ //… } } 33 / 54


Check definitions (for C and C++)

Boolean expression check – true passed

CHECK(boolean_expression);

CHECK(list.size==0);

Equality/array equality check – values are equal passed

CHECK_EQUAL(expected, actual);

CHECK_ARRAY_EQUAL(array_exp, array_act, count);

Close check – difference less than threshold passed

CHECK_CLOSE(expected, actual, threshold);

CHECK_ARRAY_CLOSE(array_exp, array_act, count, thr);

34 / 54


Exception checks (for C++ only)

Exception throw check –

exception of assumed type thrown – test passed

CHECK_THROW(expression, expected_exception_type);

Exception assertion check –

checks if object of the UnitTest::AssertException class

has been thrown

UnitTest::AssertException derives from std::exception

CHECK_ASSERT(expression);

35 / 54

Test example 1

Creating a unit test for linkedList project

Add a separate header file for each tested file

Create mylist_test.h file

In added header file import "*.c" file which will be tested

We want to test all functions, not only those declared in "*.h"

#include "..\linkedList\mylist.c"

Create test cases and test suites using applicable macros

Compile, build and run test application

36 / 54

Test example 1.1

Creating the initialTests suite

Create a simple parameter-checking test

Name: checkInitialParams

Purpose: check if size of created list is 0 (zero)

Create a fixture – empty list variable

Name: fixCreateEmptyList

Purpose: create a list variable

Create a test fixture 1

Name: checkPointers

Purpose: check if both start and end pointers point at NULL

Create a test fixture 2

Name: addFirstElement

Purpose: check if an element can be added and retrieved properly 37 / 54

Test example 1.1

Code correction – initialTests suite

Problem 1

Size of newly created list is not zero

Solution: set initial size of the list to 0 (zero)

Code: mylist.c, line 55: list.size = 0;

Problem 2 and 3

Wrong size after adding element & element can’t be retrieved

Solution: increment of size after adding the element is missing

Code: mylist.c, line 14: list.size++;

Problem 4

Pointer start or end wrongly assigned

Solution: assign pointers correctly

Code: mylist.c, line 13: list->end = N; 38 / 54

Test example 1.2

Creating the advancedTests suite

Create a fixture – a filled list variable

Name: fixCreateFilledList

Purpose: create a list variable with 100 nodes added

Create a text fixture 1

Name: getElement

Purpose: check size, check the program behaviour when trying to get elements from outside the list and from the list

Create a text fixture 2

Name: listClear

Purpose: check, if the list is cleared properly (size and pointers)

39 / 54

Test example 1.2

Code correction – advancedTests suite

Problem 1

When trying to acquire element from outside the list, result is not NULL

Solution: return NULL also when index is negative

Code: mylist.c, line 19:

if((n<0)||(n>=list->size))return NULL;

Problem 2 and 3

Problems with clearing the list: size and end pointer wrongly assigned

Solution: add missing assignments

Code: mylist.c, line 48: list->end = NULL;

Code: mylist.c, line 49: list->size = 0;

40 / 54

Test example 1.3

Creating the variousTests suite

In mylist.c create a method for calculating the π value double calculatePI();

Based on the Leibnitz equation: 𝜋 = 4 ∙ (−1)𝑖∙1

2∙𝑖+1 ∞

𝑖=0

Create a simple test of floating-point values

Name: testPiValue

Purpose 1: check if calculated value is equal to the M_PI

constant (math.h)

Purpose 2: check if the error is less than given threshold

(0.001)

41 / 54

Test example 1.3

Code correction – variousTests suite

Problem 1,2

Calculated value of π is not equal to library value

Failure 1: Expected 3.14159 but was 3.13659

Failure 2: Expected 3.14159+/- 0.001 but was 3.13659

Solution: increase number of iterations from 200 to 2000

Code: for(i=1; i<2000; i++)res+=pow(-1.,i)/(2.*i+1);

Failure 2: disappeared

Failure 1: still exists

Conclusion:

always compare floating point values with a defined accuracy

42 / 54


6th Essential Computer Law

Computers within the network usually can not connect with others.

Exception is the moment of the virus spread.

43 / 54

UnitTest++ - generate tests for class

C++ only feature

Semi-automatic test generator

for class methods

The procedure

1. Create a C++ project

2. Create a UnitTest++ project

3. Create a test and add test

to the UnitTest++ project

44 / 54

UnitTest++ in Code::Blocks

Initial requirements

CodeBlocks 13.12

UnitTest++ 1.3

compiled using gcc

Path to UnitTest++.h

Path to libUnitTest++.a

A CodeBlocks project

(i.e. ScorchedEarth )

45 / 54

Steps to create test project in Code::Blocks

We assume, that tested project is in the current workspace

Add a new project, i.e. "Scorched_test"

File New Project Console application

Language: C++, project title: Scorched_test, compiler: GNU GCC

In the build options (Project Build options…)

„Linker settings” tab: add library file

<UnitTest++ dir>\Debug\libUnitTest++.a

Search directiories Compiler tab: add the Src directory:

<UnitTest++ dir>\Src

Search directories Linker tab: add the Debug directory:

<UnitTest++ dir>\Debug

46 / 54

Test project in CodeBlocks: main.cpp

The main.cpp is extremaly simple:

#include <UnitTest++.h>

#include "shot_final_equation_test.h"

using namespace std;

int main()

{

return UnitTest::RunAllTests();

}

47 / 54

Test project in CodeBlocks:

shot_final_equation_test.h

Include file, which will be tested (shot_final_equation.c)

Create tests

Compile, link and run the test program

Results will appear in the console window

48 / 54

Test 1

Check if missile variable is created properly

TEST(initializeMissileTest){

missile_data* MISSILE = initializeMissile(10, 10);

CHECK(NULL != MISSILE);

CHECK(strcmp(MISSILE->name, "Super Light Missile")==0);

CHECK_CLOSE(1.8, MISSILE->weight, 1e-6);

CHECK(MISSILE->initial_velocity == 0);

// other params …

}

49 / 54

Test 2

Functions sinDegrees() and cosDegrees()

TEST(cosAndSinDegrees){

CHECK_CLOSE(0.7071, sinDegrees(45), 1e-4);

CHECK_CLOSE(-0.7071, sinDegrees(-45), 1e-4);

// cosDegrees() in the same way

}

50 / 54

Test 3 1/2 Creating a fixture – initial state of missile

struct fixCreateMissileData{ fixCreateMissileData(){

MISSILE.weight = 1.8; MISSILE.initial_velocity = 30.; MISSILE.x_turret_position = 5;

MISSILE.y_turret_position = 5; MISSILE.shot_angle = 45; }

~fixCreateMissileData(){ }

missile_data MISSILE; int test_data[3][2] = { {25, 20},{54, 34},{91, 45} }; };

51 / 54

Test 3 2/2 Using the fixture to test missile position

Check whether calculated values are compatible with reference values

Calculate using fixed values of missile and wind parameters

TEST_FIXTURE(fixCreateMissileData, test){

int i;

float data[2];

for(i=0; i<3; i++){

missile_data M = MISSILE;

M.initial_velocity = 30.+6*i;

xCoordinate(&M, i);

yCoordinate(&M);

data[0] = M.x_vector_coordinate[50*(i+1)];

data[1] = M.y_vector_coordinate[50*(i+1)];

CHECK_ARRAY_EQUAL(test_data[i], data, 2);

}

} 52 / 54

Short summary

UnitTest++ is a free, simple tool for testing,

It can be used with many different compilers

Allows us to carry out basic unit tests

Both C and C++ language programs can be tested

53 / 54


Weinberg’s law:

If buildings were built in the same manner in which

programmers develop applications

Then one woodpecker would destroy the entire civilization

54 / 54

c / c++ testing - metropolia ammattikorkeakoulu · features of c/c++ testing environments ......

Documents