24/10/2014 1 © Nokia 2014

SW testing in large SW project Presentation for TTY course TIE-21200 Ohjelmistojen testaus

Sanna Järvenpää

NAdC / Leena Rasinaho

24/10/2014 2 © Nokia 2014

Nokia Networks business in Finland

NAdC / Leena Rasinaho

24/10/2014 3 © Nokia 2014

Networks business in Finland

• Head office in Espoo; other offices in Oulu and in Tampere

• One of the main R&D locations for Networks

• Full range of operations including manufacturing of base station products in Oulu

• Key customers: Nationwide operators Elisa and TeliaSonera; Finnish Railway Administration (GSM-R)

24/10/2014 4 © Nokia 2014

Employees

• 6000 employees

•Over 10% of total Nokia employees

• 3700 R&D employees

• Approximately 400 manufacturing employees

• Employees by location:

•Espoo 3 000 •Oulu 2 300 •Tampere 700

24/10/2014 5 © Nokia 2014

Locations

Espoo •Head office functions •End-to-end Mobile Solution Center

Oulu • Radio Research & Design Center •Base Station manufacturing

Tampere •End-to-end Network Operability Center

24/10/2014 6 © Nokia 2014

Ecosystem • Local universities providing access to latest technologies which

utilized via coop projects and thesis workers for our R&D projects

• Lots of user experience and process automation expertise

in local companies. Coop projects with TUT, UTA, TAKK and TAMK

• Close, regular and trustful relationship with Tampere City officials and

mayor to discuss the future plans mutually. Visible e.g. via Tampere

spirit Training where public funding available

Highlights

• End to end Operability, User Experience and Process Automation

• Operations Support Systems

• GSM Base Station Controller and WCDMA Radio Network Controller

• SW maintenance services

• Real-time data processing for CEM via Traffica

• Shared Reference Network lab and Global OSS Lab

Technology Expertise

• System integration competences for OSS and end-to-end operability

• Customer experience management, Traffica, Service assurance

• GSM / WCDMA Controller Specification, SW and I&V

• System Modules, System on Chip development

Tampere – SoC Development, Operations Support Systems and Customer Experience Center

24/10/2014 7 © Nokia 2014

SW testing in Nokia NetAct product

NAdC / Leena Rasinaho

24/10/2014 8 © Nokia 2014

NetAct

General Architecture in NetAct

3RD party tool

Other

vendors

NEs

Alarms

Configurations

Measurements

GSM

WCDMA LTE

MVI

24/10/2014 9 © Nokia 2014

NetAct functionality

• New features

• Regression testing for functionality in old releases

Network element support in NetAct

• Basic support

• Regression testing for functionality in old releases

• New feature support

What to test

24/10/2014 10 © Nokia 2014

Test Planning

FC / US definition into

backlog

FC analyze/design

phase

FC Test Plan definition

FC / US implementation and testing

Development team +

architect(s) +

spec people +

documentation people

iterative

process

FC = Feature Component US = User Story

24/10/2014 11 © Nokia 2014

Component testing

Unit testing – Junits – first level of regression testing

Integration testing

How components work together

System Testing

Functional WHAT – Testing in development teams, Network Verification, System Verification

Non-functional HOW – Performance, Reliability, Load, Sequrity, Usabilility

Acceptance testing

Customer pilots, development teams defined SAT cases

(SAT = System Acceptance Test)

Validation: Building right product - Verification: Building product right

24/10/2014 12 © Nokia 2014

Static testing code review specification review test case review documentation review

Dynamic testing black-box testing ~ specification based, use cases white-box testing ~ implementation based experience based testing ~ regression testing, add-hoc testing

Static – Dynamic testing

24/10/2014 13 © Nokia 2014

NetAct product contains several building blocks that needs to work together

1. Each building block runs there own test 2. System verification tests that all building blocks work together 3. Network verification tests that network elements work together with

NetAct

Building one NetAct product

NetAct

SWM

PM CM

FM

Trace

iSON

(1) (2)

(3)

24/10/2014 14 © Nokia 2014

TestAutomation in NetAct

NAdC / Leena Rasinaho

24/10/2014 15 © Nokia 2014

Test environment installation

NetAct

Pre-tests Reporting

NetAct installation

System component

upgrade

24/10/2014 16 © Nokia 2014

Jenkins-builds

Copy to System build

Upgrade to CI lab

Execute smoke cases

Execute regression cases Automatic promotion

to NetAct product

Fix for the faulty component.

Responsible team

to check and fix

Execute daily regression cases

Execute weekly regression cases

Failing

Success

Test Automation in system component build creation

Responsible team

to check and fix

24/10/2014 17 © Nokia 2014

Building one NetAct product

NetAct

24/10/2014 18 © Nokia 2014

Different Testautomation types

Purpose Number of tests

Execution time

Smoke Can system component build be installed to system component test environments

32 ~ 15 min

Regression Can system component build be included to NetAct build

39 ~ 15 min

Daily regression

Is the basic functionality working with system component build Tested against daily installed newest system component

118 + 87 + 320 +143

4 parallel runs ~ 8 h

Weekly regression

Is the basic functionality working with system component build Tested against weekly installed system component

Several runs during week

24/10/2014 19 © Nokia 2014

Test Automation reporting

24/10/2014 20 © Nokia 2014