qa in digitalized world by kari kakkonen at kds2015

Quality Challenges of the Digitalized World – How Does Quality Assurance Change? Kari Kakkonen, Director, Quality and Competences, Knowit Oy, Finland at Knowit Developer Summit, 14.11. 2015

© Copyright Knowit Oy 2014

Kari Kakkonen, Knowit

• Speaks, train, coach and mentor regularly about

• ISTQB Advanced, Foundation and Agile Testing + Knowit Quality Professional

• Quality & Test process and organization development

• Agile testing, Scrum, Kanban, Lean

• Metrics

• Leadership

• Test automation, mobile, cloud, DevOps

• Quality, Cost, Benefits

• Speaking & writing highlights

• EuroSTAR and Iqnite several times

• ASTQB in USA, OOP in Germany, TEST-IT in South-Africa, Nordic Testing Days in Estonia, Testing Days in Czech, Israel Testing Week

• Numerous times in Finland at Testing Assembly, Aalto Testing Days, Tieturi Testing, Talentum Testing Forum, Quality Assurance & Software Testing, ICT Expo, TestIT Summit, Microsoft, HP, IBM, Borland etc. events

• Testing Experience magazine, Quality and Testing magazine, Sytyke-magazine, Tietoviikko

• Education

• ISTQB Expert Level Test Management Full & Advanced Full & Agile Tester certified

• SPICE provisionary assessor certified

• M.Sc, Helsinki University of Technology / Aalto-university

• Marketing studies, University of Wisconsin-Madison

• Professional achievements

• Wide spread of business domain knowledge

• Embedded, Industry, Public,

• Training, Telecom, Commerce,

• Insurance, Banking, Pension

• ISTQB Treasurer, Executive Committee 2015-

• Finnish Software Testing Board FiSTB, chairman

• TestausOSY/FAST founding member

• Knowit, Director, Quality and Competences

• Chairman of research project STX, Lappeenranta University of Technology

• Finnish Software Measurement Association FiSMA ry ex-board member

• Ranked in 100 most influential IT-persons in Finland

© Knowit Oy

Twitter: @kkakkonen

LinkedIn: fi.linkedin.com/in/

karikakkonen/

[email protected]

100+ Mobile apps

20+ Extranet services

50+ Intranet services

25+ Web stores 500+

Web sites

100+ Service design

projects

Knowit – We are known for our work We develop and grow our customers’ business.

© Copyright Knowit Oy 2015 | Confidential | Version 1.0

#1 In Quality Assurance

Revolution of User Experience

Mobile is the new reality – research tells us

13.11.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 5

www.shopify.com/blog

“Mobile Now Accounts for 50.3% of All Ecommerce Traffic”

Change of Business

1. Services are digitalized

2. Services processes are

digitalized

3. Internet of Things, Internet

of Everything

7

Central Quality Challenges

Security Usability

Performance Speed of software development

8

Security Testing Everything in focus

Security Testing


Preventive Security

• Expert in the team

Application level security

• Every level must be secure

Interfaces

• Internet

• Machine-2-Machine

Testing

• Vulnerability scans

• Audits

Up-to-date

• OWASP etc.

10

Usability testing Think about the UX


Surface

Skeleton

Structure

Scope

Strategy

Applied from source:

The Elements of

User Experience

By Jesse James Garrett

• Visual design

• Micro-interactions

• Service structure

• Page structure

• Content structure

• Content structure

• Information architecture

• Functionality

• Content needs

• Business needs

• User needs

Botto

m -

Up

Abstract

Concrete

Usability testing can be…

Usability assessments

Comparing service to User

Experience Levels

Assessing usability problems criticality

Reporting usability problems

Suggesting solutions


User device coverage – check usability with test automation

• Different devices, same functionality, same user experience

• Responsiveness

• One test library, many test objects

• Little to maintain

• Increasing number of objects

• Start early

• Repeat daily / by build

• Tests differ just a little

• Reporting

• Also other interfaces than UI

© Copyright Knowit Oy 2015

| Confidential | Version 1.0 13.11.2015 13

Apps to be tested (Web)

Test automation from cloud or devices

Performance testing Test and Measure, don’t get surprised

Performance testing

Testing tools

• Commercial

• Open source

Continuous monitoring

• Common goal with performance testing

Identify bottlenecks

• Fix

• Tune

Test coverage

• Devices

• Cloud

• Scenarios

Repeatability

• Include in regression tests


Speed of software development Balance between speed and quality

Fast software development process

DevOps

• As a culture

• Technically

Agility

• In organization

• In projects

Test automation

• Supporting the Team

• Speeding the Process

• Expand the Coverage

Exploratory Testing

• Defect finding effectiveness

• Efficiency

Competences

• Versatile: testing, development, operations, business


DevOps principles

• DevOps is a method which emphasizes

communication, cooperation and integration between software development teams (Dev) and IT-

support / operations teams (Ops)

• Method requires agile software development process

• It connects operations organization to software

development by using virtualized environments

• Production environments are part of software

development and as agile as the software

development


Software

Development

Quality

Assurance

Operations /

Production

DevOps


Reporting

DevOps

Automated tests and installation

Deploy Build

Integration Testing Performance

Agile development

End-user

monitoring

Production

Deploy Deploy

Virtualized back-end services

Test data management

Development Services

Version

management Continuous

Integration

Business

Code

Testing Development

Agile goes forward, what fits together?


Scrum

Kanban

Waterfall, V-Model

Scaled Agile

Framework Lean

Lean Six Sigma

DevOps

Continuous

Deployment

Hybrid

Knowit Research 2014: Agile test planning on company level

Strategic level

No plansPlan in high levelPlan in detail

• Almost a quarter doesn’t plan agile

testing at all (is that agile anymore…)

• Mainly testing is planned in high level

(this is maybe the definition of agile

testing…)

• 85 % of company level testing is

documented (to some extent)

Company level

• Not just

• Scrum

• Kanban

• …

• Organization level

• UVF

• M3.0

• SAFe

• Nexus

• Spotify Way

• Capo

• Agility Path

• Lean

• …


SAFe 3.0

24

Cloud (MS, IBM…)

Example of test automation and continuous integration

13.11.2015 © Copyright Knowit Oy 2015 | Confidential

Robot Framework

Quality Center

Jmeter

Python libraries

Selenium

Jenkins

GIT

Quick Test Pro

SoapUI

Sharepoint

Assure

Use Exploratory testing

Source: Sinkkonen, Elisabeth Hendrickson, Cem Kaner, James Bach, Lisa Crispin

“In exploratory testing you learn from previous tests, and put that

learning into your next tests”

Plan

•Test charter

Test session

•Notes

•Bugs

Debriefing

•Dashboard

Testing

Opinion-forming

Reporting

Designing

actions

Observations

Learn versatilely about business, development, testing, operations

Courses, certificates Coaching

Pair work Workshops

Increase competences


Prototype and test

Quality assurance in Digitalized World

Build Quality, then also Test

• Security

• Usability

• Performance

Get faster feedback

• Team velocity

• Support of testing

• Speed to Operations

Questions?

[email protected]

mailto:[email protected]

mailto:[email protected]