quality assurance in distributed continuous delivery

Quality Assurancein Distributed Continuous DeliveryByJin Ming｜@金明i

Continuous Delivery Lead, ThoughtWorks

Wednesday, August 8, 12

TextBackground: Top One Property Listing Site in OZ


Agency

Seller Consumer

Business Overview


Architecture Overview


How to Do QA?


Quality is Value to Some Person.---Gerald Weinberg


Challenges We’re Facing


Legacy Systems


Cross Systems


Distributed Development


Also, We Want to Be


We Need “Total Quality”


Functional Tests Examples

Story Tests Prototypes Simulations

...

Exploratory Testing Scenarios

Usability Testing UAT(User Acceptance Testing)

Alpha / Beta ...

Unit Tests Component Tests

...

Performance & Load Testing Security Testing

...

Automated & Manual

Manual

Tools Automated

Business Facing

Technology Facing

Critique Product

Supporting Team Q1

Q2 Q3 Q4

Test Quadrants


Functional Test Boundaries


We Need to Ensure End-to-End


We Need to Ensure Single System


Units Inside Single System


Automatic Those Tests


End-to-End Test�

System Test�

Unit Test

Test Pyramid

Scope

End-User Simulation

EfficiencyCost

EfficiencyCost

Scope

End-User Simulation


Functional Test Samples

Capybara

Selenium

WebDriver


Performance Test Samples


Deployment Validation


Test Plan


Release Test Plan Iteration Test Plan Daily Test Plan

$

$

$

$

Level Stories

Vision & High-

Iteration

Release

Test Plan


Iterations

Automated Regression Test

Performance Profiling Test

Full Regression

Test

Integration Test

Acceptance Testing

I1

Exploratory, Automation

I1

Test Analysis I1

I0 I1 I4 … In I2 I3

Acceptance Testing

I2


I2

Test Analysis I2

Acceptance Testing

I3


I3

Test Analysis I3

Stabilize (opt)

Pre-production Production

UAT

�iIities operations

&Compliance

Deployment

Data Migration

PVT

End-To-End Test

Load/Stress Load/Stress

End-To-End Test

Integration Test


Good Enough?


It’s Just Validation.

Where is Verification?


Stories Behaviour Application

Exercise expectations against Matches to

Customer QA Domain Expert Developer

BDD


Co-Education


Agile Quality Assurance

Business Analytics

Test Strategy

Risk Assessmen

t

Test Analytics

Process Improveme

nt

Test Framework

Common Vision on Quality

Exploratory Test

Showcase

Quality Assurance Activities


Build Quality In


Shared Responsibility


Communication & Collaboration


QA is NOT JUST Testing


Eradicating Non-Determinism in Tests


Visualizing Quality


? ? ? ? ?

? ?
