Nicman GroupTest Data Management 2.0

Leveraging Copy Data Virtualization Technology in QA

for

SQuAD

November 2016

Agenda Test Data Management 2.0Leveraging Copy Data Virtualization Technology in QA

CDM: Testing

Copy Data Management, (CDM) – History, Issues, Problems

CDM: Continuous Integration / DevOps

LIVE! – Replication of large data set for Dev and Test

Confidential | Copyright Nicman Group LLC

Today’s Storage EnvironmentsImmense Growth Curve

0

2

4

6

8

10

12

14

16 Industry has responded by addressed this situation by economics and technology

• Better, faster, cheaper hardware

• Higher density hard drives

• Make it affordable to buy more and more storage

Storage Explosion

Production Data

Copies of Production Data C

opy

DEV/TEST

COPYCOPY

DR/BC

COPY

SNAP SHOT

COPY

• NBU• ComV• Avamar

• EMC• Hitachi• NetApp

• EMC SRDF• Host Based• App Based

• Manual

BACKUP

Application Layer with Storage

Make a Master CopyCopy Data Storage at App

Layer

Method of Maintaining via Block Level Deltas

Usable Format, Native Language

Real World Example

5

Next Generation Data Management

6

NETWORKVIRTUALIZATION

2012+

SERVER & COMPUTE VIRTUALIZATION

2010+

DATAVIRTUALIZATION

2014+

COMMODITIZATION OF HARDWARE AND VIRTUALIZATION OF SERVICE

Enterprise Private & Public Cloud

Testing and DevOps Four Problems

7

COST

QUALITY

SECURITY

SPEED

The SPEED ProblemTHE PROCESS THE PHYSICS

8

Change Management RequestSLA 1–2 weeks

SLA 1 week

SLA 1 week

Tester requests copy. Tickets opened.

Compute and storage is provisioned

DBA Creates copies and masks data

Developer gets a masked copy

Ex: The initial copy + 5 physical copies of 5TB over 1 Gbps will take 5 days

First Copy

Prototype Unit TestBuild

IntegrationQA Test UAT

ProductionData

Competitive Disadvantage, Opportunity Cost Of Revenue From New Apps

Low Fidelity Data Leads to QUALITY Problems

9

Prototype Unit TestBuild

IntegrationQA Test UAT Production

CRITICAL STAGES

Unit Testing Works Great!! Integration Testing Works! QA Testing Works! It Does Not Work!!!!!!

PROBLEMS USING Low-Fi DATAX Can’t Do Scalability Tests ‘Early’X Bugs Surface Too Late In DEV Cycles

Poor Quality. Low Customer Satisfaction. High Cost Of Support

The SECURITY / CONTROL Problem

CSO/CISO Concerns• Data not masked• No control on who can access copies• No audit track-ability

10

Non-Compliance, Liability From Sensitive Data Exposure

The COST Problem

11

First Copy

ProductionData

All these physical copies drive up costs

Large burden on DBAs1. Clone DB2. Mask DB3. Copy Masked DBs4. Bring DB Online

Prototype Unit TestBuild

IntegrationQA Test UAT

Uncontrolled Infrastructure Cost

Solution

12

Dev, QA, Testers Get Self Service & Instant Access

13

Prototype Unit Test Build Integration QA Test UAT / STAGING

Virtual DB Virtual DB Virtual DB Virtual DB Virtual DB

Production Data

Faster TTM Leads To Competitive Advantage & Faster Growth

Catch Defects Early

14

Prototype Unit Test Build Integration QA Test UAT / STAGING

Virtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TB

Production Data

50 TB

Scalability Tests

Performance Tests

Regression Tests

Less Surprises. More Predictability

High Quality Release On Time, On Budget

High Fidelity Test Data With Lowest Costs

15

Prototype Unit TestBuild

Integration QA Test UAT / STAGING

Virtual DB Virtual DB Virtual DB Virtual DB Virtual DB

Test Environments1. Instant Virtual Copies2. Self Service3. Automated Refresh

IT Operations1. Role Based Access

Control2. Data Masking3. Multi-Hypervisor, Cloud

Production Data

Incremental Forever Capture

Masked Data

CDM

Control & Security

16

Ops: Specify how often to ingest data.

Ops: Specify RBAC for testers

Specify Data Masking Scripts

UNIT TEST

BUILD INTERGRATION

QA TEST

UAT \ STAGING

SUPPORT

EXAMPLE: WHO can access SQLDB on WHAT serversWHO can access Oracle DB on WHATservers

ORACLE DB

MS SQL DB

1

2

3

Reduce Risk Of Non-Compliance & Ensures Data Masking

DATAMASKED

DATA

At No Extra Storage !!

17

Prototype Unit Test Build Integration QA Test UAT / STAGING

Virtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TBVirtual DB

0 TB

Production Data

50 TB

Example cases for DevOps

18

Central Admin by Ops

19

GOLDEN COPY

Ops: Specify how often to ingest data.

Data Ingested in an incremental for ever fashion

Ops: Specify RBAC for testers

Library of virtual copies for various PIT

Unit Test

Build Integration

QA Test

UAT \ STAGING

SUPPORT

Prototype

MS SQL DB

V1V2

Vn

ORACLE DB

V1V2

Vn

EXAMPLE: WHO can access SQLDB on WHAT serversWHO can access Oracle DB on WHATservers

ORACLE DB

MS SQL DB

2

1

3

4

Self Service by Dev

20

Golden Copy

Unit Test

Build Integration

QA Test

UAT \ STAGING

SUPPORT

Prototype

MS SQL DB

V1V2

Vn

ORACLE DB

V1V2

Vn

Dev: Instant Virtual DB

Dev: Self Service: LOGIN . BROWSE. SELECT PIT. PROVISION

Databases Come Up Online. DB Parameters Customizable.No DBA Needed.No Storage Admin Needed.

Dev: Protect test environment.

Production Support: Debug \ conduct RCA on high fidelity data.

MS SQL DB

ORACLE DB

5

6

7

8

9

Automated Test Data In Continuous Integration

21

Production DB

DEV Test Server 1

Golden Copy

Code Check In

Frequent Builds

Test Server 2

Execute Automated Test Cases

Virtual DB

2

3

6

7

5

Write Code.Unit Test Using Virtual DB

Virtual DB Virtual DB

1

Invoke Actifio API To Provision Virtual DB

Test Automation Framework

Deploy Latest Build

Automates High Fidelity Test Data Provisioning

Report Integration Test Results

8

MASKED DATA

Use Chef To Provision Platform 4

CDM

Test Data Promotion

22

Golden Copy

DEV BUILD INTEGRATION

QA TEST UAT \ STAGING

V1

V2 V2 V2

V3 V3 V3 V3

PRODUCTION

Virtual Copy from golden copy

Development wants to modify schema / data structures and they want all testing to happen off this modified schema

Thus dev creates a new data branch (snapshot from v1) and makes the changes

Multiple virtual copies are created off V2 for testing by multiple teams

Dev can create further branches if needed

Multiple virtual copies are created off V3 for testing by multiple teams

After UAT, changes are rolled out in production using 3rd party tools

Continues to protect production and refresh data for Dev, QA, UAT for next iteration

MS SQL DB

1

3

28

4

6

5

7

Parallel Testing Increases Productivity

23

9:00 AM EST

10:00 AM EST

Rare issue reproduced.Need DEV to do RCA.But DEV is in 7 am PST.

QA also has to execute rest of these test cases

10:05 AM EST

TEST CASES

Golden Copy

MS SQL DB

V1V2

Vn

12:00 PM EST

Golden Copy

MS SQL DB

V1V2

VnGolden Copy

MS SQL DB

V1V2

Vn

QA uses this PIT version for testing

TEST SERVER 1

TEST SERVER 2

QA uses the same PIT version on another test server.Continues with other test cases

TEST SERVER 1

TEST SERVER 2

DEV starts RCA, debugs, fixes problem, tests the fix

QA executesthe rest of the

test cases

1

2

3

4

Bookmarks Enable Negative Test Case Execution

24

QA sets up testing environment

Golden Copy

MS SQL DB

V1V2

Vn Tester provisions a PIT image of the database on a test machine.

TEST SERVER VM

1

A snapshot/bookmark is set at this PIT

Golden Copy

MS SQL DB

V1V2

Vn

TEST SERVER VM

Before corrupting the data, tester issues a bookmark

2

Testing is performed through completion

Golden Copy

MS SQL DB

V1V2

Vn

TEST SERVER VM

Data can be modified, corrupted and negative test cases can be executed

3

QA test environment is “rolled back” to starting point

Golden Copy

MS SQL DB

V1V2

Vn

TEST SERVER VM

After the testing is done, tester rewinds the state to the bookmark

4

Nested Bookmarks, Multi –Level Grandchildren, Daisy Chained Snapshots

Data Masking

25

MS SQL DB

Golden Copy

Unit Test

Build Integration

QA Test

UAT \ STAGING

SUPPORT

Prototype

ORACLE DB

Ops: Specify data masking scripts and data masking server

Live Clone

Configure Actifio to create a live clone

Mounts the data to data masking server. Data is masked here.

Testers can select ONLY masked copies

Data Refresh happens automatically

DATAMASKINGSERVER

2

3

1

5

6

After Masking is done, control returns.

4

ABOUT USNicman Group: Denver Colorado, Secaucus New Jersey, Philadelphia Pennsylvania

50+ IT consultants and professional services experts specializing in DevOps, software development, business analytics, deployment strategies, performance testing and QA services.

Cloud services provider with focused expertise in infrastructure services, private cloud, hosting technologies and data warehousing.

Global relationships, partnerships and offshore resources.

Confidential | Copyright Nicman Group LLC