pre-con ed: high performance masking and test data generation for ims and db2

World®’16

HighPerformanceMaskingandTestDataGenerationforIMSandDB2PrashantPandeyCATechnologies

MFX59E

MAINFRAMEANDWORKLOADAUTOMATION

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

Withfinesaveragingmillionsofdollars,andprohibitivelyhighinfrastructurecosts,maskingandsubsetting themainframeisamust.ThissessionwillcoverhowtheCATestDataManagerarchitectureavoidsthediresecurityandperformanceimplicationsofmovingdatatoandfromz/OS.

PrashantPandeyCATechnologiesSr PrincipalConsultant,Presales


TestDataonz/OS(theBadNews)

§ z/OSsystemofrecord(70%datastillresideshere)§ ManyorganisationsrelyoncopiesofProductionfortesting§ Secureandreliableinproduction– whatabouttest?§ WhatandWhereisPIIdata§ Legislation– GDPR,demonstratecompliance§ BreachNotification– reputation&finesrisk§ Costofmovingtestdatato/fromplatform

(theGoodNews)BestPracticeTestDataManagementalreadyaddressesthisimpact


WhatisTestDataManagement(TDM)?

‘…isthepracticeofapplyingastructuredandcentralizedapproachtothemanagementoftestdataatanenterpriselevel,inordertoreducecostwhilstincreasingefficiencyandqualityindevelopmentandtesting’


CA-TestDataManagerDeployment&Process

1. Centralize2. Discover3. Subset4. Mask5. Synthesize6. Find&Reserve7. Automate

Synthetic/CloneData

Masked

Subset

Created

ProductionDevelopm

ent

Web- SOATestHarness

Virtualization

DataMasking

Subsetting/DataSlicing

DataProfilingCoverageMetrics

Agile-Designer

TestDesign

DataDesign

TestDataonDemand

UI- LoadTestHarness

TestMart

FindandReserve

REQUEST/RESPONSEPOOLS

VERSIONCONTROLEXPECTEDRESULTS

REGRESSIONPACKS

DATADRIVENTESTPACKS

TEMPLATES

Agile-DataHPALM/QC

Automation

DataVisualization

Javelin

–Da

taOrche

stratio

n


CA-TestDataManagerDeployment&Process

Adabas

z/Series

IMSDB2

VSAM

Unisys

DMSII

LUW

Informix

MSSQLOracle

Teradata

DB2/400 FlatFiles(CSV,Excel,XMLetc.)

i/Series

WindowsUI

ODBC/JDBCCOBOL/JCL

JAVASQL

LUW

CA– TestDataManager

Register&Profile

DefineTestDataEngineeringOperation

ProcessRun-time

EnableasOn-DemandRequest

TestDataOn-Demand

IMGR&ServiceLayerOracleorSQL

ServerRepository


SecureMaskedSubsetsof

Data

MainframeEnvironment

DB2z/OSVSAMIMSnon-indexedflatfilesOtherDBs

ValidationProfilingPIIDiscoveryDataSubsetDefinitionBuilt-InMaskingFunctionsSyntheticDataRules

RulesandDefinitions

MetaData

SecureandeffectivetestdataengineeringonMainframe

CATDMWindowsWorkbench

DataneverhastoleavetheMainframe

MainframeBatchOperation

Rules

Repository(RulesandMetaStore)

PDSJCL&CTL)

FTP

ODBC/Copybooks


End-EndIntegrity

§ Acrossmultiplesource(andplatforms!)– z/Series:DB2,IMS,VSAM(KSDS,ESDS,etc.)orflatfiles(PS)etc.– i/Series– AS400:DB2/400– Distributed:Oracle,SQLServer,DB2UDBetc.– Filestructures:CSV,Excel,XML,SOAP,REST,JSONetc.

Maintainingdatabase,businessandlogicalintegrity

CustID Item# OrderDate

07221 80-25442 2015-01-25

07221 80-25442 2015-03-14

CustID Item# OrderDate

10002 80-25442 2015-02-15

10002 80-25442 2015-05-19

Dataretainsstructure,contextandreferentialintegritybutdoesnotcontain“real”information.

CustID Name SSN Street

08052 AndrewChapman 181-23-7171 87Featherbed Lane

09112 Brian James 112-32-4435 12ParkLane

08052 AndrewChapman 181-23-7171 87Featherbed Lane

07221 TomAngle 771-11-8972 8Boardwalk

CustID Name SSN Street

10000 Hikaru Sulu 545-95-1245 87Featherbed Lane

10001 Nyota Uhura 802-41-3411 12ParkLane

10000 Hikaru Sulu 545-95-1245 87Featherbed Lane

10002 PavelChekov 325-74-6517 8Boardwalk


ProjectandVersionCentralizedControl

§ Re-usableassetsinaTestDataWarehouse

§ Providesacentrallibraryoftestdataengineeringoperations

§ Assetscanbeupgradedacrossversions(releases)

§ Regressionpackscanbestored

CATestDataManagerTestDataWarehouse(Repository)


Automaticallydiscoversensitivedatastoredenterprise-wide

§ Powerful,mathematically-basedalgorithmsdiscoverandtagallsensitivedata,enterprise-wide

§ Simple, easytousefilteringbytagwhenselectingcolumnsortablestomask

§ Supportforeverymajordatabasetype,mainframeplatforms andflatfiles

CATestDataManagerDataProfiler


Subsetting:datasetsareextractedbasedonexactcriteriaCATestDataManagerDataSubset

Referentiallyintactsubsetsofdata

ProductionAttributes

ScenarioSubset

CoveredSubsetProduction&Enhanced

Attributes


De-sensitizemillionsofrowsofreferentiallyintactdatainminutesCATestDataManagerFastDataMasker

Theseincludesubstitution,randomization,hashingandseeddata.Oncecreated,maskingrulescanbestoredandre-usedfromacentralrepository.

GTFastDataMaskerwillonlydisplaysuitablemaskingroutines,basedontheselectedcolumnandtables

Deterministicmaskingfunctionsandbuilt-incross-referencingensureconsistency.Thereferentiallyintact,realisticdata,canbeinjectedintomultiplesystemsatonce

NumberofRows Timeto Mask

201,722,392 9minutes, 42seconds

453,877,152 11minutes,22seconds

768,088,071 7minutes,57seconds

17,422,541 1minute

46,579,485 1minute, 25seconds

1,759,612 13seconds

47,895 5seconds

GTFastDataMaskerusesnativedatabasescriptstoproducethehighestpossibleperformancewhenmaskingOracle,MSServer,Teradataandmainframeplatforms

§ Maskmillionsofrowsofdata inminutesusingnativemaskingengines

§ Over 80built-indatamaskingfunctions

§ Replacementswithrealistic,referentiallyintact, appliedtomultiplesystems

§ DemonstrateCompliancewiththeEUGDPR,GLBA,HIPAA,PCIDSS,PIPEDA,andmore


GenerateRequiredDataScenariosCATestDataManagerSyntheticDataGeneration

Thismeansthatnewdatacanefficientlyandquicklyinjectedintoexistingdatabasesforexactlytherequirementsyouneed

§ ExtensivealgorithmsforSyntheticdataappliedinaDataPainterstyle

§ UseSymbolic values,combine functions,include conditional valuesandmine datafromothersourcesintothenewdata

§ Publish(inject/load)singledataobjectasmultiple/variednewrowsinexistingdatabase

§ Storedataobjectscentrally,asre-usableassetsinaTestDataWarehouse

§ Datacanbere-usedacrossversionsandreleases,withversionedregressionpacks


Find&ReserveExistingDataCATestDataManagerTestMatching

SAP-1

DataAttributeMining

DataTestConditions

DatamakerTestMatching

DB-2

XML

MatchedTest

QTPScripts


CombineTDMOperationstoProvideComprehensiveSetofData

Referential Integrity

Profiling

PII Discovery

Native DB Engines

Seed Replacement

Audit

CATestDataManager

1 2 3SecureDataSubsets

Test/DevEnvironments

Gold Copy

Subset

Seed Tables

Built-in Masking Functions

Cross Referencing

Masked

Generated Scenarios

Find & Reserve Data

SyntheticContent

SyntheticContent

Productiondatasourcesandfiles

XMLFilesExcelFilesSQLFiles

CSVFiles FixedDefinitionFiles

HTMLFiles

VSAM/ISAMSwift

TXTFiles

TestData

Warehouse


Avoidcostlymistakeswhenmanagingz/OSTestData

Avoiduseofrawproductiondataisamandatoryoperationformodernorganizations.Theaveragecostperincidentofdatabreachworldwideis$3.5m.IntheUS,theriskwasashighas$5.85m1.

Inasmanyas59%oftheseincidents,theriskcouldhavebeenmitigatedbyimplementingTestDataManagementbestpractices

1 Ponemon Institute,2014

CATDMallowsuserstoapplyTestDataManagementpracticesdirectlyonthemainframe reducingtherelianceonslow,expensiveExtract,Transformandreloadtechnologiestoensure complianceandtakeadvantageoffit-for-purposetestdata.

Processingdatadirectlyonthemainframe canprovideteamswithmillionsofrowsofhighquality,meaningful,referentiallyintactdata,inashortperiodoftimeandatlessrisktoquality!


RecommendedSessions

SESSION# TITLE DATE/TIME

MFX58ECCATestingToolsEducationalSession(Part3of3):CAMainframeApplicationTunerDemoandWhat'sNew 11/15/2016at2:00pm

MFX112S MainframeApplicationDevelopmentPortfolioReview 11/16/2016at12:45pm

MFT148SAchieveDevOpsontheMainframeforFasterTimetoMarket 11/16/2016at4:30pm


MustSeeDemos

ContinuousDevelopmentandTestingMainframeTheater

ContinuousDelivery

MainframeTheater


Questions?


Thankyou.

Stayconnectedatcommunities.ca.com

pre-con ed: high performance masking and test data generation for ims and db2

Technology