leveraging hpe alm & querysurge to test hpe vertica

WEBINAR

Leveraging

ALM & QuerySurge

to test

Vertica

Bill Hayduk

Founder & CEORTTS

Chris Thompson

Senior Domain Expert

data quality & testing

QuerySurge™a software division of

• a custom view of ALM to store source-to-target mappings

• data validation tests in QuerySurge

• The execution of QuerySurge tests from ALM

• The results of data validation tests stored in ALM

• custom ALM reports that show data validation coverage of Vertica

• how we improve your data quality while reducing your costs & risks

QuerySurge™

a software division of

WEBINAR

Here is what you will see:

QuerySurge™

About FACTS

Founded:

1996

Location:

New York, NY

Software Division:

QuerySurge

Customer profile:

• Fortune 1000

• 700+ customers

Alliance Partners:IBM, Microsoft, HPE,

Oracle, Teradata,

HortonWorks,

Cloudera, MongoDB

RTTS is the parent company of QuerySurge

and is the premier software & services firm in the QA & Testing space

that specializes in test automation

a software division of

Testing a Data Warehouse

QuerySurge™

a software division of

Business Intelligence & Analytics

CxOs are using Business Intelligence & Analytics to make critical business decisions – with the assumption that the underlying data is fine.

“The average organization loses

$14.2 million annually through

poor Data Quality.”

- Gartner

ETL

Data Architecture

BUSINESS CASE: Executive Office & Critical Data

potential problem areas

Mainframe

ETL

Extract

Data Warehouse & the ETL process

Source Data

ETL Process Target Data Warehouse/

Data Mart

Transform

Load

Mainframe

QuerySurge™a software division of

Which Data Goes Where? Source-to-Target Mapping doc

Elements of Mapping Document• Attributes of original source data

(table name, column name, source type)

• Attributes of the target data store (table name, column name, source type)

• Transformation rules, if any(aggregation, combining records, etc.)

Definition: Source-to-Target Mapping• Specification or requirements doc that shows how

data in one system is mapped to another system• Typically created and stored in Word or Excel

QuerySurge™ a software division of

Issues with Mapping Document & Testing• No easy way to link mappings to tests for tracability• No easy way to store test results • No easy way to report on data testing coverage

Data Process: Developer & Tester

Developer: Codes data movement based on Mapping Requirements

Tester: Tests data movement based on Mapping Requirements

Data Mart

ETL

Source Data

ETL Process Target DWH

Mainframe

ETL

ETL Process

About…

Application Lifecycle Management

QuerySurge™

About HPE VerticaHPE Vertica is one of the most advanced SQL database analytics portfolio built from the very first line of code to address the most demanding Big Data analytics initiatives.

Vertica's column-oriented storage approach significantly increases query performance in data warehouses

About HPE ALMApplication Lifecycle Management (ALM)

is the industry standardfor modern quality management.

It is a centralized management system that provides reporting and traceability throughout

the application delivery lifecycle.

is the leading Data Testing

solution for automated testing & validation of Big Data and

Data Warehouses

QuerySurge

Use Cases

About QuerySurge

QuerySurge™

Automate the entire testing cycle Automate the launch, tests, comparison, auto-emailed results

Query Wizards - no coding needed Query Wizards can automate about 80% of your tests

Test across different platforms EDWH, Hadoop, NoSQL, database, flat file, XML, mainframe

Analyze & Collaborate Data Health dashboard, shared tests & auto-emailed reports

Verify more data & do it quickly verifies up to 100% of all data up to 1,000 x faster

Integrate for Continuous Delivery (DevOps) Integrates with most Build, ETL & QA management software

a software division of

the QuerySurge advantage

Web-based…

Installs on...

Linux

Connects to…

…and any JDBC compliant data source

QuerySurge™

QuerySurgeController

QuerySurgeServer

QuerySurgeAgents

Flat Files

a software division of

QuerySurge Architecture

SQL

HQL

SQL

HQL

SQL

SQL

QS pulls data from data sources

QS pulls data from target data store

QS compares data quickly

QS generates reports, auto-emails

Reports, Data Health Dashboard, auto emails

Source Data

• Databases

• Data Warehouses • Data Marts

Flat Files• Fixed Width• Delimited• Excel

Mainframe

Target Data

Big Data stores• Hadoop

• NoSQL

Data Warehouses

How QuerySurge Works

QuerySurge™

QuerySurge Modules

Design Library

SchedulingRun

DashboardDeep-Dive

Reporting

Data Health

Dashboard

Query

Wizards

a software division of

Fast and Easy. No programming needed.

QuerySurge™

• Perform 80% of all data tests with no SQL coding

• Opens up testing to novices & non-technical members

• Speeds up testing for skilled coders

• provides a huge Return-On-Investment

a software division of

QuerySurge Modules

QuerySurge™

a software division of

QuerySurge Modules

Design Library• Create custom Query Pairs (source & target

SQLs for tests that have transformations)

Scheduling Build groups of Query Pairs Schedule Test Runs

• Run immediately• Run at set date/time• Have event kick it off

QuerySurge™

a software division of

QuerySurge Modules

Deep-Dive Reporting Examine and automatically

email test results

Run Dashboard View real-time execution Analyze real-time results

QuerySurge™

a software division of

QuerySurge Modules

QuerySurge™

• view data reliability & pass rate

• add, move, filter, zoom-in on any data widget & underlying data

• verify build success or failure

a software division of

QuerySurge Modules

QuerySurge™

Drive QuerySurge execution from your Test Management Solution

See QuerySurge Pass/Fail results in your Test Management solution

Click link to drill into detailed results in QuerySurge

Integration with leading Test Management Solutions

• HPE ALM (Quality Center)

• Microsoft Team Foundation Server

• IBM Rational Quality Manager

a software division of

QuerySurge Test Management Connectors

QuerySurge & DevOps: Continuous Integration

QuerySurge™

Automated Testing

Automated Reporting

Automated Launch

Data Integration/ ETL solutions

QuerySurge™

and many others…

email report

and many others…

QuerySurge™

Build/Configuration solutions

email report

Test Management solutions

QuerySurge™

email report

a software division of

USE CASE

QuerySurge™

Testing VERTICA using ALM & QuerySurge

a software division of

USE CASE: Data Process for Developer & TesterDeveloper: Codes data movement based on Mapping Requirements

Tester: Tests data movement based on Mapping Requirements

ETL

Source Data ETL Process Target DWH

Target Data

ALM Custom-built Mapping

Requirements

SQL

SQL

SQL

SQL

SQL

SQL

Schedule ALM to kick off QuerySurge through our API

QuerySurge automatically reports back all data failures

ALM Custom-builtIssue Tracking

& Coverage reports

QuerySurge runs unattended• executing all tests• comparing all data• compiling reports

Testing VERTICA using ALM & QuerySurge

Source Data

Custom-built Source-to-Target Data Mapping report

Target HP VerticaSource MySQL

Custom-built Data Mappings Module

Custom-built Data Mapping Info tab: Data Specification details

Source MySQL Target HP Vertica

Custom-built Mapping Summary Coverage by Table

2/9/2017 30

Custom-built Mapping Summary Coverage by Status

Demonstration

built by

QuerySurge™

Chris Thompson

Senior Domain Expertdata quality & testing

a software division of

Want to see the video of the webinar?Go to:

https://youtu.be/bKtUPtva9cE

QuerySurge™