WEBINAR
Leveraging
ALM & QuerySurge
to test
Vertica
Bill Hayduk
Founder & CEORTTS
Chris Thompson
Senior Domain Expert
data quality & testing
QuerySurge™a software division of
• a custom view of ALM to store source-to-target mappings
• data validation tests in QuerySurge
• The execution of QuerySurge tests from ALM
• The results of data validation tests stored in ALM
• custom ALM reports that show data validation coverage of Vertica
• how we improve your data quality while reducing your costs & risks
QuerySurge™
a software division of
WEBINAR
Here is what you will see:
QuerySurge™
About FACTS
Founded:
1996
Location:
New York, NY
Software Division:
QuerySurge
Customer profile:
• Fortune 1000
• 700+ customers
Alliance Partners:IBM, Microsoft, HPE,
Oracle, Teradata,
HortonWorks,
Cloudera, MongoDB
RTTS is the parent company of QuerySurge
and is the premier software & services firm in the QA & Testing space
that specializes in test automation
a software division of
Testing a Data Warehouse
QuerySurge™
a software division of
Business Intelligence & Analytics
CxOs are using Business Intelligence & Analytics to make critical business decisions – with the assumption that the underlying data is fine.
“The average organization loses
$14.2 million annually through
poor Data Quality.”
- Gartner
ETL
Data Architecture
BUSINESS CASE: Executive Office & Critical Data
potential problem areas
Mainframe
ETL
Extract
Data Warehouse & the ETL process
Source Data
ETL Process Target Data Warehouse/
Data Mart
Transform
Load
Mainframe
QuerySurge™a software division of
Which Data Goes Where? Source-to-Target Mapping doc
Elements of Mapping Document• Attributes of original source data
(table name, column name, source type)
• Attributes of the target data store (table name, column name, source type)
• Transformation rules, if any(aggregation, combining records, etc.)
Definition: Source-to-Target Mapping• Specification or requirements doc that shows how
data in one system is mapped to another system• Typically created and stored in Word or Excel
QuerySurge™ a software division of
Issues with Mapping Document & Testing• No easy way to link mappings to tests for tracability• No easy way to store test results • No easy way to report on data testing coverage
Data Process: Developer & Tester
Developer: Codes data movement based on Mapping Requirements
Tester: Tests data movement based on Mapping Requirements
Data Mart
ETL
Source Data
ETL Process Target DWH
Mainframe
ETL
ETL Process
About…
Application Lifecycle Management
QuerySurge™
About HPE VerticaHPE Vertica is one of the most advanced SQL database analytics portfolio built from the very first line of code to address the most demanding Big Data analytics initiatives.
Vertica's column-oriented storage approach significantly increases query performance in data warehouses
About HPE ALMApplication Lifecycle Management (ALM)
is the industry standardfor modern quality management.
It is a centralized management system that provides reporting and traceability throughout
the application delivery lifecycle.
is the leading Data Testing
solution for automated testing & validation of Big Data and
Data Warehouses
QuerySurge
Use Cases
About QuerySurge
QuerySurge™
Automate the entire testing cycle Automate the launch, tests, comparison, auto-emailed results
Query Wizards - no coding needed Query Wizards can automate about 80% of your tests
Test across different platforms EDWH, Hadoop, NoSQL, database, flat file, XML, mainframe
Analyze & Collaborate Data Health dashboard, shared tests & auto-emailed reports
Verify more data & do it quickly verifies up to 100% of all data up to 1,000 x faster
Integrate for Continuous Delivery (DevOps) Integrates with most Build, ETL & QA management software
a software division of
the QuerySurge advantage
Web-based…
Installs on...
Linux
Connects to…
…and any JDBC compliant data source
QuerySurge™
QuerySurgeController
QuerySurgeServer
QuerySurgeAgents
Flat Files
a software division of
QuerySurge Architecture
SQL
HQL
SQL
HQL
SQL
SQL
QS pulls data from data sources
QS pulls data from target data store
QS compares data quickly
QS generates reports, auto-emails
Reports, Data Health Dashboard, auto emails
Source Data
• Databases
• Data Warehouses • Data Marts
Flat Files• Fixed Width• Delimited• Excel
Mainframe
Target Data
Big Data stores• Hadoop
• NoSQL
Data Warehouses
How QuerySurge Works
QuerySurge™
QuerySurge Modules
Design Library
SchedulingRun
DashboardDeep-Dive
Reporting
Data Health
Dashboard
Query
Wizards
a software division of
Fast and Easy. No programming needed.
QuerySurge™
• Perform 80% of all data tests with no SQL coding
• Opens up testing to novices & non-technical members
• Speeds up testing for skilled coders
• provides a huge Return-On-Investment
a software division of
QuerySurge Modules
QuerySurge™
a software division of
QuerySurge Modules
Design Library• Create custom Query Pairs (source & target
SQLs for tests that have transformations)
Scheduling Build groups of Query Pairs Schedule Test Runs
• Run immediately• Run at set date/time• Have event kick it off
QuerySurge™
a software division of
QuerySurge Modules
Deep-Dive Reporting Examine and automatically
email test results
Run Dashboard View real-time execution Analyze real-time results
QuerySurge™
a software division of
QuerySurge Modules
QuerySurge™
• view data reliability & pass rate
• add, move, filter, zoom-in on any data widget & underlying data
• verify build success or failure
a software division of
QuerySurge Modules
QuerySurge™
Drive QuerySurge execution from your Test Management Solution
See QuerySurge Pass/Fail results in your Test Management solution
Click link to drill into detailed results in QuerySurge
Integration with leading Test Management Solutions
• HPE ALM (Quality Center)
• Microsoft Team Foundation Server
• IBM Rational Quality Manager
a software division of
QuerySurge Test Management Connectors
QuerySurge & DevOps: Continuous Integration
QuerySurge™
Automated Testing
Automated Reporting
Automated Launch
Data Integration/ ETL solutions
QuerySurge™
and many others…
email report
and many others…
QuerySurge™
Build/Configuration solutions
email report
Test Management solutions
QuerySurge™
email report
a software division of
USE CASE
QuerySurge™
Testing VERTICA using ALM & QuerySurge
a software division of
USE CASE: Data Process for Developer & TesterDeveloper: Codes data movement based on Mapping Requirements
Tester: Tests data movement based on Mapping Requirements
ETL
Source Data ETL Process Target DWH
Target Data
ALM Custom-built Mapping
Requirements
SQL
SQL
SQL
SQL
SQL
SQL
Schedule ALM to kick off QuerySurge through our API
QuerySurge automatically reports back all data failures
ALM Custom-builtIssue Tracking
& Coverage reports
QuerySurge runs unattended• executing all tests• comparing all data• compiling reports
Testing VERTICA using ALM & QuerySurge
Source Data
Custom-built Source-to-Target Data Mapping report
Target HP VerticaSource MySQL
Custom-built Data Mappings Module
Custom-built Data Mapping Info tab: Data Specification details
Source MySQL Target HP Vertica
Custom-built Mapping Summary Coverage by Table
2/9/2017 30
Custom-built Mapping Summary Coverage by Status
Demonstration
built by
QuerySurge™
Chris Thompson
Senior Domain Expertdata quality & testing
a software division of
Want to see the video of the webinar?Go to:
https://youtu.be/bKtUPtva9cE
QuerySurge™