test data management system (tdms) - itea · disjointed data management 3 •limited situational...
TRANSCRIPT
Test Data Management System
(TDMS)
Richard Haire, Ravi Sundaar, Ashi Chaturvedula,
Aretus and TRAX Int’l
May 17, 2012
Overview
• Problem Statement: Disjointed Data Management
• Data Acquisition on the Ranges
• A Comprehensive Approach to Data Management
– Highly Integrated Virtual Environment (HIVE): Vision & Benefits
– Design Considerations
• Test Data Management System (TDMS)
– Attributes, Current State, Next Steps
• Summary
• TDMS Current State Demonstration (Optional)
2
Disjointed Data Management
3
• Limited Situational Awareness
– Parochial view of testing; a focus on the individual event test
– Limited visibility of testing status, testing results, test data management (e.g.
islands of excellence)
• Lack of Operational Efficiency
– Scheduling test resources and services
– Setting up test and coordination among various stakeholders
– Assessing the impact of IA policies (DCN, ICAN, SIPR)
– Executing the Concept of Operations (CONOPS)
• Inadequate test data management processes – Directing data and content management/architecture strategy
– Securing test resources and ensuring IT alignment
RTSS
TrackEye
DWDM
Data Acquisition
= Current Initiatives
Report
Customers: Test Officer, Analyst, etc.
Data Center • Security
• Monitoring & management
• Applications, content & DBs
• Storage & backup
• Temp control
• Power conditioning
• Fire detection & suppression
Power,
HVAC,
Upgrade
Storage
Upgrade
TDMS
ADSS
SharePoint
Transport
Plan & Data Management
Analyze
Data Acquisition
Android-based devices and iPhones
for data processing in near real-time.
FSTE
Pedestal
Network
Wireless
Mesh
RRRP
LMR
1
2
3
4
5
5
Enterprise Service Bus (ESB)
Platform
ADSS
(PMES)
Application
Service
Platform
RTSS
Application
Service
Platform
TDMS
Application
Service
Platform
UFR/TDAP
Application
Service
Platform
TQD/WDB
Application
Service
Platform
ACAS
Application
Service
Applications & Analysis
Test Officers
Dashboard
Investments Financials
<TBD> Situational
Awareness
Joint Testing
Reporting
Range
Operations
Asset
Management
Strategic
Planning Application
ABC Application
XYZ
HIVE Certification
Web Services
Infrastructure
(WSDL)
HIVE is a platform that
allows global access to
content through user
developed applications.
Highly Integrated Virtual Environment Envisioned End-State
HIVE Architecture Benefits
6
Provides consistent, accurate, and reliable query results.
Meets ACID (atomicity, consistency, isolation, durability) requirements of transactions.
Unified interface and access-controlled via existing Active Directory & functional roles.
Significantly improves operational efficiency.
Automates data aggregation across support elements.
Dramatic improvements collaboration among Test Officers and support elements.
Establishes foundations for:
Collaboration across T&E installations.
Horizontal and vertical scaling to support multiple T&E installations.
Expansion of architecture (“hybrid” architecture) – precise nature of SQL-based structured
content, and NoSQL-based unstructured content.
Providing “What if” scenario analysis that leverages structured and unstructured data
sources across and within Test Centers.
Provides minimal disruption to current Test Center operations.
Comprehensive Data Management Design Considerations
7
• Coexistence
― Avoid major disruptions to existing test environment (allow JAVA & Microsoft)
― No adverse impact on established applications and processes at each Test Center
• Flexibility
― Ensure ACID of User Queries
― Provide dynamic metadata & metadata curation
― Enable horizontal & vertical scaling of underlying infrastructure
• Presuppositions
― Test data resides at source (avoid transfers of large volumes of data)
― Require each Test Centers’ current applications to “plug into” Enterprise Service
Bus to enable a Service Oriented Architecture
― “Push” Application/Code to source of data for Computation & Analytics
TDMS Attributes
8
YPG INITIATIVE WITHIN THE HIVE FRAMEWORK
• Provides Test Center-centric lifecycle management
of test data. (“cradle to archive to final purge”)
• Enables metadata lifecycle management.
(“standardize as you proceed without losing integrity”)
• Allows for a provider and consumer of information in a Service
Oriented Architecture (SOA) framework.
• Provides seamless authentication and authorization.
• Addresses specific needs of organization stakeholders to include
test officers, data analysts/technicians, and decision makers for
limited situational awareness.
Platform
TDMS
Application
Service
YPG INTEGRATED TEST DATA REPOSITORIES
File Systems SQL Server
CORE TDMS FUNCTIONS
Data Acquisition
Search/Data Integration
Taxonomy & Metadata
Normalization & Aggregation
USER & SECURITY MANAGEMENT
User Management
Authorization/Authentication
BUSINESS SERVICES
Basic Situational Awareness
Enterprise Search & Retrieval
Collaboration/Publishing/Distribution
Reporting &Metrics
Basic Business Analytics
Test Asset Management
Indexing
Archiving
Personalization
Security & Audit
ORG Accepts
New Test Program
Test Officer develops
Test Plan,
Schedules
Results
Compilation
Support Elements
collaborate &
Perform Test Data Acquisition
Test Data
Manipulation
and Analysis
Data and Results
Distribution
TDMS: Current State
(PMES)
Note: Expanded services and growth in several commodity areas underway.
(automotive, electronics, optics, air delivery, etc.)
Data Management – Next Steps…
10
What is needed?
• Allow the user community to execute specific queries that require
consistent and accurate results, for example,
– Enable user to retrieve “MRAP braking data” to get a big picture of what
has been done to date, and sample data to confirm testing approach.
– Enable user to locate and download all “MRAP mass property data” for
date range 1-Dec-07 to present.
• Allow the user community to conduct computation and predictive
analysis, and “What if” analysis, for example,
– Before an unfunded project gets approved, the decision makers need to
understand the impact on test capabilities should that request not be
funded.
• Expansion of the current architecture to leverage newer technologies
(Hadoop, NoSQL), and further design elaboration to accommodate
other Test Centers or MRTFBs.
– Expand people, organizations, tests, assets, & investments domains to
accommodate the larger T&E community.
NoSQL DB
YPG INTEGRATED TEST DATA REPOSITORIES
File Systems SQL Server
CORE TDMS FUNCTIONS
Data Acquisition
Basic Search & Retrieval
Taxonomy & Metadata
Normalization & Aggregation
USER & SECURITY MANAGEMENT
User Management
Authorization/Authentication
BUSINESS SERVICES
Situational Awareness
Enterprise Search & Retrieval
Collaboration/Publishing/Distribution
Reporting &Metrics
Business Analytics
Test Asset Management
Indexing
Archiving
Personalization
Security & Audit
Next Steps…
Users (JMETC, Evaluators, HQ), Applications & Analysis
Test Officers
Dashboard
Investments Financials
Near Real-Time
Test Validation Situational
Awareness
Joint Testing
Reporting
Range
Operations
Asset
Management
Strategic
Planning
Application
ABC Application
XYZ
Application
XYZ
Enterprise Service Bus (ESB)
Hadoop Framework (MAP-REDUCE) NoSQL Database For Metadata
NoSQL DB
Summary
• The HIVE platform can support the needs of T&E community;
HQs, Test Centers, evaluators, and PMs. This is possible
because:
– Underlying architecture and framework is flexible and leverages the strengths of
“Structured Content Management,” and “Unstructured Content Management” with
newer technologies.
Newer technologies include Hadoop/Map-Reduce/NoSQL used by Facebook, Twitter, Google, &
Amazon
– Platform enables individual Test Centers to Develop “Apps” to support their
stakeholders needs
• Implementation with minimal disruption to existing operations at
installations.
12
Q&A
13
TDMS Current State Demonstration (duration ≈10 minutes)
14
FOCUS AREAS
• Define concept of test program / test events / test items (system
under test (SUT)) / test data (csv, .xls, .txt, .ppt, .doc, .pdf, graphs,
images, video, etc.).
• Definition of metadata (under admin screens) – demonstrate the
flexibility of adding, disabling, creating new metadata types.
• Demonstrate the integration with another application via Web
Services (ATEC’s ATEC Decision Support System (ADSS)).
• TDMS scheduling framework is used to manage back-end jobs
(data import jobs across multiple sources on the Data Control
Network (DCN), database maintenance jobs, etc.).
Alternate Architecture
Investigated
15
GOOGLE APPLIANCE with Existing Test Data File Shares
• Test Data is typically stored in file shares in various formats
(unstructured content).
• Google appliance may be used to crawl content to build an index,
while a search interface is used to search against this index to
retrieve content.
• Uncertainty in meeting ACID (Atomicity, Consistency, Isolation,
and Durability) transaction requirements.
• Low reliability and consistency of query results due to lack of
“semantic” consistency of metadata (National Cancer Institute’s
CaDSRProject).
16
Enterprise Service Bus (ESB)
Hadoop Framework (MAP-REDUCE)
PlatformAD
SS
(PMES)
Application
Service
Platform
RTSS
Application
Service
PlatformTD
MS
Application
Service
PlatformUF
R/TDAP
Application
Service
PlatformTQ
D/WDB
Application
Service
PlatformAC
AS
Application
Service
Applications & Analysis
Test Officers
Dashboard
Investments Financials
Near Real-Time
Test Validation
Situational
Awareness
Joint Testing
Reporting
Range
Operations
Asset
Management
Strategic
Planning Application
ABC Application
XYZ
HIVE Certification
NoSQL Database For Metadata
Domain Data, Data
Location, Summary,
Identifying Data,
Metadata Extraction
Web Services
Infrastructure
(WSDL)
Time Series
Analysis,
Predictive
Analytics
HIVE is a platform that
allows global access to
content through user
developed applications.
Highly Integrated Virtual Environment Envisioned End-State