data & analytics applied to test strategy
DESCRIPTION
Data & Analytics Applied to Test Strategy. Brad Waggle. Board Test Workshop. September 9th, 2014. Here in Texas, everything’s big, so we just call it data. Michael Dell … http://www.dell.com/learn/us/en/19/power/ps2q14-20140230-michael. Agenda. Context Data usage behavior shift - PowerPoint PPT PresentationTRANSCRIPT
Data & AnalyticsApplied to Test Strategy
Brad Waggle
Board Test Workshop
September 9th, 2014
Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Michael Dell … http://www.dell.com/learn/us/en/19/power/ps2q14-20140230-michael
Here in Texas, everything’s big, so we just call it data.
Cisco Confidential 3© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Context
Data usage behavior shift Facilitating the shift
Technology Stack
What can we do now that we couldn’t do before?
What can’t we do yet but are working on?
Agenda
Cisco Confidential 4© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Test / Supply Chain Complexity 1000s of testers, 10’s-of-thousands of product IDs, 100’s of
thousands of component part numbers
Billions and Billions of rows of data (over years and years) Test and Repair
6.1B Measurement
45.5B Component
65.8B
And GROWING FAST!
Context
Cisco Confidential 5© 2013-2014 Cisco and/or its affiliates. All rights reserved.
In the past Small product sample Short time frame Solve point problems Lack of learning
Going Forward All products (n = all) All time frames Solve strategic problems Learn and prevent problems
Data usage behavior shift
Cisco Confidential 6© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Migrating from: Excel based aggregation / manipulation Limited custom web pages IT driven support model Traditional database architecture
To: Best in class front-end analytics (Spotfire) User driven support model / self-service Big data architecture
Facilitating the shiftTest
Repair
Field
Component
Cost
Product
Qualification
Cisco Confidential 7© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Migrating from: Excel based aggregation / manipulation Limited custom web pages IT driven support model Traditional database architecture
To: Best in class front-end analytics (Spotfire) User driven support model / self-service Big data architecture
Facilitating the shiftTest
Repair
Field
Component
Cost
Product
Qualification
Cisco Confidential 8© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Big Data Cloudera Hadoop Hive / Impala
Traditional database MSSQL
Spotfire Analytics R Statistical Programming (TERR)
Technology Stack
Cisco Confidential 9© 2013-2014 Cisco and/or its affiliates. All rights reserved.
What can we do that we couldn’t do before?
Cisco Confidential 10© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detect technology & quality trends
Consistently apply strategies to products in the NPI Phase
Apply statistical models at scale
Use big data and analytics to:
Cisco Confidential 11© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Detect Technology & Quality Trends Drove a major shift in test strategy
Multi-year view of a design / component margin Trend for 1 electrical test parameter (65 products, 6 years, 100M measurements)
Cisco Confidential 12© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Consistently apply test strategies to a new product Drives the most cost effective test solution up front in the NPI cycle Launch Lean vs. launch and optimize
Test strategy dashboard that facilitates choosing the best test optimization strategy for a new product
Cisco Confidential 13© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Apply statistical models / algorithms at scale Determine component variability vs. design margin with higher
confidence Reduce non-value-add tests
Cpk analysis to determine critical measurement parameters
Cisco Confidential 14© 2013-2014 Cisco and/or its affiliates. All rights reserved.
What can’t we do yet but are working on?
Cisco Confidential 15© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Leverage unstructured / semi-structured data
Perform predictive modeling
Use big data and analytics to:
Cisco Confidential 16© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Leverage Unstructured, Semi-Structured Data
ICT test log
Enhance
Cisco Confidential 17© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Predictive Modeling
Mock up of linear regression attempting to correlate component types to overall product failure rates
Cisco Confidential 18© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Spatial Analytics
Cisco Confidential 19© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Questions?