copyright © 2012 by gan wang, lori saleski and bae systems. published and used by incose with...
TRANSCRIPT
Copyright © 2012 by Gan Wang, Lori Saleski and BAE Systems. Published and used by INCOSE with permission.
The Architecture and Design of a Corporate Engineering Data Repository
Dr. Gan Wang
Ms. Lori Saleski
27th International COCOMO Forum
16-18 October 2012
Software Engineering Institute, Pittsburgh, PA
Outline
What is the Engineering Data Repository? CONOPS Architectural and Design Considerations User Interfaces Data Collection Process Lessons Learned
2
What’s the Engineering Data Repository?
A suite of tools, comprised of– Database of current and historical program data
– Data search and query portal
– Estimating tools and process
– Data collection and process
– Analysis and calibration tools
3
Identify historical programs• Quality program• Establish program technical and financial
POCs
Conduct initial data collection training session
• Learn the Data Collection Template• Learn metrics definitions
Collect raw data• Financial database• Technical data records
Interview and data review sessions• Recollect questionable data• Additional interview sessions
Conduct analyses• Normalization• Statistical analysis
Engineering Program Metrics
Database
Data Collection ToolData Search
Portal
Estimating Tools
EPB Process Flow – Proposal Phase
Entry Criteria
Prepare and Hold Proposal Kick-Off Meeting
• Customer’s Proposal Materials
• Approved Phase 1B Package
• Approved PTW
• Proposal Shared Repository
A
Perform Quick Look Cost Estimate
Perform Cost Estimation and Schedule Assessment
ExecuteRequest for Bid
Approval
Review Program- matic
Baseline
2A DR
Review Tech. Solution
/ Baseline
Establish the Programmatic Baseline
ExecuteIndependent Design
Review
Establish the Technical Solution / Baseline
TSR
Prepare and Hold Estimate Kick-Off Meeting
Independent Cost Estimate
TBR
RBA2A
Exit Criteria• RBA Briefing and Certificate
• TBR Package and Certificate
• Phase 2A Package and Certificate
• 2A IDR Package and IDR Certificate
• Risk & Opportunity Registers & Plans
• Assumptions
• CBB Provisioning Plan and MR/TC Generation Plan
• Engineering Assessment Scorecard
• Proposal Shared Repository
May be combined
Run Concurrent
Define Top LevelCost TargetAllocations forNRE and RE Efforts
Pass
Pass
Pass
Pass
B
E
J
D
F
K
C
G
H
PassI
ML
Activity Review
Key:
OptionalReview
Optional
Flow
Execute the Technical Bid
Review
ExecutePhase 2A Phase
Review
What’s in the Repository?
Historical Data:
Project metadata– Project and program level descriptions
– Attributes and characteristics
Engineering financial actuals– Actual (historical) efforts
– Estimated (in-execution) efforts
System-level design metrics, e.g.,– Requirements
– Interfaces
– Algorithms
– System complexities
Subsystem/component-level metrics, e.g.,– SLOC counts
– SWaP
– Drawing counts
– Supportability
Tools and Processes:
Engineering (parametric) estimating tools Data collection and analysis tools Procedures, templates, checklists Training materials, videos References
4
What’s Motivation?
The dilemma we ran into in the past…
5
Design.
Adapted from Dilbert, © Scott Adams, Inc.
Provide enterprise-wide, central and easy-to-use data store for all engineers– Consistent use of historical data in system design and business winning
– Supporting entire project life cycle
– Improved estimation accuracy, confidence and credibility
– Save cost and reduce cycle time
– Strategic competitive discriminator
Lifecycle Operational Data Needs
6
In support of: Cost estimating for Project baseline and rebaseline
(budget estimates), ETCs / EACs / ECP development, product design trades, risk & opportunity, engineering
productivity measurement
Detailed cost history and metric data by development phase, calibrated
estimating models
INNOVATION SELECTION SPECIFICATIONDEFINE/
REFINE
VALI-
DATIONPROD.
SUP-
PORT
DIS-
POSAL
Con
trac
t A
ccep
ted
Mob
ilize
Inte
grat
ion
of S
yste
m
System Qualification
Pro
duc
tion,
In
stal
latio
n an
d C
om
mis
sion
ing
Phase Reviews1B 2A 2B 2C 3A 3B 4 5 6 7 8
Iden
tify
Op
tions
Dev
elo
p O
ptio
n D
efin
ition
s
Ass
ess
Eac
h
Op
tion
Do
wn
Sel
ect
Op
tions
Det
ail
Rem
ain
ing
Op
tions
Pre
par
e P
rop
osa
l
Pro
po
sal
Acc
epte
d
Dep
loy
Tec
hn
olog
y
Pro
duc
e S
yste
m
Sp
ecifi
catio
n
Ag
ree
Sys
tem
S
pec
ifica
tion
Fre
eze
En
gin
eerin
g D
efin
ition
Pro
duc
t M
fact
urin
g D
efin
ition
Val
ida
te, B
uild
&
Cle
ar f
or O
p.
Tria
ls
Pro
ve O
p.
Cap
abili
ty
Cer
tify,
Bui
ld,
& P
rod
uct
Op
erat
iona
l C
lear
ance
Sup
po
rt
Pro
duc
t an
d S
ervi
ce
Dec
om
-m
issi
onin
g P
roce
dur
es
Design Reviews SRR
SDR/PDR
CDR
TRR
PRR
Detailed cost history, metric data, calibrated
estimating models
In support of:
Firm Cost Estimates / Independent Cost
Evaluations (ICE), LCCE, Engineering bid estimation
In support of: ROM & Life Cycle estimates, “Price to Win“ and feasibility
analysis
Project/Product level data for
ROMs/Quick Looks
Prepare Bid/No Bid Information
Prepare Proposal Initial Design
Des
ign
Sys
tem
In support of: ECP Development, ROMs & Life Cycle
estimates
High Level & Detailed cost
history, metric data, calibrated
estimating models
In-F
ield
S
uppo
rt
Disposal
Use of historical data essential in system design and program execution
Typical Use Cases
7
Query Single Data
Point
Query Single-
Attribute Statistics
Query Group of
Data Points
Ingest Single Data
Point
Data Repository
+
+
+
Data Collector
Data Analyst
Everyone
Three Interconnecting Logical Databases
8
In-Progress Program Database
Estimate Database
Post Mortem Database
Use Past Results to Predict Future Performance
Project Engineering EACs Project Engineering Actuals
Cost Proposals
Cost Proposals
Time
Project Bid / Proposal Project Life Cycle Project Post Mortem
Engineering Estimates
Cost Element Structure
9
1.0 – System/Project1.1 – Integrated Project Management1.2 – Systems Engineering1.3 – Prime Mission Product (PMP)1.4 – Platform Integration1.5 – System Test & Evaluation (ST&E)1.6 – Training1.7 – Data Management 1.8 – Peculiar Support Equipment1.9 – Common Support Equipment1.10 – Operational / Site Activation1.11 – Industrial Facilities
MIL-STD 881 CompliantM
app
ing
R
ules
Project/Organization Specific Structures
Dynamic Data Structure Design
10
Project 1
Project 2
Project 3
Project N
… …
Data RecordsData Instances
Data Record Closed Instance
In-Edit Instance
Record Closed
Dynamic linked lists used to support different data types and periodic collections throughout project and system life cycles
Dynamic linked lists used to support different data types and periodic collections throughout project and system life cycles
Database Workflow Supporting Collection of Multiple Data Instances of Same Program Data
11
Built-in Data Ingestion and Review Functions to Support Role-based Interactions
Acc
ess
Con
trol
by
Rol
es
Data Query Portal – Search by Program Attributes
12
Search Criteria
Accessible from Corp.
Intranet
Search Dashboard
Search Relevance
Detail Data Queries
13
CalculatedStatistics
Detail Project Data
Consistent Process is a Cornerstone
14
Qua
lific
atio
n of
Dat
a P
oint
Practical Lessons Learned
Data quality– Consistency, consistency, consistency!
– “Apples-to-oranges” has little to no benefit
Plan for additional time– Assumptions/definitions of data
– Analysis in aggregate
Problem solving exercise Corporate expertise Management championship
15
In Closing
Enterprise-level Engineering Data Repository– Database of current and historical program data
– Data search and query portal
– Estimating tools and process
– Data collection and process
– Analysis and calibration tools
Used throughout project and system design life cycles Proven effective
– Easy-to-use, popular with engineers
– Improves accuracy, confidence and credibility
– Saves cost and reduce cycle time
– Enhances business competitiveness
16
17
Dr. Gan Wang
703-668-4259
Ms. Lori Saleski
603-885-6353
Questions & Comments