dr. martin falk mr. glen logan systems engineering and open systems

Dr. Martin Falk Mr. Glen Logan

Systems Engineering and

Open Systems

Systems Engineering

• Recent OSD & Service Initiatives

• NASA Accident Experience

• Current Challenges Related to Technical Management

Uneasy Feeling that You are Headed for Trouble?

Perception

• Growing realization that systems engineering focus was lost in DoD, NASA, and NRO during last decade.

• Systems engineering capabilities have deteriorated in both government and industry.

• Substantial efforts and initiatives now underway to fix the problem.

0.6

1.0

1.4

1.8

2.2

2.6

3.0

0% 2% 4% 6% 8% 10% 12% 14% 16% 18%

SE Effort = SE Quality * SE Cost/Actual Cost

Act

ual

/Pla

nn

ed C

ost

Source: SECOE 01-03, INCOSE 2002

Average Cost Overrun

90% Assurance

Cost OverrunRisk vs. Systems Engineering Effort

0.6

1.0

1.4

1.8

2.2

2.6

3.0

0% 2% 4% 6% 8% 10% 12% 14% 16% 18%

SE Effort = SE Quality * SE Cost/Actual Cost

Act

ual

/Pla

nn

ed S

ched

ule

Source: SECOE 01-03, INCOSE 2002

Average Schedule Overrun

90% Assurance

Schedule OverrunRisk vs. Systems Engineering Effort

Systems Engineering Process

REQUIREMENTSANALYSIS

FUNCTIONALANALYSIS &

ALLOCATION

DESIGNSYNTHESIS

SYSTEMS ANALYSIS&

CONTROL

Objectives• Track progress• Ensure requirements traceability

• Evaluate alternatives• Manage risk

• RISK MANAGEMENT • TPM/METRICS• COST-PERFORMANCE TRADES• TEST EFFECTIVENESS REVIEWS• CONFIGURATION MANAGEMENT• TECHNICAL REVIEWS

Systems EngineeringFundamentals, DAU Press

System Development ‘V’

SYSTEM LEVELDESIGN REQUIREMENTS

ITEM LEVEL DESIGN REQUIREMENTS

DESIGN REQUIREMENTS COMPLETE

FAB

, IN

TEG

RA

TE &

TES

T

COMPONENTS

ASSEMBLIES

ITEMS

SYSTEM LEVEL

SFR

CDR

TRR

SVR

DESIG

NPDR

System Integration

System Demo

System Dev & Demonstration

IOC

The Defense Acquisition Management FrameworkDoDI 5000.2 (Spring 2003)

The Defense Acquisition Management FrameworkDoDI 5000.2 (Spring 2003)

DesignReadiness

Review

Funding:

LRIP

Operations& Support

• Process entry points at Milestone A, B, or C

• Entrance criteria met before entering phase

• Evolutionary Acquisition or Single Step to Full Capability

• Integrate subsystems, complete detailed design, and reduce system-level risk

SystemIntegration

• Conduct AoA, refine initial concept & develop Technology Development Strategy

ConceptRefinement

FRPDecision Review

Full-Rate Prod &Deployment

• Full rate production

• Deployment of system

Rqmnts:

ConceptRefinement

TechnologyDevelopment

Concept Decision

FRP & Deployment

Production & Deployment

Low-Rate InitialProduction

• Create efficient manufacturing capability

• LRIP• IOT&E, LFT&E

of prod-rep articles

C

Pre-Systems Acquisition Systems Acquisition Sustainment

(BA 1 & 2)

User Needs & Technology

Opportunities

FOC

• Complete development• Demonstrate ability of

system to operate in useful way consistent with KPPs

• Combined DT/OT

SystemDemonstration

• Reduce technology risk & determine the appropriate set of technologies to be integrated into a full system

• Demo the technologies in a relevant environment

Technology Development

BA 3/4 BA 5 BA 5/Procurement Proc/Operations & MaintenanceBA 5

CDD CPD

Validated & approved by operational validation authority

ICD

Increment II

Increment III B DRR C FRP

BA

B DRR C FRP

Systems EngineeringSystems EngineeringAs Applied ToAs Applied To

Concept RefinementConcept Refinement

Interpret User Needs,Analyze Operational

Capabilities & Environmental Constraints

Develop Concept Performance (& Constraints)

Definition & VerificationObjectives

Decompose Concept Performance into Functional Definition &

Verification Objectives

Develop Component Concepts, i.e., Enabling/Critical Technologies,

Constraints & Cost/Risk Drivers

Assess/Analyze System Concept Versus Functional Capabilities

Configuration ItemExpectations

Assess/Analyze Concept & Verify System Concept’s

Performance

Analyze/Assess Concepts Versus Defined User Needs &

Environmental Constraints

System FunctionalExpectations

OperationalExpectations

Trades

Trades

Trades

Analyze& Fix

Analyze& Fix

Analyze& Fix

Decomposition&

Definition

Integration&

Verification/Validation

Decompose Concept Functional Definition into Component Concepts

& Assessment Objectives

Trades

Assess/Analyze Enabling/CriticalComponents Versus Capabilities

nalyze& Fix

ComponentExpectationsComponent Engineering

Responsibility,Systems EngineeringParticipation

Systems EngineeringResponsibility,Component EngineeringParticipation

•ICD•AoA Plan•Exit Criteria•Alternative Maintenance & Logistics Concepts

•Prelim Sys Spec•T&E Strategy•SEP•Support & MaintenanceConcepts & Technologies•Inputs to: -draft CDD, -TDS, -AoA- Cost / Manpower Est

ASR

MS A

Systems EngineeringSystems EngineeringAs Applied ToAs Applied To

Technology DevelopmentTechnology Development

Interpret User Needs.Analyze Operational

Capabilities & Environmental Constraints

Develop System Performance (and Constraints) Specification

and Critical Technology Verification Plan

Develop Functional Definitions for Enabling/ Critical Technologies &

Associated Verification Plan

Develop System Concepts,i.e., Enabling/Critical Technologies,

Update Constraints & Cost/Risk Drivers

Demo SystemFunctionalityVersus Plan


Demo/ModelIntegrated System Versus

Performance Spec

Demo & Validate SysConcepts & Technology

Maturity VersusDefined User Needs



Trades

Trades

Trades

Analyze& Fix

Analyze& Fix

Analyze& Fix

Decomposition&

Definition

Integration&


Decompose Functional Definitions into Critical Component Definition

& Tech Verification Plan

Trades

Demo Enabling/Critical TechnologyComponents Versus Plan

Analyze& Fix

ComponentExpectations

Component EngineeringResponsibility,Systems EngineeringParticipation


SRR

•ICD & Draft CDD•Preferred Sys Concept•Exit Criteria•T & E Strategy•Support & MaintenanceConcepts & Technologies•AoA •SEP•TDS

•Sys Performance Spec•LFT&E Waiver Request•TEMP, SEP, PESHE, PPP, TRA•Validated Sys Support &Maintenance Objectives &Requirements•Footprint Reduction•Inputs to : - IBR, -ISP,- STA, -CDD, - Acq Strategy--Affordability Assessment-Cost / Manpower Est

MS A

Systems Engineering As Applied Systems Engineering As Applied To System Development and To System Development and

DemoDemo




Trades

Trades

Trades

Analyze& Fix

Analyze& Fix

Analyze& Fix

Decomposition&

Definition

Integration&


Trades

Analyze& Fix

ComponentExpectations

Component EngineeringResponsibility,Systems EngineeringParticipation


SRR

SFR

PDR

CDR

OTRR

SVR/PRR

DRR

MS B

•Sys Performance Spec•Exit Criteria•Validated Sys Support &• Maintenance Objectives & requirements•APB, CDD, SEP•ISP, TEMP

•Initial Prod Baseline•Test Reports, TEMP Elements of Product Support•Risk Assessment•SEP, TRA, PESHE•Inputs to:-CPD,-STA, - ISP-Cost / Manpower Est

Interpret User Needs, Refine System

Performance Specs &Environmental Constraints

Develop SystemFunctional Specs &

System Verification Plan

Evolve FunctionalPerformance Specs into CI Functional (Design to)

Specs and CI Verification Plan

Fabricate, Assemble, and Code to “Build-to”Documentation

Integrated DT&E, LFT&E & EOAs Verify Performance

Compliance to Specs

System DT&E, LFT&E & OAs,Verify System Functionality& Constraints Compliance

to Specs

Combined DT&E/OT&E/LFT&EDemonstrate System toSpecified User Needs &

Environmental Constraints

Evolve CI FunctionalSpecs into Product

(Build to) Documentationand Inspection Plan

Individual CI Verification DT&E

NASA Small SatellitesAccident Investigation

• A concise set of mission success criteria should be established early in program

• Solid engineering discipline is required at all program stages, especially transition to ops

• Continuous risk management is a must. Risk should be considered on a par with cost, schedule, and performance

• Entire team must take personal responsibility for the product

Areas Most Common to Failed Programs

• Poorly structured and conducted technical reviews (6 of 8)

• Poor risk management practices (6 of 8)

• Inadequate testing, simulation, and V&V (6 of 8)

• Poor communications (5 of 8)

How Did We Get Here?

• Acquisition reform expectations were not clearly communicated to working level program managers, engineering and contractor staffs.

• We convinced ourselves that we could have it all – faster, better, and cheaper.

• We compromised performance and management discipline to save time and money.

Challenges in Systems Engineering Management

Technology Change Rate

Insight vs OversightSystems of Systems

Design for…everythingTechnical expertise in short supply

Commercial-NDI

Interoperability

Systems of Systems

Missile Defense System

Detection C3I Weapon

• SE Issues– Capstone Capabilities vs. System Level Requirements– Interoperability– Commonality– Modular Designs

CooperatingSystems

CapstoneSYSTEM

Interoperability

• Systems of Systems demand interoperability among systems

• Requires ‘reasoned sub-optimization’ to select interface standards that benefit the integrated system.

• “Systems Thinking” must be a way of life.• Demands that systems be integrated from the

outset – a top down process vice the system up process used to date

• Gives rise to JCIDS and Joint Technical Architecture

Problem: IER Scalability

One-to-OneCurrent Interoperability KPP

centers around one DoD architectural view (OV-3) that contains “Information Exchange Requirements” (IERs)– One-to-one relationship

(point-to-point)

This example: 10 systems

IERs 10(10-1) = 90

Solution: The Net-Ready Approach

Net Ready approach centers on central network:

– Focus on organizational contributions and consumption of information

– One-to-network paradigm

One-to-Many

This example: 1 systemhas to deal with 1 interface

Network

Net-Ready KPP

*

“Consists of measurable and testable characteristics and/or performance metrics required for timely, accurate, and complete exchange and use of information to satisfy information needs for a given capability.” CJSI 6212.01C

Comprised of compliance with:Net-Centric Operations and Warfare Reference ModelGlobal Information Grid Key Interface ProfilesDOD information assurance requirementsSupporting integrated architecture products

J-6 certifies NR-KPP

JITC evaluates and certifies Joint interoperability

C4ISP replaced by Information Support Plan

Architectures

DoD Architectural Framework

Source for direction and information on the development of Operational, System, and Technical Architectures to support system developments.

DoD Architecture Framework Working Group

DoD Architecture Framework Version 1.0

Volume II: Product Description

9 February 2004

Syst

ems Technical

Operational

Syst

ems Technical

Operational

Integrated Architecture:One Architecture - 3 Views

The Systems View describes and interrelates the existing or postulated technologies, systems, and other resources intended to support the operational requirements.

The Operational View describes and interrelates the operational elements, tasks and activities, and information flows required to accomplish mission operations.

The Technical View describes the profile of rules, standards, and conventions governing systems implementation.

Operational

Systems

Technical

Insight vs. Oversight

• Concept:– Contractor flexibility to innovate, seek improved means to

deliver improved products (faster, better, cheaper)– Government retains control of top level requirements, has

required visibility to make informed key decisions

• Practice:– Technical managers in some cases believe they have little or

no leverage to require information or to effect change– Frustration with perceived responsibility absent authority

Government has obligation to ensure that technical management practices are sound; may at times feel like oversight.

Commercial and Non-Developmental Items

• CI-NDI offer real benefits:– Shorter development cycles– Development costs already amortized– Continuing benefits from commercial competition

• …but also have real (sometimes not apparent) costs:– May bias requirements to favor current, known capabilities– Integration difficulties may be overlooked– Updates and sustainment subject to marketplace– Testing is a continuous activity– Modified commercial items lose most of perceived benefits

Design for...everything

• [Some of the] design considerations (per Defense Acquisition Guidebook):

– Producibility – Interoperability– Quality – Survivability– Supportability – System Security– Open Systems – Electromagnetic

Effects– Software – Value Engineering– Environmental, Safety, Health – Unplanned Stimuli– Human Systems Integration – Vertical Integration– Standardization– Reliability, Maintainability, Availability

Corrosion

Technical Expertisein Short Supply

• Fewer technical experts available in government– Downsizing – Hard to attract needed expertise– Aging workforce eligible for retirement

• Inevitable consequence is more dependence on contractors to make technical management judgments

• Substantial evidence that, when budget and schedule pressures are applied, contractors may not make good technical management decisions

Technology Change Rate

• Computer and communications technologies have very short life cycles; much shorter than systems of which they are a part.

• Designs must accommodate frequent updates - during development and after

• Open systems architectures and modular designs are recommended design approaches to address problem

• While conceptually easy to grasp, these strategies are often poorly understood at the implementation level.

• Change vs. Time– Requirements– Designs– Testing

• Specs and ICDs: Approved vs. PlannedDevelopment Specs: 50% by PDR; 100% by CDR

• Subcontracts: Signed vs. Planned30-50% by PDR; 100% by CDR

Technical ManagementLeading Indicators

USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programshttp://cse.afit.edu

5-10% max deviation


• Engineering Manpower: Total vs. Plan

• Design Documentation: Actual vs. PlanDrawing release: 40% by PDR; 90% by CDR

• Trade Studies: Completion75-100% design definition studies complete by PDR

• Software Products: Complete vs. Time10% variance threshold typical

• EVMS: Completion and Complexity

USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programs

• HW and SW Deliveries vs. Plan• Technical Performance Measures:

Technical performance vs. plan profile

• Deficiency Reports: Actual vs. expected• Maintenance and Support Facility Changes

Minimal after Milestone C

• Action Item Closure after Design Reviews• Key Characteristics Defined

70-90% by PDR; 100% by CDR



• Testing Activities: Approved vs. Planned

• Proposed schedule vs. Actual

• Subcontract ManagementListed indicators monitored by prime for subs



Systems Engineering“Things to Look For”

• Systems Engineering Management Plan

• Requirements Management/Traceability

• Risk Management Plan/Process

• Technical Performance Measures

• Configuration Management Plan/Process

• Systems Engineering Training

Summary

• Systems engineering is in the process of a comeback as a discipline central to acquisition success.

• Today’s issues and challenges are complex – made more so by virtue of joint warfighting and systems of systems.

• There is no free lunch – this stuff is hard and it takes time and money to get it right.

Backup

Systems Engineering Process

• Operational• Function• Performance• Interfaces• Support

• Funding• Technology• Legal• Regulation• Compliance

• Mission Area Analyses– Comprehensive, analytic evaluations of capability vs.

requirements in an assigned mission area

Operational Requirements

• Mission Needs Statement (MNS),Initial Capabilities Document (ICD)

• Non-specific statement of operational capability required to accomplish a given mission

Operational Requirements Document, Capabilities Development Document (CDD)

• Objectives and minimum acceptable thresholds for a specific concept or system.

Capabilities Production Document (CPD)

Requirements Analysis

REQUIREMENTS ANALYSIS

OBJECTIVE: Define the Problem

• Functions – WHAT must system do?

• Performance – HOW WELL?

• Interfaces – UNDER WHAT CONDITIONS?

• Other Constraints

Performance Specifications describe products and servicesIn terms of

• Function• Performance• Interface

Climb

Requirements Analysis – Two Part ProcessOperational Concept Definition

Takeoff

Cruise

Attack

DistanceSurface

RateAltitude

AltitudeSpeedDistance

timeline0 2 12 70

…to get agreement between users and developerson key mission requirements and parameters

functions

performance

Requirements AnalysisDesign Requirements Definition

TAKE-OFF CLIMB CRUISE ATTACK

• Identify and confirm explicit requirements

• Identify key implicit requirements – function, performance, and interface

• Trace flow down of requirements

• Establish design constraints

ORD

DistanceSurface

Detect Ident

Requirements Flowdown

ORD

SYSTEM SPEC

AIRFRAMESPEC

ENGINESPEC

• Aircraft will be capable of air operations from carrier...

• Aircraft landing weight NTE 50,000 lb...• Approach/landing speeds NTE 150

and 110 kts respectively...• System will be supportable by

existing USN logistics system...

• Empty gross weight NTE 15,000 lb...• Absorb shock ...15 fps rate of descent...• ...tailhook...

• Weight NTE 15,000 lb...• Thrust NLT 30,000 lb at max power (SL/std day)...

ITEM SPECS

Functional Analysis & Allocation


DESIGNSYNTHESIS

ANALYSIS &CONTROL

FUNCTIONALANALYSIS &ALLOCATION

INPUTS:• Functions• Performance• Interfaces• Other Products From Requirements Analysis

OUTPUT: Functional Architecture – Logical Description, Performance Allocated to Functions

TAKEOFFCLIMB

CRUISE

ATTACK ATTACK

TRACK

IDENTIFYDETECT AIM

• Target Types

• PK

• Time

Example

Requirement: Attack light armored vehicles within 30 minutes of identification under <defined> weather conditions,

with a Pk not less than 0.85. Coordination and control

exercised by Army Fire Direction System.

function

perform

ance

interface

Attack• 30 min• Pk 0.85

Detect ID Track Aim FireComm

TD

PD

TC

PC

TID

PID

TT

PT

TA

PA

TF

PF

AFDS

Alternative Architecture

Or…?

• Key functions accomplished by off-board system• May require less development if capabilities already exist • On board attack function requires less time for this system• More interfaces, more complexity

Is this a better choice? Depends on technical and cost issues


TDPD

TCPC

TIDPID

TTPT

TAPA

TFPF

AFDS


TDPD

TCPC

TIDPID

TTPT

TAPA

TFPF

AFDS

Detect ID

Track Aim Fire

Communicate

AFDS

Off-board

Steps in Functional Analysis and Allocation1. Decompose top level functions to next

lower level

2. Allocate performance requirements to functions (higher level to lower level)

3. Evaluate alternative architectures

4. Select and refine preferred architecture

Functional Analysis & Allocation

Result: a logical description of the product with performance parameters such that specific hardware and software elements capable of performing the required functions (at the required levels of performance) can be identified- the Functional Architecture

Design Synthesis

Detect ID

Track Aim Fire

Communicate

AFDS

Off-board

Once a preferred architecture isselected, design alternatives canbe developed and compared.

GroundVehicle

Chassis Propulsion Radar Weapon

Missile

Propulsion Guidance WarheadAirframe

A selected Functional Architecture should suggest severalalternative concepts or designs that could conceivably do the job.

The issue becomes cost-effectiveness. Which will be most effectivefor the mission in terms of both performance and cost?

Design Synthesis


ANALYSIS &CONTROL

FUNCTIONALANALYSIS

DESIGNSYNTHESIS

INPUTS:• Functional Architecture

ATTACK

TRACK

IDENTIFYDETECT AIM

• Target Types

• PK

• Time

ATTACK

TRACK

IDENTIFYDETECT AIM

• Target Types

• PK

• Time

OUTPUT:• Physical Architecture

An architecture of physicalelements capable of performingthe functions required at the levels of performance specified

AIR VEHICLE

AIRFRAME PROPULSION AVIONICS WEAPON

This is the product element of the WBS!

Design RequirementsPrime Item Specification Development

• As functions and performance parameters are associated with physical elements, a set of design criteria are developed for those elements

GroundVehicle


FunctionsPerformanceInterfaces

• Detection

• Tracking

• Range

• Probability Detect

• Weapon, Chassis

Design RequirementsPrime Item Development Specification

• A performance spec is developed for each major configuration item to be developed.

• These specs set out the design requirements for the items• Together these specs document the Allocated Baseline for

configuration control purposes

GroundVehicle


ItemSpec

ItemSpec

ItemSpec

ItemSpec

dr. martin falk mr. glen logan systems engineering and open systems

Documents

systems engineering

open systems

ability of system

glen logan systems engineering

technical management

technology risk

initial concept

b drr c frp ba slide