week 5 – special topics. risk management best practices
TRANSCRIPT
Week 5 – Special Topics
Risk Management Best Practices
Traps, Alarms & Escapes From Navy Best Practices
Traps - think have risks covered by following procedures
Alarms – assumptions that cause trouble
Consequences – if nothing is done
Risk Checklist
• Personnel • Availability • Experience Levels • Mix of Disciplines • Requirements • Definition • Stability • Complexity • Resource Availability • Facilities • Hardware • Personnel • Funding Profile • Communications • Technology • Availability • Maturity Levels • System Complexity • Operating Environment • Design • Methods • Complexity • Software Tools • Testing, Modeling • Language • Operational Interfaces • Hardware • Sensitivities to Other Risks • Estimating Error • Schedule • Cost • Number of Critical Path Items
Potential Risk Item
Risk Management Training
Why RM Training is Needed Weak backgrounds in risk
management Mistaken concepts
Assessment is RM – skip planning & monitoring
Mitigation is only handling strategy All risk can be eliminated Focus on performance, omit cost &
schedule
Tailoring RM Training Adjust to different project team groups
Sr. management Working level engineers
Address issues each group is likely to face
Address tailoring RM activities to meet program needs – not one-size fits all
Software Risk Management
SW Risk as a Special Case Virtual
Can’t physically touch/feel to assess History of latent defects
Many interfaces Hardware-to-software Operating system-to-operating code Incompatible combinations
Update frequency Multiple versions Upward/downward compatibility
Taxonomy of SW RiskRisk Grouping Risk Issue
Project Level Requirements - excessive, immature, unstable, unrealisticLack of user involvementUnder estimation of complexity or dynamic nature
Project Attributes Performance - errors, qualityUnrealistic cost or schedule
Management Ineffective project management
Engineering Ineffective integration, assembly, testUnanticipated difficulties across user interface
Work Environment Immature design, process, technologiesInadequate configuration controlInappropriate methods, inaccurate metricsPoor trainiing
Other Legal, contractual issuesObsolescence (incl. excessive duration)Difficulties with subcontracted itemsUnanticipated maintenance & support costs
Boehm’s Top 10 Software Risks
Risk Item Management Techniques
Architecture, performance,quality
Architecture trade-off analysis, simulations,benchmarking, modeling, prototyping
Requirements changesChange thresholds, incrementaldevelopment, information hiding
Legacy softwareRestructuring, design recovery, wrappers,phase-out analysis
Externally-performed tasksReference checking, pre-award audits,award fee contracts, competitive design orprototyping, team building
Straining computer sciencecapabilities
Technical analysis, cost benefit analysis,prototyping
Boehm’s Top 10 Software Risks
Identification Strategies Review Schedule & Networks Review Cost Estimation Parameters Perform Interviews Revisit Lessons Learned Develop a Risk taxonomy Brainstorm and play “what if” Re-sort the watch list (e.g., by source)
Identification- Schedule Risks
Use the planning or baseline schedule Evaluate the activity network view Look for nodes with:
High fan-in (many activities terminate at a single node)
High fan-out (many activities emanate from a single node)
No predecessors
ID- Cost Risk Drivers
Consider specific areas of concern that can lead to problems: Personnel experience, availability Requirement complexity, firmness Scheduling and prediction of task and
partition times Hardware requirements, interfaces,
constraints
0.1Low
0.3Moderate
0.5High
0.7Very High
0.9Extremely
High
StabilityFactorMaturity Factor
ComplexityFactor
Technology exists andcan be used “as is”
Technology requiresminor change beforeuse (<25%)
Technology requiresmajor change beforeuse (<50%)
Technology requiressignificant design andengineering beforeuse (<75%)
State of the art, someresearch done
Simple relative tocurrent environment
DependencyFactor
Minor complexityrelative to currentenvironment
Moderately complexrelative to currentenvironment
Significantly complexrelative to currentenvironment
Extremely complexrelative to currentenvironment
Entirely within project control
Depends on existingproduct supplied fromoutside organization
Depends on supplyand modification ofexisting product fromoutside organization
Depends on newdevelopment fromoutside organization
Depends on findingdevelopment fromoutside organization
External factorswill not make anychanges
External factorswill make minorchanges (<25%)
External factorswill make majorchanges (<50%)
External factors willmake significantchanges (<75%)
External factorswill make constantchanges
ISO Risk Probability Table
ISO Risk Consequence Table
ISO Risk Contour
SW Risk Handling Avoidance - de-scoping objectives Assumption – latent defects Control – user acceptance testing Transfer – from software to
firmware or hardware
SW Metrics
S o f t w a r e R i s k I t e m s
Co
un
t
T i m e
C l o s e d R i s k s
N e w R i s k s
Commercial vs. DoD/NASA Perspective on Risk
Management
Commercial vs. Gov’t Perspective
Different market conditions Different best practices Different likelihoods for similar
issues
As always, tailor RM to program needs
Market Differences How is risk impacted?
Commercial DoD / NASA
Many small buyers Fewer large buyers
Many small suppliers Typically few suppliers of a given item
Market sets price Oligopoly pricing - biased to availble budget
Free movement in/out market Barriers to entry
Prices set by marginal costs Prices proportional to total cost
Once funding secured, usually stable May have unanticipated disruptions to funding
Capcity to supply adjusts to demand Moderate to large excess capacity
SW Development Best Practices
Commercial DoD / NASA
Evolutionary upgrades of existing systems Little reuse, many unique systems
Heavy buyer involvement (as team member) Formal development model - buyer oversees
Informal reviews Very formal reviews
Heavy user involvement Limited user involvement; buyer involved
Based on one or more industry stds Use gov't and industry stds
Prototyping common Limited prototyping
How is risk impacted?
Risk Category LikelihoodCategory Commercial DoD / NASA CommentCost Highly Likely Highly Likely Whenever new development is
requiredDesign Possible Likely Degree of design enhancement
requiredIntegration Possible Likely Driven by complexitySupport Possible Likely Commercial life cycles generally
shortManufacturing Likely Likely Varies with production rate &
resource availabilityTechnology Possible Likely Increases as push state of artManagement Possible Possible Depends on program complexity,
performance expectations
Political Unlikely Likely External issue to program
Overview of Risk Management Tools
Cautions in Tool Selection A good tool for one organization may
not be a good match for another Tailor RM to the program needs
The tool should never dictate the process Define process, then choose compatible
tool Be compatible with program culture
Effective Use of a Tool RM is more than using a RM tool Tool must efficiently & effectively
integrate into program Resources required Level of detail, complexity Focus of tool – e.g., program phase
RM Data Base Considerations Provide sufficient configuration control
Accessible to all team members Ability to accept anonymous comments
Support program needs Reporting Monitoring Captures lessons learned Fulfills contractual requirements
Balance costs/value
Tools Comparison @Risk & Crystal Ball – licensed
software Monte Carlo simulation add-in for Excel
Select desired distribution function & define parameters
Provide data and generate a plausible distribution function
Provides statistics and graphical output User provides risk analysis structure
Probability-Consequence Screening
Developed by the Air Force Risk events assigned a probability &
consequence for performance, schedule & cost
Position in consequence screening matrix determines risk score
User assigns Hi, Med, Low ranges Generates reports and graphical output www.pmcop.dau.mil
Probability-Consequence Screening
Probability-Consequence Screening
Risk Matrix Excel – based model
Collects inputs in watch list format Uses best practices (ordinal) breakout for
probability & consequence Orders events by Borda rank & assigns risk
level Generates action plan reports and
graphical output www.mitre.org
Risk Matrix
Risk Matrix
Risk Radar Access – based, licensed software
Can establish standard values for risk categorization
Manual or automatic risk prioritization Complies with ISO, SEI CMMI & Government
standards Generates detailed, summary & metrics
reports Demo available:
www.iceincusa.com/products_tools.htm
Risk Radar
Risk Radar
Risk Radar
TRIMS Technical Risk ID & Mitigation
Knowledge-based system Utilizes SEI & Navy Best Practices to collect
data on past experiences Measures technical risk rather than cost &
schedule Most applicable to design efforts Can tailor categories, templates & questions
Generates status, next action & overdue action reports
www.bmpcoe.org
TRIMS Technical Risk ID & Mitigation
TRIMS Technical Risk ID & Mitigation
DSM – Design Structure Matrix
Knowledge & simulation-based tool Assesses complexity of dependency
relationships between project tasks Measures risk in terms of schedule impact Most applicable to design efforts Ongoing development effort at MIT
Generates suggested sub-team groupings, & probability curves for task duration ranges
www.dsmweb.org
DSM – Design Structure Matrix
DSM – Design Structure Matrix
DSM – Design Structure Matrix
Final Exam Closed book, closed notes You have 90 minutes for exam. Any questions? Turn in Part II of your project
according to the schedule discussed last week