evm+agile the darkside
TRANSCRIPT
+
Integrating Agile Software Development (Agile) on Earned Value Management ProgramsStarting with an EIA–748–C compliant Earned Value Management System, integrating an Agile Software Development Lifecycle (Agile) is straight forward when there is a Bright Line between the Performance Measurement Baseline (PMB) and the Sprints and Tasks of the Agile Software Development Process.
AGILE AT SCALE
forFAR 34.2 / DFARS 234.2Acquisition Programs
1
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+The Dark SideBoth Earned Value Management and Agile have Dark Sides. Things that are not talked about in public.But when they are Integrated, each provides a solution for the problems of the other.Assess current and desired Maturity for Agile and EVM is the starting point for integrating these two processes.
Every tool, process, and practice has a dark side. Knowing these is a Critical Success Factor to the
integration of EVM and Agile at the desired Maturity Level.
338Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
339All successful software projects must answer three critical questions …
… how much it will cost, when will it be done, and what will be delivered for that time and cost?
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ The Dark Side of EVM and AgileFixed with EVM+Agilen Both EVM and Agile have
Dark Sides.
n EVM can balance some of Agile’s Dark Sides.
n Agile can balance some of EV’s Dark Sides.
n Together they can turn the Dark into Light.
n Agile + EVM = Match Made in Heaven?
340
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
Being more Agile on Software Development Programs creates Cost Growth to support the emerging requirements in the presence of Uncertainty
+ Let’s Start with the Agile Manifesto 341
Agile Manifesto Impact on EVM Programs
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software (in non–software projects this would refer to product).
This is simply good program management. The plan for delivering the valuable software needs to be encapsulated in the Performance Measurement Baseline
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
This problematic at the EVM level of the program. Constant change prevents EVM form being effective, since the baseline is constantly be changed
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
The period of delivery provides input to the closed Loop Control system. Agile mandates fine grained samples of physical percent complete
Business people and developers must work together daily throughout the project.
This is a key Critical Success Factor for all programs. Agile mandates this
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Dark Side of EVM
The Dark Side of EVM Agile’s Fix
Uncertainty is inherent and inevitable in software development processes and product †
Incremental and iterative development control exposure to these uncertainties by producing working software every four weeks
For a new software system, the requirements will not be completely known until the users have used the product ‡
Small incremental deliverables reduce exposure to large uncertainties with testable outcomes every four weeks.
Typical implementations expect everything fully defined up front
Requirements emerge as product development proceeds
No formal measures of quality, effectiveness, and performance in EVM
Working software at end of Sprint is the only measure of progress to plan
Earned Value can be claims for intermediate work products
Working software at end of Sprint is the only measure of progress to plan
† Ziv’s Uncertainty Principle, 1996, http://www.ics.uci.edu/~ziv/papers/icse97.ps‡ Humphreys Requirements Uncertainty Principle, 1998
342
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Dark Side of Agile
The Dark Side of Agile EVM’s Fix
Estimates measured in Story Points. No Story Point fields in DID–81861
Flow down BCWS to Work Package and use Story Points as apportioned effort for each task at the Feature level
All program work is probabilistic and statistical – reducible and irreducible uncertainty. Cost and schedule margin is not a concept in Agile
Risk Register and Schedule Risk Analysis (SRA) are part of DI–MGMT–81861
Story Points are Uncalibrated Ordinal measures of Effort, not duration of cost
Use SP’s as Physical Percent Complete and multiply BCWS to get BCWP
Definition of Done is an inconsistent a check list of activities required to produce software – Passes Unit Tests is typical
MOE, MOP, TPM, and KPP’s have tangible units of measure connected to deliverables
Rebaseling after release to fit Reality causes SPI=1 and CPI=1
Work Packages stay open until the work completes
343
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Some More Issues with Agile (1)
n Dependencies in the actual work may require planning beyond the next Sprint.n The notion of INVEST is naïve in many Software Intensive System of
Systems development programs
n Percent Complete at the end of the Sprintn What happens is the total number of Features is less than the planned
number of features?
n How can BCWP be stated?
n If uncompleted work results where does it go?
n When CPI=SPI=1.0 at the end of the Sprint in Agile there is no visibility to the actual performance of the programn Fixed duration with unfinished work is moved to the right
n Fixed duration with deprecated deliverables list
344
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+
Management Reserve
n “An amount of the total budget withheld for management control purposes, rather than being designated for the accomplishment of a specific task or set of tasks” (ANSI/EIA 748–B) n Meant to address “in scope changes” that were not planned in
baseline n The Contractor has authority and control over discretionary use of
MR budget n Not included in PMB, as it has not been allocated for specific work
scope n Must be formally allocated to work packages through an internal
change control process
Agile and Agile are missing the notion of Management Reserve or Schedule Margin
Some More Issues with Agile (2) 345
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+
Schedule Margin
n … is any task not associated with specific scope or resources, and is used to increase the probability of on–time completion of contract events. The terms Contract Events includes major logical integration points, such as, contract events, major test and integration milestones, and end item deliverables.
n Schedule margin, if used, is typically set at the time the baseline is established and set with the baseline and forecast duration equal.
n The baseline without schedule margin is typically not achievable
Agile and Agile are missing the notion of Management Reserve or Schedule Margin
Some More Issues with Agile (3)Dark Side
346
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+
Risk Management
n Provide processes, methods, tools, and an infrastructure of resources and organizational responsibilities to identify and assess the risks, determine what to do about them, and implement actions to deal with them.
n Agile is missing the formality of Risk Management that is needed for complex At Scale software development projectsn Agile at Scale technical and programmatic risks
n Interdependencies between work streams
n Interdependencies between deliverables
n Systems integration dependencies
Scrum and other Agile development methods are missing the notion of Risk Management
Some More Issues with Agile (4) 347
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Epistemic Uncertainty and Aleatory Variability are both risk drivers†
EpistemicUncertainty
§ Epistemicuncertaintyisthescientificuncertaintyduetolimiteddataandknowledgeinthemodeloftheprocess
§ Epistemicuncertaintycan,inprinciple,beeliminatedwithsufficientstudy
§ Epistemic(orinternal)uncertaintyreflectsthepossibilityoferrorsinourgeneralknowledge.
AleatoryVariability
§ AleatoryuncertaintiesarisefromtheinherentrandomnessofavariableandarecharacterizedbyaProbabilityDensityFunction
§ Theknowledgeofexpertscannotbeexpectedtoreducealeatoryuncertaintyalthoughtheirknowledgemaybeusefulinquantifyingtheuncertainty
RandomnessWithKnowableProbabilities RandomnessWithUnknowableProbabilities
Theprobabilityofoccurrence canbedefinedthroughavarietyofmethods.Theoutcomeis
aprobabilityofoccurrenceoftheevent
AProbabilityDensityFunction(PDF)generatesacollectionofrandomvariablesusedto
modeldurationsandcosts† Uncertainty in Probabilistic Risk Assessment: A Review, A.R. Daneshkhan
348
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Taxonomy of Uncertainty that Drives All Project Risk
Uncertainty
Irreducible(Aleatory)
Reducible(Epistemic)
NaturalVariability
Ambiguity
OntologicalUncertainty
ProbabilisticEvents
ProbabilisticImpacts
PeriodsofExposure
Agile and Agile provides some information to manage risk, but it is not a Risk Management System. Just a piece of Risk Management
349
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
Technical RiskManagement
TrackingandControllingPerformanceDeviations
Deliberating andrecommendingadecision
alternative
Riskanalysisofdecisionalternatives,performingtradestudiesandranking
Proposingand/oridentifyingdecisionalternatives
FormulationofobjectivesHierarchyandTechnicalPerformanceMeasures
Stakeholderexpectations,requirementsdefinitionandmanagement
Designsolutions,technicalplanning
Designsolution,technicalplanning,
anddecisionanalysis
Technical planninganddecisionanalysis
Decisionanalysis,lessonslearned,knowledgemanagement
Identify
Analyze
Plan
Track
Control
Decideandimplementdecision
alternatives
Communicate
350
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Now to the REAL Problem with Agile on EVM Programsn When the threshold of $20M is tripped for Self Assessment of the
EVMS and now $100M for DCMA Validation – the program is likely a System of Systems architecture, with complexities well beyond simple Agile teams with a small number of people in the same room with their customer is no longer the paradigm.
n The project is no longer a simple straight forward development effort with Independent work activities prioritized by the customer.n Systems architecture dominates the work processes
n Interface management is a critical success factor
n Interdependences in software, processes, operations, maintenance
n Coupling and Cohesion considerations overwhelm the simple task of writing the software to meet the needs of the on site customer.
351
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ Some Key Concepts for System of Systems†
n System of Systems (SoS)n Very large systems developed by creating a framework or architecture to
integrate constituent systemsn Typically software–intensive and net–centricn SoS constituent systems independently developed and managedn SoS constituent systems have their own purposen Constituent systems can dynamically come and go from SoS
n System of Systems Engineering (SoSE)n “The process of planning, analyzing, organizing, and integrating the
capabilities of a mix of existing and new systems into a system–of–systems capability that is greater than the sum of the capabilities of the constituent parts. This processes emphasizes the process of discovering, developing, and implementing standards that promote interoperability among systems developed via different sponsorship, management, and primary acquisition processes.” [USAF 2005]
† System of Systems Engineering and Process Synchronization, Jo Ann Lane, University of Southern California, Center for Software Engineering
352
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ SoS Compared to Traditional Development (1)n Architecting
n Architecting composability vs. decomposition
n Net–friendly vs. hierarchical
n Prototypes/experimentation/tradeoffsn Early tradeoffs/evaluations of alternatives
n Intense concept phase analysis followed by continuous anticipation; aided by ongoing experimentation
n Modeling and simulation, in particular to better understand “emergent behaviors”
n First order tradeoffs above the component systems level (e.g., more optimal at the SoS level, instead of at the component system level)
n Discovery and application of convergence protocols
† System of Systems Engineering and Process Synchronization, Jo Ann Lane, University of Southern California, Center for Software Engineering
353
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ SoS Compared to Traditional Development (2)n Scope and performance
n Added “ilities” such as flexibility, adaptability, composability
n Human as part of the SoS
n Organizational scope defined at runtime instead of at system development time
n Dynamic reconfiguration of architecture as needs change
n Maintenance and evolutionn Component systems separately acquired and continue to be managed as
independent systems
† System of Systems Engineering and Process Synchronization, Jo Ann Lane, University of Southern California, Center for Software Engineering
354
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016
+ System of Systems Engineering and Agilen SoSE teams are evolving traditional methods to support the development
and on–going evolution of SoSsn Early feasibility assessments and tradeoff analyses
n Knowing when to engineer and when not to engineer
n In general, leave the constituent systems engineering to the constituent system engineers
n Constituent system monitoring to ensure that constituent system changes are not adversely impacting the SoS
n Combining agile with traditional approaches
n Increases concurrency
n Reduces risk
n Compresses schedules
† System of Systems Engineering and Process Synchronization, Jo Ann Lane, University of Southern California, Center for Software Engineering
355
Dark Side
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 - 2016