adc-bsc east 2013 keynote: reading the tea leaves: predicting a project’s future

36
WK2 Keynote 11/13/2013 12:45 PM "Reading the Tea Leaves: Predicting a Project's Future" Presented by: Payson Hall Catalysis Group, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 8882688770 9042780524 [email protected] www.sqe.com

Post on 18-Oct-2014

177 views

Category:

Technology


0 download

DESCRIPTION

Is a project’s fate preordained? Does a project’s past suggest its likely future? Can anything be done to influence that future when the current signs aren’t promising? Payson Hall has participated in and reviewed many projects during his thirty-year career in software development. Without claiming mystical or magical powers, Payson shares problem symptoms he has observed and discusses strategies for isolating and correcting them. He helps you learn to identify “problem seeds” that can grow into larger issues over time. For example, when a task exceeds its planned duration, questions that might help identify the cause include: Are the people assigned to the task working on something else? Has the schedule shifted the task into holidays, training, or vacations? Are tasks blocked awaiting information, materials, or approvals? Was the work clearly defined to begin with? Payson introduces a diagnostic framework that helps you determine the next steps in an investigation to identify root causes of project issues you observe and to formulate possible remedies.

TRANSCRIPT

Page 1: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

 

WK2 Keynote 11/13/2013 12:45 PM 

      

"Reading the Tea Leaves: Predicting a Project's Future"

   

Presented by:

Payson Hall Catalysis Group, Inc.

       

Brought to you by:  

  

340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com

Page 2: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Payson Hall Catalysis Group, Inc.

A systems engineer and project management consultant, Payson Hall is a founding member of Catalysis Group, Inc. Formally trained as a software engineer and computer scientist, Payson has performed and consulted on a variety of hardware and software systems integration projects in both the public and private sectors throughout North America and Europe during his thirty-year professional career. He has been a writer and featured speaker on topics of systems integration, project management, and risk management. Payson's rare combination of IT project management experience and communication skills has made him a valued member of many project review and project oversight teams.

Page 3: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Reading(the(Tea(Leaves:((Predic3ng(a(Project’s(Future(

Payson(Hall(Catalysis(Group,(Inc.(

Predic3on:(Supernatural(or(Science?(

2(Tea(Leaves(Catalysisgroup.com(

Page 4: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Inspiring(Dreams(

3(Tea(Leaves(Catalysisgroup.com(

Learned(from(Dreams(1...(

4(Tea(Leaves(Catalysisgroup.com(

Page 5: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Learned(from(Dreams(2...(

5(Tea(Leaves(Catalysisgroup.com(

Predic3ng(the(Future(Isn’t(Magic(

Involves:(– Recognizing(paNerns(– Grasping(underlying(rules(– Seeking(confirming(or(disconfirming(evidence(

6(Tea(Leaves(Catalysisgroup.com(

Page 6: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

What(Inspired(This(Talk?(

7(Tea(Leaves(Catalysisgroup.com(

2(Years(of(Status(in(2(Days(

8(Tea(Leaves(Catalysisgroup.com(

Page 7: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Pa3ent(Vitals(Over(6(Months(

Day$ Weight$ Systolic$ Diastolic$

1( 185( 140( 95(

30( 191( 140( 103(

60( 192( 139( 122(

90( 190( 153( 118(

120( 188( 151( 103(

150( 187( 159( 112(

180( 187( 165( 109(9(Tea(Leaves(Catalysisgroup.com(

Trend(Data(Helpful((If(Available)(

(^((((

(50((

(100((

(150((

(200((

(250((

1( 31( 61( 91( 121( 151(

Weight( Systolic( Diastolic(

10(Tea(Leaves(Catalysisgroup.com(

Page 8: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

18(Month(Project((Month(1)(

S( F(

Welcome(aboard!((Looking(forward(to(working(with(you(all.((We(will(be(procuring(a(new(facility(for(the(project(team(while(we(select(a(product(and(vendor.((

Hope(to(have(the(RFP(out(by(the(end(of(next(month.((Keep(your(fingers(crossed!((

11(Tea(Leaves(Catalysisgroup.com(

18(Month(Project((Month(2)(

S( F(

Great(work(team!((We’ve(narrowed(facility(search(to(3(loca3ons(&(everyone(enjoyed(the(tours.(Final(selec3on(this(week.((Hope(to(start(moving(next(month.(RFP(release(delayed(two(weeks(by(legal(review.((We(will(be(just(seNling(into(our(new(digs(when(RFP(goes(out.((Boxes(for(packing(will(be(

distributed(next(week.((Potluck(when(RFP(released.((

12(Tea(Leaves(Catalysisgroup.com(

Page 9: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

18(Month(Project((Month(3)(

S( F(

Final(facility(selec3on(complete,(but(phones(and(such(won’t(be(hooked(up(for(another(two(weeks.((Move(date(to(be(determined,(but(don’t(unpack(anything(you(don’t(have(to.((RFP(finally(back(from(legal(and(should(go(out(in(a(week(or(so.((Looking(

forward(to(the(potluck.((We(are(a(behind(schedule,(but(things(should(seNle(down(aeer(the(move.(

13(Tea(Leaves(Catalysisgroup.com(

18(Month(Project((Month(4)(

S( F(

RFP(released!((Potluck(was(a(hit!(Mary(promises(to(post(the(recipe(for(her(tuna(casserole(in(the(break(room(when(we(get(moved.((Moving(to(new(facility(

next(month.(

14(Tea(Leaves(Catalysisgroup.com(

Page 10: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

18(Month(Project((Month(5)(

S( F(

SeNling(in(to(our(beau3ful(new(facility.((Team(s3ll(unpacking(boxes(and(sefng(up(furniture.((Tech(

support(promises(that(the(LAN(and(printers(should(be(set(up(next(week.((More(vendor(ques3ons(than(we(an3cipated(with(the(RFP,(working(to(get(an(addendum(out(this(month(so(that(we(can(get(

vendor(final(proposals(next(month.(

15(Tea(Leaves(Catalysisgroup.com(

18(Month(Project((Month(6)(

S( F(

Legal(had(a(few(issues(with(the(addendum,(but(it(should(go(out(next(week.((All(systems(up(and(running.((I(think(it’s(3me(for(an(office(warming(

potluck!((There(is(definitely(light(at(the(end(of(the(tunnel.(Thanks(for(all(your(hard(work(with(the(

addendum(and(gefng(the(office(seNled.(

16(Tea(Leaves(Catalysisgroup.com(

Page 11: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

18(Month(Project((Month(7)(

S( F(

Proposals(are(due(next(month.(Aeer(a(brief(transi3on(period,(I(will(be(turning(project(

management(du3es(over(to(Mary(Smith.(It(has(been(a(pleasure(working(with(you(all(and(I(wish(you(

best(of(luck(on(the(project.((

17(Tea(Leaves(Catalysisgroup.com(

18(Month(Project((Month(8)(

S( F(

I(want(to(the(thank(the(team(for(making(me(feel(so(welcome.((Vendor(selec3on(is(complete(and(legal(is(finalizing(a(contract(so(that(vendor(staff(can(start(work.((We(hope(the(vendor(will(be(on(site(next(

month...((

18(Tea(Leaves(Catalysisgroup.com(

Page 12: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

18(Month(Project((Month(9)(

S( F(

Legal(promises(there(are(just(a(few(more(contract(details(and(then(the(vendor(should(be(able(to(start(

aeer(the(holidays...((

19(Tea(Leaves(Catalysisgroup.com(

Project(Basics(

Project(^(A(temporary(effort(undertaken(to(accomplish(a(defined(goal((scope)(within(specified(resource(and(schedule(constraints(

Schedule( Resources(• (People(• (Funding(• (Equipment(

Scope(• (Breadth(• (Depth(

((• (Quality(• (Quan3ty(

20(Tea(Leaves(Catalysisgroup.com(

Page 13: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project(Management(

•  Defini3on(– What(do(you(want?((When?(((What(resources((people(&(funds)(will(you(commit(to(get(it(done?(

•  Planning(– What(will(happen?((When?((Who(will(do(it?((What(will(it(cost?((What(are(the(risks?(

•  Execu3on(– What(happened?(When?((What(resources(did(it(require?((How(does(that(compare(to(expecta3ons?((Are(we(on(track(to(make(it?((Should(we(con3nue?(

21(Tea(Leaves(Catalysisgroup.com(

Early(Status...(

•  “Schedule(is(high(level(and(very(aggressive”(•  “Concerned(whether(sufficiently(skilled(and(knowledgeable(staff(available(to(support(project”(

Lee(unresolved(what(kinds(of(problems(might(you(expect(or(watch(for?(

22(Tea(Leaves(Catalysisgroup.com(

Page 14: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

What(Happened...(•  Schedule(slip(•  Quality(issues(•  Reduced(3me(for(planning(

Insufficient(Planning( Surprises(

Lost(Produc3vity(

Quality(Issues(

Unexpected(work(

Schedule(Problems( ?

23(Tea(Leaves(Catalysisgroup.com(

Key(Ideas(•  It’s(hard(to(recover(from(a(sloppy(or(late(start(•  Plans(don’t(guarantee(success,(they(set(baseline(expecta3ons(

•  Comparing(reality(to(plans(helps(ID(variance(•  Progress(WON’T(match(plan(–(material(variances(should(be(explored(and(explained(

24(Tea(Leaves(Catalysisgroup.com(

Page 15: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Looking(for(PaNerns(Mul3^dimensional(problem(– Time(– Effort/Cost(– Work(– Trends(of(these(factors(OVER(TIME(

"There(is(always(an(easy(solu3on(to(every(human(problem—neat,(

plausible,(and(wrong.”((^(H.L.(Mencken(

25(Tea(Leaves(Catalysisgroup.com(

Run(Chart((Time(Series)(

0(

10(

20(

30(

40(

50(

60(

Jan( Feb( Mar( Apr( May( Jun( Jul( Aug( Sep(

Down2me$(Hrs)$

26(Tea(Leaves(Catalysisgroup.com(

Page 16: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Control(Chart^Run(Chart(w/Limits(

0(

10(

20(

30(

40(

50(

60(

Jan( Feb( Mar( Apr( May( Jun( Jul( Aug( Sep(

Down3me((Hrs)( UCL( LCL(

27(Tea(Leaves(Catalysisgroup.com(

Control(Chart(Basics(

0(

20(

40(

60(

Jan( Feb( Mar( Apr( May( Jun( Jul( Aug( Sep(

Down3me((Hrs)( UCL( LCL(

28(

!  Upper(Control(Limit((UCL)(–(Value(going(above(this(limit(triggers(ac3on(

!  Lower(Control(Limit((LCL)(–(Value(going(below(this(limit(triggers(ac3on(

!  Agreeing(to(limits(up(front(oeen(makes(ac3on(less(poli3cally(charged(

Tea(Leaves(Catalysisgroup.com(

Page 17: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Slip(Chart((Schedule(Trends)(

120( 120(

150( 150(165( 165(

0(

20(

40(

60(

80(

100(

120(

140(

160(

180(

0( 20( 40( 60( 80( 100( 120( 140( 160(

Est.$Comple2on$Day$

29(

Project$Day$Es2mate$Published$Tea(Leaves(Catalysisgroup.com(

Quiz(–(What’s(Happening?(

120(150(

180(210(

240(270(

0(

50(

100(

150(

200(

250(

300(

0( 20( 40( 60( 80( 100( 120( 140( 160(

Est.$Comple2on$Day$

30(

Project$Day$Es2mate$Published$Tea(Leaves(Catalysisgroup.com(

Page 18: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Variance(isn’t(Good(or(Bad...(

Variance(is(informa(on)The(CAUSES(of(variance(are(interes3ng...(

31(Tea(Leaves(Catalysisgroup.com(

Using(Variance(to(Predict(Trouble(

Ques2on$ Meaning$

Are(expecta3ons((plans)(clear?( ! Are(plans(credible?( ! Is(progress(unfolding(more(or(less(according(to(plan?((If(not,(why(not?( ?(Is(the(project(iden3fying(and(dealing(with(sources(of(variance?( ?(

32(Tea(Leaves(Catalysisgroup.com(

Page 19: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

The(Project(Detec3ve(

33(Tea(Leaves(Catalysisgroup.com(

Diagnos3c(Tool(Organiza3on(

Symptom$ Diagnos2c$Ques2ons$

Possible$Causes$

Possible$Remedies$

Schedule(((((((^(((((((^(

Scope(((((((^(((((((^(

Resources(((((((^(((((((^(

Other(((((((^(((((((^(

34(Tea(Leaves(Catalysisgroup.com(

Page 20: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Symptom:(Tasks(exceed(planned(dura3on(

Diagnos3c(Ques3ons:(– Are(people(assigned(doing(something(else?(

– Has(schedule(shieed(&(pushed(tasks(into(holidays,(training(or(vaca3ons?(

– Has(resource(consump3on(remained(consistent(with(original(es3mates?(

– Are(tasks(blocked(awai3ng(material,(informa3on(or(approvals?(

35(Tea(Leaves(Catalysisgroup.com(

Possible(Causes(

•  Tasks(may(be(consuming(more(resources((than(originally(es3mated((

•  People(may(not(be(available(as(originally(an3cipated(when(dura3ons(were(calculated(

•  Inefficiency(introduced(when(team(members(work(on(mul3ple(tasks(concurrently(

•  Tasks(may(not(be(broken(down(sufficiently(to(highlight(dependencies(on(informa3on(or(approvals(

•  Task(dependencies(in(the(project(plan(may(be(incorrect(

36(Tea(Leaves(Catalysisgroup.com(

Page 21: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Possible(Remedies(•  Compare(actual(resource(consump3on(with(es3mates(–(assure(sufficient(resources(available,(revise(es3mates(if(needed(

•  Review/correct(task(dependencies((•  Look(for(trends((kind(of(task,(doer)(to(focus(further(analysis(

•  Review/refine(produc3vity(assump3ons(•  Review(es3mates(with(those(assigned(to(future(tasks(and(refine/revise((

37(Tea(Leaves(Catalysisgroup.com(

Summary(•  Predic3on(can(be(scien3fic((vs.(supernatural)((•  Project(status(is(mul3^dimensional(–  Schedule(–  Scope(–  Resources(

•  Look(for(trend(informa3on(•  Progress(against(credible(plans(helps(ID(variance(•  Causes(of(variance(lead(to(issues/problems(•  Check(out(the(Job(Aid(

38(Tea(Leaves(Catalysisgroup.com(

Page 22: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Q(&(A((Stump(the(Chump)(

Schedule( Resources(• (People(• (Funding(• (Equipment(

Scope(• (Breadth(• (Depth(

((• (Quality(• (Quan3ty(

39(Tea(Leaves(Catalysisgroup.com(

Thank(You(The(Job(Aid:(

Symptoms,$Diagnosis$&$Treatment$of$$Common$Project$Management$Problems$

(Should(be(among(the(conference(proceedings(Request(an(electronic(copy(from:(

Payson(Hall([email protected](

(916)(929^3629(TwiNer:(@paysonhall(

40(Tea(Leaves(Catalysisgroup.com(

Page 23: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 1

Symptoms, Diagnosis & Treatment of Symptoms, Diagnosis & Treatment of Common Project Management ProblemsCommon Project Management Problems

Introduction Even well managed projects encounter challenges. People closest to a project often have insights regarding project problems; however, their proximity to the consequences of problems can sometimes lead to lost perspective and overlooked causes. This reference is designed to support project management by supporting objective analysis and remediation of project problems. The table is intended as a supplement to assist project managers with problem diagnosis and treatment, not a substitute for sound project management practices. Project goals exist in three dimensions:

• Scope – Deliverables, quality goals, process constraints, overall project size and boundaries • Schedule – Delivery target dates, milestone dates, expected task completion dates • Resources – People, equipment, facilities, materials, and funding

The dimensions are inter-related and problems in one dimension tend to cause problems in one or both of the others. For example, if a key resource becomes unavailable to a project, it will likely have an impact on the delivery schedule and/or functionality and quality of work products until the problem is resolved. Because projects are complex systems of cause and effect, diagnosing root causes can be challenging. The “symptom” that is observed may be the source of the problem, a contributing factor, or the middle of a chain of causes and effects. Just as a physician must explore alternative potential causes from a patient’s presenting symptoms, this reference provides the project manager with questions to assist in the diagnosis of other causes and contributing factors. The reference presents information on troubleshooting project problems. It is roughly organized into four categories best matching the problem or “symptom” of a problem that is observed:

• Scope: Quality, performance, process, or boundary related symptoms • Schedule: Schedule slippage symptoms • Resource: Personnel performance, human resource consumption, budget issues • Other: A catch-all category for classic symptoms and issues that don’t neatly fit into the categories above

Symptoms are listed under each heading and matched with Diagnostics, questions that may be helpful to clarify or focus thinking about the symptoms. Information obtained using diagnostic questions will assist in identification of Possible Causes and selection among related Possible Remedies. While the reference is not exhaustive, it is hoped that it can aid with the rapid identification and resolution of project issues. As you use the table, it is important to keep in mind that symptoms may have multiple causes or potential causes. Tasks exceeding their schedule estimates may be a result of poor estimation, poor performance, lack of skills, lack of tools, increasing scope of the task, a key contributor getting the flu, or a combination of some or all of these causes.

Page 24: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 2

Diagnostics Possible Causes Possible Remedies

Clarifying/Focusing Questions Potential Contributing Factors Remedial Action Scope/Quality Symptoms Symptom: Project Scope appears to be expanding, but the Charter has not changed. Have users or sponsors requested functionality, performance, or quality standards beyond what was originally chartered? Is the change control process been used to document increases in project scope and assure that changes are conscious decisions to modify the charter? Have risk items occurred that increased project scope? Is the development staff receiving contradictory guidance from client staff? Have personnel changes occurred in key client or project staff? Have quality problems arisen that have resulted in increased needs for testing or rework? Have tasks from another project or undertaking been moved into this project? What tasks were added and why? Was there insufficient planning, was there insufficient detail, were there really unexpected things that arose?

Inadequate change control – Changes are being accepted without the evaluation and explicit approval of the sponsor to expand project scope. Failure to document scope and process assumptions as part of the charter or monitor those assumptions for change. Initially chartered scope was insufficient for the project’s needs (quality or performance standards were initially inadequate for actual production use for example). Staff changes may have caused a loss of organizational memory regarding the meaning of some requirements or the location of project boundaries. The impact of deferred quality issues is usually amplified the longer they are deferred. Team members may not be clear on project boundaries and expanding them to include work not originally identified as part of the project or work from other projects or operational activity.

Revisit change management process and assure that only sponsor-authorized changes to scope are allowed to modify the charter. Revisit the planning process to assure that tasks are broken to an appropriate level of detail and estimated by someone competent to do the work and that estimates and task definitions are reviewed prior to finalizing plans. Assure that assumptions underlying plans are documented and that invalid assumptions and their impact are communicated to the sponsor. Assure that mandated changes to the development environment or business processes are documented via change management to assure that sponsors understand and accept the impact of changes. Assure that the team is clear about the project boundary and understands the process to recommend adding work to the project that was initially “out of bounds” Review earlier quality metrics and determine how quality or performance problems might have been detected and remedied earlier. Look for process improvements to assure that quality problems are identified and dealt with promptly. Emphasize the importance of sticking to requirements and requirements tracking in project meetings. Use requirements tracking procedures to link requirements to test cases to user acceptance. Review requirements with lead developers often to re-affirm that only the documented requirements are being developed.

Page 25: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 3

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Explain “scope creep” and how to prevent it to the project team. Revisit risk planning and determine whether anticipated risk events have occurred or if new risk events have emerged that are affecting scope. Add lessons learned to the lessons log and avoid this in the future on other projects. Identify another project designed to hold “additional” tasks. Build a “future enhancements” list to capture good ideas and new requirements that are not part of this project so that they can be included in a subsequent release.

Symptom: Test results report a large number of errors and/or the number of errors in new work or re-work does not seem to decrease.

Are the sources of errors being identified? Are the sources of error changing over time? Are the types of errors consistent? Is the severity of the error comparable? Are the errors being detected at the earliest possible opportunity in the process? Is there schedule performance pressure that is encouraging a rush through design and/or coding? Has the quality of testing improved during the life of the project?

Mechanisms for early detection of errors may not be in place or may not be implemented effectively. Feedback from the testing and quality assurance processes might not be causing necessary changes in current process. If the source or type of error detected is changing over time, it may be that testing is being more effective at finding these errors or that process changes have shifted quality problems to new areas. Some staff may need additional training with processes, tools, the application domain, or basic development skills to improve work

Look for ways to detect errors sooner in the development process. Look for trends in error sources and in types of errors to identify high leverage interventions with processes or training. Determine how the project processes, or staff capabilities can be changed to improve the quality of products delivered to testing. Revisit project plans and verify that early error detection mechanisms in the plan (code reviews, design walkthroughs, etc.) are being conducted effectively. Monitor team morale and be alert for the natural friction that can arise between development and testing. Remind the team that development of quality work products is a team goal and that processes that identify faults early support those efforts.

Page 26: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 4

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Are there skill issues with the staff responsible for creating the work products being tested? Are there morale issues with the staff responsible for creating the work products being tested? What testing behavior is being encouraged? Finding large quantities of errors? Finding high severity errors? Testing products quickly?

product quality and worker efficiency. Morale issues among developers or testers may be causing performance or communication issues that result in either poorer quality or greater error reporting. Encouraging/rewarding testers based upon the number of faults they find may have the unintended consequence of causing multiple occurrences of the same error type to be written up as different errors, or complex errors to be written up as multiple errors.

Beware the assumptions “More errors found = worse quality” “More errors found = better testing” “Few errors found = better quality” “Few errors found = worse testing”

Symptom: Product passes all tests but the client is dissatisfied Have the client requirements changed since the inception of the project? Has the project context changed since the inception of the project (regulations, strategic goals, business environment)? Were the initial requirements clear and complete? Do test plans include demonstration of meeting requirements? Have there been key user personnel changes since the start of the project? Have there been changes in sponsorship since the start of the project?

Requirements were not captured in sufficient detail to truly define the users needs for the product. Tests that sufficiently evaluate all aspects of the requirements were not developed. Requirements were not effectively managed to adapt to changes in the project business context. Changes in key client personnel may have resulted in different user expectations. There may be insufficient ongoing involvement of users in the development process. The system may meet identified

Re-evaluate the way in which the requirements were documented. Assure that the client approves the requirements document. Affirm that requirements meet the client’s expectation. Revisit and review requirements with the client. Re-evaluate the test plan to assure that the individual test cases effectively cover identified requirements New requirements may be integrated into the current project via change management or documented for the next release of the software. Review the process for monitoring the user business environment for new/changed requirements. Look for ways to improve this process. Work closely with the users to understand and capture recent or expected changes in business processes.

Page 27: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 5

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Has user staff been involved in the development or review of designs and test criteria? How could user feedback have been solicited earlier in the development process to detect user satisfaction issues sooner? Is usability an issue? Is performance an issue? Is user training an issue? Is available documentation sufficient?

requirements but be difficult to use or have performance issues. Have users received sufficient effective training and documentation about the use of the system to enable them to utilize it effectively?

Look for ways to continually re-validate that the system being built is consistent with the written requirements and needs of system users. Gather data from the users by observing them interact with the system for an extended period of time and encourage them to comment about their frustration. Look for usability issues and places where expectations may not be consistent with requirements. Look also for opportunities to improve training or documentation. Begin building an enhancement list for the next release of the product so that users feel that concerns have been heard and to assure that good ideas are captured.

Symptom: Agreed upon processes and rules are not being consistently followed Are the rules and process requirements clear? Are the rules and process requirements necessary? Are the rationale for processes and rules clearly stated? Are project staff aware of rules and process constraints? Do project staff have sufficient skill and experience to effectively comply with rules and process constraints? Has the organization required consistent compliance with rules and process constraints? Does the team have sufficient time and

If rules or process requirements are not clear, they cannot be complied with. If people don’t understand why rules and process constraints are in place, they will not comply with them consistently. If an organization does not appear to be serious about enforcing rules and process constraints, teams will tend to ignore rules and process requirements that appear to get in the way of getting the project completed. If there is no apparent consequence for breaking rules or violating process requirements, people tend to ignore them.

Rules and process requirements should be clearly written (examples improve the effectiveness of documented rules and processes) Written rules and process requirements should include a brief description of why the rule or process is in place. The team should have ready access to documentation explaining rules and process requirements. The team should receive sufficient training and support to facilitate compliance. Periodically, rules and processes should be revisited with the team to look for process improvements and provide the team a chance to offer input to the rules. Exceptions to rules and processes should be conscious decisions made by the project manager. The team must receive an unambiguous message that

Page 28: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 6

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

resources to comply with rules and process requirements? Are the same people continuing to ignore or subvert rules and processes after repeated intervention?

If the team does not have the tools, time, or skills to comply with rules, they will tend to ignore them. If rules and process requirements are clearly communicated and understood, and team members with the skills and tools to comply with them refuse to comply after repeated intervention, this is sabotage or insubordination

compliance with rules and processes is expected, and that if this causes changes in estimates or the project plan, the changes should be communicated to the project manager. When new personnel are added to the team, rule and process information should be explicitly shared with them as part of orientation. Sabotage or insubordination are personnel issues that should be promptly handled as such. Left untreated, they undermine morale, discipline, and the cohesiveness of the project team.

Resource Symptoms Symptom: Tasks are tending to consume more human resources than estimated in the project plans. Has the scope of tasks changed since they were estimated? Do the people working on the tasks have a comparable level of skill to the ones envisioned for the tasks when it was estimated? Are people working on tasks devoted full time to the project or are they being diluted with other projects or operational responsibilities? Have initial assumptions about the work or the work environment proven to be incorrect? Have process errors or quality problems occurred that required unanticipated rework? Is resource consumption being accurately tracked at a task level?

Initial estimates were incorrect. Insufficient expert input into the identification, definition, and estimation of project tasks. Assumptions underlying task estimates were incorrect. Quality problems are being introduced earlier in the project, causing later tasks to exceed their estimates. When quality problems are not dealt with promptly, subsequent task resource needs tend to grow as deferred problems must be corrected. New tools and processes introduced after estimation sometimes invalidate earlier estimates. Task resource consumption may not be reported accurately.

Estimates should be created by individuals competent to do the work. Estimates should be written, and should specify the assumptions upon which they are based, including the team size, skill levels, processes to be used, performance environment and scope of the task. When tasks materially vary from their estimates (over or under), particularly if trends are noted among types of tasks or tasks estimated by the same estimator, or tasks performed by the same doer; estimates should be reviewed to understand whether the scope of the work has changed, the anticipated resources have changed, the assumptions underlying the estimate have changed, or the estimation process needs improvement. Review the project charter to determine appropriate optimization response. Can additional resources be allocated to problem tasks to recover schedule? Can the scope of tasks be modified to better fit the budget? Revisit similar estimates for tasks not yet completed and refine where necessary.

Page 29: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 7

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Are any tasks being reported as completing with less resource than initially allocated? Have new tools or processes been introduced that effect task doer’s efficiency either short term (climbing the learning curve or stabilizing a new development environment) or longer term (utilizing more complex tools)? Are task completion criteria clear?

If task completion criteria are not clear, task doers may be doing more than required to complete tasks.

Use the change management process to inform sponsors, users and team members of any material changes to schedule, scope or resource targets. Monitor task performance after the introduction of changes to tools and processes to identify whether, after a reasonable learning curve, estimates need to be modified up or down for work that has not been completed and will utilize the new tools or methods. Monitor the impact of changes in tools and processes to assure that the expected cost benefit of the change is achieved. Consider backing out changes that prove to be ill advised or immature. Status reporting of task resource consumption by team member should occur on a regular basis so that trends and issues can be promptly identified and handled. Clearly defined completion criteria should be specified for all tasks. This is the foundation of good estimation and also provides task doers with a discrete target so that they can stop work as soon as the completion criteria are met.

Symptom: Unanticipated tasks (tasks not included in the project plan) are consuming substantial project resources Has the scope of the project changed? Are changes being approved using the change management process? Have risk events occurred that caused unanticipated tasks? Have external forces increased the amount of work (new regulations, required changes to work environment such as new tools or modified processes)?

Plans omitted tasks that were necessary to achieving the chartered scope. Plans may not have been developed at level of detail sufficient to identify all required work. Changes in the project context may be invalidating estimates The team may be expanding the scope of the project with un-

Project plans should be developed by people competent to do the work, and developed at a level of detail sufficient to capture the work to be done. Project plans and estimates should be revisited periodically to look for trends in missed tasks or highly variant estimates. Risk events that occur should be documented and the change management process used to assure that sponsors and users are willing to incur the additional cost, schedule and scope impact of the risk event. Change management processes should be used to inform

Page 30: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 8

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Has the team discovered work that was not originally anticipated and included in the plan? Are the un-anticipated tasks work that must occur and was omitted from the original plans or optional work that is being included?

anticipated tasks. Risk events may have occurred whose consequences are being handled without conscious acknowledgment

sponsors of refinement to estimates and provide an opportunity to revisit the schedule and scope of the project. Assumptions about project context should be reviewed periodically to determine whether they impact project scope. Assure that the team is clear about the project boundary and understands the process to recommend adding work to the project that was initially “out of bounds” Risks that have occurred become project issues that may result in change management activities Risk events that have occurred and were not previously identified as potential risks may indicated deficiencies with the current level of risk planning.

Symptom: Tasks monetary costs are exceeding planning estimates Has the scope of tasks changed since they were estimated? Have initial assumptions about the work or the work environment proven to be incorrect? Have process errors or quality problems occurred that required unanticipated rework? Is resource consumption being accurately tracked at a task level? Have fees for services increased? Have fees for materials increased beyond what was anticipated/budgeted? Did budget estimates include an

Initial estimates were incorrect. Insufficient expert input into the identification, definition, and estimation of project tasks. Assumptions underlying task estimates were incorrect. Quality problems being introduced earlier in the project are causing later tasks to exceed their estimates. When quality problems are not dealt with promptly, subsequent task resource needs tend to grow as deferred problems must be corrected. Procuring products, services and equipment at the last minute frequently drives up costs.

Estimates should be created by individuals competent to do the work. Estimates should be written, and should specify the assumptions upon which they are based. When tasks materially vary from their estimates (over or under), particularly if trends are noted among types of tasks or tasks estimated by the same estimator; estimates should be reviewed to understand whether the scope of the work has changed, anticipated costs have changed, the assumptions underlying the estimate have changed, or the estimation process needs improvement. Review the project charter to determine appropriate optimization response. Can additional resources be allocated to problem tasks to recover schedule? Can the scope of tasks be modified to better fit the budget? Revisit similar estimates for tasks not yet completed and refine where necessary.

Page 31: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 9

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

inflation factor? Are materials and equipment that have been procured being used? Are overhead costs (equipment rentals or facilities charges) increasing because of schedule delays? Are products, services and travel being procured in sufficient volume and with sufficient lead time to allow discounts?

Use the change management process to inform sponsors, users and team members of any material changes to schedule, scope or resource targets. Status reporting of task budget consumption should occur on a regular basis so that trends and issues can be promptly identified and handled. Consider negotiating discounts for products and services procured in bulk or with substantial lead times.

Schedule Symptoms Symptom: Tasks are exceeding their planned duration Are the resources assigned to the task working on other project tasks or assigned to other project or operational duties? Has a schedule shift caused tasks to straddle holidays, training, or vacation of team members allocated to the task? Has the resource consumption of the task remained consistent with original estimates? Are tasks being held up waiting for approvals or sign offs? Are tasks being held up waiting for materials, equipment or information? Are tasks starting when they were scheduled?

The task may be consuming more resources than originally estimated. Tasks that consume more resources than originally estimated frequently also exceed their schedule estimates. Project resources may not have the availability that was originally anticipated when durations were calculated. Inefficiency is introduced when team members are required to work on multiple tasks concurrently. Tasks may not be broken down to sufficient level of detail to highlight dependencies on information or approvals. Task dependencies in the project plan may be incorrect.

Compare actual resource consumption of tasks with observed duration. Generally, dealing with increased resource requirements must occur before addressing corresponding schedule impacts. Review task dependencies for accuracy. Review task definitions and break down tasks to highlight external dependencies and approvals. Look for trends (kind of task, task doer) to focus further analysis. Review productivity estimates for project resources. Generally it is ill advised to expect more than 30 hours per week of productive project time from full time resources. This must be further adjusted to account for holidays, vacation, training, other project work and any other substantial resource commitments. Task duration predictions are normally derived from resource estimates and assumptions about the availability of team members to do the work. When duration estimates are

Page 32: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 10

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

What are the assumptions about team member availability during a typical work week? How were duration estimates created? Is there a disruption in the work environment that is causing interruptions in task work?

Project resource productivity may have been over-estimated. Duration estimates may not be based upon resource estimates. Interruptions in task work introduce two factors that increase schedule, the overhead of stopping and starting the task, as well as the off-task time of the interruption. Some unexpected factor may be affecting durations or performance.

inaccurate, working backward through availability assumptions, skill assumptions, and resource estimates is usually instructive. Work to minimize interruptions in task work. This can be accomplished by trying to eliminate unnecessary meetings, providing productive work space, assuring that staff have reliable and efficient equipment (computers, copy machines, networks, printers, etc.) Discuss schedule variances with the team to discover root causes.

Symptom: Tasks are not performed in the specified order Is the desired order of task performance clear? Are team members encouraged to complete one task before starting another unless blocked? Are task dependencies correct? Are team members informing their team leaders or project managers when they are blocked on a task?

The assignment of tasks is not being clearly stated, including the order in which they are to be performed. Team members may work on aspects of the project they like first. Task dependencies may not be correct or may be incomplete.

Review project schedule during the regularly scheduled project meetings. Review project task assignments with each person. Pass out task assignments in a tangible form, such as emails or a form. Link tasks assigned to their line-location in the project schedule. Print an oversized copy of the project schedule and display in a common area. Make sure that task owners understand their responsibility to perform tasks in the prescribed order.

Other Symptoms

Symptom: Stakeholders are blocking tasks (not providing timely feedback on reviews or holding up document approvals) Does the communication plan clearly identify stakeholder roles? Do tasks exist in the project plan for stakeholder review and sign off?

Stakeholders may have concerns about the work product or document they have been reviewing. Stakeholders may be unclear whether or not they have the authority to

The communication plan should identify which stakeholders have project responsibility and authority to approve work products. The impact of stakeholder delays should be communicated to stakeholders to give them a chance to provide feedback and

Page 33: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 11

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Have stakeholders been contacted to understand what concerns are holding up their tasks? Has a change order been filed to reflect the schedule, scope and resource implications of the delay? Have turn around expectations been clearly defined and communicated to stakeholders in writing? Are the planning assumptions related to document approval and turnaround clearly documented? Is the authority for review or approval clearly assigned in the project charter or task descriptions?

approve a work product. Stakeholders may be unaware of the impact that their delays have on the project. Stakeholders may be unaware of the expectations surrounding their turn around.

take appropriate action. If delays continue, change orders should be filed to communicate the impact of delays to the sponsor.

Symptom: A team member is struggling to solve a technical problem or struggling to get tasks done in a time that is realistic How is the problem being approached? Does the problem resemble one with which the team has experience? Is the approach to solving the problem rational? Are multiple avenues of resolution being sought? Are appropriate internal and external experts being sought? What has been tried thus far?

The problem may be beyond the skill of the staff members trying to solve it. Problem solving training may be necessary. External expertise may be necessary.

Provide mentoring if the issue can be narrowly defined and targeted mentoring will help. Provide opportunity to take training classes. Switch personnel and obtain someone that has a background appropriate to this technical problem or task. Consider outside expertise to assist with specialized technology. Provide problem analysis and problem solving training to the staff.

Symptom: A team member is not providing the expected amount of time to the project. What activities are competing with the project for his or her attention?

The team member may have been redirected to other tasks by his or her

Talk with the person and determine why they have not been putting in the required number of hours.

Page 34: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 12

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

Have the average hours per week expected from the team member been documented as part of computing the duration of the team member’s tasks? Is the team member’s supervisor aware of the time commitment and willing to support the project? If personal issues are competing for the team member’s time, is a replacement person available for the project?

supervisor. The team member may have other activities competing for his or her attention. The team member may not be available to the project because of personal issues.

Talk to their supervisor or other project managers to resolve scheduling issues. Emphasize the importance of their role on the project and limit their hours to only the work that requires their special effort. Seek to adjust resource assignments to back fill if possible. If the resource issue cannot be resolved, submit a change request to the sponsor to document the change in assigned resources and the schedule or scope implications to the project.

Symptom: Status reports are not being read Does the communication plan specify the content and timing of status reports? Have the content, format and frequency of status reports been reviewed with the sponsor recently? Is the project still a priority with the sponsor? Has sponsorship changed?

Status reports are too long. Status report doesn’t contain the information that is really relevant to understanding the status of all the dimensions of the project. Status reports have been too optimistic or untrustworthy. The sponsor may be become complacent

Design status reports to be exception-based. Ensure the status report contains information that is relevant to all dimensions of the project. Ensure the status reports accurately reflect project status.

Symptom: Staff from another department is not being fully compliant or cooperative. Have expectations for external staff participation and cooperation been clearly communicated to the organization?

Staff from another department may have higher priorities as assigned by their manager. Expectations may not have been clearly communicated.

Discuss the issue with the individual. Identify priority conflicts or other issues contributing to the problem. Review the priority of the project in the context of the competing priorities. If discussion with the individual does not resolve the problem or if the individual does not believe that the priority of the project is higher than other assigned work, discuss the issue with the individual’s manager or supervisor. Verify priorities, amount of time and commitment to your project.

Page 35: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 13

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

If this is not sufficient, discuss the issue with your sponsor and seek assistance with the organizational issues or changes to the project schedule boundaries to account for resulting schedule impacts.

Symptom: Team morale is low Does the team have concerns about the project’s ability to meets its objectives? Is the team being pushed hard to make unrealistic goals? Is the team being rewarded for their work? Are personnel issues being dealt with promptly and fairly? Are team members putting in excessive overtime? Are any individuals having personal issues that should be dealt with? Is the team comfortable with the quality of the work products they are producing? Is there an absence of humor on the project?

Project team members may have concerns about the success of the project. Project teams frequently know before the project manager and sponsor when a project is in trouble. Team members may not believe they are being fairly compensated or recognized for their work. Individual team members may have personnel issues that need attention before they can contribute. Failure to handle discipline or performance issues with team members who are not doing their share may be damaging morale or cohesiveness of the group. Many team members take the quality of their work very seriously and morale may suffer if they believe that the quality of their work is inadequate. Poor morale can cause a spiral of poorer morale.

Speak individually with all members of the project team on a regular basis to provide opportunities to get their perspectives on the project, what is working well, what is not working well, their concerns, and their suggestions about how the project could be better accomplished. Involve team members in the discussion of decisions that affect the way that they work. Take the time to publicly recognize team members who are performing well. Deal promptly, fairly, and privately with performance or discipline issues. Look for ways to encourage humor that do not make individual team members the focus of jokes. Regularly emphasize that team members have permission to approach the project manager with suggestions, questions or concerns. If a project is not on track to meet its schedule, scope or resource goals, communicate that information to the sponsor and assure that the team is involved and aware of any changes to the goals or approach. If financial compensation is unavailable, look for ways to use choice assignments, training opportunities, or allocation of new equipment as rewards for the team. If personal problems are causing a disruption, request additional resources be brought to bear to relieve the troubled

Page 36: ADC-BSC EAST 2013 Keynote: Reading the Tea Leaves: Predicting a Project’s Future

Project Symptom & Diagnostic Aid V1.0 www.catalysisgroup.com Page 14

Diagnostics Possible Causes Possible Remedies Clarifying/Focusing Questions Potential Contributing Factors Remedial Action

team member of project responsibilities and deal with the problem.

Symptom: Team resists timely and accurate status reporting for their tasks Are status reporting requirements clear? What is the consequence of inaccurate or untimely status? Have expectations been clearly communicated? Has the rationale for gathering status been communicated? Is status information being used constructively?

Reporting requirements may not be clear. Team may not understand how the data is used. Team may not see value in data reported. There may be a perception or history that reported status information results in negative consequences. Team may resist status because “it takes too much time” or is seen as administrative rather than project work.

Assure status reporting requirements are clear. Assure the team understands how status information is used. Assure team understands the value of status information. Assure that all information gathered is used constructively. Assure that the format of status information gathered is focused and can be reported efficiently. Consider adding brief status reporting and team meeting tasks to the project plan to reinforce the perception that status reporting is a necessary project activity.