Slide 1Intelligent Testing, Improvement and Assurance

Susan WindsorPrincipal

Gerrard Consulting Limited+44 (0) 7974 808604susan@gerrardconsunting.comWeb: gerrardconsulting.com

Test AssuranceComing of Age?

Why are we here?

• When I first started talking about Test Assurance a few years ago, there were lots of blank looks but it’s become a hot topic now. Economic climate?

• I’ll share some of my experiences with you to illustrate potential benefits and pitfalls.

• Perhaps you’re interested in this as a career option, or maybe it could help your project or organisation?

Slide 2Intelligent Testing, Improvement and Assurance

Topics

• What is Test Assurance?

• Approach to Analysis

• Delivery Options

• Stakeholder Benefits

• Building your Development Plan

• Meeting your objectives

Slide 3Intelligent Testing, Improvement and Assurance

What is Test Assurance?Definition (1)

• Three main styles– Internal function with multiple project

scope– External function to provide

independent assurance on thoroughness, completeness and quality of testing

– External function to provide guidance on improved effectiveness (e.g. project in difficulty)

Slide 4Intelligent Testing, Improvement and Assurance

What is Test Assurance?Definition (2)

• Key characteristics– Insurance policy; if everything always

went well it wouldn’t be needed– Never includes delivery responsibility– Always relates to implementation– Demands direct route to senior

stakeholders

Slide 5Intelligent Testing, Improvement and Assurance

What is Test Assurance?Definition (3)• Terms of reference for recent

assignment– Troubled project – Can send you a copy

• Example Job Description – Use now to explore scope– Use to identify career development path– Need to be credible and make a difference

quickly to be accepted– Can send you a copy

Slide 6Intelligent Testing, Improvement and Assurance

What is Test Assurance?Activities (1 and 2 of 6)

• Establishing the requirement– Meet your stakeholder and understand their goals for you– Start of project/programme or at any point for a project in

trouble

• Analysis– How will you deliver your goals?– What is the current status?– What are the typical hot spots?

• Gaps, duplication, environments, contracts, staffing, etc.– How long have you got to demonstrate your value?

• Could be an hour, a day or a week – won’t be longer– We’ll explore a model for analysis shortly

Slide 7Intelligent Testing, Improvement and Assurance

What is Test Assurance?Activities (3 and 4 of 6)

• Building your Team– Sometimes you have to do this alone– Is there administration support available?– Do you need specialists to support you? If so ask for them

immediately– Write a short terms of reference to include your requirements,

plans and deliverables - get your stakeholder to sign it and circulate to everyone on the project

• Understanding your impact– Most people will not understand why you are there– Everyone you need to talk to is busy and may also be under

extreme stress– Some people will not tell you the whole truth and may mislead

you – There is always organisational politics – Human nature to believe “we’ll catch up”, “lets fix this in-

house”

Slide 8Intelligent Testing, Improvement and Assurance

What is Test Assurance?Activities (5 and 6 of 6)

• Building relationships– Have a clear mandate that you can talk to anyone you want to

that is related to the project/programme and senior people in the organisation

– Ensure everyone knows who your sponsor is– Accept you may not be popular but you need co-operation to

be successful

• Closure– You will be asked for, and expected to give, your opinion on

• The overall testing that’s been undertaken• Which issues remain outstanding and require action• How to make a better start next time

Slide 9Intelligent Testing, Improvement and Assurance

Approach to Analysis• Consulting Model

– An assessment method designed to support formal reviews of testing effectiveness

– But now has to be second nature, rapid analysis essential– It takes a long time to write a short letter!

• Information Gathering– As a minimum read project start-up documentation; test strategy; test

plans, etc.– Look for “out of scope” and “assumptions”.

• Supplier Review– What is their brief; how are they managed; what are their objectives– Use a supplier risk assessment checklist to seek out priority areas

• Typical Project Issues – Gaps and/or duplications in test activities– Environments; Test data– Skill levels; Staffing numbers– Levels of reporting – Role of Project Office Slide 10Intelligent Testing, Improvement and

Assurance

Basic Consulting Model

Slide 11Intelligence Testing, Improvement and Assurance

Identify Actions

Plan the transition

Implement the change

BaselineCurrent

Assignment

Culture

Methods

TechnologySkills

Organization

Measurement

Improve

"How do we get there ?"

"Do it !"

"Where are we now ?"

Where do we want to go next?

Expand dimensions to suit your task

Slide 12Intelligence Testing, Improvement and Assurance

How effective is the project organisation; are there any communications issues; is the meeting culture appropriate; are there activities not owned by anyone; does everyone understand their role

Is the technology appropriate; is it implemented and supported; what issues exist on the ground using it;

Assignment

Culture

Methods

TechnologySkills

Organisation

Measurement

Are there too many/few staff assigned; do they have the skills/competencies required; is the level of management support sufficient

Are the controls and measurements sufficient and appropriate; is the timing appropriate; how does information change as it migrates; is it accurate

Does everyone on the team have a common understanding and commitment to the project goals What could inhibit these being achieved

Is everyone using the same methods; do they understand the methods; are they appropriate for this project

Identify Realistic RecommendationsThis is a Must Have! • Relate findings to the context of the project; include high level

architecture diagrams and organisation charts and mark the “hot-spots”

• Group findings into a small number of areas or “hot-spots”

• Rank the findings in each area by priority; consider impact on key project goals (always include timescale); don’t forget financial implications

• For the top “3” in each area give specific solutions; ask for exactly what you need; give options where possible

• As a minimum you must be prepared to verify that the reporting and status provided is accurate

Slide 13Intelligent Testing, Improvement and Assurance

Remember your time line – 1 hour/day/week!

Delivery Options

• External Consultant or Internal Assurance

• Single project/programme or all current projects/programmes

• Initial baseline and/or review

• Recommended actions

• Ongoing assurance

Slide 14Intelligent Testing, Improvement and Assurance

Stakeholder Benefits

• Gains a project independent and balanced view of the status and quality of testing

• Builds confidence in the predictability of implementation

• Provides rapid, accurate and valuable information to base their decisions upon

• Provides a fast path to key issues for them to resolve

Slide 15Intelligent Testing, Improvement and Assurance

Building your Development Plan?Bit of homework!• How could you develop your skills and

competencies; can you identify specific ones to work on; where can you find support for this

• How can you get relevant experience; could you use the consulting model to review existing projects; can you brief senior managers on the value of the function; can you get in a specialist to work alongside and achieve skills transfer

• Finding a mentor or an expert; who do you know that can already do this; how can you learn from them

Slide 16Intelligent Testing, Improvement and Assurance

And Finally…….

• Ask me for any support materials you’d like– Job Description– Terms of Reference– Supplier Risk Assessment– Examples of suitable reporting formats

• Feel free to ask me any questions

• Contact me on susan@gerrardconsulting.com

Slide 17Intelligent Testing, Improvement and Assurance