evaluating the effectiveness of gov't digi comms p-njoku 15-05-2014

Evaluating the Effectiveness of Government Communications

UK Perspective

Paul NjokuGovernment Communication Service - Evaluation LeadCabinet Office, UK

15th May 2014

John WanamakerInventor of mass

retailing in the United States

"Half the money I spend on advertising is

wasted;the trouble is I don’t

know which half."

3

Why evaluate?

Context• Austere times

Need to make every £ or $ count View as an investment not an expense

• Media landscape & consumption patterns

Evidence of what works and what does not Optimise use of scarce resources

• The role of communications under scrutiny

How it supports achievement of policy outcomes Business planning & activity prioritisation

Government Communications Review:

Summary findings• Tactically strong, strategically weak

Half of departments had no communication strategy

• Poor links between policy and communications

• Major activity rarely evaluated

• Inconsistent and variable standards and approaches

• Little join up across departments

• Digital skills falling behind private sector

66

Barriers – stopping it happening

Insufficient time / resource / budget

Lack of SMART or unrealistic policy

objectives / targets

Difficulty accessing the right data / tools

Culture & entrenched behaviours

Gaps in evaluation standards & capability

Lack of integrated communication

strategy linked to policy objectives

7

Strategic Alignment

Start with the policy aim, then develop communications objectives that help deliver this.

Objectives should be measurable, focused on outcomes not outputs, and related to changing attitudes and/or behaviour.

Setting the right objectives is crucial

So what are we doing? Mandatory Evaluation Project (MEP) is a key part of Government Communication reform programme.

Aim: To create a ‘step change’ in organisations’ approach to evaluation of Government communication activities

Four Key deliverables:

1. Visual performance frameworks & dashboards

2. Evaluation training courses & Evaluation guide.

3. New set of evaluation capability standards.

4. ‘Centre of excellence’ & promote best-practice.

Evaluation stages – The Big IDIA

IdentifyThe scope of your project

DevelopYour evaluation plan

ImplementGather data to measure

performance

Analyse & reportPerformance against plan

1

2

3

4

Task 1: Define what you need to evaluate by asking:• What activity am I evaluating?• What do I know & what factors could affect the outcome?• What is my evaluation expected to achieve?Output: Summary of your proposed evaluation approach

Task 2: Define how you’ll measure success:• Set SMART objectives & define your target audience• Map out how activity will work• Set performance metrics (KPIs) & agreeing targets Output: Draft evaluation plan

Task 3: Identify and gather evaluation data:• Make most of existing data• Gather additional data (research, feedback & proxies) • Review data gaps (more budget ?) manage expectations Output: Completed evaluation plan

Task 4: Assess the success of your activity:• Analyse effectiveness & provide insights for future• Demonstrating efficiency and value for money • Demonstrating role of communications in supporting

policy objective delivery(outcome)Output: Final evaluation report

1. Strategic alignment – Ensure activity objectives are SMART, aligned and supports policy delivery.

2. Business impact – Always measure true business impact (outcome) rather than outputs or out-takes.

3. Big IDIA – Try to adopt the suggested frameworks, principles and follow the IDIA evaluation stages.

4. Continuous improvement – Ensure results drive appropriate actions and learnings inform future activities.

5. Best practice – Be objective, share results and make evaluation an integral part of your communications approach.

Top Tips

11

Thank you!

Contact: Paul NjokuEmail : paul.njoku@cabinet-office.gsi.gov.ukWeb link to guide: https://gcn.civilservice.gov.uk/guidance/evaluation/