i

DesigningCluster

EvaluationHow design can support

creative collaboration

Contents Foreword 2Timeline and other notes 4The first workshop 6The second workshop 12Designing new ways to engage 20Hot topics: targets 24Reflections and next steps 26Appendix A: FAQs 28

1

2

1 Foreword:Sharing new approaches in cluster evaluation

As interest in clusters has grown amongst

companies, practitioners and policy makers, the

need for effective evaluation has also risen, not least

to be able to show the return on investment from

such initiatives. However, cluster measurement and

evaluation has long been a source of frustration:

how to properly capture the impact of the

investment in clusters.

Evaluation of clusters is thus a complex area,

involving different levels and diverse audiences.

Cluster evaluation frameworks need to not only

focus on measuring the final outcome (the effect),

but also the softer, more qualitative elements,

including the benefits of greater trust and

collaboration, as well as the process of change.

In order to explore this further the Cluster

Evaluation Working Group was formed in 2013 to

share experience, capture joint learning, and work

together to develop new and better approaches to

show the value of collaboration and demonstrate

return on investment.

The Institute of Design Innovation (InDI) took a lead

role in driving forward this working group because

of our interest and involvement in cluster

development. InDI uses design to build complex

3

collaborations and help organisations deliver things

together that they cannot do alone, but we also

explore how to better evidence the more intangible

effects of clustering - the ‘how’ as well as the ‘why’-

as there are parallels with evidencing the value of

design. In addition, we have been exploring the role

of design in helping diverse expertise be effectively

shared on a complex subject through the working

group activities.

This booklet summarises the first two years of the

working group, during which time over 50 people

from more than 25 nations and regions have

participated in sessions. We hope this helps capture

our collective knowledge and work so far, and

informs future strands of work. True to the nature

of clusters, by sharing and collaborating we can do

more than by working alone.

Madeline Smith,

Head of Strategy, Institute of Design Innovation,

Member of the Board of Directors, TCI

4

2 Timelineand other notes

Cluster evaluation fundamentalsEvaluation of clusters is a complex area, involving

different levels and diverse audiences. Evaluation

is a learning process, the outputs of which should

feedback into future policy and interventions. It is

not just audit.

At a glance: Clusters evaluation learningsAs cluster evaluation approaches have become

more sophisticated they have expanded to include:

Looking at activity (numbers involved and

engaged with the cluster)

Capturing the difference that this activity is

making, both to the companies and to

the region

June 2013First meeting Forres, Scotland

September 2013TCI conference – Plenary Presentation, Conference exhibition, subgroup meeting

5

April 2014Working group meeting Belfast NI

November 2014TCI conference presentation session

Understanding the processes that help build a

successful cluster, and how the social capital and

trust generated in clusters can be maximised

Cluster evaluation frameworks therefore need to

not only focus on measuring the final outcome

(the effect), but also the softer, more qualitative

elements, including the benefits of greater

trust and collaboration, and also the process of

change (how this happened)

6

7

3 The first workshopForres, 2013

Cluster evaluation enthusiasts from nine different

countries gathered in the beautiful north of

Scotland for the first meeting of the TCI

working group.

Key themes for discussion included the fundamental

role of evaluation:

What has been achieved by the cluster?

(Effectiveness)

How well have they done this? (Efficiency)

Were these the right things to be doing?

(relevance: different circumstances)

How was this achieved? (Understanding process)

To design an evaluation there needs to be an

understanding of:

The cluster actors: who is involved, participants,

partners, stakeholders

The cluster organisation: how is this organised

The conditions in which the cluster operates

(external context )

The behaviours, including trust (internal context)

Germany:‘This was one of the

best organised

European workshop

experiences ... I found

it very interesting to

see how different kinds

of methods are used...

I hope we can catch up

with this group further

to come up with... a

unique evaluation

system for European

clusters.’

8

Overall evaluations should consider capturing:

The Why: Ultimately the impact must be measured

in sector growth and the change in the regional

economic structure.

The What: Cluster interventions focus on building

a strong business environment and improving the

competitiveness of the companies (the ‘what’ of

a cluster programme: access to finance, skills and

talent, infrastructure, knowledge and innovation,

and market access and global positioning.)

The How: Social capital is the core of the cluster

approach building competitiveness by bringing

partners together. Capturing the level of partnership

and collaboration as a cluster develops can help

show how these relationships have matured and

deepened to give greater returns.

The group had previously prepared “homework”

looking at their experience of evaluation, as well

as new approaches and key gaps or areas in need

of development. The group discussed common

themes and challenges and used these to isolate

three key questions worthy of future focus:

9

Why?

What?

How?

Regional Benefit/Impact

ClusterProjects

PartnershipConnectedness

and Global Capital

Q1: How can we develop and share a TCI ‘fruit

salad’ of current good practice and techniques in

cluster evaluation?

Q2: How do we factor in the ‘human element’ into

cluster evaluations

Q3: How do we evidence, back up and

communicate the critical success factors in clusters?

Strategic Operational

Indicators and themes

— Jobs

— GVA

— Industrial Structure

— Distribution of

opportunity

Business environment i.e.

— Innovation performance

— Physical assets

— Finance and Funding

— Skills

— Talent attraction/retention

— Internationalisation

— Who is involved

— Type of partnership

— Depth of partnership

— Behavioural change

10

Right: ‘homework’ boards designed for the fi rst workshop

Com

peti

tive

le

vel o

f pl

atfo

rm

inno

vati

on

Sti

mul

atio

n

and

Acq

uisi

tion

of

rel

evan

t

rese

arch

and

de

velo

pmen

t

Sha

ring

of

cr

itic

al s

ecto

ral

know

ledg

e

Deg

ree

of

enha

ncem

ent

of

com

peti

tive

in

nova

tion

Mot

ivat

ion

fo

r co

mpe

titi

ve

inno

vati

on

Glo

bal

com

peti

tion

and

ex

tern

al m

arke

t pr

essu

re

Com

peti

tive

po

wer

of

cl

uste

r

Type

of

cu

ltur

al

cont

ext

Ext

ent

of

colla

bora

tive

in

itia

tive

s

Ext

ent

of

shar

ed r

espo

nse

to

com

mon

ch

alle

nges

Per

form

ance

of

indi

vidu

al fi

rms

Inte

nsit

y of

diff

eren

tiat

ed

com

peti

tive

per

form

ance

Ove

rall

perf

orm

ance

of

all

firm

s in

zon

e

L1 In

ter-

Firm

Riv

alry

Loo

p

L4 V

entu

re

Att

ract

ivne

ss L

oop

L3 C

olla

bora

tive

A

dvan

tage

Loo

p

L5 F

utur

e

Foc

us L

oop

L2 In

ter-

Firm

Co-

oper

atio

n Lo

op

Gro

up w

ith

com

mon

age

nda

but

littl

e fo

rmal

or

info

rmal

inte

ract

ion

Col

lect

ion

Giv

ing

and

exch

angi

ng o

f in

form

atio

n.

Indi

vidu

al P

rogr

amm

es s

till

tota

lly s

epar

ate

Com

mun

icat

ion

Join

t ac

tivi

ties

and

com

mun

icat

ions

, giv

ing

gene

ral s

uppo

rt a

nd

endo

rsem

ent

of e

ach

othe

r’s

prog

ram

mes

, ser

vice

s or

obj

ecti

ves

Co-

oper

atio

n

Join

t ac

tivi

ties

and

com

mun

icat

ions

, joi

nt

plan

ning

and

syn

chro

nisa

tion

of

sche

dule

s,

acti

viti

es, e

vent

s an

d ob

ject

ives

Co-

ordi

nati

on

Indi

vidu

als,

gro

ups,

org

anis

atio

n re

linqu

ish

som

e au

tono

my

and

shar

e ri

sk, f

or m

utua

l gai

ns. T

rue

colla

bora

tion

res

ults

in

cha

nges

to

beha

viou

r to

sup

port

col

lect

ive

goal

sC

olla

bora

tion

Pot

enti

al w

ider

ben

efits

incl

uded

:P

oten

tial

reg

iona

l ben

efits

incl

uded

:

–

Uni

vers

ity li

nkag

es w

ith in

dust

ry

–

Ski

lls d

evel

opm

ent

–

Kno

wle

dge

tran

sfer

s

–

Enc

oura

ging

For

eign

Dire

ct In

vest

men

t

–

Ent

repr

eneu

rshi

p

–

Red

uctio

n of

‘Bra

in D

rain

’

–

No

of lo

cal/

regi

onal

jobs

indi

rect

ly d

epen

dent

on

the

proj

ect

–

Deg

ree

of h

ighe

r m

anag

emen

t be

ing

inje

cted

–

Deg

ree

of R

&D

bei

ng in

ject

ed

–

Inno

vativ

e na

ture

of

the

proj

ect

–

Con

side

r di

strib

utio

nal i

ssue

s su

ch a

s

Are

as o

f D

isad

vant

age

and

Equ

ality

.

Clu

ster

to

Clu

ster

C

olla

bor

atio

nFo

cus

for

BS

R is

Tra

nsna

tiona

l Col

labo

ratio

n.

–

The

clu

ster

dyn

amic

s qu

estio

n se

t w

ith a

n ad

ded

‘tran

snat

iona

l’ le

vel

–

Add

ition

al q

uest

ions

in in

terv

iew

s w

ith p

artn

ers

in t

he fi

ve

tran

snat

iona

l net

wor

ks o

n th

e su

ppor

t (w

hat’s

bee

n m

ost/

leas

t he

lpfu

l, w

hat

pref

erre

d ‘ti

min

g’ o

f di

ffer

ent

supp

ort,

etc

.)

–

Ann

ual e

valu

atio

n/co

achi

ng s

essi

ons

with

the

five

net

wor

ks

(foc

used

on

thei

r jo

int

‘str

ateg

ic a

ctio

n pl

ans’

)

–

Con

tinua

l ‘pr

oces

s su

ppor

t’

Reg

iona

l B

enefi

t/Im

pact

Clu

ster

P

roje

cts

Par

tner

ship

–

Con

nect

edne

ss a

nd S

ocia

l Cap

ital

Str

ateg

ic

Ope

rati

onal

Fram

ewor

k C

ondi

tion

s–

Inn

ovat

ion-

frie

ndly

env

iron

men

t

– R

egul

atio

n, t

axes

– A

vaila

bilit

y of

qua

lified

per

sonn

el

– I

nfra

stru

ctur

e

– A

ttra

ctiv

e en

viro

nmen

t

–

Acc

ess

to p

riva

te a

nd p

ublic

fina

ncin

g

Clu

ster

Org

anis

atio

nS

ervi

ces

and

acti

viti

es t

o su

ppor

t:

– N

etw

orki

ng a

nd c

o-op

erat

ion

– A

cces

s to

R&

D p

artn

ers

– I

nter

nati

onal

isat

ion

– T

rain

ing

and

Coa

chin

g

– P

R

Figu

re 1

: The

nut

shel

l mod

el o

f cl

uste

r in

terv

entio

n S

ourc

e: V

DI/

VD

E-IT

201

0

Fram

ewor

k

Clu

ster

A

ctor

s

Clu

ster

O

rgan

isat

ion

Con

cept

ual m

odel

for

the

clus

ter

prog

ram

s

Rel

atio

nsh

ip b

asis

–

The

gro

up’s

abi

lity

and

will

ingn

ess

to

expl

oit

syne

rgie

s

Pot

enti

al s

yner

gie

s–

Pot

enti

al b

enefi

ts o

f co

llabo

rati

on r

esul

ting

fro

m

exte

rnal

eco

nom

ies

of

scal

e an

d co

mpl

emen

tari

ty

Col

lab

orat

ion

pro

cess

es–

In

tern

al a

nd e

xter

nal

linka

ges

wit

hin

the

busi

ness

env

iron

men

t

Targ

et a

chie

vem

ent

–

Inno

vati

on

–

Pro

duct

ivit

y –

In

tern

atio

nalis

atio

n –

G

row

th

–

Pro

fita

bilit

y

Clu

ster

pro

gra

ms’

inte

rven

tion

in b

usi

nes

s en

viro

nm

ent

In t

he lo

nger

ter

m, s

truc

tura

l cha

ract

eris

tics

and

colla

bora

tion

pote

ntia

l will

be d

evel

oped

as

a re

sult

of c

olla

bora

tion

proc

esse

s

11

Com

peti

tive

le

vel o

f pl

atfo

rm

inno

vati

on

Sti

mul

atio

n

and

Acq

uisi

tion

of

rel

evan

t

rese

arch

and

de

velo

pmen

t

Sha

ring

of

cr

itic

al s

ecto

ral

know

ledg

e

Deg

ree

of

enha

ncem

ent

of

com

peti

tive

in

nova

tion

Mot

ivat

ion

fo

r co

mpe

titi

ve

inno

vati

on

Glo

bal

com

peti

tion

and

ex

tern

al m

arke

t pr

essu

re

Com

peti

tive

po

wer

of

cl

uste

r

Type

of

cu

ltur

al

cont

ext

Ext

ent

of

colla

bora

tive

in

itia

tive

s

Ext

ent

of

shar

ed r

espo

nse

to

com

mon

ch

alle

nges

Per

form

ance

of

indi

vidu

al fi

rms

Inte

nsit

y of

diff

eren

tiat

ed

com

peti

tive

per

form

ance

Ove

rall

perf

orm

ance

of

all

firm

s in

zon

e

L1 In

ter-

Firm

Riv

alry

Loo

p

L4 V

entu

re

Att

ract

ivne

ss L

oop

L3 C

olla

bora

tive

A

dvan

tage

Loo

p

L5 F

utur

e

Foc

us L

oop

L2 In

ter-

Firm

Co-

oper

atio

n Lo

op

Gro

up w

ith

com

mon

age

nda

but

littl

e fo

rmal

or

info

rmal

inte

ract

ion

Col

lect

ion

Giv

ing

and

exch

angi

ng o

f in

form

atio

n.

Indi

vidu

al P

rogr

amm

es s

till

tota

lly s

epar

ate

Com

mun

icat

ion

Join

t ac

tivi

ties

and

com

mun

icat

ions

, giv

ing

gene

ral s

uppo

rt a

nd

endo

rsem

ent

of e

ach

othe

r’s

prog

ram

mes

, ser

vice

s or

obj

ecti

ves

Co-

oper

atio

n

Join

t ac

tivi

ties

and

com

mun

icat

ions

, joi

nt

plan

ning

and

syn

chro

nisa

tion

of

sche

dule

s,

acti

viti

es, e

vent

s an

d ob

ject

ives

Co-

ordi

nati

on

Indi

vidu

als,

gro

ups,

org

anis

atio

n re

linqu

ish

som

e au

tono

my

and

shar

e ri

sk, f

or m

utua

l gai

ns. T

rue

colla

bora

tion

res

ults

in

cha

nges

to

beha

viou

r to

sup

port

col

lect

ive

goal

sC

olla

bora

tion

Pot

enti

al w

ider

ben

efits

incl

uded

:P

oten

tial

reg

iona

l ben

efits

incl

uded

:

–

Uni

vers

ity li

nkag

es w

ith in

dust

ry

–

Ski

lls d

evel

opm

ent

–

Kno

wle

dge

tran

sfer

s

–

Enc

oura

ging

For

eign

Dire

ct In

vest

men

t

–

Ent

repr

eneu

rshi

p

–

Red

uctio

n of

‘Bra

in D

rain

’

–

No

of lo

cal/

regi

onal

jobs

indi

rect

ly d

epen

dent

on

the

proj

ect

–

Deg

ree

of h

ighe

r m

anag

emen

t be

ing

inje

cted

–

Deg

ree

of R

&D

bei

ng in

ject

ed

–

Inno

vativ

e na

ture

of

the

proj

ect

–

Con

side

r di

strib

utio

nal i

ssue

s su

ch a

s

Are

as o

f D

isad

vant

age

and

Equ

ality

.

Clu

ster

to

Clu

ster

C

olla

bor

atio

nFo

cus

for

BS

R is

Tra

nsna

tiona

l Col

labo

ratio

n.

–

The

clu

ster

dyn

amic

s qu

estio

n se

t w

ith a

n ad

ded

‘tran

snat

iona

l’ le

vel

–

Add

ition

al q

uest

ions

in in

terv

iew

s w

ith p

artn

ers

in t

he fi

ve

tran

snat

iona

l net

wor

ks o

n th

e su

ppor

t (w

hat’s

bee

n m

ost/

leas

t he

lpfu

l, w

hat

pref

erre

d ‘ti

min

g’ o

f di

ffer

ent

supp

ort,

etc

.)

–

Ann

ual e

valu

atio

n/co

achi

ng s

essi

ons

with

the

five

net

wor

ks

(foc

used

on

thei

r jo

int

‘str

ateg

ic a

ctio

n pl

ans’

)

–

Con

tinua

l ‘pr

oces

s su

ppor

t’

Reg

iona

l B

enefi

t/Im

pact

Clu

ster

P

roje

cts

Par

tner

ship

–

Con

nect

edne

ss a

nd S

ocia

l Cap

ital

Str

ateg

ic

Ope

rati

onal

Fram

ewor

k C

ondi

tion

s–

Inn

ovat

ion-

frie

ndly

env

iron

men

t

– R

egul

atio

n, t

axes

– A

vaila

bilit

y of

qua

lified

per

sonn

el

– I

nfra

stru

ctur

e

– A

ttra

ctiv

e en

viro

nmen

t

–

Acc

ess

to p

riva

te a

nd p

ublic

fina

ncin

g

Clu

ster

Org

anis

atio

nS

ervi

ces

and

acti

viti

es t

o su

ppor

t:

– N

etw

orki

ng a

nd c

o-op

erat

ion

– A

cces

s to

R&

D p

artn

ers

– I

nter

nati

onal

isat

ion

– T

rain

ing

and

Coa

chin

g

– P

R

Figu

re 1

: The

nut

shel

l mod

el o

f cl

uste

r in

terv

entio

n S

ourc

e: V

DI/

VD

E-IT

201

0

Fram

ewor

k

Clu

ster

A

ctor

s

Clu

ster

O

rgan

isat

ion

Con

cept

ual m

odel

for

the

clus

ter

prog

ram

s

Rel

atio

nsh

ip b

asis

–

The

gro

up’s

abi

lity

and

will

ingn

ess

to

expl

oit

syne

rgie

s

Pot

enti

al s

yner

gie

s–

Pot

enti

al b

enefi

ts o

f co

llabo

rati

on r

esul

ting

fro

m

exte

rnal

eco

nom

ies

of

scal

e an

d co

mpl

emen

tari

ty

Col

lab

orat

ion

pro

cess

es–

In

tern

al a

nd e

xter

nal

linka

ges

wit

hin

the

busi

ness

env

iron

men

t

Targ

et a

chie

vem

ent

–

Inno

vati

on

–

Pro

duct

ivit

y –

In

tern

atio

nalis

atio

n –

G

row

th

–

Pro

fita

bilit

y

Clu

ster

pro

gra

ms’

inte

rven

tion

in b

usi

nes

s en

viro

nm

ent

In t

he lo

nger

ter

m, s

truc

tura

l cha

ract

eris

tics

and

colla

bora

tion

pote

ntia

l will

be d

evel

oped

as

a re

sult

of c

olla

bora

tion

proc

esse

s

12

4 The second workshopBelfast, 2014

The latest meeting of the Cluster Evaluation

Working Group was hosted by TCI organisational

members Invest NI.

25 participants from 11 different countries and

regions participated in two days of sharing,

developing and vigorous discussion.

In previous meetings the working group had shared

approaches around the ‘why’, ‘what’ and ‘how’ of

cluster evaluation. Three key questions that tackle

some of the biggest challenges in cluster evaluation

had been identified:

HomeworkIn order to maximise the time together participants

contributed ‘homework’ beforehand, to give some

insight into current practices, and share some of the

remaining challenges. Across the three questions

this pre-work explored:

How has your approach to cluster evaluation

helped address the key questions?

What new approaches have worked well?

What are still the gaps/difficulties?

This was then collated into an exhibition format to

allow everyone to look across the different

Scotland:‘I think the value of

coming together with

colleagues from

different experiences

has been quite

worth while. I say

that because I think I

changed my mind on

a few things through

the day... I have

found some different

approaches...’

13

responses and see common themes and further

areas for development.

The groups then divided into three, to look more

closely at the three key questions, unpicking further:

What does success look like?

What do we already know?

What gaps remain?

The following day the group embarked on the

Cluster Evaluation Game. It was developed by InDI

for this meeting to try and garner as much collective

knowledge as possible from such an experienced

and practiced group. The outputs from this game

are shown in appendix A.

Finally, the group re-gathered for a final discussion.

A few mini project ideas had been generated and

those will now be taken forward by members of

the group.

14

Right: boards designed for the second workshop

How

can

we

dev

elo

p a

nd

sh

are

a T

CI “

Fru

it S

alad

” of

cu

rren

t g

oo

d p

ract

ice

and

tec

hn

iqu

es

in c

lust

er e

valu

atio

n?

We

are

all u

nder

taki

ng e

valu

atio

ns u

sing

som

e br

oadl

y si

mila

r ap

proa

ches

(m

easu

ring

act

ivit

y, o

utpu

ts, i

mpa

cts)

. H

owev

er, e

ach

clus

ter

cont

ext

is d

iffer

ent

(diff

eren

t lo

cal

envi

ronm

ents

, sec

tor

stru

ctur

es, g

over

nmen

t su

ppor

t) a

nd

so e

valu

atio

n ne

eds

to b

e fl

exib

le–

hen

ce t

he n

eed

for

a “f

ruit

sal

ad”

- al

low

ing

each

clu

ster

/reg

ion

to p

ick

and

choo

se t

he b

est

com

bina

tion

for

the

ir c

onte

xt a

nd s

itua

tion

.

Can

we

dev

elo

p a

Kn

ow

led

ge

rep

osi

tory

/co

mm

un

ity

of

pra

ctic

e to

pro

vid

e co

nte

nt,

ex

amp

les

and

co

nta

ct p

eop

le

for

the

fru

it s

alad

, st

ruct

ure

d

aro

un

d:

H

ow t

o sh

ow b

enefi

t of

co

mpa

nies

’ inv

olve

men

t in

cl

uste

r in

itia

tive

s?

How

to

show

val

ue o

f cl

uste

r in

itia

tive

/clu

ster

or

gani

sati

on?

How

to

show

rel

evan

ce a

nd

effec

tive

ness

of

clus

ter

prog

ram

me

appr

oach

?

How

to

show

ben

efit

/im

pact

on

regi

onal

/nat

iona

l e

cono

my

(inn

ovat

ion

syst

em)?

Log

ic m

odel

sW

ith

indi

ctor

s ac

ross

all

leve

ls (

e.g.

# o

f co

mpa

nies

, did

it le

ad t

o co

mpa

ny g

row

th, e

xpor

ts a

nd jo

bs e

tc, o

r ot

her

asso

ciat

ed im

pact

s e.

g. f

undi

ng,

colla

bora

tion

s w

ith

univ

ersi

ties

)

Res

our

ces

& In

puts

Act

ivit

ies

Out

puts

Out

com

eIm

pact

Pla

nn

ed w

ork

Inte

nd

ed d

eliv

erab

les

Gat

her

the

frui

ts

to m

ake

it e

dibl

e!

Dev

elop

Cas

e st

udy

exam

ples

.”

“Diff

eren

t te

chni

ques

ar

e ne

eded

for

diff

eren

t la

yers

, aud

ienc

es a

nd

diff

eren

t co

ntex

ts.

Ult

imat

ely

eval

uat

ion

is

ab

ou

t le

arn

ing

an

d in

form

ing

th

e n

ext

step

s.

Que

stio

n...

Ap

pro

ach

es

Th

e W

hy

Sec

tor

grow

th a

nd

the

chan

ge in

the

reg

iona

l ec

onom

ic s

truc

ture

.

Th

e W

hat

Clu

ster

in

terv

enti

ons

e.g.

acc

ess

to

fina

nce,

ski

lls, i

nfra

stru

ctur

e,

know

ledg

e an

d in

nova

tion

, and

m

arke

t ac

cess

.

Th

e H

ow

Soc

ial c

apit

al i.

e.

how

clu

ster

s gr

ow, t

owar

ds

coop

erat

ive

and

colla

bora

tive

be

havi

ours

.

Pro

ject

rep

orti

ng a

gain

st s

peci

fic

stra

tegi

c go

als

and

revi

ew d

irec

tion

Bas

elin

e (w

here

we

star

ted)

and

fol

low

up

(sho

w p

rogr

ess

and

chan

ge/i

mpr

ovem

ent)

Div

erse

typ

e of

info

rmat

ion

– e

cono

mic

, tr

ust

leve

l, m

otiv

atio

n, t

ime

inve

stm

ent,

sa

tisf

acti

on, i

mag

e

Gov

erna

nce

– h

ow e

ffec

tive

is t

he c

lust

er

orga

nisa

tion

Mea

sure

ben

efits

for

eac

h ki

nd o

f st

akeh

olde

r: c

ompa

nies

, fun

ders

go

vern

men

ts, a

cade

mic

inst

itut

ions

Clu

ster

map

ping

, sta

tist

ical

and

ec

onom

ic a

naly

sis

Why

, Wh

at, H

ow?

Cap

turi

ng p

rogr

ess

acro

ss a

gain

st w

hy, w

hat

and

how

Bal

anci

ng q

uant

itat

ive

and

qual

itat

ive

Sel

f as

sess

men

t/re

port

ing

or

inde

pend

ent

revi

ew

In d

epth

eva

luat

ion

take

s in

vest

men

t

Set

ting

app

ropr

iate

goa

ls a

nd r

elev

ant

indi

cato

rs o

f pr

ogre

ss a

nd s

ucce

ss?

Und

erst

andi

ng c

lust

er m

atur

ity

–

thin

gs t

ake

tim

e to

hap

pen

Con

trol

gro

ups

– c

an b

e ha

rd t

o fi

nd

com

para

ble

grou

ps

Bal

anci

ng d

eliv

ery

prio

riti

es w

ith

eval

uati

on d

eman

ds

Con

diti

ons

(Ext

erna

l con

text

)B

ehav

iour

s (I

nter

nal c

onte

xt)

Clu

ster

Act

ors

Clu

ster

Org

anis

atio

n

Dev

elop

ing

a no

t to

o ge

neri

c to

ol b

ox

from

whi

ch o

ne c

an

choo

se t

he s

uita

ble

met

hod

or t

o be

com

e in

spir

ed”

So

nja

Kin

d

“

Trap

s an

d d

ifficu

ltie

sW

hen

is a

goo

d ti

me

to e

valu

ate

- to

o so

on -

not

eno

ugh

resu

lts,

too

la

te a

ttri

buti

on is

sues

Cau

salit

y

Dat

a co

llect

ion

Com

preh

ensi

ve f

ram

ewor

k –

Mix

of

stat

isti

cal d

ata,

sur

veys

, int

ervi

ews,

w

orks

hops

/foc

us g

roup

s, q

uant

itat

ive

and

qual

itat

ive

Ben

chm

arki

ng –

how

we

com

pare

wit

h co

mpe

tito

rs

Fir

m le

vel p

erfo

rman

ce (

prod

ucti

vity

, in

nova

tion

, int

erna

tion

alis

atio

n, q

ualit

y et

c.)

Par

alle

l app

roac

h to

und

erst

and

the

mec

hani

sms

– h

ow t

his

happ

ened

Mea

suri

ng c

hang

e in

per

form

ance

, ca

pabi

lity

and

unde

rsta

ndin

g –

ac

ross

indi

vidu

als,

gro

up/c

lust

er a

nd

regi

onal

leve

ls

Wh

at n

ew a

pp

roac

hes

h

ave

wo

rked

wel

l?

C

ase

stud

ies

and

“sto

ries

of

chan

ge”

H

ard

data

on

soft

issu

es –

e.g

.

incr

ease

d en

gage

men

t al

so

m

easu

re #

com

pani

es e

ngag

ed;

#

SM

Es;

# r

esea

rch

node

s

enga

ged

etc.

In

terv

iew

s an

d pa

rtic

ipat

ory

ap

proa

ches

enc

oura

ge

en

gage

men

t

D

iscr

imin

ant

anal

ysis

, CG

E m

odel

Tr

iang

ulat

ing

appr

oach

es

Ye

arly

vis

its

and

dial

ogue

mee

ting

s

In

tern

atio

nal p

eer

revi

ew

15

How

can

we

dev

elo

p a

nd

sh

are

a T

CI “

Fru

it S

alad

” of

cu

rren

t g

oo

d p

ract

ice

and

tec

hn

iqu

es

in c

lust

er e

valu

atio

n?

We

are

all u

nder

taki

ng e

valu

atio

ns u

sing

som

e br

oadl

y si

mila

r ap

proa

ches

(m

easu

ring

act

ivit

y, o

utpu

ts, i

mpa

cts)

. H

owev

er, e

ach

clus

ter

cont

ext

is d

iffer

ent

(diff

eren

t lo

cal

envi

ronm

ents

, sec

tor

stru

ctur

es, g

over

nmen

t su

ppor

t) a

nd

so e

valu

atio

n ne

eds

to b

e fl

exib

le–

hen

ce t

he n

eed

for

a “f

ruit

sal

ad”

- al

low

ing

each

clu

ster

/reg

ion

to p

ick

and

choo

se t

he b

est

com

bina

tion

for

the

ir c

onte

xt a

nd s

itua

tion

.

Can

we

dev

elo

p a

Kn

ow

led

ge

rep

osi

tory

/co

mm

un

ity

of

pra

ctic

e to

pro

vid

e co

nte

nt,

ex

amp

les

and

co

nta

ct p

eop

le

for

the

fru

it s

alad

, st

ruct

ure

d

aro

un

d:

H

ow t

o sh

ow b

enefi

t of

co

mpa

nies

’ inv

olve

men

t in

cl

uste

r in

itia

tive

s?

How

to

show

val

ue o

f cl

uste

r in

itia

tive

/clu

ster

or

gani

sati

on?

How

to

show

rel

evan

ce a

nd

effec

tive

ness

of

clus

ter

prog

ram

me

appr

oach

?

How

to

show

ben

efit

/im

pact

on

regi

onal

/nat

iona

l e

cono

my

(inn

ovat

ion

syst

em)?

Log

ic m

odel

sW

ith

indi

ctor

s ac

ross

all

leve

ls (

e.g.

# o

f co

mpa

nies

, did

it le

ad t

o co

mpa

ny g

row

th, e

xpor

ts a

nd jo

bs e

tc, o

r ot

her

asso

ciat

ed im

pact

s e.

g. f

undi

ng,

colla

bora

tion

s w

ith

univ

ersi

ties

)

Res

our

ces

& In

puts

Act

ivit

ies

Out

puts

Out

com

eIm

pact

Pla

nn

ed w

ork

Inte

nd

ed d

eliv

erab

les

Gat

her

the

frui

ts

to m

ake

it e

dibl

e!

Dev

elop

Cas

e st

udy

exam

ples

.”

“Diff

eren

t te

chni

ques

ar

e ne

eded

for

diff

eren

t la

yers

, aud

ienc

es a

nd

diff

eren

t co

ntex

ts.

Ult

imat

ely

eval

uat

ion

is

ab

ou

t le

arn

ing

an

d in

form

ing

th

e n

ext

step

s.

Que

stio

n...

Ap

pro

ach

es

Th

e W

hy

Sec

tor

grow

th a

nd

the

chan

ge in

the

reg

iona

l ec

onom

ic s

truc

ture

.

Th

e W

hat

Clu

ster

in

terv

enti

ons

e.g.

acc

ess

to

fina

nce,

ski

lls, i

nfra

stru

ctur

e,

know

ledg

e an

d in

nova

tion

, and

m

arke

t ac

cess

.

Th

e H

ow

Soc

ial c

apit

al i.

e.

how

clu

ster

s gr

ow, t

owar

ds

coop

erat

ive

and

colla

bora

tive

be

havi

ours

.

Pro

ject

rep

orti

ng a

gain

st s

peci

fic

stra

tegi

c go

als

and

revi

ew d

irec

tion

Bas

elin

e (w

here

we

star

ted)

and

fol

low

up

(sho

w p

rogr

ess

and

chan

ge/i

mpr

ovem

ent)

Div

erse

typ

e of

info

rmat

ion

– e

cono

mic

, tr

ust

leve

l, m

otiv

atio

n, t

ime

inve

stm

ent,

sa

tisf

acti

on, i

mag

e

Gov

erna

nce

– h

ow e

ffec

tive

is t

he c

lust

er

orga

nisa

tion

Mea

sure

ben

efits

for

eac

h ki

nd o

f st

akeh

olde

r: c

ompa

nies

, fun

ders

go

vern

men

ts, a

cade

mic

inst

itut

ions

Clu

ster

map

ping

, sta

tist

ical

and

ec

onom

ic a

naly

sis

Why

, Wh

at, H

ow?

Cap

turi

ng p

rogr

ess

acro

ss a

gain

st w

hy, w

hat

and

how

Bal

anci

ng q

uant

itat

ive

and

qual

itat

ive

Sel

f as

sess

men

t/re

port

ing

or

inde

pend

ent

revi

ew

In d

epth

eva

luat

ion

take

s in

vest

men

t

Set

ting

app

ropr

iate

goa

ls a

nd r

elev

ant

indi

cato

rs o

f pr

ogre

ss a

nd s

ucce

ss?

Und

erst

andi

ng c

lust

er m

atur

ity

–

thin

gs t

ake

tim

e to

hap

pen

Con

trol

gro

ups

– c

an b

e ha

rd t

o fi

nd

com

para

ble

grou

ps

Bal

anci

ng d

eliv

ery

prio

riti

es w

ith

eval

uati

on d

eman

ds

Con

diti

ons

(Ext

erna

l con

text

)B

ehav

iour

s (I

nter

nal c

onte

xt)

Clu

ster

Act

ors

Clu

ster

Org

anis

atio

n

Dev

elop

ing

a no

t to

o ge

neri

c to

ol b

ox

from

whi

ch o

ne c

an

choo

se t

he s

uita

ble

met

hod

or t

o be

com

e in

spir

ed”

So

nja

Kin

d

“

Trap

s an

d d

ifficu

ltie

sW

hen

is a

goo

d ti

me

to e

valu

ate

- to

o so

on -

not

eno

ugh

resu

lts,

too

la

te a

ttri

buti

on is

sues

Cau

salit

y

Dat

a co

llect

ion

Com

preh

ensi

ve f

ram

ewor

k –

Mix

of

stat

isti

cal d

ata,

sur

veys

, int

ervi

ews,

w

orks

hops

/foc

us g

roup

s, q

uant

itat

ive

and

qual

itat

ive

Ben

chm

arki

ng –

how

we

com

pare

wit

h co

mpe

tito

rs

Fir

m le

vel p

erfo

rman

ce (

prod

ucti

vity

, in

nova

tion

, int

erna

tion

alis

atio

n, q

ualit

y et

c.)

Par

alle

l app

roac

h to

und

erst

and

the

mec

hani

sms

– h

ow t

his

happ

ened

Mea

suri

ng c

hang

e in

per

form

ance

, ca

pabi

lity

and

unde

rsta

ndin

g –

ac

ross

indi

vidu

als,

gro

up/c

lust

er a

nd

regi

onal

leve

ls

Wh

at n

ew a

pp

roac

hes

h

ave

wo

rked

wel

l?

C

ase

stud

ies

and

“sto

ries

of

chan

ge”

H

ard

data

on

soft

issu

es –

e.g

.

incr

ease

d en

gage

men

t al

so

m

easu

re #

com

pani

es e

ngag

ed;

#

SM

Es;

# r

esea

rch

node

s

enga

ged

etc.

In

terv

iew

s an

d pa

rtic

ipat

ory

ap

proa

ches

enc

oura

ge

en

gage

men

t

D

iscr

imin

ant

anal

ysis

, CG

E m

odel

Tr

iang

ulat

ing

appr

oach

es

Ye

arly

vis

its

and

dial

ogue

mee

ting

s

In

tern

atio

nal p

eer

revi

ew

16

Right: boards designed for the second workshop

Que

stio

n...

How

do

we

fact

or

in t

he

Hu

man

Ele

men

t in

to C

lust

er

Eva

luat

ion

s?

The

thi

ng t

hat

mak

es a

clu

ster

diff

eren

t is

the

bui

ldin

g of

tr

ust

and

rela

tion

ship

s th

at m

eans

the

gro

up g

oes

from

just

be

ing

a co

llect

ion

to b

uild

ing

deep

er le

vels

of

colla

bora

tion

an

d do

ing

thin

gs t

oget

her

that

the

y co

uld

not

do

indi

vidu

ally

. “W

e ca

n d

o n

ew t

hin

gs

tog

eth

er –

we

can

o

nly

do

th

ese

thin

gs

tog

eth

er” Ho

w d

o y

ou

cap

ture

bu

ildin

g

tru

st a

nd

so

cial

cap

ital

wit

hin

a

clu

ster

?

Obj

ecti

ve o

f cl

uste

r is

to

get

inte

ract

ion

wor

king

, cat

alys

ing

the

acto

rs t

o ge

t th

ings

wor

king

Und

erst

andi

ng h

ow b

est

to d

o th

is is

fun

dam

enta

l

The

ong

oing

ev

alua

tion

alr

eady

sh

ows

whe

re t

he

hum

an f

acto

r is

a

wei

ght

on t

he

clus

ter

acti

vity

”

Rh

ôn

e-A

lpes

“

It’s

qui

te d

ifficu

lt t

o m

easu

re t

rust

, or

evid

ence

cha

ngin

g be

havi

ours

, but

it h

as

valu

e al

ongs

ide

the

econ

omic

ben

efit

– it

is

qua

ntif

ying

it t

hat

is d

ifficu

lt.

The

re is

a g

ener

al

acce

ptan

ce t

hat

soci

al c

apit

al a

nd

trus

t is

a c

riti

cal

elem

ent

for

effec

tive

cl

uste

r de

velo

pmen

t.

Thi

s is

a v

ery

impo

rtan

t pa

rt o

f cl

uste

ring

, but

one

th

at is

ver

y po

orly

ca

ptur

ed in

ev

alua

tion

.

It is

a q

uest

ion

abou

t ch

ange

– in

un

ders

tand

ing,

min

dset

, in

rela

tion

s in

tern

al a

nd e

xter

nal”

Ja

n M

essi

ng

“

Ap

pro

ach

esT

he p

artn

ersh

ip jo

urne

y –

w

hat

is t

he e

vide

nce

of

chan

ge f

rom

col

lect

ion

to

clus

ter

Clu

ster

dyn

amic

s m

odel

Evi

denc

e of

gro

wth

in

part

ners

hips

(be

twee

n co

mpa

nies

, com

pany

to

univ

ersi

ty; w

ith

com

pani

es in

di

ffer

ent

sect

or/m

arke

t et

c)

Em

ail t

raffi

c be

twee

n cl

uste

r or

gani

sati

ons

Co

llect

ion

Co

mm

un

icat

ion

Co

-op

erat

ion

Co

-ord

inat

ion

Co

llab

ora

tio

n

Clu

ster

dyn

amic

s (a

nd o

ther

pa

rtic

ipat

ory

met

hods

) to

o “b

ulky

” –

dev

elop

fur

ther

for

qu

ick

easy

sel

f as

sess

men

ts

Can

we

use

lead

ersh

ip

liter

atur

e to

und

erst

and

the

role

of

boun

dary

spa

nner

s et

c?

Defi

ne d

escr

ipti

on a

nd le

vels

of

pro

gres

s be

tter

– t

hen

self

ass

essm

ent

– w

here

are

yo

u on

thi

s sc

ale?

As

trus

t de

velo

ps, w

hat

wou

ld y

ou

expe

ct t

o se

e?

Bal

anci

ng t

ime

spen

t ca

ptur

ing

the

data

wit

h th

e va

lue

of t

hat

data

Mai

ntai

ning

neu

tral

ity–

en

gage

men

t an

d un

ders

tand

ing

as a

n ev

alua

tor

vs in

depe

nden

ce

Sha

ring

indi

cato

rs /

vis

ual i

s us

eful

to

show

pro

gres

s

Trap

s an

d d

ifficu

ltie

s

Wh

at n

ew

app

roac

hes

hav

e w

ork

ed w

ell?

U

nder

stan

ding

the

im

port

ance

of

fa

cilit

atio

n an

d

inve

stin

g in

tha

t ro

le

C

aptu

ring

cat

alyt

ic

m

omen

ts, u

niqu

e

stor

ies,

ane

cdot

es

an

d qu

otes

C

onne

ctin

g

conn

ecto

rs –

boun

dary

spa

nner

s

T

ime

inve

sted

by

cl

uste

r pa

rtic

ipan

ts

Soc

ial n

etw

ork

anal

ysis

Is t

here

evi

denc

e of

ch

ange

in

beha

viou

r/at

titu

des

Tang

ible

ben

efits

of

clus

teri

ng e

.g. H

as t

he

colla

bora

tion

led

to

new

/diff

eren

t se

rvic

es,

attr

acte

d ne

w

cust

omer

s, le

d to

new

pa

rtne

rshi

ps e

tc.

17

Que

stio

n...

How

do

we

fact

or

in t

he

Hu

man

Ele

men

t in

to C

lust

er

Eva

luat

ion

s?

The

thi

ng t

hat

mak

es a

clu

ster

diff

eren

t is

the

bui

ldin

g of

tr

ust

and

rela

tion

ship

s th

at m

eans

the

gro

up g

oes

from

just

be

ing

a co

llect

ion

to b

uild

ing

deep

er le

vels

of

colla

bora

tion

an

d do

ing

thin

gs t

oget

her

that

the

y co

uld

not

do

indi

vidu

ally

. “W

e ca

n d

o n

ew t

hin

gs

tog

eth

er –

we

can

o

nly

do

th

ese

thin

gs

tog

eth

er” Ho

w d

o y

ou

cap

ture

bu

ildin

g

tru

st a

nd

so

cial

cap

ital

wit

hin

a

clu

ster

?

Obj

ecti

ve o

f cl

uste

r is

to

get

inte

ract

ion

wor

king

, cat

alys

ing

the

acto

rs t

o ge

t th

ings

wor

king

Und

erst

andi

ng h

ow b

est

to d

o th

is is

fun

dam

enta

l

The

ong

oing

ev

alua

tion

alr

eady

sh

ows

whe

re t

he

hum

an f

acto

r is

a

wei

ght

on t

he

clus

ter

acti

vity

”

Rh

ôn

e-A

lpes

“

It’s

qui

te d

ifficu

lt t

o m

easu

re t

rust

, or

evid

ence

cha

ngin

g be

havi

ours

, but

it h

as

valu

e al

ongs

ide

the

econ

omic

ben

efit

– it

is

qua

ntif

ying

it t

hat

is d

ifficu

lt.

The

re is

a g

ener

al

acce

ptan

ce t

hat

soci

al c

apit

al a

nd

trus

t is

a c

riti

cal

elem

ent

for

effec

tive

cl

uste

r de

velo

pmen

t.

Thi

s is

a v

ery

impo

rtan

t pa

rt o

f cl

uste

ring

, but

one

th

at is

ver

y po

orly

ca

ptur

ed in

ev

alua

tion

.

It is

a q

uest

ion

abou

t ch

ange

– in

un

ders

tand

ing,

min

dset

, in

rela

tion

s in

tern

al a

nd e

xter

nal”

Ja

n M

essi

ng

“

Ap

pro

ach

esT

he p

artn

ersh

ip jo

urne

y –

w

hat

is t

he e

vide

nce

of

chan

ge f

rom

col

lect

ion

to

clus

ter

Clu

ster

dyn

amic

s m

odel

Evi

denc

e of

gro

wth

in

part

ners

hips

(be

twee

n co

mpa

nies

, com

pany

to

univ

ersi

ty; w

ith

com

pani

es in

di

ffer

ent

sect

or/m

arke

t et

c)

Em

ail t

raffi

c be

twee

n cl

uste

r or

gani

sati

ons

Co

llect

ion

Co

mm

un

icat

ion

Co

-op

erat

ion

Co

-ord

inat

ion

Co

llab

ora

tio

n

Clu

ster

dyn

amic

s (a

nd o

ther

pa

rtic

ipat

ory

met

hods

) to

o “b

ulky

” –

dev

elop

fur

ther

for

qu

ick

easy

sel

f as

sess

men

ts

Can

we

use

lead

ersh

ip

liter

atur

e to

und

erst

and

the

role

of

boun

dary

spa

nner

s et

c?

Defi

ne d

escr

ipti

on a

nd le

vels

of

pro

gres

s be

tter

– t

hen

self

ass

essm

ent

– w

here

are

yo

u on

thi

s sc

ale?

As

trus

t de

velo

ps, w

hat

wou

ld y

ou

expe

ct t

o se

e?

Bal

anci

ng t

ime

spen

t ca

ptur

ing

the

data

wit

h th

e va

lue

of t

hat

data

Mai

ntai

ning

neu

tral

ity–

en

gage

men

t an

d un

ders

tand

ing

as a

n ev

alua

tor

vs in

depe

nden

ce

Sha

ring

indi

cato

rs /

vis

ual i

s us

eful

to

show

pro

gres

s

Trap

s an

d d

ifficu

ltie

s

Wh

at n

ew

app

roac

hes

hav

e w

ork

ed w

ell?

U

nder

stan

ding

the

im

port

ance

of

fa

cilit

atio

n an

d

inve

stin

g in

tha

t ro

le

C

aptu

ring

cat

alyt

ic

m

omen

ts, u

niqu

e

stor

ies,

ane

cdot

es

an

d qu

otes

C

onne

ctin

g

conn

ecto

rs –

boun

dary

spa

nner

s

T

ime

inve

sted

by

cl

uste

r pa

rtic

ipan

ts

Soc

ial n

etw

ork

anal

ysis

Is t

here

evi

denc

e of

ch

ange

in

beha

viou

r/at

titu

des

Tang

ible

ben

efits

of

clus

teri

ng e

.g. H

as t

he

colla

bora

tion

led

to

new

/diff

eren

t se

rvic

es,

attr

acte

d ne

w

cust

omer

s, le

d to

new

pa

rtne

rshi

ps e

tc.

18

Right: boards designed for the second workshop

Que

stio

n...

The

sta

rtin

g po

int

of a

ny c

lust

er

eval

uati

on s

houl

d be

to

unde

rsta

ndin

g th

e au

dien

ce..

..and

w

hat

deci

sion

s or

ou

tcom

es y

ou a

re

tryi

ng t

o in

flue

nce”

Eliz

abet

h R

edm

an

“

Com

mun

icat

ing

resu

lts

rela

ted

to

goal

s is

eff

ecti

ve. T

he d

ifficu

lt p

art

is e

stab

lishi

ng r

elev

ant

goal

s /e

xpec

tati

ons.

” V

inn

ova

“How

do

we

evid

ence

an

d

com

mu

nic

ate

the

Cri

tica

l S

ucc

ess

Fac

tors

in C

lust

ers?

How

do

we

diss

emin

ate

and

evid

ence

the

val

ue o

f cl

uste

rs

and

com

mun

icat

e th

is e

ffec

tive

ly t

o ou

r pa

rtne

rs, p

oten

tial

cl

uste

r m

embe

rs a

nd m

ost

impo

rtan

tly

to o

ur f

unde

rs. I

f w

e ca

n pu

t ac

ross

rea

l arg

umen

ts a

s to

the

val

ue o

f cl

uste

ring

th

is c

an h

elp

supp

ort

furt

her

inve

stm

ent.

Ho

w h

ave

you

dis

sem

inat

ed a

nd

co

mm

un

icat

ed r

esu

lts

to y

ou

r ke

y au

die

nce

s?

Usi

ng c

oncr

ete

indi

cato

rs t

o sh

ow p

rogr

ess

agai

nst

goal

s

Com

bine

ane

cdot

al a

nd h

ard

evid

ence

Hor

ses

mou

th is

ver

y po

wer

ful

i.e. c

ompa

ny t

o co

mpa

ny

Sho

wca

sing

the

clu

ster

s as

an

attr

acti

on t

o re

gion

– a

hub

, gi

ving

acc

ess

to r

esou

rces

(m

oney

and

tal

ent)

and

wit

h lim

ited

ris

k

Imag

e an

d re

puta

tion

– h

as t

he

clus

ter

help

ed?

Des

crib

ing

good

pra

ctic

e an

d th

e pr

oces

s/de

cisi

ons

that

ga

ve t

he b

est

resu

lts

Pub

lishi

ng a

nd c

onfe

renc

es t

o sh

are

appr

oach

es a

nd r

esul

ts

Dir

ect

disc

ussi

on a

nd s

hari

ng

wit

h po

licy

mak

ers

and

clus

ter

prac

titi

oner

Clu

ster

pol

icy

is

not

a ho

mog

eneo

us

bloc

k bu

t in

clud

es

a va

riet

y of

dy

nam

ics”

tech

no

lop

lis

“

Ap

pro

ach

es

Sti

ll di

fficu

lt t

o ex

plai

n th

e pr

oces

s an

d sh

ow h

ow

clus

ter

orga

nisa

tion

led

to

the

resu

lts

Rol

e of

lead

ersh

ip a

nd d

rive

in

a c

lust

er is

impo

rtan

t –

ho

w d

o yo

u as

sess

pro

perl

y?

Pol

icy

mak

ers

wan

t “a

n an

swer

”, p

refe

rabl

y an

“i

mpa

ct fi

gure

” to

just

ify

spen

d an

d in

vest

men

t

Clu

ster

pol

icie

s on

ly o

ne

elem

ent

in a

n in

nova

tion

la

ndsc

ape.

Und

erst

andi

ng t

he e

xter

nal

cont

ext

– a

flat

per

form

ance

in

a d

eclin

ing

sect

or c

ould

be

a su

cces

s or

sm

alle

r w

orkf

orce

but

mor

e hi

ghly

sk

illed

Trap

s an

d d

ifficu

ltie

s

Wh

at n

ew a

pp

roac

hes

hav

e w

ork

ed w

ell?

Lead

ersh

ip a

nd v

isio

n (c

ompe

lling

) an

d th

en

show

ing

prog

ress

Agr

eem

ent

on c

lari

ty /

tra

ject

ory,

dir

ecti

on o

f tr

avel

Und

erst

andi

ng a

nd r

epor

ting

impa

ct a

nd im

port

ance

of

cont

ext

and

cond

itio

ns (

.e.g

. in

coun

try/

regi

on)

on t

he

resu

lts

achi

eved

Hig

hlig

hts

fro

m S

tarD

ust

pro

ject

res

ult

s

200

%

Targ

et

100

%

of g

oal

New

str

ong

R&

I nod

es

enga

ged

New

too

ls f

or

colla

bora

tion

MN

Es

enga

ged

thro

ugh

inno

vati

on

node

s

SM

Es

enga

ged

thro

ugh

inno

vati

on

node

s

New

pro

duct

s an

d se

rvic

esC

all f

or

EU

-ten

ders

ap

piie

d

Cal

l for

E

U-t

ende

rs

won

19

Que

stio

n...

The

sta

rtin

g po

int

of a

ny c

lust

er

eval

uati

on s

houl

d be

to

unde

rsta

ndin

g th

e au

dien

ce..

..and

w

hat

deci

sion

s or

ou

tcom

es y

ou a

re

tryi

ng t

o in

flue

nce”

Eliz

abet

h R

edm

an

“

Com

mun

icat

ing

resu

lts

rela

ted

to

goal

s is

eff

ecti

ve. T

he d

ifficu

lt p

art

is e

stab

lishi

ng r

elev

ant

goal

s /e

xpec

tati

ons.

” V

inn

ova

“How

do

we

evid

ence

an

d

com

mu

nic

ate

the

Cri

tica

l S

ucc

ess

Fac

tors

in C

lust

ers?

How

do

we

diss

emin

ate

and

evid

ence

the

val

ue o

f cl

uste

rs

and

com

mun

icat

e th

is e

ffec

tive

ly t

o ou

r pa

rtne

rs, p

oten

tial

cl

uste

r m

embe

rs a

nd m

ost

impo

rtan

tly

to o

ur f

unde

rs. I

f w

e ca

n pu

t ac

ross

rea

l arg

umen

ts a

s to

the

val

ue o

f cl

uste

ring

th

is c

an h

elp

supp

ort

furt

her

inve

stm

ent.

Ho

w h

ave

you

dis

sem

inat

ed a

nd

co

mm

un

icat

ed r

esu

lts

to y

ou

r ke

y au

die

nce

s?

Usi

ng c

oncr

ete

indi

cato

rs t

o sh

ow p

rogr

ess

agai

nst

goal

s

Com

bine

ane

cdot

al a

nd h

ard

evid

ence

Hor

ses

mou

th is

ver

y po

wer

ful

i.e. c

ompa

ny t

o co

mpa

ny

Sho

wca

sing

the

clu

ster

s as

an

attr

acti

on t

o re

gion

– a

hub

, gi

ving

acc

ess

to r

esou

rces

(m

oney

and

tal

ent)

and

wit

h lim

ited

ris

k

Imag

e an

d re

puta

tion

– h

as t

he

clus

ter

help

ed?

Des

crib

ing

good

pra

ctic

e an

d th

e pr

oces

s/de

cisi

ons

that

ga

ve t

he b

est

resu

lts

Pub

lishi

ng a

nd c

onfe

renc

es t

o sh

are

appr

oach

es a

nd r

esul

ts

Dir

ect

disc

ussi

on a

nd s

hari

ng

wit

h po

licy

mak

ers

and

clus

ter

prac

titi

oner

Clu

ster

pol

icy

is

not

a ho

mog

eneo

us

bloc

k bu

t in

clud

es

a va

riet

y of

dy

nam

ics”

tech

no

lop

lis

“

Ap

pro

ach

es

Sti

ll di

fficu

lt t

o ex

plai

n th

e pr

oces

s an

d sh

ow h

ow

clus

ter

orga

nisa

tion

led

to

the

resu

lts

Rol

e of

lead

ersh

ip a

nd d

rive

in

a c

lust

er is

impo

rtan

t –

ho

w d

o yo

u as

sess

pro

perl

y?

Pol

icy

mak

ers

wan

t “a

n an

swer

”, p

refe

rabl

y an

“i

mpa

ct fi

gure

” to

just

ify

spen

d an

d in

vest

men

t

Clu

ster

pol

icie

s on

ly o

ne

elem

ent

in a

n in

nova

tion

la

ndsc

ape.

Und

erst

andi

ng t

he e

xter

nal

cont

ext

– a

flat

per

form

ance

in

a d

eclin

ing

sect

or c

ould

be

a su

cces

s or

sm

alle

r w

orkf

orce

but

mor

e hi

ghly

sk

illed

Trap

s an

d d

ifficu

ltie

s

Wh

at n

ew a

pp

roac

hes

hav

e w

ork

ed w

ell?

Lead

ersh

ip a

nd v

isio

n (c

ompe

lling

) an

d th

en

show

ing

prog

ress

Agr

eem

ent

on c

lari

ty /

tra

ject

ory,

dir

ecti

on o

f tr

avel

Und

erst

andi

ng a

nd r

epor

ting

impa

ct a

nd im

port

ance

of

cont

ext

and

cond

itio

ns (

.e.g

. in

coun

try/

regi

on)

on t

he

resu

lts

achi

eved

Hig

hlig

hts

fro

m S

tarD

ust

pro

ject

res

ult

s

200

%

Targ

et

100

%

of g

oal

New

str

ong

R&

I nod

es

enga

ged

New

too

ls f

or

colla

bora

tion

MN

Es

enga

ged

thro

ugh

inno

vati

on

node

s

SM

Es

enga

ged

thro

ugh

inno

vati

on

node

s

New

pro

duct

s an

d se

rvic

esC

all f

or

EU

-ten

ders

ap

piie

d

Cal

l for

E

U-t

ende

rs

won

20

21

5 Designing new waysto engage: the cluster evaluation board game

Developed by InDI researchers for the TCI working

group meeting in Belfast in April 2014, this game

was designed to extract high quality information

from a diverse group of experts in a rapid and

engaging way.

The meeting in Belfast involved twenty-five

participants from eleven different countries and

regions across Europe bringing diverse and deep

practical expertise on this topic. A key challenge

was how to use limited time to garner that

experience, in a way that could be easily sharable.

The ‘rules’The questions used in the game were informed

by the initial discussion in the workshop, and the

activity delivered a fun and energetic form of

focused brainstorming.

Broad rules were put in place. Three games ran

in parallel, with two teams on each. The aim of

the game was to ‘reach success’, and teams could

decide to move forward or move the other team

back. On landing on a space the team had two

minutes to answer the question. There were also

a few wild card questions to keep participants on

their toes.

Poland:‘...it gives a bigger

picture, a broader

perspective, which is

very useful.’

22

The use of this game proved hugely beneficial

in quickly drawing out and capturing the joint

expertise gathered in the meeting. The activity

generated over forty flipchart pages of answers

to specific questions. By using a blank game

the questions could be made more specific and

bespoke to the event underway.

In evaluation afterwards, the game was rated very

highly as an excellent tool to release and capture

expertise in a fun and engaging way. It generated

some very successful outputs for the design

researchers because it was physical and tangible,

but also because the playful nature helped energise

the process.

The collective expertise gathered through the

game is captured as part of the FAQ section in

this booklet.

23

24

6 Hot topics: targetsA tale of indicators and owls

There is often a strong pressure, particularly from

funders, to set targets for a cluster to achieve.

The working group discussed this at length, and

agreed that they can be useful to show tangible

progress, and to give comfort to funders. However,

targets must reflect overall goals and objectives

therefore care must be taken to identify the right

targets to drive the right behaviours. The owl

example helped illustrate this.

Imagine you are in charge of a forest, and your

overall goal and objective is to create a strong

ecosystem within that forest.

A good indicator of the health of the ecosystem is

the number of owls within the forest. Owls are at

the top of the food chain. They eat the rodents,

who eat the insects, who live within the fallen leaves

and debris on the forest floor. So, if the owls are

healthy, this indicates that all those elements are

working well and the forest ecosystem is strong.

However, if the number of owls becomes a target

and that becomes the driver to success, it can

encourage the wrong behaviours. For example,

additional owls may be brought in externally.

The debris might be cleared from the forest floor to

make it easier for owls to catch their prey.

25

New food sources might be added. All of these in

the short term may help owl numbers, but in the

longer term will distort and weaken the ecosystem –

the opposite of the overall objective and goal.

Thus owls are a really good indicator, but a lousy

target. We must be careful when setting our targets

that we do not lose sight of our objectives.

26

7Reflections and next steps

The first year of activity for the Cluster Evaluation

Working Group has really made progress in sharing

expertise and starting to develop new approaches.

As well as the main group workshops, a few mini

project ideas have been generated and those will

be taken forward by members of the group, for

example, how can we better capture the ‘human

element’ of clustering? There is potential for

further meetings, but the key focus is currently to

collate the outputs from the working group and

share that learning.

An online area for the working group has been

set up in new TCI website to share activity

(including the banners produced during the

Belfast workshop), and other ways to continue the

discussion, for example, webinars on specific topics,

are being explored.

The role of designAt InDI we continue to be interested in

researching the role of design in clusters, in building

complex collaborations, and evaluating those

intangible benefits.

We have also successfully used design to help elicit

the depth of knowledge and expertise across this

TCI working group, helping to to bring knowledge

27

together to share with other cluster practitioners.

This is a fertile area for further investigation.

Taking it forwardAs Christian Ketels, TCI President wrote recently,

‘Increasingly, we seem to be on the cusp of a real

breakthrough in this area. TCI can and should play

its role in facilitating this process.’

As ever, TCI is most successful when members

take forward areas that are really important to

them, so we look forward to many more projects

and conversations, both as formal meetings of the

working group, and as important activity driven by

members priorities.

South Africa:‘I think this is still the

most important TCI nut

to be cracked’—Nigel Gwynn Evans

28

Appendix A FAQs

These questions were identified by the working

group as important issues to consider when

designing a cluster evaluation. Through the

workshop this group of experts generated some

informed answers, to help inform your evaluation

planning.

Q1 – How can we develop and share a TCI ‘fruit salad’ of current good practice and techniques in cluster evaluation?

1. What should you ask regional funders in an

interview

What are your three main expectations for the

cluster?

What are your expected deliverables, and any

timescales for achieving these?

How does the cluster fit with wider strategies

and policies?

2. How would you benchmark clusters

Use similar sectors or similar individual

portfolios, at a similar development stage

(although you can learn from those further

ahead)

Benchmark against best in class, and using

tools like cluster observatory, cluster

collaboration platform

29

Benchmark against not just current stage but

also aspirations (whilst remaining realistic)

3. How would you capture additionality (and why is

it important)

Use comparison/control groups counterfactual

Use a baseline case – and then show what is

different from business as usual

Storytelling – what you’re getting in addition,

e.g. enhanced capability by working together

4. Who should be included in a cluster survey

All cluster members - companies, academia,

relevant public sector associations

Policy makers and funders

Companies from the supply chain

Stakeholders from committee

Facilitator

Other cluster organisations in the region

Peer review similar clusters in similar sector

5. How would you capture firm level performance?

Standard data on firm performance and softer

data (through survey) from firms

Hard data – on productivity, R&D, employment

Export trends level and trend over time

Examples of how they innovate

What makes a good case study

Open and honest and succinct

30

Shows steps on journey (start and progress

from/to) and decision process

Shows impact and critical success factors and

Onward strategies

Personal – pictures, videos, testimonials,

verbal quotes

Q2 – How do we factor in the human element into cluster evaluations

1. What would you see in a cluster with very good

social capital

Joint projects and initiatives and investments

Trust between stakeholders i.e. willing to rely

on, take risk with, believe in

A lot of communication – daily interaction,

Good networking (informal as well as structured)

Sharing problems and working together to solve

Sharing resources e.g. human capital e.g.

lobbying e.g. IP e.g. learning and skills

Collaboration both between and beyond

the cluster

31

2. What would you expect to see in a good

‘boundary spanner’

Outgoing and proactive in engaging wide

groups in different organisations/geographies

Seeking out opportunities and open to new

ideas and change

Credibility and legitimacy and ability to

stay neutral

Broad knowledge of different organisations and

culturally aware

Not just in it for themselves

3. How would you evidence regional leadership

Consistent political support

Co. financing and investment of time

Participating in steering board

Ambassador outside – e.g. communicate

regional opportunities to national government

Mentioned in strategic docs

Consultation with clusters when making policy

Help build new linkages

32

4. What are the good/ bad things about

self assessment

Good:

— Cost efficiency

— Easy (sometimes only feasible approach)

— Honest

— Learning/involvement of most informed

Stakeholders

— Starting point for other techniques

— Could visualise the internal discussion

Bad:

— Lack of objectivity (perceived bias)

— Often more qualitative – (perceived lack

of accuracy)

— Influenced by own agenda

— Pick out the things that show results in

good light

— Not challenged on perspective – internal

view

5. How would you evidence an increase in

‘linkages’ (i.e more collaborative behaviour)

People involved in meetings and who from an

organisation

Types of collaborative projects changing to

more in depth

Evidence of collaboration – behaviours (before

and after)

Collaboration space

33

Joint venture – partners involved

Formal analysis e.g. SNA (social network

analysis)

6. Outline the stages/ levels you might see to

describe increasing (deeper) collaboration

behaviour

Start/emerging

Deciding and investment

Maturity – function well/collaborate well, strong

social capital

Reinvention – cluster can fund itself

Exploring new fields and activities – cluster to

cluster collaboration

Developing new strategy – survive through crisis

7. Draw three ways you usually present progress

Presenting progress (rockets/chart etc)

Summarise key messages for regional

stakeholders

Show an example of success/possible future

What needs to be done and responsibility for

progress

8. Describe some critical ‘human element’ roles

that might be important for a cluster

Leadership (cluster manager or initiative leader)

Management (cluster organization/process

support team)

34

Regional backing

Engagement (of companies, knowledge

institutions and other relevant actors)

(Internal) Collaboration and how this develops

to deeper trust

(External) Interaction/linkages, including

transnational and cross sectoral linkages

Q3 - How do we evidence and communicate the critical success factors in clusters?

1. What are the key messages for regional

stakeholders

What’s in it for them?

New perspectives for development and new

investment opportunities

Renewal potential for the future - visionary/

future context messages

Attraction for region to new companies and

talent

Access to people you want to influence

Look at clusters as high performance innovation

systems

Cluster map – look at regions strengths/

weaknesses

Communicate both success and failures – good/

bad reliable story

35

2. How would you factor in the external content in

an evaluation

Understanding current marketplace – look at

sector context, locally, nationally, internationally

Policy – mix/change (e.g. election)

Benchmark within country/outside other regions

Direct effects and indirect issues/challenges

Compare with a control group

Time series – multi method story with different

controls

Pestle analysis and other tools (political,

economical, social, technology, legislation,

environmental).

3. What are the key messages for companies

The benefits of working in a cluster including

Improve products

Market place access and market intelligence

New knowledge and abilities to innovate

Improving firm performance, cost reduction and

increased profit by working together

Attraction – credibility, reputation and

recognition of markets

Influence policy

Access to talent and skills and shared workforce

Access to funding

Ability to fill gaps in own capability

New and enhanced skills

New thinking and new perspectives (open eyes)

36

Value of collaboration and being many instead

of one

4. How can company stories help

Show good examples of benefits

Provide real evidence and encouragement to

others

Good marketing and pride/recognition and

testimonials

Better understanding – feedback to cluster org

and policy

Outlines the expected journey – high/low/

emotions

Shows what works well/doesn’t (learning)

5. How would you set cluster goals

Understanding what cluster members deem to

be successful

Setting key objectives and a realistic workplan

What are the critical success factors and the deal

breakers

Ensure shared vision, talk to stakeholders and

align with policy goals

Innovation goals – new products, new services,

renewal of industry, new connections, start ups

Understanding goals and interim goals,

including timings/expectation

Have a range of goals, finance, human etc and

balance quantitative and qualitative goals

37

System Impact

Clear, measurable, SMART (not just activities,

also output, outcome, impact)

Involves everyone who is key to success?

Balance cluster needs and cluster demands

6. How can clusters help the image/ reputation of

a region/ sector

Show strength of a region

Demonstrate evidence of collaboration and its

value

Examples of new joint products which are

relevant globally

Demonstrate diversity and long term thinking

(and investment)

Develop trademark to be respected locally and

regionally

Packaged and communicate success stories

Promote/appoint best innovation award

7. Describe the role of a good facilitator in a

cluster

Encourages participation

Manage conflict and achieves consensus

Drives strategy process and managing project

Commercial focus

Fundraiser

Opens doors / boundary spanner

Good listener

38

Visionary / horizon scanning

Energetic/energise

Power plant on two legs

8. What is ‘unique’ about clusters (compared to

other innovation/ sector interventions)

Bringing together organisations that may have

been competitors

Draw together number of factors/functions on

one/neutral platform

Intersection – between region/sector policy

Boundary spanning between sectors

Bridge building between firms/research small-

small, small-large

Provide forced and un-bureaucratic money

seed funding

39

40

41

42