1

Workshop 3

Revealing the causal chain

Laura PolverariEPRC

2

Three papers on Theory-Driven Evaluation: distinct but complementary• "Making sound judgements about the effects of

public policies: how to explore the missing links between cause and effect?" (Petri Uusikylä, Net Effect Ltd., Finland)

• "Theory driven evaluation: tracing links between assumptions and effects" (Karol Olejniczak, Warsaw University, Poland)

• "Systems constellations in theory based evaluations – tools and experiences“ (André Martinuzzi, Vienna University of Economics and Business, Austria)

3

Key messages (from papers)• TDE can be a useful approach for evaluating Cohesion

policy programmes:– Makes programme hypotheses explicit (and the potential

weaknesses of these) useful for complex, composite OPs– Clarity of research problem: verification of programme hypotheses– Aims to explain why objectives have been achieved (or haven’t)– Focuses on effects: implementation one explanatory factor – Links programme evaluation with wider theories, contexts, actors’

behaviours and path dependence– Encourages methodological pluralism and a participative approach

• Variety of methodologies available (examples in Uusikylä)• Awareness of some shortcomings: over-simplification,

generalisation, level of abstraction, deadweight neglect

4

Key messages (from discussion)• Conceptual limitations of TDE

– Performance not necessarily linked to soundness of programme theory

– Need to look at unintended effects– May help justify under-performing programmes?– Inward orientation (programmes ‘in isolation’ from other policies)

other policies may displace Cohesion policy• Evaluator’s role

– Should he/she make programme theory explicit?– As policy developer– As ‘bridge’ between policy-makers and stakeholders

• Risk of evaluation becoming self-referential • Need to adapt TDE to evolving Cohesion policy context outwith large

Convergence MS programmes area) Part of a ‘bigger picture’ need to integrate evaluation of Cohesion

policy programmes with that of domestic policies under which they are subsumed

b) More narrowly focussed potential to undertake ‘segmented’ rather than programme-wide evaluations (smaller thematic studies)