Measuring Outcomes in Relation to SCP Core Elements

NEWB, Green Street 13th January 2011

Dr. Paul Downes

Director Educational Disadvantage Centre Senior Lecturer Education (Psychology)

St. Patrick’s College

Why discuss this?!

Limits to SMART outcomes (Downes 2007) Wider UN Framework of Structural, Process and Outcome Indicators Structural Indicators: International Conventions and National Strategies Examples from the context of human trafficking Core Structural Indicators for SCP Agreed at National Level? Anticipated Outcomes Approach (Ivers, McLoughlin & Downes 2010) Process Indicators - Examples Previous Critiques of SCP (Downes, Maunsell & Ivers 2006; Downes & Gilligan

2007) Outcomes Wider than Literacy and Numeracy for SCP Aim: An Agreed Language of Goals (SI, PI and OI) for Each of the SCP Core

Elements

Limits to SMART outcomes (Downes 2007)

A range of contexts have been identified where the dominion of SMART outcomes has less relevance

(Downes 2007).

The tendency to overlook background contingent

conditions for the cause to ‘work’

SMART outcomes as behavioural indicators: Limits to behaviouristic assumptions

Beyond simple causality in SMART outcomes to complex causality and a systemic focus

An individual learner-centred focus may be in tension with a generic outcomes focus

The tendency to overlook background contingent conditions for the cause to ‘work’

SMART outcomes provide the temptation to select those with more stable background conditions in order to improve the chances of causal impact of the intervention thereby excluding the most disadvantaged groups from the intervention.

SMART outcomes bring the danger that the most disadvantaged children and families may become filtered out of focus as it is these groups which may be most resistant to measurable gains

SMART outcomes as behavioural indicators: Limits to behaviouristic assumptions

A focus on behavioural indicators may tend to minimise focus on emotional indicators that are less easily measurable and may require a longer time period for change to be manifested.

Behaviourist assumptions of a one to one correspondence between an antecedent (intervention) and a consequent (outcome) have been questioned within behaviourism by Rachlin (1984). This one to one correspondence is oversimplified and requires acceptance of a systemic interaction between a range of antecedents and a range of consequents

Beyond simple causality in SMART outcomes to complex causality and a systemic focus

There is a need for more than single dimensional interventions as part of an integrated intervention strategy. What is called an outcomes focus is often a uni-dimensional outcome focus.

A uni-dimensional outcome focus is in tension with an integrated strategy and integrated interventions focus as they are part of a complex system of interactions.

An individual learner-centred focus may be in tension with a generic outcomes focus

There is a need to start from where the learner is at and a SMART outcomes focus may invite a tendency to impose an agenda on the learner that is not necessarily shared with and owned by the learner.

The learner’s pace may not fit within the limits of the SMART outcome timeframe. Commitment to generic outcomes may be in tension with the disparate starting points of the range of individuals involved in the particular intervention.

SMART outcomes may be overly narrowly focused on individual gains as opposed to gains in community development or family relations.

Wider UN Framework of Structural, Process and Outcome Indicators

The need for wider indicators and benchmarks for social inequality through a focus on structural, process and outcome indicators In the words of the UN Special Rapporteur on the international right to health (2006): ‘54. Structural indicators address whether or not key structures and mechanisms that are necessary for, or conducive to, the realization of the right to health, are in place. They are often (but not always) framed as a question generating a yes/no answer. For example, they may address: the ratification of international treaties that include the right to health; the adoption of national laws and policies that expressly promote and protect the right to health; or the existence of basic institutional mechanisms that facilitate the realization of the right to health…’;

55. ‘Process indicators measure programmes, activities and interventions. They measure, as it were, State effort’; 57 ‘Outcome indicators will often be used in conjunction with benchmarks or targets to measure change over time’. This framework moves beyond the qualitative/quantitative framework as process indicators can be quantified It allows for a more dynamic focus on how a system is changing in relation to indicators – this is necessary for complex, changing environments It provides a focus on a system and subsystems in relation to indicators as benchmarks for progress within a system and subsystem

(See also Downes 2007 and Downes 2008 for applications of these indicators to wider contexts such as socioeconomic exclusion and human trafficking in the Baltic States)

a structural, process and outcomes indicators approach is highly suited to assessing progress at a community

development level, as is recognized by the UN Special Rapporteur on the Right to Health (2005, 2006)

Structural Indicators

International Conventions and National Strategies

UN Convention on the Rights of the Child (1989)

International Right to Health, including Mental Health (2002)

National Children’s Strategy (2000)

National Drugs Strategy (2010)

EU Commission Documents on Lifelong Learning: Includes Goals of Active Citizenship, Personal Fulfillment

Resource materials

Structural Indicators (Downes et al 2008)

Prevention – Structural Indicator

Strict State Monitoring of Standards for Labour Agencies Facilitating Employment Abroad

Prevention – Structural Indicator

Restriction of Advertising of the Sex Industry in Both the State and Private Sphere, including Taxis, Airports, Newspapers – Advertising Arguably Contrary to the UN Convention on the Elimination of All Forms of Discrimination Against Women

Core Structural Indicators for SCP Agreed at National Level ?

Anticipated Outcomes from CASPr (Ivers, McLoughlin & Downes 2010)

After school to modify the impact of poverty (Posner and Vandell

1994; Friel & Conlon 2004)

After school and early school leaving (Posner & Vandell 1994; Hennessy & Donnelly 2005)

After school and social skills (Posner & Vandell 1994; Ivers 2008)

After school and social support for positive mental health in contexts of psychological stress (Levitt 1991; Antonucci 1990, Downes 2004, Downes, Maunsell & Ivers 2006; Downes & Maunsell 2007; Downes 2008)

Afterschool and overcoming fear of failure (Glasser 1969; Warnock 1977; Handy & Aitken 1990; Casby 1997; Kellaghan et al 1995; MacDevitt 1998; Kelly 1999)

After school and positive climate (Halpern, 2000).

After school and self-directed learning (Halpern, 2000).

After school and language development (NESF 2009) After school and safety (Halpern 1999)

After school and the Arts (McNeal 1995, Downes, 2006) After school and supports for parents minding children

(Hennessy and Donnelly 2005, Mulkerrins 2007)

After school and staff quality (Graham 2006, Downes 2006)

Process Indicators

Engagement of marginalized parents (also as an outcome indicator – challenge fatalism, fear of failure (Downes 2003)and anomie)

Process indicators (Downes et al 2008)

Prevention – Process Indicator The Governments of the Estonia, Latvia and Lithuania have focused heavily on

trafficking prevention campaigns in schools. This is important but trafficking is a problem of the whole of society and unfortunately in Estonia and Latvia in particular, a campaign to target the whole of society has often been neglected

Prevention – Process Indicator Links are now made between poverty, education, prostitution and trafficking.

Gender inequality is presented as ‘the second most important factor in creating conditions for prostitution and trafficking’

Prevention – Process Indicator Specific Supports for the Vulnerable Group of Children in Orphanages and

Strategies to Support Orphans after they leave the Orphanage.

Previous Critiques of SCP (Downes, Maunsell & Ivers 2006; Downes &

Gilligan 2007)

Outcomes Wider than Literacy and Numeracy for SCP

Outcome Indicators (Downes et al 2008)

Prosecution - Outcome indicator

Numbers of arrests of traffickers per annum.

Prosecution - Outcome indicator

Average length of sentences of traffickers per annum.

Prosecution - Outcome indicator

Numbers of arrests of users of sex services per annum.

Aim: An Agreed Language of Goals (SI, PI and OI) for Each of the SCP Core

Elements

16 Core Elements

Extra-curricular activities / Sports

After school clubs / homework clubs

Individual support / one-to-one / key work / personal development / group work

Breakfast clubs / lunch clubs / school meals

Counselling / therapies

Summer programmes / educational trips

Learning support / Literacy and Numeracy support

Attendance tracking / monitoring / awards / rewards

Transfer programmes / transition / induction Staff Interagency collaboration / co-operation / liaising /

networking Family support / home visits

Behaviour management / anger management / suspension intervention

Targeting / Prioritisation of young people at risk of early school leaving

Mentoring Transport

REFERENCES: Antonucci, T.C. (1990). Social supports and social relationships. In R.H. Binstock & L.K. George (Eds.) Handbook of Aging and the Social Sciences (3rd Ed). Burlington, MA.: Academic Press. Casby, A. (1997). Making connections: Access to education in Ballyfermot. Ballyfermot Partnership Co. Ltd Downes, P. (2003). Living with heroin: HIV, Identity and Social Exclusion among the Russian-speaking minorities in Estonia and Latvia. English version. Legal Information Centre for Human Rights, Tallinn, Estonia/ Educational Disadvantage Centre, St. Patrick's College, Drumcondra, Dublin. Downes, P. (2004) Present and Future Psychological Support Services for Ballyfermot. Dublin: URBAN Downes, P. (2004a) Voices of Children: St. Raphael’s Primary School, Ballyfermot. Commissioned report for URBAN Ballyfermot Downes, P. (2006) Quality Development of Out-of-School Services: An Agenda for Development. Dublin: QDOSSDownes, P. (2007). Intravenous drug use and HIV in Estonia: Socio-economic integration and development of indicators regarding the right to health for its Russian-speaking population. Liverpool Law Review, Special Issue on Historical and Contemporary Legal Issues on HIV/AIDS, Vol.28, 271-317 Downes, P. (2007). Why SMART outcomes ain’t always so smart… pp.57-69. In Beyond Educational Disadvantage (2007), (P. Downes & A-L Gilligan, Eds.), Institute of Public Administration: Dublin Downes, P. (2008). Mental health strategy for deprived children missing from education plan. Action on Poverty Today No. 21. pp. 4-5 Downes, P. (2009). Prevention of bullying at a systemic level in schools: Movement from cognitive and spatial narratives of diametric opposition to concentric relation. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds). The Handbook of Bullying in Schools: An International Perspective. New York: Routledge Downes, P, Maunsell, C & Ivers, J (2006) A Holistic Approach to Early School Leaving and School Retention in Blanchardstown: Current Issues and Future Steps for Services and Schools. Dublin: BAP Downes, P., Maunsell, C & Ivers, J. (2007) The Jolt between Primary and Post Primary in Downes, P & Gilligan, A.L. (eds) Beyond Educational Disadvantage Dublin: Institute of Public Administration Downes, P & Maunsell, C (2007) Count us in: Tackling early school leaving in South West Inner City Dublin: An integrated response. Commissioned by the South Inner City Community Development Association.

Farrelly, G (2007) Bullying and Social Context: Challenges for Schools in Downes, P and Gilligan, AL (eds) Beyond Educational Disadvantage. Dublin: IPA Fingleton, L. (2003). Listen B 4 I Leave: Early school leavers in the Canal Communities area and their experiences of school. Canal Communities Partnership Ltd. Friel, S. & Conlon, C. (2004). Summary of Food Poverty and Policy. Dublin: Combat Poverty Agency & St. Vincent de Paul Society Glasser, W. (1969). Schools without failure. New York & London: Harper & Row Graham, I (2006). Developing qualifications for school age childcare staff ChildLinks - School Age Programmes - Issue 3. pp. 27-31 Halpern, R (1999). After –school programs for low income children: Promises and challenges. The Future of Children Vol. 9, Pt. 2, pp 81-95 Halpern, R (2000). The promise of after-school programs for low-income children. Early childhood Research Quarterly 15, No. 2, 185-214 Handy, C. & Aitken, R. (1990). Understanding schools as organizations. Harmondsworth: Penguin Books Hennessy, E & Donnelly, M. (2005). After-school in disadvantaged areas: the perspectives of children, parents and experts. Dublin: Combat Poverty Agency [Working Paper Series 05/01] Ivers, J (2008.) Fear of Success among North Inner City Youth. Master’s thesis, Educational Disadvantage Centre, St. Patrick’s College, Drumcondra Ivers, J., McLoughlin, V & Downes, P. (2010). Current Steps and Future Horizons CASPr: Review of CASPr North-East Inner City After Schools Project. Dublin: CASPr Kellaghan, T., Weir, S., O’hUallachain, S. & Morgan, M. (1995). Educational disadvantage in Ireland. Dublin: Department of Education/ Combat Poverty Agency / Education Research Centre Kelly, A.V. (1999). The curriculum: Theory and practice. London: Paul Chapman Publishing Ltd. Levitt. M.J. (1991). Attachment and close relationships: A life-span perspective. In J.L. Gewirtz & W.M. Kurtines (Eds.), Intersections with attachment. Hillsdale, NJ: Erlbaum Lynch, K. and Lodge, A (2002) Equality and Power in Schools: Redistribution, Recognition and Representation. London: Routledge/ Falmer. Mayock, P., Kitching, K & Morgan, M. (2007) Relationships and Sexuality Education in Ireland in the context of social, personal and health education (SPHE): An Assessment of the Challenges to Full Implementation of the Programme. Dublin: Crisis Pregnancy Agency Morgan, M. (2001). Drug use prevention: Overview of research. National Advisory Committee on Drugs. Mulkerrins, D (2007). The transformational potential of the Home School Community Liaison Scheme. In Downes, P & Gilligan A L (Eds), Beyond Educational Disadvantage. Dublin: IPA

MacDevitt, D. (1998). Measures to combat early school-leaving in EU countries. In Educational disadvantage and early school leaving. Dublin: Combat Poverty Agency McNeal, R.B. (1995). Extracurricular activities and high school dropouts. Sociology of Education, 68, 63-81 NESF (National Economic and Social Forum) (2007). Mental health and social inclusion (Forum report no. 36). Dublin: NESF NESF (2009). Child literacy and social inclusion: implementation issues. Dublin: NESF Posner, J K & Vandell, DL. (1994). Low-income children's after-school care: are there beneficial effects of after-school programs? Child Development. 1994; 65 (2 spec no.):440-456 Rachlin, H. (1984). Mental yes. Private no. Behavioral and Brain Sciences, 7, 566-7. Sharkey, S. (2007) Children’s participation in decision‐making: A combined systems theory approach to student councils at primary school level. Unpublished thesis Educational Disadvantage Centre, St. Patrick’s College UNITED NATIONS Economic and Social Council 21st February 2005 COMMISSION ON HUMAN RIGHTS, ECONOMIC, SOCIAL AND CULTURAL RIGHTS. Report submitted by the Special Rapporteur on the right of everyone to the highest attainable standard of physical and mental health, Paul Hunt, MISSION TO ROMANIA UNITED NATIONS Economic and Social Council 3 March 2006 COMMISSION ON HUMAN RIGHTS ECONOMIC, SOCIAL AND CULTURAL RIGHTS. Report of the Special Rapporteur on the right of everyone to the enjoyment of the highest attainable standard of physical and mental health, Paul Hunt UN Convention on the Rights of the Child (1989) OHCHR: Geneva UN International Covenant on Economic Social and Cultural Rights (ICESCR), article 2, para 1 (United Nations, Treaty Series, vol. 993, p. 3) Warnock, M. (1977). Schools of thought. London: Faber. Wehlage, G.G & Rutter, R.A. (1986). Dropping out: how much do schools contribute to the problem? Teachers College Record, 87, 374-392