tracking, monitoring and evaluating
DESCRIPTION
A presentation to Teach First teachers on the 3rd November 2015TRANSCRIPT
www.derby.ac.uk/iCeGS
Tracking, monitoring and evaluating
Why counting is an important part of your school’s career development programme
Tristram Hooley
www.derby.ac.uk/iCeGS
Overview
By the end of this session participants will have:• Considered what information they should be keeping to
support the effective deliver of their school’s career development programme.
• Discusses the difference between tracking, monitoring and evaluation.
• Identified a range of different kind data that they can use to evaluate their careers programme.
• Developed an action plan to improve their school’s use of data.
www.derby.ac.uk/iCeGS
What is tracking?
www.derby.ac.uk/iCeGS
What to track
• Progress• Engagement• Interventions• Decidedness• Follow through• Destination
Which of these do you track?
How do you track them?
www.derby.ac.uk/iCeGS
Approaches to trackingWho How WhenTeachers
Careers lead
All school staff
Admin/temp staff
Notes and card
Databases
School systems
Throughout school
At key decision points
After school
www.derby.ac.uk/iCeGS
Tracking software
• Schools Intelligence http://www.schoolsintelligence.co.uk/
• Pixl Edge http://www.pixl.org.uk/edge
• UCAS Adviser Track https://www.ucas.com/advisers/managing-applications/adviser-track-and-schools-reports-2015-16
• Bespoke system development
www.derby.ac.uk/iCeGS
Approaches to practice
• Integration into personal development planning (e-portfolios)
• Form tutor responsibilities• Central administration and flagging• Career interviews (including group interviews)• Triage processes• Integration with existing school processes e.g. VLE,
school reports• RONIs
www.derby.ac.uk/iCeGS
Your tracking approach
Discuss what you are going to change/develop in your school.
www.derby.ac.uk/iCeGS
Monitoring and evaluation
• Monitoring allows you to check that you are doing what you said that you were going to do.
• Formative evaluation allows to you inform what is being done while the activity is still in progress.
• Summative evaluation creates a summary of what has been achieved and what the impacts have been.
www.derby.ac.uk/iCeGS
Why evaluate?
Evaluation enables us to:
– examine what we do
– think about how we can improve it
– decide on whether it was worth doing
– provide others with a summary to help them to understand what was done.
www.derby.ac.uk/icegs
www.derby.ac.uk/iCeGS
Discussion: Evaluation data
• What data do you have which could allow you to make a judgement about the impact of your practice?– What level of impact would it describe?
• What data could you collect which could allow you to make a judgement about the impact of your practice?– What level of impact would it describe?
www.derby.ac.uk/icegs
www.derby.ac.uk/iCeGS
Key principles
• Granular
• Linkable
• Analysable
What is your counter-factual?
Levels of impactReturn on investment
Results
Behaviour
Learning
Reaction
Take-up
Investment
www.derby.ac.uk/iCeGS
Publish or perish
• Too little evaluation and impact work on careers work is published.
• Writing up your evaluation for broader circulation is an important way to support the development of the sector. • Self publication• Journal publication• Partnership with academics• Using external consultants
www.derby.ac.uk/icegs
www.derby.ac.uk/iCeGS
Write it down
“My main reason for writing is simple: I do not know what I think until I have written it. In conversation one can get away with loose, exploratory thinking, but in writing it down one has to weigh up the arguments and the evidence, and decide what it all means and where one stands. It is hard work, but important; and if published, it adds to the body of knowledge on which others can draw.” Tony Watts
www.derby.ac.uk/iCeGS
On working with academics and researchers• What matters to them
– Time– Money– Authorship– Impact
• What you can give them– Your time– Funding– Authorship or co-authorship– Data– Access to interventions and research populations– The change to impact on practice
www.derby.ac.uk/iCeGS
Your monitoring and evaluation approach• Discuss what you are going to change/develop in your
school.
www.derby.ac.uk/iCeGS
Useful resources
• Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE.
• Hooley, T. (2014). The Evidence Base on Lifelong Guidance. Jyväskylä, Finland: European Lifelong Guidance Policy Network (ELGPN).
• Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic.
• Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU.Hughes, D., Lang C. and Popham I. (2001).
• Taylor, A.R. & Hooley, T. (2014). Evaluating the impact of career management skills module and internship programme within a university business school. British Journal of Guidance & Counselling, 42(5): 487-499.
www.derby.ac.uk/iCeGS
About me
Tristram HooleyProfessor of Career EducationUniversity of Derbywww.derby.ac.uk/icegs
[email protected] @pigironjoe https://adventuresincareerdevelopment.wordpress.com/