Introducing Undergraduate Mentors (UMs) in Year 1 Courses

Rachel Ferris, Joy Moloney, Jim Anderson, Jim Andrews, Rachel Mills26 June 2008

The Project Team – who we are

Dr Rachel Ferris & Mrs Joy Moloney (LTCs), Faculty of Engineering, Science and Mathematics

Dr Jim Anderson (DHoSE), School of MathematicsDr Jim Andrews (Senior Tutor) & Dr Rachel Mills (DHoSE), School of

Ocean & Earth Science (SOES)

In 2006-07, awarded funding from the Learning & Teaching Enhancement Fund for implementing project in 2007-08

Introduction• Undergraduate mentoring is a work relationship where

students with a proven set of knowledge and skills, help other students to fully understand and apply course content.

• Our rationale for using undergraduates is:

• There is a larger cohort of UGs (more assistance)

• UMs will improve their own skills

• It will incentivise UGs (only the best are UMs)

• It will enhance UMs CVs (employability)

• It will provide income for UGs

• It may reduce staff workload

Aims & Outcomes• Aims: To ‘pilot’ employing trained UMs (3rd/4th year

UGs) in some Maths & SOES first year courses

• Outcomes of the study enabled UMs to:• describe their role and state the most important

skills/qualities required to be an effective UM• reflect upon their development in the role

• Outcomes included evaluation by UMs & participants

• Broader Outcomes: to embed the model in other courses

Mentored Course UnitsYear 1 courses employing UMs

MATH1046 Consolidation Class (Voluntary) – core modules alongside were Calculus, Linear Algebra 1, Elements of Pure Maths and Number Theory and Cryptography (Semester 1) – c. 186 students in year 1

SOES1002 The Dynamic Earth (Semester 2)

SOES1005 Introduction to Ocean Biogeochemistry (Semester 1)

SOES1010 Quantitative Earth & Ocean Science (Semester 1)

Project Timescales

• May/June ‘07: Advertised & selected 3rd/4th year UM students & PGR trainers

• September ‘07: Wrote UM Resources Booklet; Planned Training session

• October ’07: Week 0, UMs trained by PGR trainers

• October ’07 – June ’08: UMs employed

• March ’08: Evaluation for School of Maths

• September ’08: Completion

Mentor Selection• Advert: 300 words on ‘why they wanted to

be a UM’, ‘the skills they could bring to the job’, ‘the challenges the job would bring’ & a ‘summary of grades’ & any ‘work experience in education’

• Prerequisite: Attend UM Training

• Prerequisite: 1st/2i marks in relevant courses

• UMs employed: 21 SOES, 15 MATHS

• Payment: £12.70/hr

4th October, 2007

Time (p.m.) Topic

1.00 Welcome

1.10 Card Sort exercise on UM role

2.00 Learning Styles

2.30 Listening Skills

3.30 Break

3.45 Feedback Skills

4.15 Asking Questions

5.15 Dealing with challenging students

6.00 Feedback on session

UM Training Programme

Evaluation

1. Selection of open (comparative analysis) and closed (Likert-scale) questions

2. Quantitative & qualitative evaluation of Training Day

3. Written feedback from UMs after each UM session; focus group – verbal comments

4. Written feedback via email questionnaire from UMs and mentees at the end of the pilot

Training Day Evaluation (n = 36) A B To what extent do you feel that your understanding

of the following key principles has been developed? C

Content: Relevance to future role within the University

The UG mentor role

Learning Styles

Key principles of listening

Key principles of effective feedback

Key principles of questioning

Dealing with challenging students

How well do you feel the induction was delivered?

Summary statistics

Responses Mean 3.89 3.64 4.14 4.03 4.19 3.89 3.89 4.11 Mode 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00

Training Day Quantitative Evaluation

Likert-Style: e.g. 5 (‘Very Good’) to 1(‘Very Poor’)

Training DaySnapshot of Comments

• “Practical Sessions were really good”

• “Was good learning about different learning skills”

• “Would be very good to know exact modules we’ll be working with”

• “Confidence in tackling difficult scenarios; liked the fact that the training was engaging and involved everyone”

Qualitative Evaluation(From MATHS UMs weekly sessions)

a) In terms of a positive experience, most common remarks centred around help and communication:

• “Students conversed amongst themselves to help each other”• “The students in my group are willing to ask questions”• “Explained/helped students to understand/solve Maths problems”• “I was able to answer questions e.g. proof, logic, eigen vectors”

b) To improve upon the session, most common remarks:• “See tutorial sheets before the session”• “I should have prepared better”• “Better seating arrangements”• “Encourage group to work together”

Qualitative Evaluation Cont …c) What did UMs learn? Most common remarks centred

around the approach to learning:• “Assume nothing, start from first principles”; “Good to think on my feet”• “Its hard to explain simple ideas – I never struggled understanding”• “Group discussion – students helped each other”• “Use different teaching styles”

d) What did mentees learn? Most common remarks: a better understanding of subject/how to solve problems:

• “Understanding basic vectors”; “Thinking outside the box”• “Increased confidence with their own abilities!”; • “A new way of solving partial fractions”• “Learnt to break down problems into smaller chunks”• “Improved communication skills with other members in the group”• “Different methods of learning i.e. parrot fashion/understanding”

Qualitative Evaluation Cont …e) Plans for the next peer meeting, most common

remarks related to helping / explaining concepts

• “Continuation of addressing student problems”

• “Next week’s tutorial sheets”

Early Shared Findings•‘Timetable clashes’ & ‘location’ a problem (i.e. SOES mentors getting to Highfield)

•Maths UMs wanted tutorial sheets / answers in advance; more space / movable seating in room

•Good attendance by UMs & mentees keen: c. 1/3 of large 1st year cohort (c. 186) at Consolidation Classes

End of course: Views of Mentors on their role in MATH1046 (n=12)

0

10

20

30

40

50

60

70

80

StronglyAgree

Agree Disagree StronglyDisagree

Response Category

Perc

en

tag

e (

%)

of

stu

den

ts Helped students to become more familiarwith key course concepts

Provided guidance and encouragement forstudents to work out the problem, rather thangiving answers

Helped students to be prepared forforthcoming assignments

Having taken the courses myself, I hadgreater empathy for the first year experience

Provided a friendly environment to askquestions and discuss topics of concern

Helped to develop the students confidencefor the subject

Helped to achieve active participation andinteraction within the group

Development of my own study skills (e.g. timemanagement)

Reinforced my own learning of the subject

End of MATH1046 Key Messages from UG Mentors

• Positive benefits – “improving communications / skills / ways of solving problems”

• Most challenging - “thinking on my feet / on top of first year material”

• Changes needed – “provide solutions to UMs before session / advertise scheme widely”

• Least useful – “if UM lacked subject knowledge”• Other – “Scheme should continue next year”

End of Course: Views of Mentees on MATH1046 UM sessions

0

10

20

30

40

50

60

70

80

90

Strongly Agree Agree Disagree Strongly DisagreeResponse Category

Perc

en

tag

e (

%)

of

stu

den

ts

Improved my understanding of subject materialImproved my preparation for assignmentsImproved my marks in assignmentsI could clarify difficult concepts in a friendly settingDeveloped my confidence in the materialAn opportunity to get academic support other than from teaching staffDevelopment of my study skills (e.g. time management, revision techniques)Support gained that met my individual needs

End of MATH1046 Key Messages from UG Mentees

• Attendance – “Many UGs attended more than 5 sessions”

• Primary motivation - “to improve grades & confidence”

• Most useful – “lots of help/discussions”• Least useful – “if UM lacked subject knowledge”• To improve – “UMs see problem sheets prior to

session” (MATHS)• Most help – “linear algebra / pure maths”• Other – “very useful class, worth carrying on”

Improved myunderstanding of subjectmaterialImproved my preparationfor assignments

Improved my marks inassignments

I could clarify difficultconcepts in a friendlysettingAn opportunity to getacademic support otherthan from teaching staff

Helped students tobecome morefamiliar with keycourse concepts

Provided a friendlyenvironment to askquestions anddiscuss topics ofconcernReinforced my ownlearning of thesubject

Statements with 100% Agreement

From UG Mentees From UG Mentors

Summary Statistics – Linear algebra Category Average (± se) % difference in marks between T1 and T2

Average (± se) % difference in marks between DQ and T1

Average (± se) % difference in marks between DQ and T2

Whole Class

-16.9 (± 0.1)

+18.6 (± 0.2)

+1.7 (± 0.2)

Not Mentored

-17.3 (± 0.1)

+16.6 (± 0.1)

-0.7 (± 0.1)

Mentored -15.5 (± 0.1)

+26.1 (± 0.8)

+10.6 (± 0.7)

≤60/100 on DQ

-19.1 (± 0.4)

+35.0 (± 0.9)

+15.9 (± 0.5)

DQ = diagnostic quiz; T1 = test 1; T2 = test 2; n – varied.

A Brief Look at Assessment

Homework Marks - Calculus Course

0.0

20.0

40.0

60.0

80.0

100.0

120.0

31-Oct 07-Nov 14-Nov 21-Nov 05-Nov 12-Dec 09-Jan 23-Jan

Date

Pe

rce

nta

ge

(%

)

Whole Class Not Mentored Mentored ≤60/100 on DQ

Average (±SD) marks for a Calculus course; DQ = diagnostic quiz; n - varied.

Any Coursework Trends?

Our Learning, Teaching & Enhancement Strategy

The model supports key themes:• Student Centredness: e.g. well-timed feedback

• Employability: UMs developed skills

• Development of the infrastructure to support education: UM role – extra educational layer

Conclusions & the Future• Major Successes/Learning:

• First years & UMs value the model; assists staff where student numbers are high

• Role reinforced UMs own learning and skills• Provided Mentees with a friendly environment to discuss

concepts and to develop subject confidence and understanding

• Changes:• In SOES, timetabling/location resolved for 2008-09• At training, both Schools provide information on modules • Reconsider payment, as generous

• The Future:• Both Schools are running schemes next year• Both Schools will train more mentors• Hope to expand the model in the University

References

• http://www.southampton.ac.uk/lateu/individuals/2007projects.html

• The project team thanks the University of Southampton for the financial support, the module convenors who involved their units in this study, our UM trainers and all our mentors.

Acknowledgements

Any Questions?