campus quality assurance office (office of the principal) july 2013 first impressions, early...
TRANSCRIPT
Campus Quality Assurance Office(Office of the Principal)July 2013
First impressions, early opinions & evidence-based interventions:
The UWI Cave Hill CampusNEW STUDENT SURVEY
The University of the West Indies, Cave Hill Campus
2
OBJECTIVES
Campus Quality Assurance Office
By the end of this session, you will be able to:
Outline the aims and methodology of New Student Survey Describe the process of survey administration (including questionnaire
development) and the approach to reporting Assess the usefulness of the New Student Survey for decision-making Identify opportunities for improving the design and administration of the
survey
The UWI Cave Hill New Student Survey
3
ABOUT THE SURVEY
Campus Quality Assurance Office
Background & Objectives
Introduced in September 2011 Designed to:
— Explore new students’ early impressions and opinions of Cave Hill— Assess new students’ satisfaction with processes and amenities
related to joining the Campus community— Enable decision-making re: future provision of information, training,
services, amenities related to the application and orientation period
The UWI Cave Hill New Student Survey
4
ABOUT THE SURVEY
Campus Quality Assurance Office
Methodology
Online survey (administered through ) Duration: 3-4 weeks Repeated annually Target population: all first year undergraduate and graduate students ID login required Self-selecting sample Incentivised: $500 University Bookshop voucher (Office of the Principal)
The UWI Cave Hill New Student Survey
5
ABOUT THE SURVEY
Campus Quality Assurance Office
Methodology
The UWI Cave Hill New Student Survey
6
ABOUT THE SURVEY
Campus Quality Assurance Office
Administration
Questionnaire development, programming and testing Contact list
— Key fields: Student ID, Name, Gender, Age, Full time/Part time, Domicile, Nationality, Level of study, Major
— Timing: 1st run at end of orientation week, re-run every week until survey closes
Approval of incentive, invitation letter and reminder from Principal Promotional plan Prize distribution
The UWI Cave Hill New Student Survey
7
THE QUESTIONNAIRE
Campus Quality Assurance Office
Development Two questionnaires: graduate and undergraduate (minor differences) Drafted by Campus Quality Assurance Office Amended based on feedback from:
— Faculties— Campus IT Services— Admissions— School for Graduate Studies &
Research— Main Library— Law Library
— Office of Student Services— Bursary— Accommodation Office— International Office— Marketing Office— University Bookshop
The UWI Cave Hill New Student Survey
8
THE QUESTIONNAIRE
Campus Quality Assurance Office
Development Eight (8) sections Follows the application, orientation, registration process:
— Your background and application process— Offer period— Orientation— Paying your fees— Registration – selecting your courses— IT Services— Accommodation— The University Bookshop
Uses routing/filters Save responses; resume later
The UWI Cave Hill New Student Survey
9
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments Amended annually based on:
— Requests from departments— Campus developments— Implementation updates— Previous year’s analysis
Most questions repeated to enable year-on-year comparison
The UWI Cave Hill New Student Survey
10
THE QUESTIONNAIRE
Campus Quality Assurance Office
Year-on-year comparisons
The UWI Cave Hill New Student Survey
11
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments – Campus developments ‘Graduate Information Session’ held in March 2013, replacing Graduate
Fair (only open to 3rd year undergraduates)
The UWI Cave Hill New Student Survey
12
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments – Previous year’s analysis Include question on reasons for non-attendance at orientation events such
as Main Library orientation session, Matriculation Ceremony, Orientation Fair.
The UWI Cave Hill New Student Survey
MAIN LIBRARY UNDERGRADUATE REPORT Orientation events 1. Attendance at the Matriculation Ceremony and
Orientation Fair have increased (from 30% and 39%, respectively to 45% and 44%, respectively), suggesting that efforts to heighten awareness (for example by the Office of Student Services) are bearing fruit. However, participation in the Main Library Orientation session has declined. It may be worth investigating the level of awareness of this event, in particular.
Implementation update Choose an item. Other Comments: Click here to enter text.
13
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments – Requests from departments Request from Head of Department:
The UWI Cave Hill New Student Survey
14
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments – Requests from departments Revised questionnaire:
The UWI Cave Hill New Student Survey
15
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments - Implementation updates Add question for more detailed insights
The UWI Cave Hill New Student Survey
16
THE QUESTIONNAIRE
Campus Quality Assurance Office
Adjustments - Implementation updates Additional open-ended question:
The UWI Cave Hill New Student Survey
17
SURVEY PROMOTION
Campus Quality Assurance Office
2012-13
Principal’s invitation via email to MyCaveHillMail inbox (invitation and
weekly reminders) Links published at www.cavehill.uwi.edu/newstudentsurvey (ID log in
required) Electronic banners: Cave Hill main website spotlight; Facebook page;
Student Portal; Graduate Studies website. Posters at points of service delivery Notices during First Year Experience workshops
The UWI Cave Hill New Student Survey
18
SURVEY PROMOTION
Campus Quality Assurance Office
2012-13
The UWI Cave Hill New Student Survey
19
SURVEY PROMOTION
Campus Quality Assurance Office
2013-14
More attractive incentive (corporate sponsorship?) Flyer in New Student’s Handbook Notices during Orientation Fair, Guild events Notices during Orientation Fora
The UWI Cave Hill New Student Survey
20
RESULTS & ANALYSIS
Campus Quality Assurance Office
Response rates
The UWI Cave Hill New Student Survey
Year Undergraduate Graduate
2011-12 22% 30%
423 students 101 students
2012-13 22% 26%
368 students 93 students
21
RESULTS & ANALYSIS
Campus Quality Assurance Office
Representativeness
The UWI Cave Hill New Student Survey
22
RESULTS & ANALYSIS
Campus Quality Assurance Office
Reporting approach
Objectives: accurate, timely, actionable, user-friendly Reports designed to aid:
― Reading: executive summary, full-colour, bookmarks (main contents and appendices), text plus graphics
― Decision-making: separate undergraduate and graduate reports, structure, chapter summaries, year-on-year comparisons, cross-tabulations, percentages and absolute numbers, coding and statistical analysis of open-ended responses, appendices, ‘sign-posted’ & evidence-linked Conclusions & Recommendations
― Action and follow-up: itemised Conclusions & Recommendations
The UWI Cave Hill New Student Survey
23
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Reporting level Separate undergraduate and graduate reports
The UWI Cave Hill New Student Survey
24
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Report structure Perspective of decision-makers
The UWI Cave Hill New Student Survey
25
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Chapter summaries Focus attention on key findings
The UWI Cave Hill New Student Survey
26
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Year-on-year comparisons Decision-making in context
The UWI Cave Hill New Student Survey
27
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Cross-tabulations by Faculty More precise
decision-making Decisions in context
The UWI Cave Hill New Student Survey
28
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Percentages and absolute numbers Decisions in context
The UWI Cave Hill New Student Survey
29
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Open-ended responses – coding and analysis Priority for action based on statistics
The UWI Cave Hill New Student Survey
30
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Appendices – Students’ comments & suggestions Detailed insights; specific suggestions
The UWI Cave Hill New Student Survey
31
USEFULNESS FOR DECISION-MAKING
Campus Quality Assurance Office
Conclusions & Recommendations Evidence-linked, actionable recommendations. Sign-posted call to action
The UWI Cave Hill New Student Survey
32
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Follow-up process
June: CQAO extracts report recommendations and relevant survey sections for each participating department. Prepares templates.
June/July: CQAO writes to Heads of Department/Deans for:― Requests for amendments to questionnaire― Updates on implementation of recommendations (updates by email or by telephone, with email confirmation)
July: CQAO prepares summary report to Principal(presented for noting at Academic Board in September)
The UWI Cave Hill New Student Survey
33
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Follow-up process
The UWI Cave Hill New Student Survey
34
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Targeting prospective students
Findings:
Conclusion / Recommendation:
“Promotional channels used by UWI to reach prospective students are effective ..The exception is the teleconference information session for regional students… It may be worth re-assessing the value of this session and/or its current format, particularly in terms of its ability to deliver information that is clear and easily accessible.”
Relevant to: Marketing
The UWI Cave Hill New Student Survey
35
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Targeting prospective students
Outcome:
The UWI Cave Hill New Student Survey
In 2012, the Marketing Office revised the format of the teleconference session by hosting different countries/regions on
separate days in an effort to better facilitate questions from participants. It planned to further segment sessions for the OECS
countries in 2013.
36
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Application experience
Findings:
Conclusion / Recommendation:
“Applicants would also welcome clearer guidance on the overall process and clearer instructions on completing the form. As such, recommendations for adding FAQs, online tutorials and sample forms should be seriously considered.”
Relevant to: Admissions, Campus IT Services
The UWI Cave Hill New Student Survey
37
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Application experience
Outcome:
The UWI Cave Hill New Student Survey
CITS confirmed that FAQs were added to the Admissions website.
38
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Orientation experience
Findings:
Conclusion / Recommendation:
“There may be scope for greater effort to encourage and facilitate higher participation in general (i.e. non faculty-specific) orientation events, since, with the exception of the main undergraduate orientation session, these attract less than half of their target audience.”
Relevant to: Office of Student Services, Main Library
The UWI Cave Hill New Student Survey
39
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Orientation experience
Outcome:
The UWI Cave Hill New Student Survey
The Office of Student Services (OSS) confirmed plans to promote other departments’ services during the main orientation session and through student volunteers who interact with students. In 2012, the
OSS also convened a multi-disciplinary Orientation Planning Committee in an effort to achieve a more integrated approach to
that year’s activities.
40
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Academic advice
Findings:
Conclusion / Recommendation:
“Participation in these sessions in general seems relatively low (66%) and appears to be especially low at the faculty of Law. Individual faculties may need to assess their arrangements for delivering academic advice, to ensure, in particular, that the Campus meets its commitment to offering this level of guidance and support to students. “
Relevant to: All Faculties
The UWI Cave Hill New Student Survey
41
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Academic advice
Outcome:
The UWI Cave Hill New Student Survey
The Faculty of Science & Technology confirmed continuing efforts to increase student participation rates but noted the challenge of ensuring that all staff
meet their commitments in this regard.
The Faculty of Law confirmed plans to assign advisers to students earlier than in previous years and actively encourage students to participate.
The Faculty of Medical Sciences met with the Medical Students’ Association re: earlier assignment of advisors and intensified promotion of the exercise
among both staff and students.
42
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: IT services
Findings:
Conclusion / Recommendation:
“There is scope to augment usage of and/or sign-up to some of the newer IT facilities designed to enhance students’ access to online university services, especially the Self-Service Password Reset service.”
Relevant to: CITS
The UWI Cave Hill New Student Survey
43
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: IT services
Outcome:
The UWI Cave Hill New Student Survey
CITS confirmed that it was investigating alternative Password Reset solutions, including a web version of the current service, which
permits off-campus access.
44
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Accommodation services
Findings:
Conclusion / Recommendation:
“The off-campus accommodation support service is largely under-subscribed. As such, it may be worthwhile to assess whether this is sufficiently advertised and, if not, to explore options for boosting awareness.”
Relevant to: Accommodation Office
The UWI Cave Hill New Student Survey
45
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Accommodation services
Outcome:
The UWI Cave Hill New Student Survey
The Accommodation Office confirmed plans for the Accommodations Assistant to intensify promotion of the service at the orientation session. He also continues to deliver five-minute
presentations on his services at the AGMs of student island associations.
46
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Bookshop services
Findings:
Conclusion / Recommendation:
“Recommendations such as separate queues for different modes of payment or types of goods should be reviewed for feasibility and implemented, where appropriate and practicable. Possibilities of adding more cashiers, even if on a temporary basis during peak times, should also be seriously explored.
Relevant to: University Bookshop
The UWI Cave Hill New Student Survey
47
EVIDENCE-BASED INTERVENTIONS
Campus Quality Assurance Office
Service enhancements: Bookshop services
Outcome:
The UWI Cave Hill New Student Survey
The Bookshop confirmed plans to introduce two additional cash registers at peak time and to implement an express line.
48
LOOKING FORWARD …
Campus Quality Assurance Office
Survey limitations
Delayed assessment: service during application Separate reports for some non-differentiated services e.g. paying fees,
Bookshop, IT services Limited feedback from departments on usefulness
The UWI Cave Hill New Student Survey
49
LOOKING FORWARD …
Campus Quality Assurance Office
Opportunities for improvement
Additional cross-tabulations and regression analysis for deeper insights, esp. re: Marketing information.
Optimise formatting for use on tablets and / or publish in html format. Detailed contents Survey link available via e-learning
The UWI Cave Hill New Student Survey