ontario’s experience in ahelo’s civil engineering strand
DESCRIPTION
Ontario’s experience in AHELO’s Civil Engineering Strand. Presented by: Mary Catharine Lennon Senior Research Analyst Higher Education Quality Council of Ontario Presented to:State Higher Education Executive Authority August 9, 2012 Chicago, Illinois. - PowerPoint PPT PresentationTRANSCRIPT
Presented by: Mary Catharine LennonSenior Research AnalystHigher Education Quality Council of Ontario
Presented to: State Higher Education Executive AuthorityAugust 9, 2012Chicago, Illinois
Ontario’s experience in AHELO’s Civil Engineering Strand
Informing the Future of Higher Education
2
Canada is 1 of 9 countries participating in EngineeringAustraliaCanadaColumbiaEgyptJapanMexicoRussiaSlovakiaUnited Arab Emirates
Informing the Future of Higher Education
3
HEQCO is implementing AHELO on behalf of MTCU
Informing the Future of Higher Education
Council of Ministers of Education (CMEC) is the Canadian representative to OECD’s Education
Branch
Ministry of Training, Colleges and Universities (MTCU) is a member of
CMEC
HEQCO is an agency of MTCU
What is HEQCO?
“The objective of the Council is to assist the Minister in improving all aspects of the postsecondary sector, including improving the quality of education provided in the sector, access to post-secondary education and accountability of post-secondary institutions.”
5
Ontario activities timeline
Informing the Future of Higher Education
July 2011
HEQCO joined project
October
Recruited institutions
November
Hired lead scorer
December
Vetted and adapted the documents
January 2012
IC training and ethics
February
Recruitment, IT
preparations and TA training
March
Student testing
April
Faculty and institutional
survey collection
May
Scoring
6
AHELO’s Civil Engineering strand• Pilot managed by the Australian Council on
Educational Research (ACER)• Expected learning outcomes were developed
through Tuning process• Assessment tool developed by ACER, Japan’s
National Institute for Educational Policy Research (NIER), the European and Global Engineering Education network (EUGENE), in collaboration with international Engineering education experts
Informing the Future of Higher Education
7
The Engineering Test frameworkUnderstanding and application of design methodologies to meet requirements
Practical competencies required to solve problems, conducting investigations, and designing engineering devices and processes. Covers non-technical elements of civil engineering practice like professional ethics, responsibilities and the impact of engineering solutions in a global economic, societal and environmental context
Using analytical methods to identify, formulate and solve problems
Effective communication and awareness of the wider civil engineering context
Knowledge and understanding of underlying scientific and mathematical principles – general sciences; materials and construction; structural engineering; geotechnical engineering; hydraulic engineering and urban and rural planning
Informing the Future of Higher Education
Engineering design
Engineering practice
Engineering analysis
Generic engineering
skills
Basic engineering
skills
8
The test• Students are provided 2 of 3 performance
tasks, each requiring 30 minutes to complete• Students are provided 20 of 40 multiple choice
questions, together requiring 30 minutes to complete
• Students complete the context survey
Informing the Future of Higher Education
9
How it works
• Online tests and contextual questionnaires are completed by students submitted directly to ACER for evaluation
• Short written responses are graded by trained Ontario-based Civil Engineering faculty and submitted to ACER
• All analysis of Civil Engineering strand occurs at ACER
• ACER provides result of Engineering strand to OECD
Informing the Future of Higher Education
10
9 of 10 Ontario universities participated
Informing the Future of Higher Education
11
Student recruitment• Having specific and small
population simplified recruitment
Recruitment examples: emails, websites, classroom announcements, meetings of class and civil engineering student society representatives
Informing the Future of Higher Education
12
Student incentivesLimited funds and lots of creativity!
Incentive examples: class pictures, voucher for formal end of term dinner, donation to the engineering societies, individual gift cards, increased amount on gift card if certain percentage of participation achieved, food after exam, raffles for ipad3
Informing the Future of Higher Education
13
The resultsOntario Institutions Students who wrote
test/populationFaculty who completed survey/population
Institution 1 61/77 = 79.2% 2/18 = 11.1%
Institution 2 23/34 = 67.6% 18/20 = 90%
Institution 3 87/137 = 63.5% 17/18 = 94.4%
Institution 4 57/100 = 57% 16/18 = 88.9%
Institution 5 65/114 = 57% 25/34 = 73.5%
Institution 6 43/90 = 47.8% 27/40 = 67.5%
Institution 7 45/77 = 58.4% 19/32 = 59.4%
Institution 8 26/44 = 59.1% 15/18 = 83.3%
Institution 9 35/56 = 62.5% * 16/17 = 94.1%
Total 442/729 = 60.6 155/215 = 72%
Informing the Future of Higher Education
*Note: Institution 9 had 62.5% of their students write the test, but only 5.4% were electronically captured, altering the final numbers for analysis to 56%*
14
AHELO Transmission of Results
• ‘Institution Report’: Institutions receive a report detailing the performance of the cohort of students from their individual institution
• ‘Jurisdictional report’: will outline the performance of Ontario’s Civil Engineering programs
• ‘Inter-jurisdictional report’: Australia and Canada have agreed to share data for comparative purposes
• ‘International Report’: The OECD will provide a final report reviewing the feasibility study
Informing the Future of Higher Education
15
Challenges and successesChallenges Successes
Informing the Future of Higher Education
• Institutional Research Ethics Board approval made it data linking impossible.
• Technical issues at one institution
• Considerable ‘buy-in’ from all levels
• Discipline-strand simplified sampling and recruitment
• Weekly Institutional Coordinator teleconferences were extremely useful
Lessons Learned• Get to institutions early in the academic year• Allow time for Ethics approval• Ensure adequate IT support