a longitudinal study of the abet outcomes-based
TRANSCRIPT
![Page 1: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/1.jpg)
A Longitudinal Study
of the
ABET Outcomes-Based Accreditation
Model
INQAAHE Madrid 2011
![Page 2: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/2.jpg)
Outline of the Presentation
• Context– What was the Impetus for Change?
– What was the Change?
– What is the Impact - The Longitudinal Study?
• Key findings of the Longitudinal Study
− Program Changes
− Student Experiences
− Student Learning Outcomes
• Conclusions and Implications− The good and the opportunities for improvement
2
![Page 3: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/3.jpg)
Impetus for Change
Circa 1990…
• Industry seeks graduates with quality technical AND professional skills, but dense accreditation criteria leave programs no room cover both.
• Deans attempt to innovate engineering education to meet the needs of industry, but face brick wall in prescriptive accreditation criteria.
• ABET adopts new leadership philosophy and strives to ensure quality AND stimulate innovation in engineering education.
![Page 4: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/4.jpg)
New Philosophy
• Institutions and programs define mission and objectives to meet the needs of their constituents – enables program differentiation.
• Emphasis on outcomes – What students learn, less on what they were taught.
• Programs demonstrate how criteria and educational objectives are being met
• Practice of Continuous Quality Improvement– Input from Constituencies
– Process focus
– Outcomes and Assessment Linked to Objectives
![Page 5: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/5.jpg)
Basic Level Criteria
1. Students
2. Program Educational Objectives
3. Program Outcomes and Assessment
4. Professional Component
5. Faculty
6. Facilities
7. Institutional Support & Financial
Resources
8. Program Criteria
9
5
![Page 6: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/6.jpg)
Student Learning OutcomesEC2000: Criterion 3, a-k
a. An ability to apply knowledge of mathematics, science, and engineering
b. An ability to design and conduct experiments, as well as to analyze and interpret data
c. An ability to design a system, component, or process to meet desired needs
d. An ability to function on multi-disciplinary teams
e. An ability to identify, formulate and solve engineering problems
f. An understanding of professional and ethical responsibility
g. An ability to communicate effectively
h. The broad education necessary to understand the impact of engineering solutions in a global and societal context
i. A recognition of the need for and an ability to engage in life-long learning
j. A knowledge of contemporary issues
k. An ability to use the techniques, skills and modern engineering tools necessary for engineering practice
→ A twelfth outcome, “ability to manage a project,” was added to the research study because it is frequently mentioned in the literature on engineering education 6
![Page 7: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/7.jpg)
Tracking Student Learning Outcomes
0
1
2
3
4
5
1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012
Year
Gra
du
ati
ng
Se
nio
rs' P
rep
ara
tio
n
= 1994: Benchmark (Pre-EC2000)
1998: Implementation of EC2000
2001: EC2000 Mandatory
= 2004: Engineering Change Study
(1st Post-EC2000 Data Point)
7
![Page 8: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/8.jpg)
Key Questions
1. What impact, if any, has EC2000 had on
graduating seniors’ preparation to enter the
engineering profession?
2. What impact, if any, has EC2000 had on
practices that may be related to changes in
student preparation?
8
![Page 9: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/9.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
EC2000Outcomes-
Based
OUTCOMES
Continuous Quality
Improvement
Employer
Ratings
Student
Learning
(3.a-k)
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
STUDENT
EXPERIENCES
In-
Class
Out-of-
Class
![Page 10: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/10.jpg)
Data Sources and Response Rates
Data Sources
Target
Population
Number of
Responses
Response
Rate
Programs 203 147 72%
Faculty 2,971 1,243 42%
Deans 40 40+ 98%
1994 Graduates
(Pre-)13,054 5,494 42%
2004 Graduates
(Post-)12,921 4,330 34%
Employers unknown 1,622 N/A 10
![Page 11: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/11.jpg)
Arizona State University
Case Western
Clemson University
Cornell University
Georgia Tech
Howard University
Illinois Institute of Tech.
Iowa State University
Lehigh University
Marquette University
MIT
Ohio State University
Princeton University
Purdue University
Syracuse University
Temple University
Texas A&M University
University of Arkansas
UCLA
University of Florida
University of Illinois, Chicago
University of Michigan
University of Missouri, Columbia
University of Notre Dame
University of Texas, Arlington
University of Texas, Austin
University of the Pacific
Virginia Tech
Western Michigan University
Worcester Polytechnic Institute
Participating Institutions: Doctoral
11
![Page 12: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/12.jpg)
Participating Institutions:
Master’s and Bachelor’s
Bachelor’s and Others
South Dakota School of Mines
Tri-State University
Union College
United States Military Academy
Master’s
Cal. Polytechnic, Pomona
Cal. State, Sacramento
Embry-Riddle, Daytona
North Carolina A & T
Tuskegee University
Youngstown State Univ.
12
![Page 13: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/13.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
EC2000Outcomes-
Based
OUTCOMES
Continuous Quality
Improvement
Employer
Ratings
Student
Learning
(3.a-k)
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
STUDENT
EXPERIENCES
In-
Class
Out-of-
Class
![Page 14: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/14.jpg)
Significant Findings:
Curriculum and Instruction at Course Level
Faculty report:
• Increased emphasis on engineering tools, design,
teamwork, and contemporary issues and
contexts.
• Increased use of active learning methods.
• Greater increases in emphasis on teamwork,
communication skills, and use of engineering
tools.
• Faculty and chairs report little change in
emphasis on basic math and science. 14
![Page 15: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/15.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
EC2000Outcomes-
Based
OUTCOMES
Continuous Quality
Improvement
Employer
Ratings
Student
Learning
(3.a-k)
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
STUDENT
EXPERIENCES
In-
Class
Out-of-
Class
![Page 16: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/16.jpg)
Significant Findings:
Faculty Culture
• More than 70% of program chairs indicate high
levels of faculty support for continuous
improvement.
• 88% of faculty report at least some personal effort
in program assessment.
• 68% of faculty consider their level of effort in
assessment to be “about right.”
• 20 - 25% of faculty say they have increased their
personal efforts to improve their courses.16
![Page 17: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/17.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
EC2000Outcomes-
Based
OUTCOMES
Continuous Quality
Improvement
Employer
Ratings
Student
Learning
(3.a-k)
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
STUDENT
EXPERIENCES
In-
Class
Out-of-
Class
![Page 18: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/18.jpg)
Significant Findings:
Students’ In- and Out-of-Class Experiences
Compared to 1994 graduates, 2004 graduates reported:
• Greater active engagement in their own learning
• More interaction with instructors
• More feedback from instructors
• More time spent in cooperative or internship experiences
• More international travel
• More involvement in engineering design competitions
• Greater emphasis on openness to new ideas and people
• Some uncertainty about changes in diversity climate.
18
![Page 19: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/19.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
EC2000Outcomes-
Based
OUTCOMES
Continuous Quality
Improvement
Employer
Ratings
Student
Learning
(3.a-k)
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
STUDENT
EXPERIENCES
In-
Class
Out-of-
Class
![Page 20: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/20.jpg)
Math, Science, and Engineering Skills Cluster
4.023.73
3.56
4.073.91 3.95
1.00
2.00
3.00
4.00
5.00
Applying Math
and Science
(Criterion 3.a)
Experimental
Skills
(Criterion 3.b)
Applying
Engineering Skills
(Criterion 3.k)
Ad
juste
d M
ea
n S
co
re
*** p<.001
1994 Graduates (Pre-) 2004 Graduates (Post-)
******
***
20
![Page 21: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/21.jpg)
Project Skills Cluster
3.67 3.74 3.833.89 3.974.22
1.00
2.00
3.00
4.00
5.00
Design and Problem-
Solving Skills
(Criterion 3.c,e)
Communication
Skills
(Criterion 3.g)
Group Skills
(Criterion 3.d)
Ad
juste
d M
ea
n S
co
re
1994 Graduates (Pre-) 2004 Graduates (Post-)
*** p<.001
******
***
21
![Page 22: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/22.jpg)
Contexts and Professionalism Cluster
2.95
3.663.40
3.65
4.04
3.49
1.00
2.00
3.00
4.00
5.00
Societal and
Global Issues
(Criterion 3.h,j)
Ethics and
Professionalism
(Criterion 3.f)
Life-long
Learning
(Criterion 3.i)
Ad
juste
d M
ea
n S
co
re
1994 Graduates (Pre-) 2004 Graduates (Post-)
*** p<.001
***
***
***
22
![Page 23: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/23.jpg)
Deans’ Comments
• Interviews with Deans of participating institutions
resonated with many of the findings.
– EC2000 credited with promoting good educational
planning processes.
– EC2000 “enabled” change.
– Deans comment “…ABET is one of several
important influences on curriculum, teaching, and
learning in engineering programs.”
23
![Page 24: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/24.jpg)
Deans’ Comments
• Deans rarely reported that EC2000 changed
their priorities or direction.
• A few worried EC2000 might have misdirected
faculty efforts.
• Many reported EC2000 increased administrative
and/or faculty workloads.
• Typically, Deans reallocated existing funds for
EC2000-related activities.
• Few reported EC2000 affected promotion and
tenure policies.24
![Page 25: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/25.jpg)
The Longitudinal StudyStudying the Impact of EC2000
PROGRAM
CHANGES
OUTCOMESSTUDENT
EXPERIENCES
In-
Class
Out-of-
Class
Continuous Quality
Improvement
EC2000Outcomes-
Based
Curriculum&
Instruction
Faculty
Culture
Policies &
Practices
Student
Learning
(3.a-k)
Employer
Ratings
![Page 26: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/26.jpg)
The Employer Respondents
Diversity
• Industry Sectors:
– Respondents represent all 19 industry sectors
– About half work in companies engaged in manufacturing
or providing scientific and technical services.
• Geographic Spread:
– Respondents represent all US states, territories, and 24
foreign countries.
• Company Sizes:Less than 50 employees 25%
50-499 39%
500-3,000 24%
More than 3,000 13% 26
![Page 27: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/27.jpg)
Significant Findings: Employers
• Greatest increases seen in teamwork and
communication skills and in life-long learning.
• About 1 of 4 employers report decreases in
problem-solving skills and understanding of social
and environmental contexts.
• Large national employers are more positive in their
Pre- and Post-EC2000 ratings than are smaller local
and regional employers.
• Majority of employers rate nearly all the a-k criteria
as highly important or essential for new hires. 27
![Page 28: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/28.jpg)
Conclusions and Implications
• America’s engineers are measurably better prepared than
their peers of a decade ago.
• Some differences are substantial:
– Societal and Global Issues
– Applying Engineering Skills
– Group Skills
– Ethics and Professionalism
• Reported decreases in technical skills areas from some
faculty and employers may suggest where more work
needs to be done. 28
![Page 29: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/29.jpg)
Conclusions and Implications
• 25% of the employers also report decreases in
problem-solving skills…but 75% report recent
graduates adequately or well-prepared in problem-
solving.
• Fewer employers than faculty report decreases in
abilities to apply math, science, and technical skills.
• More than 90% report recent graduates adequately or
well-prepared to apply math, science and technical
skills. 29
![Page 30: A Longitudinal Study of the ABET Outcomes-Based](https://reader033.vdocuments.net/reader033/viewer/2022050415/6270e8d954517926c00154fe/html5/thumbnails/30.jpg)
Thank you
on
Behalf of