copyright © 2006 by abet, inc. abet

53
Copyright © 2006 by ABET, Inc. A-1 December 2005 ABET Program Evaluator Training (Session 1 of 2) Stephen J. Ressler, P.E., Ph.D. Who I Am Professor & Vice Dean for Education, U.S. Military Academy at West Point, NY Former CE Program Director at USMA Chairman, USMA ABET Committee ABET Evaluator ASCE Committees: Educational Activities Curriculum & Accreditation Academic Prerequisites for Professional Practice Faculty Development Instructor, ASCE ExCEEd Teaching Workshop

Upload: others

Post on 03-May-2022

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-1

Decem

ber 2005

ABETProgram Evaluator Training

(Session 1 of 2)

Stephen J. Ressler, P.E., Ph.D.

Who I Am� Professor & Vice Dean for Education, U.S.

Military Academy at West Point, NY

� Former CE Program Director at USMA

� Chairman, USMA ABET Committee

� ABET Evaluator

� ASCE Committees:� Educational Activities

� Curriculum & Accreditation

� Academic Prerequisites for Professional Practice

� Faculty Development

� Instructor, ASCE ExCEEdTeaching Workshop

Page 2: Copyright © 2006 by ABET, Inc. ABET

Administrative Notes

� Purpose & intended audience

� Attendance sign-in rosters

� ABET slides vs. my slides

� Pace, your questions, and my questions

� Breaks

� Homework

Copyright © 2006 by ABET, Inc.

A-4

Decem

ber 2005

Learning Objectives

� Evaluate a program using the appropriate criteria

� Recommend an accreditation action

� Complete all required reports and forms

By the end of our two sessions, you will be able to:

Page 3: Copyright © 2006 by ABET, Inc. ABET

Plan for the Training

� Session 1:

� Program Evaluator Responsibilities

� Assessment Fundamentals

� Criteria - Basis for Accreditation

� Session 2:

� Decision Making

� The Accreditation Process

� Forms and Reports

Copyright © 2006 by ABET, Inc.

A-6

Decem

ber 2005

Program Evaluator Responsibilities

“Program evaluators are ABET’s most important asset. They are the ‘face of ABET’ to the programs that we accredit.”

George D. PetersonExecutive Director

Page 4: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

Decem

ber 2005

You Provide

As an ABET team member you provide knowledge concerning:

• Professional practice

• Professional preparation

• Continuous quality improvement

Copyright © 2006 by ABET, Inc.

A-8

Decem

ber 2005

Your CommitmentYour commitment is critical to a successful visit!

�Exercise sound judgement.

�Base all decisions on the criteria.

�Do careful pre-visit preparation.

�Perform on-site visit thoroughly.

�Develop and communicate succinct

conclusions.

�Participate in team decisions.

�Promptly submit all reports and forms.

�When in doubt or when a question arises,

use you team chair as a resource.

Page 5: Copyright © 2006 by ABET, Inc. ABET

Sequence of Events (1 of 4)

� Complete your training by participating as an Observer on an evaluation visit. � Work under the mentorship of an experienced PEV.

� Complete all forms and reports.

� Receive an assessment from the ASCE Committee on Curricula & Accreditation.

� Be assigned as a PEV the following year.

As a PEV, you will:

Sequence of Events (2 of 4)

� Receive notification of assignment to an evaluation team.

� Receive the program Self-Study, student transcripts, and other materials.

� Perform a complete evaluation of the program, based on the Criteria.

� Initiate a dialog with the evaluated Department Chair. � Request additional materials, as required.

� Resolve as many issues as possible before the visit.

As a PEV, you will:

Page 6: Copyright © 2006 by ABET, Inc. ABET

Sequence of Events (3 of 4)

� Participate in the Evaluation Visit:� Review student work and other documentation.

� Interview students and faculty.

� Evaluate facilities.

� Evaluate institutional areas, as directed by the Team Chair.

� Coordinate your findings with the entire team.

� Complete reports and draft Exit Statement

As a PEV, you will:

Sequence of Events (4 of 4)

� Participate in post-visit coordination:

� Submit final reports to Team Chair.

� Submit copies of all reports to ASCE for assessment of your performance.

� Be responsive to your Team Chair on all “Due Process” issues.

� Report travel expenses.

As a PEV, you will:

More details in Session 2

Page 7: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-1

3Decem

ber 2005

What the Visit Team Produces

� Team submits a Visit Report� Basis for the Draft Statement to Institution

� Criteria-based

� All visit forms feed into this document

� Includes statements of program strengths, concerns, weaknesses and deficiencies for each program based on documentation and observations from pre-visit and on-site

� Sample Draft Statement included in your course materials

At the end of the visit…

Copyright © 2006 by ABET, Inc.

A-1

4Decem

ber 2005

ABET CODE OF CONDUCT

� Approved by the ABET Board of Directors in October 1999

� Governs all volunteers and staff involved in ABET activities

� PEVs agree to abide by when completing on-line bio form

� Code of Conduct included in your course materials

Page 8: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-1

5Decem

ber 2005

Conflict of Interest

Expectations for ABET representatives:• Behave in an ethical and professional

manner• Disclose real or perceived conflicts of

interest• Recuse yourself from discussions or

decisions related to real or perceived conflicts of interest

Copyright © 2006 by ABET, Inc.

A-1

6Decem

ber 2005

Confidentiality

� Information supplied by institution and derived from the visit is for confidential use of ABET and its commissions

� ABET has specifically authorized Professional Societies to participate in the accreditation process

� General information about ABET and the Commissions is available on the ABET Web Site at http://www.abet.org

Page 9: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-1

7Decem

ber 2005

Conflict of Interest Exercise

Do this for homework,

prior to Session #2

Copyright © 2006 by ABET, Inc.

A-1

8Decem

ber 2005

Assessment Fundamentals

Page 10: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-1

9Decem

ber 2005

Continuous Quality Improvement

� A SYSTEMATIC PURSUIT OF EXCELLENCE

� AND SATISFACTION OF THE NEEDS OF CONSTITUENCIES

� IN A DYNAMIC AND COMPETITIVE ENVIRONMENT.

Copyright © 2006 by ABET, Inc.

A-2

0Decem

ber 2005

Foundation of CQI is Assessment

� Assessment of inputs & process only establishes the “capability” or “capacity”of a program

� Assessment of “outcomes” determines what is done with that capability

� Outcomes assessment improves:

� Institutional effectiveness

� Learning

� Accountability

Page 11: Copyright © 2006 by ABET, Inc. ABET

Assessment

Process of reviewing the results of data collection and analysis and making a determination of the value of findings and action to be taken.

Evaluation

Evaluation

Processes that identify, collect, use and prepare data that can be used to evaluate achievement.

Assessment

Standards, rubrics, specifications,

metrics, outcomes, etc.

Specific, measurable statements identifying the performance(s) required to meet the outcome; confirmable through evidence.

Performance Criteria

Objectives, standards, etc.

Statements that describe what students are expected to know and able to do by the time of graduation.

Outcomes

Goals, outcomes, etc.

Statements that describe the expected accomplishments of graduates during the first few years after graduation.

Objectives

Some other terms for same concept(could be used by programs)

DefinitionTerms(Typically used by

ABET)

Competency-Based

InstructionAssessment-Based Curriculum

Individual Perf. Tests

PlacementAdvanced Placement Tests

Vocational Preference Tests

Other Diagnostic Tests

“Gatekeeping”

Admissions Tests

Rising Junior Exams

Comprehensive Exams

Certification Exams

Campus and Program Evaluation

Program Reviews

Retention Studies

Alumni Studies

“Value-added” Studies

Program Enhancement

Individual assessment

results may be aggregated to

serve program evaluation needs

Level

of

Assessment

(Who?)

Individual

Group

K

N

O

W

L

E

D

G

E

S

K

I

L

L

S

A

T

T

I

T

U

D

E

S

&

V

A

L

U

E

S

B

E

H

A

V

I

O

R

Object of

Assessment

(What?)

Learning/Teaching

(Formative)

Accountability

(Summative)

Purpose of Assessment (Why?)

(Terenzini, JHE Nov/Dec 1989)

Taxonomy of Approaches to Assessment

Page 12: Copyright © 2006 by ABET, Inc. ABET

Assessment for Continuous Improvement

Program Outcomes

Constituents

Assessment: Collection & Analysis of Evidence & Data

Evaluation:Interpretation of Evidence & Data

Feedback for Continuous Improvement

Measurable Performance

Criteria

Institutional MissionEducational Objectives

Program Learning Practices/Strategies

AssessEvaluate

Student

Precollege

Traits

Educational

Outcomes

Institutional Context

Adapted from Terenzini, et.al. 1994,1995

Program Assessment

Classroom Experience

Pedagogy;

Facilities; Climate;

Faculty & Student

Characteristics

Out-of-class

Experiences

Co-curricular; co-

ops; internships;

support services

Coursework

& Curricular

Patterns

Classes chosen;

major

EnvironmentalFactors

Page 13: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-2

5Decem

ber 2005

Role of ABET Accreditation

The role of ABET accreditation is to provide periodic external assessmentand evaluation in support of the continuous quality improvementprogram of the institution.

PEV role and responsibilities:

� ABET Conflict of Interest and Confidentiality Policy

� Conflict of interest issues

� Fundamentals of assessment

Questions?

Page 14: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-2

7Decem

ber 2005

Accreditation Basics

Copyright © 2006 by ABET, Inc.

A-2

8Decem

ber 2005

Objectives of Accreditation

(1) Assure that graduates of an accredited

program are adequately prepared to enter and continue the practice of engineering

(2) Stimulate the improvement of engineering education

(3) Encourage new and innovative approaches

to engineering education and its assessment

(4) Identify accredited programs to the public

Page 15: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-2

9Decem

ber 2005

ABET Accredits Programs

For purposes of accreditation review

� All programs lead to degrees

� All paths of study must be accreditable

� A program is described by

�Objectives

�Outcomes

� Curriculum

� Transcript is primary evidence of degree

Copyright © 2006 by ABET, Inc.

A-3

0Decem

ber 2005

Philosophy

� Institutions and programs define

mission and objectives to meet the

needs of their constituents—enable

program differentiation

� Emphasis on outcomes—preparation for

professional practice

� Programs demonstrate how criteria and

educational objectives are being met

Page 16: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-3

1Decem

ber 2005

Emphasis

� Practice of continuous improvement� Input of constituencies

� Process focus

� Outcomes and assessment linked to objectives

� Knowledge required for entry into the engineering profession

� Student, faculty, facilities, institutional support, and financial resources linked to program objectives

Copyright © 2006 by ABET, Inc.

A-3

2Decem

ber 2005

Key Documents

� Program Self-Study (not included in training packet)

� Criteria for Accrediting Engineering Programs (current and proposed revisions)

� Accreditation Policy and Procedure Manual

� PEV Report (forms)

Page 17: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-3

3Decem

ber 2005

Self-Study Report

A good program self-study should include:

� A complete description of how and the extent to which the program satisfies each of the criteria requirements:

� Students

� Program Educational Objectives

� Program Outcomes and Assessment

� Professional or Technical Component

� Faculty

� Facilities

� Institutional Support and Financial Resources

� Program Criteria

Copyright © 2006 by ABET, Inc.

A-3

4Decem

ber 2005

ABET Definitions

� The institution is free to define its own terms but must meet the philosophy of Criteria 2 and 3 definitions

� Make sure you understand their terminology

� Be aware the programs in a given institution may not use the same terms and definitions

Page 18: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-3

5Decem

ber 2005

ABET Definitionscontinued

� Program educational objectives –broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve.

Copyright © 2006 by ABET, Inc.

A-3

6Decem

ber 2005

ABET Definitionscontinued

� Program outcomes – narrower statements that describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire from the program.

Page 19: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-3

7Decem

ber 2005

ABET Definitionscontinued

� Assessment – one or more processes that identify, collect, and prepare data to evaluate the achievement of program outcomes and program educational objectives.

Copyright © 2006 by ABET, Inc.

A-3

8Decem

ber 2005

ABET Definitionscontinued

� Evaluation – one or more processes for interpreting the data and evidence accumulated through assessment practices. Determines the extent to which outcomes and objectives are being achieved and results in actions to improve the program.

Page 20: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-3

9Decem

ber 2005

Basis for Evaluation

� Criteria for the current visit cycle

� Accreditation policies and procedures for the current visit cycle

� Both on the ABET web site

� http://www.abet.org

Copyright © 2006 by ABET, Inc.

A-4

0Decem

ber 2005

PEV Report

� Curriculum Analysis

� Transcript Analysis

� Faculty Analysis

� Program Evaluator Worksheet

Page 21: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-4

1Decem

ber 2005

Purpose of the PEV Report

� Documentation (evidence) backs up the statement to the institution

� Filling out the forms moves the evaluator through the criteria thoroughly

Questions?

Accreditation Basics:

� Objectives of Accreditation

� Self-Study Report

� PEV Report

� Review of Definitions

Page 22: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-4

3Decem

ber 2005

Accreditation Criteria

Standards to be applied

with judgment!

Copyright © 2006 by ABET, Inc.

A-4

4Decem

ber 2005

CriteriaWhere do they come from?

� The ABET commissions and member societies representing the professions develop general and program criteria

� The commissions recommend criteria to the ABET Board

� The Board approves criteria

Page 23: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-4

5Decem

ber 2005

CriteriaSo what are they?

Standards by which programs are evaluated for accreditation

Copyright © 2006 by ABET, Inc.

A-4

6Decem

ber 2005

Engineering Criteria

I. General Criteria for Basic Level Programs

II. Criteria for Advanced Level Programs

Page 24: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-4

7Decem

ber 2005

I. Basic Level Accreditation Criteria

1. Students

2. Program Educational Objectives

3. Program Outcomes and Assessment

4. Professional Component

5. Faculty

6. Facilities

7. Institutional Support & Financial Resources

8. Program Criteria

Copyright © 2006 by ABET, Inc.

A-4

8Decem

ber 2005

Students – Criterion 1

The program or its institution must:

� Evaluate student performance

� Advise students regarding curricular and career matters

� Monitor students progress to foster their success in achieving outcomes

Page 25: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-4

9Decem

ber 2005

Students – Criterion 1 -continued

� The program or its institution must have and enforce

� policies for transfer students

� policies for validation of courses taken for credit elsewhere

� procedures to assure that all students meet program requirements

Copyright © 2006 by ABET, Inc.

A-5

0Decem

ber 2005

Student Transcripts

� Provide direct evidence that the institution’s program requirements are met

� Provide evidence that the curricular requirements of Criteria 4 and 8 (if applicable) are met

Page 26: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-5

1Decem

ber 2005

Transcript Evaluation

� Determine if curriculum is being followed

� Note that curriculum may differ from the one being evaluated

� If no “Worksheet” is provided, ask if it exists

� Does institutional process assure that course substitutions meet ABET requirements?

� The transcript should indicate the title of the program being evaluated

� Consult team chair for programs “in transition”

Copyright © 2006 by ABET, Inc.

A-5

2Decem

ber 2005

Issues for Criterion 1

� Problems with student advising (often cited with Criterion 5 - faculty)

� Ineffective and inconsistent advising

� Lack of understanding of curricular requirements especially if many options are available

� Ineffective monitoring

� No documentation of course substitutions or missing prerequisites

Page 27: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-5

3Decem

ber 2005

Issues for Criterion 1 -continued

� Problems with transfer students� No documentation on acceptability of transfer credits (primarily for engineering topics courses)

Questions?

Criterion 1

Page 28: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-5

5Decem

ber 2005

Program Educational Objectives - Criterion 2

� Program Educational Objectives:� Broad statements that describe the career and

professional accomplishments that the program is preparing graduates to achieve

� Consistent with the mission of the institution

� Allows differentiation between programs

Copyright © 2006 by ABET, Inc.

A-5

6Decem

ber 2005

Program Educational Objectives - Criterion 2

� Each program must have� Detailed published educational objectives consistent with the mission of the institution and these criteria

� Process based on needs of constituencies in which objectives are determined and periodically evaluated

Page 29: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-5

7Decem

ber 2005

Program Educational Objectives - Criterion 2

� Each program must have� An educational program, including a curriculum that prepares students to attain program outcomes

� A process of on-going evaluation of the extent to which these objectives are attained and uses results to develop and improve the program outcomes

Copyright © 2006 by ABET, Inc.

A-5

8Decem

ber 2005

Issues for Criterion 2

� Educational objectives not published or readily accessible to the public

� Educational objectives not related to institutional mission

� No evidence of constituency input in setting or periodic evaluation of objectives

Page 30: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-5

9Decem

ber 2005

Issues for Criterion 2 -continued

� No process for evaluating the extent to which objectives are attained

� No data available on the attainment of educational objectives

� Results of evaluation not used to develop and improve the program outcomes

Copyright © 2006 by ABET, Inc.

A-6

0Decem

ber 2005

� Objectives� How were the objectives determined?� Are they consistent with the mission statement?� Are they measurable? How do you accomplish your

objectives?� What is your process to review and update?� How do you know when you have achieved your

objectives?

� Constituencies� Who are your constituencies?� How do you involve your constituencies in the

continuous improvement process?

Questions You May Wish to Ask

Page 31: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-6

1Decem

ber 2005

Evaluation of Program Objectives

� The following tools are useful in evaluation of program objectives:� Alumni Surveys

� Industrial/Professional Advisory Boards

� Employers’ Surveys and Placement Data

� Graduate programs at other universities

� Programs may choose to use these or other appropriate assessment methods

Questions?

Criterion 2

Page 32: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-6

3Decem

ber 2005

Program Outcomes & Assessment - Criterion 3� Program outcomes:

� Narrower statements that describe what students are expected to know or be able to do by the time of graduation. Relate to skills, knowledge, and behaviors that students acquire from the program.

� The achievement of all outcomes indicates that the student is equipped to achieve the program educational objectives

� ABET designated (a-k) included in some way

Copyright © 2006 by ABET, Inc.

A-6

4Decem

ber 2005

Program Outcomes and Assessment - Criterion 3

� Programs must demonstrate their graduates have achieved outcomes “a to k”

� Programs must have an assessment process with documented results that indicate the degree to which outcomes are achieved

� Programs must provide evidence that the results of the assessment process are applied to the further development and improvement of the program

Page 33: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-6

5Decem

ber 2005

Program Outcomes

� Engineering programs must demonstrate that their graduates have:

a. An ability to apply knowledge of mathematics, science, and engineering appropriate to the discipline

b. An ability to design and conduct experiments, analyze and interpret data

Copyright © 2006 by ABET, Inc.

A-6

6Decem

ber 2005

Program Outcomes

c. An ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability.

d. An ability to function on multi-disciplinary teams

Page 34: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-6

7Decem

ber 2005

e. An ability to identify, formulate, and solve engineering problems

f. An understanding of professional and ethical responsibility

g. An ability to communicate effectively

Program Outcomes (continued)

Copyright © 2006 by ABET, Inc.

A-6

8Decem

ber 2005

Program Outcomes (continued)

h. The broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context

i. A recognition of the need for, and an ability to engage in, life-long learning

j. A knowledge of contemporary issues

k. An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice

Page 35: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-6

9Decem

ber 2005

Issues for Criterion 3

� No evidence demonstrating one or more outcomes

� Outcomes not assessed objectively (student performance)

� Anecdotal versus measured results

� Reliance on course grades as assessment of outcomes

� Over-reliance on self-assessment (e.g., surveys)

Copyright © 2006 by ABET, Inc.

A-7

0Decem

ber 2005

Issues for Criterion 3 (continued)

� No systematic assessment process� No process or process not documented

� Plans developed but not implemented

� Little or no faculty support for the process

� No evidence that assessment results are being applied to develop and improve the program� Assessment results not used

� Assessment & development cycle not complete

Page 36: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

1Decem

ber 2005

Assessment Tools and Measures

� The primary outcomes assessment should be based on direct measures of student learning

� Senior exit surveys, alumni surveys, and employer surveys as means of outcomes assessment are indirect measures. These data could be used as supportive evidence; their use as the primary means of assessment is discouraged.

Copyright © 2006 by ABET, Inc.

A-7

2Decem

ber 2005

Evidence to Be Provided—

� Self-study should describe the materials that will be available for review during the visit to demonstrate achievement of the program outcomes

� The evaluators will be looking for results of assessment and evaluation. Evaluators will not assess student work to determine what outcomes were covered

� Taking a course does not guarantee achievement of outcomes

Page 37: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

3Decem

ber 2005

Objectives versus Outcomes—based on Criteria Definitions

� Objectives require longer term evaluation

� Will be based on needs of constituencies

� May be based on alumni and employer surveys

� Evaluation is required

� Program development is required

� Outcomes assessment is shorter term

� Should involve student work as the primary means of assessment

� Surveys and other “opinion polls” can be used as secondary evidence

� Outcomes should be measured against an expectation

� Closing the loop is required

Copyright © 2006 by ABET, Inc.

A-7

4Decem

ber 2005

Questions You May Wish to Ask

� Outcomes– What should students know and be able to do upon graduation?

– Are students familiar with the required outcomes?

� Processes– What is your process to achieve your outcomes? How is it documented?

– What does it do? How do you know it does what you say it does?

– Who maintains and improves the process? What is your involvement?

Page 38: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

5Decem

ber 2005

� Assessment– Are data being collected for each outcome?

– What is measured? How often?

– How do you use the data you collect?

– Does your evaluation provide the information that you

need to improve the program?

– What is your feedback process to continuously

improve the program?

– What actions have been taken to improve the

program as a result of the assessment process?

Questions You May Wish to Ask

Copyright © 2006 by ABET, Inc.

A-7

6Decem

ber 2005

� Results– How do you demonstrate that outcomes are attained?

– What is your evaluation of the quality of the program?

– What evidence do you have to demonstrate that your efforts to improve the program are producing results?

– Based on your evaluation of assessment data, what are your plans for additional improvement?

Questions You May Wish to Ask

Page 39: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

7Decem

ber 2005

Level of Expectation� Exactly which outcomes must each

graduate attain?� A system must be in place to ensure that all graduates have, to some minimum extent, achieved the required Criteria 3 and 8 outcomes and all elements of the Professional Component

� The level of achievement may vary among students, so long as the variation is consistent with program objectives

Copyright © 2006 by ABET, Inc.

A-7

8Decem

ber 2005

What to Look For

� Processes in place that provide for:

�Definition of desired, measurable outcomes

�Collection of data linked to the outcomes

�Analysis of data and evaluation of results

� Implementation of change

�Repeat cycle and review

Page 40: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-7

9Decem

ber 2005

What to Look For (continued)

� Documentation of results and evidence that results are being used to develop the program, for example:� Student portfolios

� Subject content examinations

� Performance observations

� Performance evaluations of internships and/or co-ops

Copyright © 2006 by ABET, Inc.

A-8

0Decem

ber 2005

Keep in Mind

� You don’t have to be an expert on assessment. The institution must provide evidence that it has a working and effective system in place

� The institution must describe a clear relationship between program objectives, outcomes, and measurable indicators of success with required levels of achievement

� You are assessing the program based on the criteria and the strength of the evidence provided by the institution, not your own personal preferences

Page 41: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-8

1Decem

ber 2005

Criterion 3

Questions?

Copyright © 2006 by ABET, Inc.

A-8

2Decem

ber 2005

Professional Component –Criterion 4

� Faculty must ensure that the curriculum devotes adequate attention and time to each component, consistent with the outcomes and objectives of the program and institution

� Preparation for engineering practice

� Major design experience

� Subject areas appropriate to engineering

Page 42: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-8

3Decem

ber 2005

Major Design Experience

• A culminating experience, based on

knowledge and skills acquired in earlier

course work

• Must incorporate appropriate engineering

standards and multiple realistic

constraints.

Professional Component

Copyright © 2006 by ABET, Inc.

A-8

4Decem

ber 2005

Professional Component

Subject Areas� One year of a combination of college-level

mathematics and basic sciences (some with experimental experience) appropriate to the discipline

� One and one-half years of engineering topics, consisting of engineering sciences and engineering design appropriate to the student’s field of study

� A general education component that complements the technical content of the curriculum and is consistent with the program and institution objectives

Page 43: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-8

5Decem

ber 2005

Criterion 4 - Issues

� Quality of the major design experience� No culminating experience - analysis or

research instead of design; several courses with elements of design

� Multiple capstone courses with widely varying quality

� Design experience does not address multiple constraints

� Engineering topics satisfied by electives, but advising doesn’t assure adequate coverage

Copyright © 2006 by ABET, Inc.

A-8

6Decem

ber 2005

Criterion 4

Questions?

Page 44: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-8

7Decem

ber 2005

Faculty -Criterion 5

� Sufficient in number and competencies to cover all curricular areas

� Sufficient in number to accommodate adequate levels of student-faculty interaction, advising and counseling, service, professional development, and interactions with industrial and professional practitioners and employers

� Ensure proper guidance of the program and its evaluation, development, and improvement

Copyright © 2006 by ABET, Inc.

A-8

8Decem

ber 2005

Criterion 5 - Issues

� Insufficient number:� To support concentrations, electives, etc.

� To provide student advising

� Poor faculty morale affecting program� Lack of professional development

� Excessive workloads

� Retention/turnover rate

� Salaries as it relates to retention and recruiting (cited with Criterion 7)

Page 45: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-8

9Decem

ber 2005

Criterion 5 Issues - continued

� Faculty Quality

� For teaching design (program criteria issues)

� Excessive reliance on adjuncts

� Heavy reliance on temporary faculty appointments, potentially jeopardizing program stability (cited with Crit. 7)

Copyright © 2006 by ABET, Inc.

A-9

0Decem

ber 2005

Criterion 5

Questions?

Page 46: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-9

1Decem

ber 2005

Facilities - Criterion 6

� Classrooms, laboratories, and associated equipment must be adequate to support attainment of program objectives and provide an atmosphere conducive to learning� Opportunities to learn the use of modern

engineering tools

� Computing/information infrastructure to support scholarly activities of the students and faculty and the educational objectives of the institution

Copyright © 2006 by ABET, Inc.

A-9

2Decem

ber 2005

Criterion 6 - Issues

� Insufficient Space� Overcrowded laboratories and classrooms

� Laboratories � Unsafe conditions

� Inoperable equipment

� Lack of modern instrumentation

� Lack of funds for upgrading (cited with Crit. 7)

� Computing/Information Infrastructure� Lack of funds for upgrading (cited with Crit. 7)

Page 47: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-9

3Decem

ber 2005

Institutional Support and Financial Resources - Criterion 7

� Institutional support, financial resources, and constructive leadership must be adequate to assure quality and continuity of the program� Attract, retain, and provide for professional

development of a well-qualified faculty

� Resources to acquire, maintain, and operate equipment and facilities

� Adequate support personnel

� Support of quality-improvement efforts

Copyright © 2006 by ABET, Inc.

A-9

4Decem

ber 2005

Criterion 7 - Issues

� Unstable leadership affecting programs� Dean/Dept Chair positions open or filled by

“interim” appointments for an extended period

� Frequent turnover of university administration and engineering school leadership

� Inadequate operating budget affecting:� Acquisition and maintenance of laboratories

and computing equipment

� Faculty salaries, promotions, and professional development affecting hiring and retention

Page 48: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-9

5Decem

ber 2005

Criterion 7 - Issues (continued)

� Insufficient Support Staff

� Teaching assistants

� Technicians for instructional laboratories, machine shops, and laboratory services

� Administrative/clerical

Copyright © 2006 by ABET, Inc.

A-9

6Decem

ber 2005

Criteria 6 and 7

Questions?

Page 49: Copyright © 2006 by ABET, Inc. ABET

Copyright © 2006 by ABET, Inc.

A-9

7Decem

ber 2005

Program Criteria –Criterion 8

� Each program must satisfy applicable Program Criteria

� Curricular topics

� Faculty qualifications

� Current Program Criteria are on the ABET server (www.abet.org)

� Must satisfy all Program Criteria implied by title of program

CE Program Criteria

� Proficiency in math, thru differential equations, prob & stats, calculus-based physics, and general chemistry

� Proficiency in at least 4 recognized CE areas

� Ability to conduct lab experiments…in more than one recognized CE area

� Ability to perform design by means of design experiences integrated throughout the curriculum

� Understanding of professional practice issues…

1. Program must demonstrate that graduates have:

Page 50: Copyright © 2006 by ABET, Inc. ABET

CE Program Criteria

� Faculty teaching courses that are primarily design in content must be qualified by:

� Licensure or

� Education and design experience

� The program is not critically dependent on one individual.

2. Program must demonstrate that:

The ASCE CE Commentary

� Definitions and discussion of:

� Proficiency

� Recognized major civil engineering areas

� Engineering design

� Design integration

� Specific examples of how programs might demonstrate compliance

� Guidance on faculty qualifications

http://www.asce.org/community/educational/eng.cfm

CE Program Evaluators must read and understand this document!

Page 51: Copyright © 2006 by ABET, Inc. ABET

CE Program Criteria - Issues

� Failure to ensure that all students

attain proficiency in 4 CE areas

� Inadequate student understanding of

professional practice issues

� Inadequate design integration

� Faculty teaching design who are not

qualified to do so

� Faculty expertise inadequate to teach

4 CE areas

Related Disciplines

� Construction Engineering

� ASCE is lead society

� Commentary available

� Architectural Engineering

� ASCE is lead society

� No Commentary yet

� Environmental Engineering

� AAES is lead society

� Combined degree titles must satisfy all applicable Program Criteria

Page 52: Copyright © 2006 by ABET, Inc. ABET

CE Program Criteria

Questions?

Copyright © 2006 by ABET, Inc.

A-1

04

Decem

ber 2005

II. General Criteria for Advanced-Level Programs

� Completion of a program satisfying the general criteria for basic-level engineering programs

� One year of additional study

� An engineering project or research activity with a report that demonstrates:� Mastery of the subject matter

� High level of communication skills

Page 53: Copyright © 2006 by ABET, Inc. ABET

Session 1 Wrap-Up� Print & sign the sign-in roster, and fax to Dion

Coward at 703-295-6132 � For homework:

� Study all remaining Session 1 slides.� Formulate responses to the Conflict of Interest

Exercise� Read the ABET Decision Exercises� Print the ASCE “State Tech Report” and have it

available for our next session.� Send questions to [email protected]

� Next session:� Log in 15 minutes early!� Review Conflict of Interest Exercise� Review Criteria� ABET Decision Exercises� The Accreditation Process & PEV Reports

“See” you next week!