accreditation of engineering, technology and computing programs moshe kam ieee vice president for...

Accreditation of Engineering, Technology and Computing Programs

Moshe KamIEEE Vice President for Educational Activities

First Edition – October 2007

Version 003

IEEE EAB on accreditation2

Contact Information

Moshe KamRobert G. Quinn Professor and Department Head

Drexel University

Electrical and Computer Engineering

3141 Chestnut Street

Philadelphia, PA 19101

[email protected]


DISCLAIMER

This presentation was prepared by the IEEE Educational Activities Board for a broad, general discussion of accreditation of engineering, computing, and technology

Material is provided for illustrative purposes only

Description of various rules and regulations are made in general descriptive terms and are not intended for operational or legal use

Material is not purported to represent the official policy of any accrediting body or any other governmental or non-governmental agency outside of IEEE


Outline

Purpose Accreditation in Engineering,

Computing and Technology• Definition, aims, uses and misuses,

models

Mutual recognition agreements Building new accrediting bodies in

the early 21st Century


Outline



models




Purpose

To provide an overview of the accreditation process

To present different models and principal trends

To review existing international agreements and accords in the area of accreditation


A Few Words about IEEE

IEEE is the largest multinational professional engineering association in the world • 367,000 members in 150 countries• A 501(c)3 organization in incorporated in New York

Originally concentrating on power engineering and communications, IEEE at present spans technical interests across the spectrum of technology• From nanotechnology to oceanic engineering

In many respects IEEE has become “the steward of Engineering”


Early Presidents

Alexander G. Bell Elihu Thomson Charles Steinmetz Frank Sprague


A few more recent Presidents

Leah Jamieson Joseph Bordogna Michael Lightner Wallace Read


Why is IEEE interested in Accreditation?

Because it is in IEEE’s stated mission

Because accreditation has significant impact on the content of the curriculum in IEEE’s fields of interest• And hence on the future of the profession

Because IEEE’s involvement introduces the voice of the profession and its practitioners into the decision making process of educational institutions


Why is IEEE interested in Accreditation?

IEEE considers accreditation a strategic objective and supports accrediting bodies worldwide

• The IEEE BoD allocates funds and human resources to accreditation on an annual basis

About 500 volunteers $2M/year in direct expenditures in 2007


Outline



models

Mutual recognition agreements Building a new accrediting body in



Operational Definition of Accreditation by CHEA (US)

Accreditation in higher education is defined as a collegial process based on self- and peer assessment for public accountability and improvement of academic quality

[Peers = group of peer faculty and staff, professionals, and public members]

Peers assess the quality of an institution or academic program and assist the faculty and staff in improvement


Three Major Activities

The faculty, administrators, and staff of the institution or academic program conduct a self-study using the accrediting organization’s set of expectations about quality (standards, criteria) as their guide

A team of peers, selected by the accrediting organization, reviews the evidence, visits the campus to interview the faculty and staff, and writes a report of its assessment, including a recommendation to the commission of the accrediting organization


The third step…

Guided by a set of expectations about quality and integrity, the commission • reviews the evidence and

recommendation• makes a judgment• communicates the decision to the

institution and other constituencies if appropriate


A Broader Definition of Accreditation

Formal recognition of an educational program by an external body on the basis of an assessment of quality

An evaluation process in which an objective group (accrediting body) examines an educational program to ensure that it is meeting minimum standards established by experts in the field• The outcome of the process is binary: program is

either accredited or not accredited


Challenges to the Traditional Definition (1)

Should the accreditation be done by an “external body”?• Is it possible to conduct accreditation by peer groups

E.g., peer institutions

Should the result of accreditation be binary?• Some groups in Europe have called for providing

evaluation in four categories with respect to every criterion

Fails to meet minimum requirements Meets minimum requirements Exceeds minimum requirements Excels in meeting this criterion


Challenges to the Traditional Definition (2)

Should we strive to meet minimum standards rather than achieving continuous improvement and excellence?

Will the current system of accreditation be useful to industry in the long term?• The mobility of labor has challenged traditional

credentials• The ECE industry had already rejected the licensing

process E.g., the industrial exemption in the United States


Looking Forward: Traditions

The trends we observe in accreditation will challenge traditional models

It is unlikely that over-prescriptive accreditation models will survive

It is unlikely that models that are based solely on minimum thresholds will survive


The Constituencies of an Educational Program

Past, present and prospective students

Prospective employers

Other bodies of higher education

Licensing bodies

Government

The public at large


The Various Functions of Accreditation (1)

Provide constituencies of the educational program with a guarantee that an educational program… • meets (minimum) standards• continues to evolve in order to incorporate

best practices

Put a stamp of approval on graduates – they are ready to practice• Raises the issue of First Professional Degree

in Engineering


The Various Functions of Accreditation (2)

Provide educational programs with opportunities for self-definition and self-reflection • and with feedback on program content and

direction

Provide opportunities for continuous improvement of education programs


Misuse of Accreditation

Coercion• The process needs to be voluntary

Disciplinary action

Ranking and comparison of schools

Controlling the school

Serving the interests of one constituency on the expense of others

Homogenizing higher education

Control competition

Limit enrollments


Accreditation is not indispensable…

Industry can replace accreditation by other mechanisms of quality assessment• University rankings by academic bodies• University rankings by the popular press• Internal lists of “acceptable institutions” • Entry exams and interviews of graduates

If accreditation is to survive it should be• Pertinent • Transparent• Fair• Economical • Adaptive to the business climate


The Basic Structure of the Process: Accrediting Body

Accrediting body defines its accreditation philosophy and publishes criteria and process

Accrediting body identifies and trains program evaluators

Bodies that recognize accrediting bodies require proof of decision independence• The funding mechanism and accreditation

decisions should be independent


A Word of Caution: Independence

In several instances, IEEE observed loss of independence of accrediting bodies

In all of these cases, the accrediting body was discredited• Schools preferred foreign accrediting bodies over the

local one

It is not clear whether government controlled accrediting bodies will be recognized in the future by international accords


The Basic Structure of the Process: Program

Program studies accrediting body literature Program collects required material and verifies

presumption of accreditability

Program requests an accreditation visit

Program gets organized to provide information to accrediting body and visiting team

Self study


Interaction Between Accrediting Body and Program

Mutual agreement on visiting team

Agreement on dates and logistics• Within published guidelines

Pre-visit communications

Accrediting visit and preliminary reporting

Post-visit communications

Report preparation and determination of outcome

Post-report communications – possible appeals


Two important caveats

The accreditation visit is supposed to provide “no surprises”• All concerns that program evaluators have on

the basis of submitted data are supposed to be discussed ahead of the visit

On site visit focuses on the accreditation criteria and their implementation• This is not the time for ‘free advice’ or

planning of the program future by the visiting group


Looking Forward: Implementation

At present most accreditation bodies operate on 5-7 year cycles

A series of “dramatic events” followed by long periods of low or no activity

Elaborate visits requiring significant preparation

It is possible to design a much simpler process that takes advantage of progress in information technology• Information is posted and updated continually• Visits are shorter

focus only on the few items that do not require face to face interaction


Selected accrediting bodies (1)

Engineers Australia

Engineers Ireland

Canadian Engineering Accreditation Board of the Canadian Council of Professional Engineers

France: Commission des Titres d'Ingénieur

Germany: ASIIN

Hong Kong Institution of Engineers

For additional details see www.Accreditation.org



Japan: Japan Accreditation Board for Engineering Education

Korea: Accreditation Board for Engineering Education of Korea

Malaysia: Board of Engineers Malaysia

Mexico: Council of Accreditation of the Education of Engineering

New Zealand: Institution of Professional Engineers of New Zealand




Singapore: Institution of Engineers Singapore

South Africa: Engineering Council of South Africa

United Kingdom: Engineering Council United Kingdom

United States: ABET



Characteristics of Accreditation (1)

Voluntary

Performed by an external agency• Based on the locale of the program• Uses representation of all major constituencies

Government inspection is not Accreditation

Based on clear published standards

Evaluative – not regulatory• It is not the place of the visiting team to provide the visited

program with detailed prescriptions and methodology


Characteristics of Accreditation (2)

Requires continuous maintenance

Binary (at present, in most cases)

Cognizant of program objectives and goals• One size does not fit all• Accreditation is not meant to homogenize the

education system


What are the Factors That May be Considered?

Content of the curriculum• Is there enough exposure to discrete

mathematics?

Size and skill base of the faculty• Does a Computer Science program have

individuals who are trained in Artificial Intelligence?

Morale and governance of the faculty


Potential Consideration Factors (2)

Facilities• Does the Microwave Laboratory in an

EE program have a Spectrum Analyzer?

Admission criteria• Do the admission criteria ensure that

incoming students have the basic skills required to attend the program?


Potential Consideration Factors (3)

Support services• Do the program’s computing facilities enjoy

professional system administration?

Graduate placement• Do the majority of the program graduates find

gainful professional employment within 6 months of graduation?

Budgets and expenditures


Looking Forward…

Many of the traditional factors required a site visit for verification

A model that considers most of the factors on line is now possible

Accreditation may become continuous rather than a discrete event• Shift from reaching the minimum to

continuous improvement


Level of Specificity

The degree to which criteria are defined in terms of numerical goals or specific coverage methodologies

The degree of specificity depends on the accreditation model• The general trend in the last 10 years is

away from specifics


Level of Specificity: Examples

Non-specific• A computer science program needs to show that

graduates were exposed to the principles of database organization and have used this knowledge in open- ended exercises and projects

Specific• A computer science program will include at least 24

hours on in-class instruction on databases which includes: database models (at least 3 hours); relational models (at least 1 hour)…

• A minimum of two 6-hour laboratory exercises on databases must be included. These exercises include…


A Two-Tier Process

Usually accreditation of engineering, computing and technology programs relies on a “general accreditation” of the institution

Another accrediting agency ascertains compliance with laws and regulations, basic fiscal solvency, and preservation of human rights

If the first tier is missing, the “technical” accreditation needs to include it• This may be a challenge for new accrediting bodies


Most of the Work Does Not Involve the Accrediting Body

Program must establish mechanism to collect data on its activities

Program must establish mechanism to use data to reaffirm or reform its activities

Program must undergo a thorough self-study• This is often the most important outcome of

the accreditation process


Different Approaches and Styles of Accreditation

The Minimal Model

The Regulatory Model

The Outcome-Based Model

The Peer-Review model

The Program Club model


The “Minimal Model”

Ascertains basic characteristics of the school and program• Often numeric and law-based

Does the school satisfy basic legal requirements? Does the school have enough budget, infrastructure and

reserves to conduct the program?

Ascertains existence of the fundamental basics in the school and program• Physical conditions, size and skill base of the faculty,

coverage of basic topics in the curriculum

Provides a prescription for a minimal core and very general parameters for the rest of the curriculum


Reflections on the Minimal Model

It is easy to install and maintain as long as it adheres to the “minimal” philosophy

Not a bad way to start an accrediting body

Does not encourage continuous improvement

The biggest danger is “mission creep”• More and more requirements


The Regulatory Model

Requires strict adherence to a core curriculum• E.g., defines the minimum requirements for a Software

Engineering curriculum

Specifies parameters for the rest of the curriculum• E.g., at least 6 credit hours of post WWII history

Often involving direct prescriptions of curriculum and faculty composition• E.g., “at least three faculty in manufacturing are required

if the body of students exceeds 120”


Reflections on the Regulatory Model

Makes the accrediting process uniform and potentially fair• Criteria are unambiguous and often numeric

Difficult to establish and update• Leads to endless strife over what the “core” means

Relatively easy to maintain• The key to success is adherence to clear rules

Was shown to stifle innovation and creativity in the curriculum• This was the philosophy of the pre-2000 ABET model


The Outcome-Based Model

Prescribes a “small” core and basic requirements

Prescribes basic parameters for the goals of the program• But does not specify the specific goals of the program

Focuses on the goals and objectives of the program• E.g., to maximize the number of graduates who continue to

Medical or Law school• E.g., to maximize the number of graduates who become

program managers in the construction industry

Requires evidence of measurement of goals

Requires evidence of using the measurements to feed a quality improvement process


Reflections on the Outcome-Based Model

Provides for significant diversity in goals and objectives• Very different from the regulatory model

Puts a lot of responsibility and risk in the hands of the program leaders• E.g., some programs may try to achieve goals that are

unattainable

Sophisticated and hard to evaluate• Very difficult to avoid complaints on inconsistent evaluations

This is the basic philosophy of the current ABET EC2000 and TC2000 criteria• Though implementation does not always follow the

philosophy


A Word of Caution: Outcome-Based Accreditation

While outcome-based accreditation is the most popular paradigm for accreditation, it is not problem-free

The prescriptive nature with respect to course content can be replaced by a prescriptive process with respect to assessments

Too much data may be collected and analyzed in order to prove that methods were assessed

Adherence to the process by zealous program evaluators may cause strong disagreements about methodology• E.g., the debate about Direct Assessment


Goal Evidence

Graduates are ready to enter the workforce

Placement figures of recent graduates

Programs develops future leaders of industry

Statistically-valid evidence of leadership positions for graduates of the last 20 years

Programs provides the Province with computer scientists needed to accomplish national R&D goals

Evidence that graduates settle in the Province and maintain employment in Computer Science

Program serves students who wish to gain engineering background before they develop careers in medicine or law

Evidence that graduates turn to practice of law and medicine in statistically meaningful numbers

Collection of Data


Evidence Use of Evidence

Employer survey indicates graduates have difficulties using modern computing tools for control

Junior class in Control Systems now includes a section and lab on symbolic computation for control

Percentage of students who choose graduate school is dropping

Department commissions a survey of recent graduates to understand their new post-graduation profile

Graduation rates are dropping; exam grades in mathematics are dropping

Department institutes an “entrance exam” in mathematics. Low achievers are directed to remedial classes.

Use of Data for Improvement


The Peer Review Model

A coalition of schools organizes in group of peers • Schools select their peers • Members from other constituencies are added

Government, Industry, professional associations

The peer groups conduct the review in evaluator teams

Model requires an arbitrator and facilitator• Ideally a professional association


Reflections on the Peer Review Model

This is the way accreditation was done in the US in the early 20th Century• E.g., Princeton and Johns Hopkins came to visit the

College of Engineering at Drexel University in 1904

Difficult to organize

Considered less confrontational and more collegial

Risk a drift in the direction of unpublished mandates

Risks clashes of philosophies and program rivalry

Selection/acceptance of peers may be complicated


The “Program Club” Model

Group of peer institutions create a “program club”• Use a common website for communication

Programs that wish to join create a website with requested information

Programs report continually on progress and experimentation in education

New ideas are discussed and tried by members of the club

Few on-site visits (possibly during an annual conference)


Reflections on the “Program Club” Model

Continuous accreditation model

Difficult to organize

Considered less confrontational and more collegial

Risks clashes of philosophies and program rivalry

Selection/acceptance of peers may be complicated

“High maintenance”


Key to Success: Consistency and Transparency

Criteria need to be clear and published• Ambiguities and potential for different

interpretations need to be flagged out and addressed

• Terminology needs to be defined

Visit and reports should follow the criteria closely• Findings and conclusions should use the

criteria and the established terminology


Accreditation and Licensing


Relationship Between Licensing and Accreditation

Accreditation is provided to educational programs

A license is provided to individuals• Graduation from an accredited

program is often a condition for licensing

• Licensing exams are often based on model curricula of accrediting bodies

Serious problem in the US since EC2000


Engineering and Computing Licensing in 2007…

Licensing of engineering and computing professionals is perceived to be “broken” in many countries• Not a needed credential in many disciplines• Poor enforcement• Further weakening due to massive redistribution of

labor among markets in the early 2000s

One possible solution - the Canadian model• Licensing = graduation from an accredited program

PLUS testing on safety, professionalism and ethics NOT on school material


Licensing and the First Professional Degree in Engineering

At present there is debate in the US and Europe about “the first professional degree in engineering”

The US National Academy of Engineering and several European organizations appears to favor a Master of Science degree• In Europe this view follows adoption of the Bologna

Process

The NCEES (USA) wants B.S. degree plus 30 semester credits

No consensus among professional organizations


Outline



models





Purpose

Mutual recognition agreements establish acceptance of accreditation decisions of one accrediting body by another

Foster mobility of professionals

Provide recognition to accrediting bodies• As well as quality control



Caution: who recognizes whom

Mutual recognition agreements are between accrediting bodies, not states or governments

It is possible, for example, that • an accrediting body will recognize that

graduates from programs recognized by another accrediting body are ready to practice at an entry level…

• But the State/Government would NOT


An agreement between accrediting bodies on mutual recognition of program accreditation• Does not cover licensure and registration

Recognizing the substantial equivalency of accreditation systems of organizations holding signatory status, and the engineering education programs accredited by them

Establishing that graduates of programs accredited by the accreditation organizations of each member nation are prepared to practice engineering at the entry level


“By virtue of the Washington Accord’s affirmation of substantial equivalence among its members, the signatories have come to acknowledge generally accepted, globally relevant attributes that graduates from accredited engineering programs are expected to possess.

This acknowledgement of substantial equivalence has the potential for facilitating mobility of practicing engineers across country boundaries.”

Source: Presentation by George Peterson to EAB, November 2006


More on the Washington Accord

Signatories may exchange observers to annual meetings or accreditation visits

Verification required at regular intervals (no more than 6 years)

Bilateral agreements by individual signatories not recognized by other signatories

Recently – accreditation “outside the home territory” is also recognized


Washington Accord Signatories

Engineers, Australia

Canadian Council of Professional Engineers

Institution of Engineers Singapore

Hong Kong Institute of Engineers

Engineers Ireland

Japan Accreditation Board for Engineering Education

Engineers, New Zealand

Engineering Council of South Africa

Engineering Council, United Kingdom

ABET, Inc.



Recent additions (2007)

ABEEK (Republic of Korea) IEET (Chinese Taipei)


Provisional Members

ASIIN Germany 2003

BEM Malaysia 2003

RAEE Russia 2005



Maintenance

Signatories routinely observe each others processes

Mandatory review of each signatory at regular intervals

Every two years there is a general review of the agreement



Upgrading

Increasing scope of the agreement

Introduction of new members

Improving operations


Other Agreements

The Bologna Declaration

Lisbon convention

MERCOSUR

Western Hemisphere Initiative


Additional Agreements

Name Area/Focus

ENAEE Europe

Engineering for the Americas

South and Central America

APEC Engineer Agreement

Asia Pacific

Sydney Accord Engineering Technology

Dublin Accord Engineering technicians



The Bologna Declaration (1999)

Signed by ministers of education of 29 European countries

Adoption of a system of easily readable and comparable degrees

common terminology and standards

Adoption of a system essentially based on two main cycles, undergraduate and graduate….


Two Cycles: 3+(2+2)

Access to the second cycle shall require successful completion of first cycle studies, lasting a minimum of three years

The degree awarded after the first cycle shall also be relevant to the European labour market as an appropriate level of qualification

The second cycle should lead to the master and/or doctorate degree


Some ramifications of the Bologna Process to Engineering Education (1)

Potential ambiguity about the duration and scope of studies toward the Bachelor of Science Degree

• In some countries a 4-year B.Sc. degree is considered the First Professional Degree (FPD) in Engineering

• In other countries a 3-year B.Sc. Degree is considered a pre-engineering degree

FPD=Master of Science


Some ramifications of the Bologna Process to Engineering Education (2)

Potentially – differences between interpretations and weight of the Bachelor of Science Degree in Engineering • FPD=B.Sc. Or M.Sc.?

Questions about mobility of engineers between Europe and the rest of the world


A Complementary Approach - Registry

In several parts of the world accrediting bodies and regulatory agencies have established an engineer registry

The registry recognizes engineers who have graduated from programs accredited by a recognized body

Process makes it easier to verify credentials• Can be perceived as being in competition with

licensing/registration


Outline



models

Mutual recognition agreements Building a new accrediting body in

the Early 21st Century


Scope

Building new accrediting bodies provides an opportunity to use about 80 years of experience with existing bodies

Buy-in needed from• Professional Associations and leaders of the

profession• Academic institutions and the faculty• Industry, especially employers of engineers

and technologists• Governmental bodies and regulators


Desired Final Outcome

A fully functional stable accrediting body, operating with clear rules and regulations, and with a transparent and simple structure

Reputation for independence in accreditation decisions

Membership of the accrediting body in the appropriate mutual recognition accords


Opportunities to learn from existing bodies…

Structure and basic processes

Criteria

Methodology • especially self studies and outcome-based techniques

Development of constituency coalitions

Decision independence• “independence from any parent entity, or sponsoring entity,

for the conduct of accreditation activities and determination of accreditation status” (CHEA 2007)


Opportunities to improve on the operations

of existing accrediting bodies

Better use of information technology and automation

A more continuous and smooth process

Experimentation with less centralized models• The Peer Review and Coalitional models


New opportunities for accrediting bodies…

Development and provisions of tools for continuous reporting, assessment and improvement

Creation and maintenance of a registries • of engineers, computer scientists, and

technologists


Who should govern the accrediting body?

Professional associations Academic institutions Industry

Institutions from the three sectors should be invited to become Members of the accrediting body

Voting Members in the annual/bi-annual assembly of Members

Governments should be invited to observe and advise


Possible structure of a new accrediting body


Member Assembly

Board of Directors

Engineering Computing TechnologyCOMMISSIONS

Program Evaluators

STAFF


How should the accreditation body be financed?

Participation fees • From professional associations and industry

Accreditation fees • From participating institutions

Grants and gifts• For special projects and research


Stakeholders

Academic institutions• Presidents, provosts, chancellors, deans

Industry• Major employers of engineers, computer scientists,

and technologists Professional associations

• Local, local sections of transnational organizations, transnational organizations with local sections

Governments• Ministries of education and industry, accreditation

oversight bodies Other Civic Organizations


Possible dangers

Not enough interest

Failure to include all major stakeholders• Especially Industry

Poor finances

Lack of decision independence

Competition/meddling by outside accrediting bodies

Political infighting


Questions or Comments?


Additional Sources

Presentations in EAB workshops on accreditation• Esp. by Lyle Feisel, see www.ieee.org/education

Public domain information provided by ABET Inc., EUR-ACE, the Washington Accord website, CHEA• Mostly from the organizations’ web sites

J.W. Prados, G. D. Peterson, and L.R. Lattuca: “Quality Assurance of Engineering Education through Accreditation: The Impact of Engineering Criteria 2000 and Its Global Influence,” Journal of Engineering Education, pp. 165-184, January 2005.

Prof. Dr. Dirk Van Damme (Ghent University, Belgium). Accreditation in global higher education. The need for international information and cooperation. Outline of IAUP approach. May 2000.

accreditation of engineering, technology and computing programs moshe kam ieee vice president for...

Documents

ieee ieee

ieee slide

ieee eab

accreditation of engineering

accreditation process

area of accreditation

ieee bod

respects ieee