abet & ncaaa accreditation course design workshop design workshop.pdf · abet & ncaaa...

ABET & NCAAA ACCREDITATION

Course Design Workshop

Prof. Ali M. Al-Bahi

Director of the Academic Accreditation Unit

Faculty of Engineering , King Abdulaziz University

Modified and Presented by

Dr. Faisal Iskanderani

UBTVR for Q&D

Sept 11, 2014

Session #1

September 11, 2014 1

Introduction

why Learning Outcomes?

Student Learning Experience


Introduction

September 11, 2014 2Course Design Workshop

CEIT will be working towards National, and International Accreditation ( NCAAA and ABET)

Time line: end of 2016

Today’s workshop will deal with one component of accreditation, i.e., Students’ Learning Outcomes delay this


Introduction

ABET and NCAAA accreditation is assurance that a college or university program meets the quality standards established by the profession for which it prepares its students.

NCAAA and ABET have similar accreditation criteria

We will work together in parallel to achieve both at the same time, using the same documentation


Introduction

ABET Documents

Criteria for Accrediting Engineering Programs

Accreditation Policy and Procedure Manual

Template for the Engineering Self Study Report

Course binders or files

NB: Documents should be downloaded for the year of the accreditation cycle from:

http://abet.org/

5Course Design WorkshopSeptember 11, 2014

NCAAA Documents

National Qualifications Framework,

NCAAA Standards for the Accreditation and Quality Assurance for Institutions and Programs (11 Standards),

Self Evaluation Scales for NCAAA Standards, and

Forms and Templates for Institutions and Programs as follows: www.ncaaa.gov.sa


NCAAA Forms and Templates

Course Specification

Course Report

Field Experience Specification

Field Experience Report

Program Specification

Annual Program Report

Template for Report on Periodic Program Self-Study


Accreditation Time-Line (a 2-3 year process)

8

1

2

35

4

9

6 8

7

Course Design WorkshopSeptember 11, 2014

Note: In addition to one year for preparations

ABET Criteria for Accrediting Eng. Programs

1. Students2. Program Education objectives (PEOs).3. Student Outcomes (SOs). We will call them

“Program Learning Outcomes”4. Continuous Improvement5. Professional Component (curriculum).6. Faculty.7. Facilities.8. Institutional Support.

+ Program Criteria.

9September 11, 2014 Course Design Workshop

1. Students2. Program Education objectives (PEOs).3. Student Outcomes (SOs). We will call them

“Program Learning Outcomes”4. Continuous Improvement5. Professional Component (curriculum).6. Faculty.7. Facilities.8. Institutional Support.

+ Program Criteria.


ABET Criteria for Accrediting Eng. Programs

NCAAA Standards for Accreditation and

Quality Assurance1. Mission and objectives (ABET C2)

2. Governance and administration (ABET C6 )

3. Management of quality assurance and improvement (ABET C4)

4. Learning and teaching (ABET C3)

5. Student administration and support services (ABET C1)

6. Learning resources (Part of ABET C7)

7. Facilities and Equipment (Part of ABET C7)

8. Financial planning and management (ABET C8)

9. Employment processes (Part of ABET C6)

10. Research (Part of ABET C6)

11. Institutional relationships with the community (ABET C2)


NCAAA Domains of Learning Outcomes

knowledge, the ability to recall, understand, and

present information including: 1. knowledge of

specific facts, 2. knowledge of concepts, principles

and theories, and 3. knowledge of procedures.

cognitive skills, the ability to apply understanding

of concepts, principles, theories and procedures in

critical thinking and creative problem solving, both

when asked to do so and when faced with

unanticipated new situations.

September 11, 2014 Course Design Workshop 13


Interpersonal skills and responsibility, the ability to: 1. take

responsibility for their own learning and continuing personal and

professional development, 2. work effectively in groups and exercise

leadership when appropriate, 3. act responsibly in personal and

professional relationships, and 4. act ethically and consistently with high

moral standards in personal and public forums.

Communication, information technology and numerical skills,

including the ability to: 1. communicate effectively in oral and written

form, 2. use information and communications technology, and 3. use

basic mathematical and statistical techniques.

Psychomotor skills involving manual dexterity (this domain applies

only to some programs).

NCAAA Domains of Learning Outcomes


NCAAA Domains of Learning Versus

ABET Outcomes• Domain of Knowledge (Not existing in ABET):

1. Ability to recall, understand, and present knowledge of facts, concepts,

principles and theories of Math and basic sciences.

2. Ability to recall, understand, and present knowledge of discipline-related

practices and procedures.

• Domain of cognitive skills (corresponding to a, b, c, e, h, & j)

3. Ability to identify, formulate and solve real-life discipline related

problems.

4. Ability to design, implement and evaluate a discipline related system or

process to satisfy desired needs.

5. Ability to identify, analyze, and critically evaluate arguments and take

informed criteria-based decisions.


ABET Outcomes

• Domain of Interpersonal skills (corresponding to i, d, & f)

6. Ability to take responsibility for their own learning and

continuing personal and professional development

7. Ability to work effectively in groups and exercise leadership

when appropriate.

8. Ability to act responsibly in personal and professional

relationships.

9. Ability to act ethically and consistently with high moral

standards in personal and public forums.


ABET Outcomes

• Domain of Communication, IT, and numerical skills (corresponding to a, g, & k)

10. Ability to communicate effectively in oral and written forms

11. Ability to use information and communications technology.

12. Ability to use basic mathematical and statistical techniques.

• Domain of psychomotor skills (not existing for engineering, IT, or managerial education)

Why Learning Outcomes?


Why Learning Outcomes?ABET Definition of Program Learning Outcomes

Statements that describe what the students are expected to know and be able to do by the time of graduation. These relate to the knowledge, skills, and behaviors that the students acquire as they progress through the program.


Why learning outcomes?

Course Learning Outcomes

Each course should have a set of specific learning outcomes that lead to:

More measurable student outcomes

Better assessment

Higher quality feedback

Improved courses and programs

Improved student learning and achievementSeptember 11, 2014 20Course Design Workshop

IntendedLearning

Outcomes ofthe Lesson

IntendedLearning

Outcomes ofthe Unit

IntendedLearning

Outcomes ofthe Course

Deliver Forward

Design Backward

Infusion of Learning Outcomes into Courses

ProgramOutcomes


22

Criterion 3 Program Learning Outcomes (a-k)

a)an ability to apply knowledge of mathematics, science, and engineering,

b)an ability to design and conduct experiments, as well as to analyze and interpret data,

c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability,

d)an ability to function on multi-disciplinary teams,

e)an ability to identify, formulate, and solve engineering problems,


23

f) an understanding of professional and ethical responsibility,

g) an ability to communicate effectively,h) the broad education necessary to understand the

impact of engineering solutions in a global and societal context,

i) a recognition of the need for, and an ability to engage in life-long learning,

j) a knowledge of contemporary issues, andk) an ability to use the techniques, skills, and modern

engineering tools necessary for engineering practice.


Criterion 3 Program Learning Outcomes (a-k)

NCAAA and ABET Definitions

for Learning and Teaching

Program Educational Objectives:

Broad statements that describe what graduates are expected to attain within a few years after graduation. Program educational objectives are based on the needs of the program’s constituencies.

Program Learning Outcomes:

Describe what students are expected to know and be able to do by the time of graduation. These relate to the knowledge, skills, and behaviors that students acquire as they progress through the program.



for Learning and Teaching

Course Learning Outcomes:

Describe what students are expected to know and be able to do by the end of the course delivery. These relate to the knowledge, skills, and behaviors that students acquire as they progress through the course.


Assessment : one or more processes that identify, collect, and prepare data to evaluate the attainment of student outcomes and program educational objectives. Effective assessment uses relevant direct, indirect, quantitative and qualitative measures as appropriate to the outcome or objective being measured. Appropriate sampling methods may be used as part of an assessment process.



for Learning and Teaching (Cont’d)

Evaluation: one or more processes for interpreting the data and evidence accumulated through assessment processes. Evaluation determines the extent to which student outcomes and program educational objectives are being attained. Evaluation results in decisions and actions regarding program improvement



for Learning and Teaching (Cont’d)

Mission, Objectives, Outcomes & Learning Experience

Learning

Experience

Curriculum &

Other Activities

Student

Outcomes

of the

Program

Program

Objectives

Mission

Accomplished

Undergraduate

Studies

Upon

Graduation

Work

environmentMission



Learning

Experience

Curriculum &

Other Activities

Program

Learning

Outcomes

Program

Objectives

Mission

Accomplished

Undergraduate

Studies

Upon

Graduation

Work

environmentMission



A path to achieve Program Learning Outcomes and foster the attainment of program educational objectives:

1. Curriculum Design (into courses and other learning activities)

2. Course Design3. Teaching and Learning4. Assessment (course-level and program-level)5. Evaluation and continuous improvement

Teaching versus Learning

• Education has become outcome-based instead of being content-based.

• Educational activities became centered around learning instead of teaching.

INPUTS OUTCOMES

INPUTS OUTCOMESLearning

Domains of Learning

Levels of Learning

Writing Course Learning Outcomes

36September 11, 2014

Session # 2



Domains of Learning


Domains of the Learning Outcomes

The outcomes of any learning activity, or the change in the learner’s behavior, may fall in one of 3 distinguished domains:

– Cognitive Domain that involves knowledge and the development of intellectual skills

– Affective Domain that includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes.

– Psychomotor Domain that includes physical movement, coordination, and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution.



Levels of Learning


Cognitive Domain (mental) – Levels of Learning

Knowledge

Comprehension

Application

Analysis

Synthesis (Creative Thinking) Evaluation (Critical Thinking)


Levels of Learning

Classical Terms New Terms• Evaluation

• Synthesis

• Analysis

• Application

• Comprehension

• Knowledge

•Creating

•Evaluating

•Analyzing

•Applying

•Understanding

•Remembering(Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8)

The names of six major categories were changed from noun to verb forms. 41September 11, 2014 Course Design Workshop

Affective Domain (Attitudes) – Degrees of

internalizationThis domain includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes.

1. Receiving: Awareness. Willingness to hear. Selected Action Verbs : chooses, asks,

replies, selects

2. Responding: Active participation. Attends to and demonstrates compliance,

willingness or satisfaction in responding. Selected Action Verbs: complies, conforms, assists

3. Valuing: Is based on the internalization of a set of values, and are expressed in overt and

identifiable behavior. Selected Action Verbs: Demonstrates, proposes, initiates, justifies

4. Organizing: Values are organized into priorities. The emphasis is on comparing, relating,

and synthesizing values. Selected Action Verbs: arranges, integrates, synthesizes, defends

5. Internalizing (characterization): A value system that controls behavior. Behavior

is pervasive, consistent, predictable, and characteristic of the values held. Selected Action Verbs : acts, proposes, discriminates.

Psychomotor Domain (levels of physical skills)

The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution. 1. Set: Readiness to act Includes mental, physical and emotional sets. Selected Action Verbs :

begins, displays, moves, proceeds2. Guided Response: Early stages in learning a complex skill. Involves imitation and trial &

error. Selected Action Verbs: copies, traces, follows, reproduces, responds3. Mechanism: Intermediate skill in learning a complex skill. Includes habituation and

proficiency. Selected Action Verbs: assembles, constructs, dismantles, displays4. Complex Overt Response: Skillful performance that involves complex movement patterns.

Proficient in speed and accuracy. Selected Action Verbs: Same as Mechanism, but will have adverbs that indicate performance is quicker, more accurate and automatic

5. Adaptation: Skills are well-developed and the individual can modify to fit special requirements. Selected Action Verbs: adapts, alters, revises

6. Origination: Creating new patterns to react to particular situations. Emphasis on

creativity based on highly developed skills. Selected Action Verbs: Builds, combines, composes.

A Fluid Mechanics Example

Level Explanation Sample Question

RememberingThe student is expected to recite memorized

information about the concept.What is a fluid?

UnderstandingThe student is expected to explain the concept

in his or her own words.

Why mercury is considered as a

fluid?

ApplyingThe student is expected to apply the concept to

a particular situation.

Calculate the position of the

shock wave in the given nozzle

Analyzing

The student is expected to separate materials or

concepts into component parts so that their

organizational structure may be understood.

Analyse the flow field inside the

test section a high speed wind

tunnel.

EvaluatingThe student is expected to make judgments

about the value of ideas or materials.

Compare the given two nozzle

designs that perform the same

task, which one is better and

why?

Creating

The student is expected to put parts together to

form a whole, with emphasis on creating a new

meaning or structure.

Design a variable area nozzle to

execute the following flight

mission.

A Computer Programming Example

Level Explanation Sample Question

RememberingThe student is expected to recite memorized

information about the concept.What is a program?

UnderstandingThe student is expected to explain the concept

in his or her own words.

How is a program similar to a

recipe?

ApplyingThe student is expected to apply the concept to

a particular situation.

What is the output of the

following program

Analyzing

The student is expected to separate materials or

concepts into component parts so that their

organizational structure may be understood.

Identify the customer high level

requirements based on the

following customer interview.

What additional questions you

may need?

EvaluatingThe student is expected to make judgments

about the value of ideas or materials.

Given two programs that

perform the same task, which

one is better and why?

Creating

The student is expected to put parts together to

form a whole, with emphasis on creating a new

meaning or structure.

Write a program to perform a

given task


Writing Learning Outcomes


Writing Learning Outcomes

• While it is possible to write learning outcomes of all types for each of the three domains, the vast majority are written for the cognitive domain.

Action Verbs (used in writing learning outcomes)

Taxonomy Categories Sample Verbs for Stating Specific Learning Outcomes

Remembering

Understanding

Applying

Analyzing

Evaluating

Creating

Cite, Define, Identify, Label, List, Match, Name, Recognize, Reproduce, Select, State

Classify, Convert, Describe, Distinguish Between, Explain, Extend, Give Examples,

Illustrate, Interpret, Paraphrase, Summarize, Translate

Apply, Arrange, Compute, Construct, Demonstrate, Discover, Modify, Operate,

Predict, Prepare, Produce, Relate, Show, Solve, Use, calculate

Analyze, Associate, Determine, Diagram, Differentiate, Discriminate, Distinguish,

Estimate, Infer, Order, Outline, Point-out, Separate, Subdivide, Model.

Appraise, Assess, Compare, Conclude, Contrast, Criticize, Discriminate, Evaluate,

Judge, Justify, Support, Weigh

Combine, Compile, Compose, Construct, Create, Design, Develop, Devise, Formulate,

Integrate, Modify, Organize, Plan, Propose, Rearrange, Reorganize, Revise, Rewrite,

Tell, Write



Learning

DomainsMeaning Suggested Verbs

Knowledge

List of basic knowledge that

students should know and

understand when they complete the

course.

list, name, record, define, label,

outline, state, describe, recall,

memorize, reproduce, recognize,

record, tell and write.

Cognitive

Skills

List of the thinking and problem

solving skills the course is intended

to develop. The list should include

the ability to:

(1) Use analytic and predictive

formulae as well as conceptual

tools that students should use when

asked to do so.

(2) Identify and use the tools that

are appropriate for new and

unanticipated problems.

estimate, explain, summarize, write,

compare, contrast, diagram,

subdivide, differentiate, criticize,

calculate, analyze, compose, develop,

create, prepare, reconstruct,

reorganize, summarize, explain,

predict, justify, rate, evaluate, plan,

design, measure, judge, justify,

interpret and appraise.

NCAAA Suggested Guidelines for Learning Outcome Verbs



Learning

DomainsMeaning Suggested Verbs

Interpersonal

Skills &

Responsibility

List of the outcomes related to

students’ interpersonal skills,

capacity for self directed learning,

and personal and social

responsibility.

demonstrate, judge, choose,

illustrate, modify, show, use,

appraise, evaluate, justify,

analyze, question, and write.

Communication,

Information

Technology,

Numerical

List of outcomes related to

students’ communication, IT and

numerical skills.

demonstrate, calculate,

illustrate, interpret, research,

question, operate, appraise,

evaluate, assess, and criticize.

NCAAA Suggested Guidelines for Learning Outcome Verbs


1st Break (10 Minutes)


Elements of Course Design

Step by Step Course Design & Assessment


Session # 3


Project

Course Goals & Course Intended

Learning Outcomes

Instruction

Lectures Labs

Institutional technologies

Student expectations

Syllabus

Assessment

Homework

Exams and Quizzes

StudentPortfolio

Surveys

Process Check

Student outcomes

Instructor's goals

Levels of Learning

Students

Plus & Delta

Modified from Felder and Brent

Learning Methods: Problem-Based, Project-based, Active Cooperative

Feedback from ABET Program Committee

To A

BET

Pro

gram

C

om

mit

tee

STEP BY STEP COURSE DESIGN& Assessment


STEP BY STEP COURSE DESIGN & Assessment

1. Determine Program Core Courses

2. Apply Course Design Methodology for each Core Course:• Define the goals of the course• Define the Course Intended Learning Outcomes• Construct the Course Mapping Matrix• Construct the Course Articulation Matrix• Prepare the course syllabus

3. Construct Mapping Matrix for all core courses

4. Apply Course Assessment Methodology for each Core Course

5. Evaluate Course Assessment results


Program Core Courses

Program core courses: a set of courses designated to address some or all of the Program outcomes.

Compulsory courses in the Program would be obvious candidates for the core.

Compulsory courses given by other Programs, such as math & physics might be included as long as they consistently address outcomes.

Elective courses or courses whose content varies from one offering to another can't be included.


Step #1: Define the goal or goals of the course

• Two to three lines :“The course is intended to……. ”

• Example (Data Base Management Systems):

Example: The course is intended to:

“Develop the student’s ability to understand, design, and implement Data Base Management tools.”


Step #2: Define The Course Intended Learning Outcomes

• Intended Learning Outcomes: specific observable (measurable) actions that the students should be able to perform if they mastered the course. i.e.

“ by the completion of this course, the students should be able to:1. …2. …………………”

The first word must be one action verb from the process descriptive verbs related to LOL (see the first part of this workshop)


• Look at the items or concepts in detailed course content.

• For each item or concept decide what are the actions or tasks you want the students to do in order to demonstrate their level of mastery. Also think of how you will measure these actions (i.e. your assessment tools) and write down a suitable task statement.

• Each task statement includes one key action verb [such as list, explain, calculate, estimate, derive, model, design, choose, and criticize] along with a definition of the task and possibly a specification of the conditions under which the task is to be performed.



• Do you want the students to define terms, explain them, apply them to known situations, to new situations, or use them to design a new artifact or process?

• Think of the levels of learning and their associated action verbs and associate a targeted level of learning to each course learning outcome.

• The best choice is to have a maximum of 9 to 12 course learning outcomes (one to 2 per course topic). The more they are detailed, the easier you could design their assessment, but you may find it difficult to have sufficient assessment tools.



Course Title: Introduction to Database Systems

Current Catalog Description:

Introduction to the database management systems, database administration, data processing, data modeling, database design, development, and implementation. Contrasts alternative modeling approaches. Includes implementation of current DBMS tools and SQL.

Major Topics Covered in the Course:

• Data modeling

• Logical database design

• Physical database design (including performance issues)

• SQL and other database access methods

• Database Administration (including Data Warehousing and Administration)


Step #2: Define The Course Intended LO (EXAMPLE)


(Good Examples)

Data modeling

1. Model data relevant to a database task, given written description, reports and other information from a systems user.

2. Model data using entity-relationship as well as object-oriented methods.

Logical database design

3. Transform entity-relationship model into a logical design, following the relational approach.

Physical database design (including performance issues)

4. Assess aspects of the physical design, and transform the logical design accordingly.



(Bad Examples)

• Understand the fundamentals of data base design (not measurable)

• Students demonstrate that they can derive and properly applythe equations of …. (redundant words- what is meant by properly apply? In which situation)

Statements of non-observable actions (such as “learn”, “know”, “understand ”, or “appreciate”) might qualify as educational objectives BUT NOT Intended learning Outcomes

Understanding cannot be directly observed; the student must do something observable to demonstrate his understanding.


Hands-on Exercise

Forming work teams

Define the course goals

Define a set of Intended course learning

outcomes

Presenting

Session # 3

Forming Teams


Forming work Teams• Now we will form teams of 3 members each.

• All members of each team belong to one

program.

• Each team will select one core course that

falls in a domain of their common interest.


Define the Course Goals


Step#1: Define the goal or goals of the course

(10 minutes)

• Use the catalogue description of the course to define its goals from the instructors view point

• Take 5 minutes to write 2 - 3 lines :“The course is intended to……. ”

• Remember the following example (Data Base Management Systems):

The course is intended to:“Develop the student’s ability to understand, Design, and implement Data Base Management tools.”


Define a set of Intended course learning

outcomes


Step#2 Define the Course Intended Learning

Outcomes (30 minutes)

• take 20 minutes to prepare 4 to 5 intended course learning outcomes (1 to

2 CLOs per major topic).

• Notice that each CLO should start by an action verb

• Define the level of learning for each CLO.

• Map each CLO to one or more of ABET Program Learning Outcomes.

Follow the following examples:

Physical database design:

• Assess aspects of the physical design, and transform the logical design accordingly. (Evaluating, level #3, outcome c).

SQL and other database access methods

• Model data relevant to a database task, given written description from a systems user. (Analyzing, level #2, outcome c).


Empty Course Articulation Matrix


Program LO


Presenting Your Work

73

• Now one team will come to podium.

• Take 5 minutes to present

September 11, 2014 Course Design Workshop

Presenting


2nd Break (30 Minutes)



Session # 4 Learning Depth and Breadth

Apply Course Design Methodology for each

Core Course

Step #3. Construct the Course Mapping Matrix

Step #4. Construct the Course Articulation Matrix

Documenting Course Design Data


Learning Depth and Breadth


1. The Depth of Learning is measured by level of learning in the cognitive domain

2. The breadth of learning is measured by the number of contact hours allocated to the learning activities targeting an outcome.

3. The 6 levels of learning are grouped into 3 levels of learning depth:• Level #1: low level of learning: Knowledge and Comprehension

or Remembering and Understanding (weight = 1).• Level #2: Medium level of learning: Application and Analysis or

Applying and Analyzing (weight = 2).• Level #3: High level of learning: Synthesis and Evaluation or

Evaluating and Creating (weight = 3).

Learning Depth and Breadth


Student’s Exposure

1. The product of depth of learning of a given student outcome by the corresponding breadth in a course indicates the student’s exposure to this outcome in that course as compared to other courses addressing the same student outcome.

2. Student outcomes that have high exposure in a given course are considered as the critical outcomes of that course and the course is considered as a key course for these outcomes.

3. Add a column to your articulation matrix containing the CLOcoverage hours.

4. Exposure of an outcome = SUM of (the Product of the alignment coefficient of the outcome with each CLO by CLO coverage).

Course Design WorkshopSeptember 11, 2014 79


Apply Course Design Methodology for

each Core Course


STEP BY STEP COURSE DESIGN & Assessment

1. Determine Program Core Courses

2. Apply Course Design Methodology for each Core Course:• Define the goals of the course• Define the Course Intended Learning Outcomes• Construct the Course Mapping Matrix• Construct the Course Articulation Matrix• Prepare the course syllabus

3. Construct Program Mapping Matrix for all core courses

4. Apply Course Assessment Methodology for each Core Course

5. Evaluate Course Assessment results

Course Design WorkshopSeptember 11, 2014 81Course Design Workshop

Step #3: Construct the course mapping matrix “Course

Learning Outcomes – Program Learning Outcomes”

1: low levels of learning in Bloom’s Taxonomy (Remembering and understanding)2: medium levels of learning in Bloom’s Taxonomy (Applying and Analyzing)2: high levels of learning in Bloom’s Taxonomy (evaluating and creating)

IE 301 LOL

1 1 1

3 1 1

2 1

Maximum

Attainable1 3 3 2

a b c d e f g h i j kCLO_1

CLO_2

….

ml n

CLO_3


Step #4: Construct the course mapping

matrix (Continued)

• Some schools fill the matrix with an indicator to demonstrate to which degree (High, medium, low) this course learning outcome (CLO) contributes to the achievement of the corresponding student outcome (using letters such as H, M, and L).

• Other schools indicate the corresponding level of learning (LOL).

• A simplest method is to use a binary system (ONE if the Course LO addresses the Program LO and ZERO if it does not).


Useful approach: Course Articulation

MatrixProgram LO



Program LO


Documenting Course Design Data


Documenting Course Design Data• A course articulation matrix assigns 1-2 Intended Course

Learning Outcome (CLOs) to each topic and competency (a total of 9 to 12 CLOs per course).

• Each CLO is associated with a certain number of hours of exposure (contact hours allocated to this CLO).

• In-class & out-of-class learning activities are indicated for each CLO.

• Classical assessment tools are indicated for each CLO.

• CLOs are mapped into Student Learning Outcomes (a-k).

• Key outcomes are identified as those having the highest exposure.

Session # 4

Hands-on Exercise

Construct the Course Mapping Matrix (20 minutes)




Program LO


Session # 5

Course Assessment & Course Binder


Course Assessment


Why Assessment

• Helps students understand where they should focus their time & energies

• Motivates students to do their best

• Helps students understand their strengths and weaknesses through feedback

• Gives students documentation of what they have learned which can be used to apply for jobs, awards, programs of advanced study, etc.


What Is Assessment? – ABET Definition

One or more processes that identify, collect, and prepare data to evaluate the attainment of student outcomes and program educational objectives. Effective assessment uses relevant direct, indirect, quantitative and qualitative measures as appropriate to the outcome or objective being measured. Appropriate sampling methods may be used as part of an assessment process.


Embedding Assessment With Regular Reflection

Course

Section 1

Teaching &

Learning

Assessment of

Section 1

Topics

Evaluation of

Achievement of

Section 1 CLOs and

Related SLOs Using

HI-CLASS

Revisions

of Course

Plan

Course

Section 2

Teaching &

Learning

Begin

Section 2Begin

Section 1

Embedding Assessment with Regular Reflection Approach Normal teaching Approach


Direct Assessment Methods

(Clear and Compelling Evidence) • “Capstone” experiences (research projects, presentations, theses, oral

defenses, exhibitions, or performances) scored using a rubric • Other written work, performances, or presentations, scored using rubrics • Portfolios of student work • Scores on locally-designed multiple choice and/or essay tests (e.g. final

examinations in key courses, qualifying exams, comprehensive exams) accompanied by test “blueprints” describing what the tests assess

• Score gains between entry and exit on published or local tests or writing samples

• Summaries/analyses of electronic discussion threads • Student reflections on their values, attitudes and beliefs, if developing

those are intended outcomes of the course

Indirect Assessment Methods

(Signs that Students Are Probably Learning, But Exactly What or How Much They Are Learning is Less Clear)

• Course grades (see next slide)

• Assignment grades, if not accompanied by a rubric or scoring guide (See an Example)

• Student ratings of their knowledge and skills and reflections on what they have learned in the course or program

• Questions on end-of-course student evaluation forms that ask about the course rather than the instructor


Assessment and Grading• They do have one common characteristic: both intend to identify what the students

have learned.

• Grades alone do not always give direct evidence: they don’t identify which specific student learning outcomes and at what levels students have learned.

• Some course grades include additional student behaviors that are not related to student learning outcomes (e.g. attendance and participation).

• For example, a grade of B in a mathematics course indicates a student learned an acceptable level of math, but not the specific math knowledge or skills that the student mastered well or what areas need improvement.

• A project or assignment grade would be a better link to the evidence of student learning, but a rubric for the project or assignment would provide the best direct evidence and the level at which the student learned.

• Grades are based on direct evidence of student learning such as the evaluations of tests, papers, and projects, but need to be clearly linked and aligned to learning outcomes and/or rubrics to suffice as direct evidence for assessment purposes.


Proposed Course Level Assessment Tools

• Each semester

– Indirect measures:• Course Learning Outcomes (CLOs) questionnaire (exit and

sometimes entry)

• Course in-class and out-of-class learning activities questionnaire (exit)

• Student Outcomes (SOs) questionnaire (exit)

– Direct measures:• Course Exams, HWs, Design projects, Written reports, Oral

presentations, Course portfolios, Lab Reports evaluated using rubrics or mapped into CLOs

• Assignments for key SOs evaluated using rubrics .


Course Level Indirect Assessment

• Course Learning Outcomes (CLOs) Questionnaire:

– Put your course learning outcomes in the form of a questionnaire to measure the students confidence level in mastering the course learning outcomes. Use the 1 to 5 Eckert scale to cover Excellent-Medium-to-Poor mastery levels.

– Add some questions to evaluate the importance of course prerequisites.

– Distribute the questionnaire twice during the semester: in the 2nd week (entry, if you admit this approach) and/or in the 12th

week (exit) of the course.

– Answer the questionnaire yourself to indicate your estimation of the students average level of mastery. (see an example)


Course Level Indirect Assessment (continued)

• Course learning activities (or instructional tools) Questionnaire:

– Ask the students to evaluate the impact of different in-class and out-of-class learning activities on their level of mastery of course learning outcomes. Use the 1 to 5 Eckert scale to cover Excellent-Medium-to-Poor impact.

– Distribute the questionnaire in 12th week.

– Answer the questionnaire yourself to indicate your estimation of the impact of the learning activities.

(see an example)


Course Level Indirect Assessment (continued)

• Student Outcomes Questionnaire:– ABET Committee should divide each outcome into

outcome elements and to define outcome attributes (measures) or KPIs associated with these outcome elements.

– A questionnaire should be developed for each outcome and used by the instructor who pretends that his course addresses this outcome.

– The questionnaire is to be distributed during the 12th

week of the semester.

(see an example of outcome attributes/indicators and outcome Questionnaire)


Course Level Direct Assessment

• Direct Assessment of:

– HI-CLASS or Similar Assessment of Course Learning Outcomes (CLOs) – Direct Approach

– Direct Rubric-Based Assessment of Key Student Outcomes.


Construct the Articulation Matrix (15 minutes)

Now let us fill the Learning Activities and Assessment tools into the Matrix

THANK YOU …

ABET & NCAAA ACCREDITATION


Prof. Ali M. Al-Bahi

Director of the Academic Accreditation Unit

Faculty of Engineering , King Abdulaziz University

Modified and presented by

Dr. Faisal Iskanderani

UBTVR for Q&D

Sept 11, 2014
