let quality guide evaluation quality: recent trends implemented in the middle east an eclectic...
TRANSCRIPT
Let Quality Guide Evaluation Quality: Recent Trends Implemented in the Middle East
An Eclectic Approach to Evaluation for Higher Education Institutions:
Lessons Learned in Saudi Arabia
Dr. Eqbal Z. DarandariNational Commission for Academic Assessment and Accreditation (NCAAA)
Mrs. Tahira Hoke Prince Sultan University (PSU)
Presented at American Evaluation Association, November 13, 2010
Overview• Recent Trends in Evaluation of Higher Education:
Middle East - Saudi Arabia• Role of HE Evaluation in Saudi Arabian Higher
Education• NCAAA’s Assessment, Quality Assurance and
Accreditation System: Built in Evaluation Approaches.• Common Evaluation Approaches Used by HE
Education Quality Assurance Directors• Program Evaluation and Paradigm Shift: Benefiting
from Other Disciplines and Quality Systems• Quality Talk and Walk: How can new trends in quality
systems enlighten evaluation quality?
Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia
• Middle East and Gulf states trends and practices
• Recent trends and practices implemented in Evaluation and quality systems in higher education. • Evaluation- Accreditation• Internal-External• Quality Management – Quality
Assurance/Improvement
• Rapid growth in post-secondary institutions.
• Establishment of Saudi National Commission for Academic Accreditation & Assessment (NCAAA) in 2004.
• In only four years, NCAAA has developed, established, tested, and implemented a new quality assurance system for higher education, benefiting from the international expertise, while keeping its unique characteristics.
Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia
Accreditation means….
• Is providing quality education outcomes• Meets standards for leadership, facilities,
teaching and learning, assessment, ……. (national or international)
• Has established an internal system of self evaluation and monitoring.
• Is committed to external peer evaluation• Is committed to continuous improvement
Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia
• Saudi Institutions Implement the New Quality Assurance System 2007-2010
• Capacity Building at National and Institutional Level
• Quality assurance has become central in educational reforms.
Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia
Self-evaluation involving negotiations- reflections with clear directions and
expectations led to collective group
decisions and forced systematic and
sustainable improvements.
Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia
Role of HE Evaluation in Saudi Arabia Higher Education
New drives behind Saudi HE Quality Assurance and Accreditation System: • Educational Reform,
Global Recognition, Public Trust,
• transformation, and integration of stakeholders and cooperation of various actors.
What is behind HE Assessment + Quality + Accreditation System?
Criteria for Quality Assurance and
Accreditation in HE in SA• Supporting materials such
(such as handbooks detailing quality assurance processes, BQF, Institution + Program Standards, and self-evaluation scales, templates for programme plans and reports, Key Performance Indicators, and student surveys).
Fitness for
purpose
Standards
Fitness of Purpose
Role of HE Evaluation in Saudi Arabia Higher Education
NCAAA 11 Standards
A) Institutional Context: 1) Mission, Goals and Objectives; 2) Governance and Administration; 3) Management of Quality Assurance and Improvement;
B) Quality of Learning and Teaching: 4) Learning and Teaching;
C) Support for Student Learning: 5) Student Administration and Support Services; 6) Learning Resources;
D) Supporting Infrastructure:7) Facilities and Equipment; 8) Financial Planning and Management; 7) Employment Processes;
E) Community Contributions: 10) Research; 11) Institutional Relationships with the Community.
Standards as A system: Reach to the Main Beneficiary (Maffet, 2010)
Systemic Thinking
NCAAA Quality Assurance and Accreditation Process: System
Environmental Scan: SWOT Analysis
Building Capacity Meeting Standards + Mission and Objectives
Initial Self EvaluationEstablishing Baseline
Establishing a continuous system of
internal evaluation and assessment
Addressing Improvement
Plan
Shift in learning, teaching, and assessment system
System based on Learning
Outcomes National Qualifications Framework: specifies
generic standards of learning outcomes based
on educational level
Learning Outcomes Courses
Course Code and Number
Knowledge Facts Concepts, theories Procedures
Cognitive Skills Apply skills when asked Creative thinking and problem solving
Interpersonal Skills and Responsibility
Responsibility for own learning
Group participation and leadership
Act responsibly-personal and professional situationsEthical standards of behavior
Communication IT and Numerical Skills
Oral and written communication
Use of IT
Basic math and statistics
Psychomotor Skills
from
To
NCAAA Quality Assurance and Accreditation Process: Outcomes
Plan
Monitor
Implem
ent
Review
Saudi NCAAA Quality Cycle
NCAAA’s Assessment, Quality Assurance and Accreditation System: Built in Evaluation Approaches
• New quality system borrowed from the effective evaluation and other disiplines tools and approaches
• Successfully integrated them into multilevel staged transformative processes through designed qualitative change.
Common Borrowed Approaches• Input-Processes-Outcome• Empowerment• Participatory• Responsive• Critical reflections• Self-Evaluation• Dialectical Evaluation• Developmental Evaluation• System thinking• Strategy Evaluation
• Depending on the context and institution quality culture, quality assurance directors employed evaluation methods most appropriate for the size, type of institution.
Common Evaluation Approaches Used by HE Quality Assurance Directors
Note: Multiple Response Item
Common Evaluation Approaches Used by HE Quality Assurance Directors : Survey Results
The Percentage Increase in Evaluation Activities in a Sample of Saudi Universities and Colleges Between March 2007 and August 2008. (Darandari, et al., 2009; Darandari, & Hoke, 2007)
0.29
0.34
0.74
0.54
0.57
0.40
0.54
0.57
0.40
0.40
0.31
0.46
0.26
0.54
0.77
0.35
0.59
1.00
1.00
0.59
0.53
0.76
0.65
0.65
0.76
0.82
0.82
0.18
0.88
0.88
0.000.200.400.600.801.001.20
15- External Reviews
14- Internal Reviews
13- Quality Assurance Committees
12- Strategic Planning
11- Curriculum Reviews
10- Satisfaction Surveys
9- Course Reports
8- Program Reports
7- Institutional Reports
6- Events/Activities Evaluation
5- Workshop Evaluations
4- Self-Evaluation
3- Peer-Observation
2- Chair Evaluation of Teachers
1-Student Evaluation of Teachers
Eva
luat
ion
Act
ivity
Percent Mar.2007 Aug.2008
Quality Assurance as An Empowerment Tool for Women: A Case from
Saudi Arabia
INQAAHE Conference, 2009 Eqbal Z. Darandari
Common Evaluation Approaches Used by HE Quality Assurance Directors : A Case Study
Vision for the Future: How can new trends in quality enlighten evaluation quality.
Program Evaluation and Paradigm Shift: Benefiting from Other Disciplines and Quality Systems
Benchmarking
Peer ReviewContext - DynamicsSystems theories and thinkingComplex SystemsChange theoryMultifaceted Need analysis
Compliance vs. Reliance
Evaluating Strategy MissionVisionFitting
purpose
TruthEngagement
ContextBeyond methodsGlobal thinking
Advocate Evaluation
Developmental Evaluation
UtilityInnovation
Visualizing Data for monitoring and decision making
Performance Measurement
Loops
Self Evaluation and Reflection
Learning ProcessSelf Reporting
Emerging Quality Themes in Program Evaluation
Evidence based
Documented
Continuous Improvement
(Plans-Indicators-benchmarks)
OwnershipCommitment
Quality Culture
Leadership
Periodic-External and
internal Review
Quality Asurance Standards +
Criteria(Evaluator-
Program – Client)
Quality Talk and Walk: How can new trends in quality systems enlighten evaluation quality?
FrameworkAlignmentConnection
?????Balance and
Direction
Quality is not just meeting Standards and following a process protocol.
It is al about comprehensive quality system.
It is about culture and built in continous improvement
Quality Talk and Walk:
Improvement Process Components
28
Plan andImplementation
What future are we
pursuing ?Vision
Profile
What is our current reality?
What actions will we take to
improve?
What have we accomplished?
Results
Monitoring
Drawing quality map clarified to all stakeholders.
Participants need to know where they are, where they want to go, how they are similar or different from others nationally and internationally, what should be changed and to what level, what the alternatives are.
29
Evaluation
Continuous improvement
Quality Evaluation Needs to work within a system and guide to incremental continuous
improvement
30
