![Page 1: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/1.jpg)
![Page 2: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/2.jpg)
Kelly Morris, CHRL
416-923-2324 x342
Exams Manager
Human Resources Professionals Association
![Page 3: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/3.jpg)
John Wickett, PhD
514-794-1396
Principal
Wickett Measurement Systems Inc.
![Page 4: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/4.jpg)
Office of the Registrar2017 Fall How-To webinar series
Wednesday,September 13, 2017
Validation of Experience Requirement Explained
Tuesday,September 19, 2017
Information for Internationally Educated Professionals (IEPs)
Thursday,September 28, 2017
Maintaining your designation through CPD
Thursday,October 12, 2017
Exam Scoring Explained
![Page 5: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/5.jpg)
On-demand webinars
• Our previously aired regulatory and how-to webinar series can be found on the Regulatory Webinars page of our website.• Listen to recording
• Download the power point slides
• View the Questions & Answers
5
![Page 6: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/6.jpg)
Housekeeping
•Webinar will be recorded and posted online
•Will post answers to questions that we could not answer in the webinar
![Page 7: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/7.jpg)
Questions involving specific individual circumstances
• This webinar is not the appropriate place and time to address specific individual circumstances
• Sometimes the correct answer depends on some details that are not provided with the question
• Please contact the Office of the Registrar with questions involving specific individual circumstances
![Page 8: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/8.jpg)
Exam Scoring Explained October 12, 2017
![Page 9: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/9.jpg)
Today’s Webinar
• Who does what with respect to exams Exam Validation Committees and their roleWhy does HRPA delegate the development and scoring of the exams
• Overview of CKE1, CKE2 and CHRP/CHRL Employment Law Exams• Exam Development process
HRPA’s Competency FrameworkTest specifications and blueprint
• Post exam analysisKey validationAngoff and EquatingCompensatory and non-compensatory scoringApproval of pass mark process
![Page 10: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/10.jpg)
Who does what with respect to examsOffice of the Registrar Staff
• Office of the Registrar staffI. Claude Balthazard, CHRL – VP Regulatory Affairs & Registrar
II. Kelly Morris, CHRL – Exams Manager
III. Thomas Callitsis – Exams Coordinator
• Manage the exam registration & administration process
• Release of exam results
![Page 11: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/11.jpg)
Examination Vendor – Computer Based Testing
• Prometric – www.prometric.com
• Exam delivery via computer based testing
• Scheduling system – www.prometric.com/hrpa
• Management of test centres and exam invigilation
• Exam accommodations
![Page 12: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/12.jpg)
Delegation of exam development and scoring
• In Ontario, the exams are prescribed pursuant to statutory authorities delegated to HRPA
• HRPA delegates to Wickett Measurement Systems the task of developing and scoring exams
• Delegation is not a transfer of authority or an abdication of responsibility, HRPA remains fully responsible and accountable for the exams that are developed and scored by Wickett Measurement Systems
• The delegation is done to manage possible conflicts of interest and relies on the psychometric expertise of Wickett Measurement Systems
![Page 13: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/13.jpg)
Exam Committees
• CHRP Exam Validation Committee – (CKE1 and ELE1)Exam item validation
Key validation and approval of the pass mark
Approval of exam blueprint
CHRL Exam Validation Committee – (CKE2 and ELE2)Exam item validation
Key validation and approval of pass mark
Approval of examination blueprint
** Adhoc activities – item writing, item review
![Page 14: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/14.jpg)
Overview of Exams Comprehensive Knowledge Exam
• Comprehensive Knowledge Exam (CKE1)Valid for the CHRP designation only 3 1/2 hours in length 175 questions, 150 scored items 25 experimental items 4 option multiple choice format Delivered via computer based testing
• Comprehensive Knowledge Exam (CKE2)Valid for CHRP and CHRL designation(s) 5 hours in length 250 questions, 225 score items 25 experimental items 4 option multiple choice format Delivered via computer based testing** coursework requirement is the same to write CKE 1 vs. CKE 2
![Page 15: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/15.jpg)
CHRP and CHRL Employment Law Exams
• 110 questions: 100 scored items, 10 experimental items
• The time limit is 3 hours (moving to 3 ½ hours in 2018)
• Items will be a mix of case and independent 3 option multiple choice questions
• Delivered via a computer
• Offered 3 times a year
• Candidates have access to relevant legislation
https://www.hrpa.ca/hrdesignations_/Pages/CHRP-CHRL-Employment-Law-Exam.aspx
![Page 16: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/16.jpg)
Exam Development Cycle
Item Writing
Editorial
Item Review
Validation
Admin #1
Admin #2
Admin #3
Annual Report
![Page 17: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/17.jpg)
HRPA Competency Framework
• The HRPA’s Competency framework is the body of knowledge for which the CKE 1 and CKE 2 is based upon
• The competency framework is used to develop exam questions• 9 functional domains
StrategyProfessional PracticeOrganizational EffectivenessWorkforce planning and Talent ManagementLabour& Employee RelationsTotal RewardsLearning and DevelopmentHealth Wellness and Safe WorkplaceHR Metrics Reporting &Financial Management
![Page 18: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/18.jpg)
Test Specifications
• Blueprint for the CKE 1 and CKE2
https://www.hrpa.ca/hrdesignations_/Pages/CKE1-CKE2-Comprehensive-Knowledge-Exam.aspx
• Blueprint for the CHRP/CHRL Employment Law Exam
https://www.hrpa.ca/hrdesignations_/Pages/CHRP-CHRL-Employment-Law-Exam.aspx
![Page 19: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/19.jpg)
Exam Administration Flow
![Page 20: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/20.jpg)
Post Exam Analysis – Key Validation
• Scoring of the exams is a multi step process
• Data extraction• Data is sent to a secure ftp site.
• Each candidate has an XML file that contains all their test information.• Form taken
• Responses
• Comments on each question (if applicable)
• Post-Exam Survey question responses
![Page 21: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/21.jpg)
Post Exam Analysis – Key Validation
• Data cleaning• Remove outliers
• Need homogeneous population to avoid skewed metrics.
• Candidate comments.
• Post-Exam Survey.
![Page 22: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/22.jpg)
Post Exam Analysis – Key Validation
• Key Validation• Difficulty and discrimination.
• Exam Validation Committee reviews items to determine which will not be used for scoring.• Too easy
• Too difficult
• Poorly discriminating
• Candidate comments
• The EVC decisions are considered final and the remaining items (150 for CKE 1, 225 for CKE 2, and 100 for ELE) will be used to score candidates.
![Page 23: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/23.jpg)
Angoff and Equating
• Periodically, standard setting is conducted using the Modified Angofftechnique.• EVC plus others.
• Establish the minimum expected performance.
• Assess every test item as to how well that ‘borderline’ candidate will perform.
• Equating is used otherwise.• Fairness is paramount.
• No matter when an exam is taken, equal chance of passing if same ability.
• Adjusts for differences in exam difficulty.
![Page 24: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/24.jpg)
Angoff and Equating
• Using some common items across test forms, can evaluate if a group of candidates is of the same, lower, or higher ability.• If higher ability, more should pass.
• If lower ability, fewer should pass.
• Using performance of all items relative to the common items, can evaluate if exam form is of the same difficulty, easier, or harder.• If the exam form is easier, the pass mark should go up.
• If the exam form is harder, the pass mark should go down.
• Using established statistical techniques, a precise adjustment to the pass mark is calculated.
![Page 25: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/25.jpg)
Compensatory and Non-Compensatory Scoring
• Pass/fail decisions can be based just on one total score, or based on performance across several domains.
• When based on one total score, this is called compensatory scoring because a candidate can make up for poorer performance in one domain with better performance in another.
• When candidates must show sufficient performance on several domains, this is called non-compensatory scoring because doing well in one domain will not make up for poorer performance in another domain.
![Page 26: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/26.jpg)
Compensatory and Non-Compensatory Scoring
• The CKE 1, ELE 1, and ELE 2 are scored using a compensatory system.• Pass/fail decisions are made based on a total score only.
• The CKE 2 is scored using a non-compensatory system.• Candidates must pass based on a total score and based on their score in each
of the 9 functional areas.
• This is done to reflect the greater emphasis placed on sufficient competence in every functional area for CHRL candidates.
• Regardless of method, scoring is done based on items approved by the Exam Validation Committee and using equating to establish thresholds.
![Page 27: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/27.jpg)
Approval of pass mark process
• Whether equating or Angoffing is used to establish the pass mark, the Exam Validation Committee reviews it.
• If they are comfortable that the process used to establish it was sound, they will recommend it to HRPA for final approval.
• Once approved, all candidates are scored against the pass mark (and against the individual functional area thresholds for CKE 2 candidates), and pass / fail decisions are made.
• Several confirmatory checks are made to verify the accuracy of the computed scores and they are then provided to HRPA.
![Page 28: Kelly Morris, CHRL - HRPA · PDF fileKelly Morris, CHRL –Exams Manager ... •Scheduling system – •Management of test centres and exam invigilation](https://reader035.vdocuments.net/reader035/viewer/2022082211/5a9f32bd7f8b9a84178c7efc/html5/thumbnails/28.jpg)
Questions