Florida Department of Elder Affairs

Assisted Living Facilities (ALF)

Core Competency Assessment System

Prepared by George MacDonald AFL Principal Investigator Assistant Director for Research and Grant Development David C. Anchin Center College of Education University of South Florida June, 2012

2

Table of Contents

Table of Contents ................................................................................................................................... 2

Standardized Assessments ...................................................................................................................... 4

Criterion-Referenced Assessment ...................................................................................................... 4

Benchmark Criterion-Referenced Assessments ................................................................................. 4

Reliability and Validity ...................................................................................................................... 5

Reliability ....................................................................................................................................... 5

Validity ........................................................................................................................................... 5

Standard Setting ................................................................................................................................. 6

The Assisted Living Facilities Core Competencies Certification Assessment (ALF-CCCA) ............... 6

Difficulty of ALF-CCCA ................................................................................................................... 6

Steps in Creating a Valid and Reliable Criterion-Referenced Mastery Cut Score Assessment ............. 6

Assisted Living Facilities Core Competencies Specifications Document .......................................... 7

ALF Core Competencies Curriculum ................................................................................................. 9

ALF Core Competencies Assessment ................................................................................................ 9

Spanish Version .................................................................................................................................. 9

3

George MacDonald is the Assistant Director for Research and Grant Development at the David C. Anchin Center, University of South Florida. He is the lead researcher for the. Anchin Center’s Edumetrics Consortium for Advanced Modeling and Assessment. The Consortium conducts advanced assessment and modeling employing Classical Test Theory, Item Response Theory (IRT) and Cognitive Diagnostic Assessment Modeling. The Consortium employs psychometric techniques such as Exploratory Factor Analysis, Confirmatory Factor Analysis, Multi-Level Modeling, Structural Equation Modeling, and Linear Logistic Test Modeling. The IRT modeling includes item and test level assessment.

Mr. MacDonald publishes and is invited to present his research at state, national and international Conferences. His research foci include Cognitive Diagnostic Assessments, Mathematics Education, instrument development and creation, reliability generalization, and United States patriotism. MacDonald recently completed a two-year term on the National Committee of the National Council on Measurement in Education (NCME). He is a member of the Assessment Committee for Evaluation, Measurement, & Statistics of the American Psychological Association (APA). He also serves as an active member of the Florida Educational Research Association (FERA).

Mr. MacDonald is the Principal Investigator (PI) for the Assisted Living Facilities (ALF) State Certification Examination which is administered through the Anchin Center under the auspices of the Florida Department of Elder Affairs. MacDonald is the PI for a number of federally funded evaluations which include the Magnet School Assistance Program, Voluntary Public School Choice, the Teacher Incentive Fund, BRIDGE Project, and the Smaller Learning Communities Initiative. In addition, MacDonald served as the lead researcher for the 2011 publication of Florida’s First Comprehensive Conditions of Education Report.

Mr. MacDonald holds a B.A. in Philosophy & English Literature, an Honors B.A. in Psychology, Suma Cum Laude, and a Masters in Divinity. He is completing a Ph.D. in Curriculum and Instruction with an emphasis in Educational Research and Measurement. His cognate is focused in Mathematics Education and Cognition.

4

Standardized Assessments

Standardized assessments have the same standard for questions, administration,

scoring and interpretation. Any assessment in which the same test is given in the same

manner to all examinees is a standardized assessment. Standardized tests are

designed in such a way that the questions, conditions for administering, scoring

procedures, and interpretations are consistent and are administered and scored in a

predetermined, standard manner. For an assessment to be standard, all of these areas

must be consistent. Some standardized assessments scores are criterion-referenced,

which determines how well students know a standard. For effective evaluation,

standardized assessments need to have reliability and validity.

Standardized assessments are perceived as being fairer than non-standardized

assessments. Consistency is integral to standardized assessment: consistency permits

more reliable comparison of outcomes across all assessment takers. Questions on

standardized assessments are usually objective, such as multiple-choice and true or

false. This allows for easy and efficient grading. Along with consistent questions, the

administration, scoring and interpretation of the assessment must be standardized.

Criterion-Referenced Assessment

Criterion-referenced assessments do not rank students against their peers; instead,

they determine how well students meet a particular performance or established

standard. These assessments show a raw score and a score summary. If a student

scored 8 out of 10 on a criterion-referenced assessment, this would indicate 80 percent

mastery of material.

Benchmark Criterion-Referenced Assessments

Benchmark Assessments are criterion-referenced standardized assessments that

are used to evaluate how well students have mastered a particular performance

standard. Criterion referenced assessments are often used to classify examinees as

masters or non-masters according to whether the assessment score exceeds or does

not exceed a cut score or criterion.

5

Reliability and Validity

The considerations of validity and reliability typically are viewed as essential

elements for determining the quality of any standardized assessment.

Reliability

A standardized assessment needs to be reliable. The term reliability refers to the

construction of an assessment in which groups who take the assessment over time

receive the same results as the assessment sample normed group. The smaller the

variance on assessment results over time the higher the reliability of a standardized

assessment. Makers of standardized assessments try to construct tests that lack errors.

Standardized assessments with major inconsistencies in assessment performance are

not considered reliable measurements.

Validity

In psychometrics, validity has a particular application known as test validity: "the

degree to which evidence and theory support the interpretations of test scores”

(American Educational Research Association, Psychological Association, & National

Council on Measurement in Education. (1999). Standards for Educational and

Psychological Testing. Washington, DC: American Educational Research Association.)

The term validity in standardized assessments refers to students actually taking the

assessment. A standardized assessment could be reliable in construction, but is not

standard if there are discrepancies in taking the assessment. For an assessment to be

valid, the instructions must be followed exactly. For example, if the instructions state the

assessment should be administered in a two-hour block, a proctor who arbitrarily gave

assessment-takers three hours would invalidate the assessment. Also, the conditions

for the assessment should be similar to those for the sample norm group. If an air

conditioner broke down and students took the assessment in a 90 degree classroom,

the validity of the assessment would be questionable.

6

Standard Setting

Standard setting is an evaluative decision. Measurement assists us but ultimately it

is an evaluative decision. Assuming you have a construct, and can therefore describe

what a person who passes has mastered, changes to the derived cut score should be

considered in terms of content. If the cut score is reduced, then we are giving up

mastery of some content. If this kind of loss is acceptable, then we proceed. Otherwise,

you fix your cut score at that point where mastery of the content is considered an

essential requirement. The weight of these decisions is qualitative and is supported by

quantitative analysis.

The Assisted Living Facilities Core Competencies Certification Assessment

(ALF-CCCA)

The Assisted Living Facility (ALF) Core Competencies Certification Assessment is a

reliable and valid criterion-referenced standardized assessment with a mastery or cut

score of seventy-five percent. The goal of the assessment is to identify ALF

administrators who have mastered the ALF Core Competencies curriculum.

Difficulty of ALF-CCCA

The ALF-CCCA is a difficult test in which approximately half of the examinees pass

on their first attempt. To identify a master of the ALF-CCCA, it is imperative that the

mastery cut score continue to be at least seventy-five percent.

Steps in Creating a Valid and Reliable Criterion-Referenced Mastery Cut Score

Assessment

A system must be in place to support a reliable and valid criterion referenced

mastery cut score assessment By system we mean a whole compounded of several

interacting and interdependent components forming an integrated whole. The

interacting and interdependent components of the ALF-CCCA examination are Core

Competency Standards, Core Competency Standard Statements, Core Competency

Standard Description, Core Competency Standards Benchmarks, Core Competency

Curriculum, and Core Competency Assessment.

7

Assisted Living Facilities Core Competencies Specifications Document

Key to this system is a Core Competencies Specifications Document. The Core

Competencies Specification Document is made up of Core Competency Standards,

Core Competency Standard Statements, Core Competency Standard Description, and

Core Competency Standards Benchmarks. The following is an exemplar of a Core

Competency Specifications Document based on Competency Standard 1:

Administration of an Assisted Living Facility.

Florida Department of Elder Affairs

COMPETENCY STANDARD 1: ADMINISTRATION OF AN ASSISTED LIVING FACILITY

It is expected there will be at least 10 questions on the examination from this competency standard. This training module is expected to take at least 4 hours to complete.

PURPOSE: Definitions, licensing and administration of an ALF, state laws and rules, staffing,

background checks, admission procedures, appropriateness of placement, residency criteria, inspections and recordkeeping. COMPETENCY STANDARD STATEMENT:

PART I: PURPOSE, DEFINITIONS, LICENSING REQUIREMENTS FOR FACILITY AND ADMINISTRATOR, STAFFING, BACKGROUND SCREENING, FORMS, RECORD LOGS AND RECORDKEEPING

PART II: ADMISSION PROCEDURES, ADMISSION-DISCHARGE, APPROPRIATENESS OF PLACEMENT, RESIDENCY CRITERIA, VIOLATIONS, WAIVERS AND FUNDING

COMPETENCY STANDARD DESCRIPTION: Competency Standard 1, Administration of an ALF, includes the administrative and

operating procedures, admission and residency criteria, admission and discharge procedures, appropriate placement, recordkeeping, forms, fines, and staffing.

In addition, this standard includes current laws and rules related to the management of an

ALF; licensure and background screening requirements; and the use of adverse incident forms. This competency module will provide participants with the following information: Part I: 1. The purpose of an ALF and the availability of services for elderly and disabled adults. 2. The types of ALFs – LNS, LMH, ECC, Standard. 3. The administrative process including the minimum requirements for licensing of an

assisted living facility and related definitions. 4. How to maintain an administrator credentials, and an administrator’s responsibilities. 5. The admission and residency criteria including appropriate placement of residents and

assistance with daily living (ADL). 6. License types, license capacity, license renewal, change of ownership, change of

administrator, closure and other required forms.

8

7. Staffing requirements and how to calculate hours properly. 8. Background screening requirements for administrative, managers, and direct staff of

ALFs. Part II: 1. Admission and discharge procedures and provision of services-- Rule 58A-5.0181,

F.A.C. 2. Resident admission criteria for facilities holding limited nursing services, limited mental

health, or extended congregate care – Rule 58A-5.030, Section 429.41, F.S., and Chapter 69A-40, F.S.

3. Facility policy and procedures regarding resident placement and medical examination form(s).

4. How to determine appropriate of placement -- Section 429.26, F.S. 5. Identify violations, grounds for moratorium, and administrative penalties and fines. 6. Facility inspection(s) and responsibilities. 7. Facility adverse incident forms/reports and how to develop a plan of action and/or a

quality assurance program. 8. Admission-discharge logs. 9. The types of waivers and assistance programs offered by federal/state funding such as

Medicaid, OSS, Assisted Living waiver, Nursing Home Diversion, and Personal Assistive Services.

BENCHMARKS: After participating in this competency standard training, the student will be able to understand and interpret the following information, concepts and ideas:

Part I: 1. Recognize the purpose of the Assisted Living Facilities Act and the availability of

services for elderly persons and adults with disabilities. – Section 429.01, F.S. 2. Comprehend definitions related to operating/owning an assisted living facility. – Rule

58A-5.0131 / Section 429.02, F.S. 3. Understand initial licensure, renewal, change of ownership applications and conditional

licenses. 4. Distinguish between licensing types -- standard license, limited nursing services license,

limited mental health license, and extended congregate care license. Section 429.04, F.S.

5. Acknowledge change of ownership instructions and how to apply for a provisional license.

6. Review the state’s regulatory agencies and their responsibilities – the Agency for Health Care Administration, the Department of Elder Affairs, the Department of Children and Families, Agency for Persons with Disabilities, Adult Protective Services, and the Department of Health.

7. Understand administrative penalties, violations, and the suspension or denial of a license renewal.

8. Discuss enforcement, timelines and types of deficiencies. 9. Review facility inspections such as county inspections relating to fire safety, hygiene,

sanitary practices, and biomedical waste – Ch. 59A-5.0161. Part II: 1. Understand the admission and discharge process; how to fill out the admission forms

and package; required files to keep on residents and staff; how long to keep discharge information; and the contracted provision of services.

9

2. Examine differences and requirements among ALF license types from standard to limited mental health, limited nursing services and extended congregate care and the types of care that are required in each type of facility.

3. Review appropriate placement procedures and regulations including examples of resident needs and continuance of stay at the facility.

4. Discuss monitoring and citing violations, moratoriums, administrative penalties, sanctions, license revocation, denial of the renewal license or denial of a change of ownership to a new operator.

5. Discuss various types of facility inspections and investigations from the Agency for Health Care Administration, Department of Health, local county health inspectors, and the Fire Marshall as they pertain to licensing.

6. Understand definition and the process of contracted care through managed care organizations (MCO), various waiver programs, and optional state supplementation (OSS).

7. Understand how to determine if an incident is adverse and guidelines for completing the adverse incident forms.

8. Identify how to develop a plan of action and quality assurance program.

ALF Core Competencies Curriculum

Once the ALF Core Competencies Specification Document has been created and

approved, the next step is to write a curriculum based on the ALF Core Competencies

Specifications Document. At the same time as the curriculum is being created a new

Assessment Instrument based on the Specifications document will be created. The ALF

Core Competencies curriculum provides trainers in the State of Florida with a

standardized version of the materials to be taught. Standardization is critical for the

reasons mentioned above.

ALF Core Competencies Assessment

While the ALF Core Competencies curriculum is being prepared, a ALF Core

Competencies Assessment Instrument will be written. This instrument will be based on

the Core Competencies Specifications Document and assess each of the Competency

Standards. We strongly recommend there be a minimum cut score established for each

standard as well as for the entire examination.

Spanish Version

To ensure standardization (discussed above) all documents are translated into

Spanish and made available by the Florida Department of Elder Affairs (DoEA). To

accomplish the necessary Spanish translation, the DoEA, in consultation with USF, is

10

establishing a state level working committee. The DoEA/USF Spanish Committee will

be staffed by USF personnel trained in World Languages. The committee will be tasked

with translating statutes, rules; the ALF Core Competency Specifications Document;

Curriculum; and Assessment materials into Spanish. Further, the DoEA/USF Spanish

Committee will work within the Spanish Assisted Living Facility community statewide to

identify, engage, and mentor key personnel in critical areas including: core competency

trainers, potential committee members, curricular experts, and administrators.

This effort within the Spanish community and the translation of materials into

Spanish, as part of the Core Competencies Assessment System, is meant to address

the differential ethnic/racial performance of Spanish examinees on the Assessment.

ALF Negotiated Rulemaking Committee Information Request ALF Test 2011 

I. Summary of Test Results by Module 

• Form 1500 and Form 1501 are in English, at sites offering the CCT in Spanish, Form 2550 is administered with Form 1500 and Form 2551 is administered with Form 1501.

Module (# items) Form 1500

Form 1501

Form 2550

Form 2551

A. General License Activity (5) 0.823 0.836 0.768 0.720

B. Administration (19) 0.704 0.733 0.603 0.591

C. Records (6) 0.640 0.693 0.626 0.694

D. Residency (13) 0.806 0.784 0.700 0.679

E. Food Service (6) 0.758 0.705 0.611 0.635

F. Medication Management (24) 0.720 0.755 0.656 0.661

G. Personal Care and Services (6) 0.794 0.676 0.797 0.610

H. Special Needs Population (6) 0.685 0.698 0.580 0.496

I. Resident Rights (9) 0.908 0.905 0.807 0.799

J. Enforcement Activities (6) 0.724 0.749 0.588 0.576

All Modules (100) 0.750 0.757 0.666 0.648

II. Pass Rates by Language of Test Administration 2011 

1. 1650 individuals registered to sit for the Core Competency Test 2. 1583 individual tests were scored, 1358 English tests (Form 1500 and Form 1501) and

225 in Spanish (Form 2550 and Form 2551).

Figure #

Language Registrations Successful Examinees Percent Successful

English 1358 799 58.8

Spanish 225 42 18.6

Total 1583 841 53.1

III. Pass Rates by Test Form 2011 

Test Form Registrations Successful Examinees Percent Successful

Form 1500 666 374 56.1

Form 1501 679 421 62.0

Form 2550 87 17 19.5

Form 2551 138 25 18.1

Form 0500 8 2 25.0

Form 0501 5 2 40.0

Total 1583 841 53.1

2011 ALF Competency Exam Trainer Pass/Fail Rates Trainer

ID Number

Registrations Percent of

Registrations Number

Examinees Successful Examinees

Unsuccessful Examinees

Successful Percent

1. 0 24 1.45 10 8 2 80.00 2. 3 376 22.79 368 251 117 68.21 3. 5 99 6.00 96 66 30 68.75 4. 6 4 0.24 4 3 1 75.00 Retakes 9 279 16.91 260 85 175 32.69

5. 10 23 1.39 22 6 16 27.27 6. 11 11 0.67 11 5 6 45.45 7. 12 101 6.12 99 69 30 69.70 8. 14 3 0.18 2 0 2 0.00 9. 16 17 1.03 15 8 7 53.33

10. 21 159 9.64 154 64 90 41.56 11. 25 3 0.18 3 0 3 0.00 12. 26 3 0.18 3 1 2 33.33 13. 27 9 0.55 8 4 4 50.00 14. 29 7 0.42 6 2 4 33.33 15. 33 84 5.09 83 21 62 25.30 16. 35 1 0.06 1 1 0 100.00 17. 36 61 3.70 59 21 38 35.59 18. 40 30 1.82 30 19 11 63.33 19. 54 1 0.06 1 0 1 0.00 20. 57 25 1.52 24 9 15 37.50 21. 58 133 8.06 132 90 42 68.18 22. 59 64 3.88 62 39 23 62.90 23. 61 5 0.30 5 3 2 60.00 24. 62 28 1.70 27 5 22 18.52 25. 63 14 0.85 13 3 10 23.08 26. 64 41 2.48 41 26 15 63.41 27. 65 35 2.12 34 28 6 82.35 28. 66 10 0.61 10 4 6 40.00

All 1650

1583 841 742 53.13

10 new trainers were approved in 2012

ALF Training Requirements                          Revised 4/19/2012 

Training Requirements for Assisted Living Facilities (ALF)  Beginning January 7, 2012 

 

Type of Training   ALF Administrator or Designee  ALF LTC Worker Is DSHS Required to 

Approve Curriculum?

First Aid And CPR  Within 30 days of employment; maintain valid card.  

Within 30 days of employment; maintain valid card. 

NO 

Orientation  2 hours 

Prior to providing care to a resident unless exempt from training.   

Prior to providing care to a resident unless exempt from training.    

YES 

Safety Training 3 hours 

Prior to providing care to a resident unless exempt from training.   

Prior to providing care to a resident unless exempt from training.    

YES 

70 hours of core basic and population specific training  

Within 120 days of employment unless exempt from training.  

Within 120 days of employment unless exempt from training.   

YES 

Cont. Ed.   10 hrs per calendar year. 12 hrs per year beginning 7/1/2012. 

12 hrs per year beginning 7/1/2012. 

YES.  Curriculum approval is not required until 7/1/2012. 

HIV/AIDs   All ALF staff, 2 hrs within 30 days of employment. Basic training includes this.  

All ALF staff, 2 hrs within 30 days of employment. Basic training includes this. 

 

Nurse Delegation Core and Diabetes 

Must be a Nursing Assistant Registered, Nursing Assistant Certified, or Certified Home Care Aide and complete ND core training before accepting a delegated task. 

Must be a Nursing Assistant Registered, Nursing Assistant Certified, or Certified Home Care Aide and complete ND core training before accepting a delegated task. 

Must use DSHS curriculum.  Instructors must be DSHS contracted community instructors. 

Specialty Training 

If an ALF serves one or more residents with special needs, the administrator or designee must complete manager specialty training and demonstrate competency by passing the DSHS test. 

If a resident develops special needs while living in an ALF without a specialty designation, the administrator or designee have 120 days to complete manager specialty training and demonstrate competency. 

If an ALF serves one or more residents with special needs, all LTC workers must receive training regarding the specialty needs of individual residents in the facility.  

 

YES 

Instructors must be DSHS contracted community instructors to provide manager specialty training. 

ALF Workgroup Training Recommendations: Crosswalk for Core Training

Core Training The following suggestions were recommended in the ALF Workgroup Final Recommendations report. • Increase the core training hours from 26 to 40 hours and include the following new topics:

Topics Recommended

Required in Current Rule

Elopement Prevention 58A-5.0191(2)(f) 1 hour

Aggression, de-escalation, behavior management, and proper use the Baker Act

Do Not Resuscitate Orders

58A-5.0191(11) 58A-5.0186 1 hour

Infection Control 58A-5.0191(2)(a)

Admission, continuing residency, and best practices Phases of caregiving and interacting with residents

Business operations including human resource management Financial management Supervision of staff

Addition to Core for Administrators: Extended Congregate Care Training - 8 hours Limited Mental Health - 8 hours

DATA AS OF: 06/13/2012 TOP 10 VERIFIED COMPLAINTS IN ASSISTED LIVING FACILITIES06/12/2011 TO 06/12/2012

PAGE 1 OF 1FILE: H:\Programs\ALF\Rulemaking\Negotiated Rulemaking\Meetings\June 26, 2012\2012-06-13 Verified complaints - Summary

TAB: Summary

Top 10 COMPLAINT DESCRIPTION CountCOMPLAINT DESCRIPTION TotalMenu (1003) 145Medications - administration, organization (0605) 134Cleanliness, pests, general housekeeping (1102) 111Equipment/Buildings (1103) 106Dignity, respect - staff attitudes (0403) 84Personal property (0503) 70Activities - choice and appropriateness (0901) 69Privacy-telephone, visitors, couples, mail (0408) 68Shortage of staff (1302) 59Billing/charges (0501) 55Grand Total 901