it’s not just numbers: implementing point-in-time counts, using hmis, and ensuring data accuracy...
TRANSCRIPT
It’s Not Just Numbers: Implementing Point-in-Time
Counts, Using HMIS, and Ensuring Data Accuracy
Erin Wilson, Abt Associates Inc.
Julie Eberbach, Iowa Institute for Community Alliances
September 23, 2008
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development2
Overview
• Learning Objectives
– Review basics of Point-in-Time (PIT) counts
– Learn how to incorporate HMIS into PIT counts
– Understand how to prepare HMIS for use in a PIT count
– Learn simple data checks for population and subpopulation information reported to HUD
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development3
What Does It Mean to “Count” Homeless People?
• A “count” = collecting information about the sheltered and unsheltered homeless population in your community
• It is possible just to do a count, but preferable to gather descriptive information too
Data on the number of homeless
people
+Descriptive
information on those counted
-Demographic
-Service Use
-Needs
=Point-in-time
count of homeless
people
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development4
Why Count Homeless People?
• For planning and program development
– Understand characteristics and needs
– Develop programs based on need
– Access resources for services and housing
• To measure progress in eliminating homelessness and to ensure accountability
• To raise public awareness
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development5
HUD Guidelines and Reporting Requirements
• According to HUD, PIT counts must be:– Actual counts or statistically reliable estimates of
homeless persons in sheltered and unsheltered locations on one night
– Conducted at least every other year during the last ten days of January (22-31) in odd calendar years
– 2009 is a required year!
• Designate one night for the count • CoCs must count:
– Sheltered and unsheltered adults, children and unaccompanied youth
– Number of households with and without dependent children
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development6
HUD Guidelines and Reporting Requirements
• At a minimum, count homeless persons according to HUD definitions and guidelines– Unsheltered, emergency, transitional
• CoCs may also want to collect additional information for local program planning – Homeless persons according to local definitions – In-depth information about a specific
subpopulation– Information about people at risk of homelessness
or precariously housed
• Must be able to report data according to HUD definitions and requirements!
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development7
Homeless Population Data
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development8
Homeless Subpopulation Data
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development9
How does HUD Use PIT Data?
• Reporting– The Annual Homeless Assessment Report
(AHAR) – Reporting the number of homeless persons in
each CoC on HUD’s website
• To make funding decisions Exhibit 1– HUD scores PIT counts based on quality of
methods and reliability of data• It’s about accurate counts, not large numbers!
– CoCs that conduct an annual PIT may receive additional credit
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development10
Counting Sheltered Homeless Persons
• Count people in emergency shelters and transitional housing programs– All programs listed on the e-HIC– Temporary emergency hotel/motel/apartment
vouchers for homeless people– Faith-based providers and VA homeless programs
• Collect data using:– Homeless Management Information System
• Requires high provider coverage + excellent data quality• All providers must enter/update information on all clients
in a bed on the night designated for the count
– Provider report or survey– Client-level survey using standardized instrument
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development11
Counting Unsheltered Homeless Persons
• Methods for counting unsheltered individuals:
– Street count using observation/interviews
• Complete coverage (block-by-block) and/or known locations
• Probability sampling
– Service-based count
• Non-shelter services (i.e. soup kitchens, healthcare centers)
• Can include outreach teams
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development12
Using HMIS for a PIT Count?
• Provides two components:– Count– Subpopulation information
• Typically used in combination with other data collection techniques to offset issues with low coverage
HMIS
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development13
Benefits and Limitations of Using HMIS for PIT Count
• Benefits of HMIS– Requires fewer resources over time than a non-
HMIS PIT count– Can reduce duplicate counting– Provides in-depth subpopulation data on persons
who are counted without intrusive or repetitive interviews
– Reinforces the value of the HMIS and contributes to year-round HMIS participation and data quality
• Limitations of HMIS– Requires high level of data quality and coverage
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development14
Key Considerations for Using HMIS
• Data quality
• Make sure information is gathered or extrapolated for each provider
• Start planning early
– 4 to 6 months prior to count
– Leave adequate time to assess data quality, improve data quality, and decide whether and how to use HMIS
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development15
Key Considerations for Using HMIS (cont.)
• Uses universal and program-specific HMIS data elements, as established by HUD
– HMIS Data and Technical Standards Final Notice (July 2004) FR 4848-N-02
– Revised HMIS Data and Technical Standards currently in public comment
• HMIS is a tool for gathering information about people who use services
– More useful for sheltered count
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development16
Using HMIS Data for an Unsheltered Count
• Most communities will need to continue street and service-based counts even if outreach workers are using HMIS
• Potential uses for unsheltered counts– Provides an opportunity to populate the HMIS with
information on unsheltered people– Helpful with de-duplicating street or service-based
counts
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development17
HMIS Data Elements for the Population Chart
Step 1: Determine if
the individual was in the
program on the night of the count…
Step 2: Classify whether
the individual is in an ES
or TH program…
2.13 Program ID2.10 Program Entry / 2.11 Program Exit
To count number of people and number of households…
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development18
HMIS Data Elements for the Population Chart (cont.)
Step 4: Determine whether the individuals should be counted as
a single or as part of a household with children…
Step 3: Determine whether the
individual is an adult or child…
2.3 Date of Birth
2.14 Household ID
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development19
HMIS Data Elements for the Subpopulation Chart
3.6 Mental Health
3.7 Substance Abuse
Severely Mentally Ill
Chronic Substance Abuse
2.6 Veteran Status
3.5 HIV/AIDS
Veterans
Persons with HIV/AIDS
3.8 Domestic ViolenceVictims of Domestic Violence
2.3 DOB + 2.14 Household ID
Unaccompanied Youth
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development20
HMIS Data Elements for Determining Chronic Homelessness
A chronically homeless person is defined as: • An unaccompanied homeless individual • with a disabling condition • who has either been continuously homeless for a year or
more OR has had at least four episodes of homelessness in the past three years.
2.3 Date of Birth
2.14 Household ID 2.13 Program ID
Determine if the individual is an
adult
Determine if the individual is
unaccompanied
Determine where the individual is
living (ES)
Step 1: Determine who is an unaccompanied homeless individual.
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development21
HMIS Data Elements for Determining Chronic Homelessness (cont.)
Step 2: Determine disability.
2.7 Disability
3.6 Mental Health
3.7 Substance Abuse
3.5 HIV/AIDS
3.3 Physical Disability
3.4 Developmental Disability
-OR-
Dis
abil
ity
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development22
HMIS Data Elements for Determining Chronic Homelessness (cont.)
Step 3: Determine chronicity.
2.10 Program Entry
2.11 Exit Across ES Programs and Stays
2.8 Residence Prior to Program Entry
Also consider if the individual was unsheltered prior to entering the program.
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development23
Examples of Using HMIS for a Count
HMIS
Option 1:
only – for count + subpopulation information
• 100% bed coverage required
HMIS
Option 2:for participating providers + extrapolation for non-participating providers
• Need 75% coverage rate for emergency shelter + transitional housing, both individuals + families• Non-participating providers need to be similar to participating providers (program type and clients served)
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development24
Examples of Using HMIS for a Count (cont.)
HMIS
Option 3:for participating providers
Option 4:
Paper/ manual survey
for non-participating providers
HMIS
for count Paper/ manual survey
for subpopulation information
Option 5:
HMIS
Participating providers complete paper/manual surveys using HMIS-generated reports Paper/
manual survey
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development25
Major Steps in Using HMIS for a PIT Count
Planning Steps and Suggested Timeline
6 months before – Assess HMIS Data Quality
3-4 months before – Finalize data collection approach and request provider participation
2 weeks before – Remind providers
3-5 days before – final data quality check
1 week after – Assess results and follow-up
In months after – Assemble findings
In months after – Evaluate the process
THE PIT COUNT
January
22nd – 31st
August OctoberSeptember November December January MarchFebruary
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development26
6 Months – Assess HMIS Data Quality
• As a first step, assess data quality at the system-level– Provider participation– HMIS data coverage
• If you think you may be able to use HMIS:– Begin in-depth data quality assessment and correct
data accuracy• Assess data quality at program level • Address data quality issues with providers
Number of people staying in shelter on a single night with data
reported in HMIS÷
Number of homeless people
served on that night overall
X 100
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development27
Ongoing – Assess and Improve Data Quality
• Develop reports to assess data quality – Client roster report: Verifies current population and
exit dates– Program and system-level occupancy reports:
Ratio of clients to beds– Missing (null) value report: Missing data fields– Program and system-level report to check for
duplicate records
• Develop a program and system-level Population and Subpopulation report
• Train program managers on using the reports
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development28
Ongoing – Assess and Improve Data Quality (cont.)
• Assess how programs are using the HMIS and adjust PIT reports as needed – Do some programs use a bed management system that
would provide more accurate PIT occupancy information than the program entry/exit dates?
– Are some programs exiting clients on the same day as entry?
– Do some programs do all data entry only once per week?
• Work with program managers to implement steps to improve data quality and clean up data errors
• Continue to run data quality reports to ensure that managers follow through as planned
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development29
3 to 4 Months – Finalize the Data Collection Approach
• Make a final decision about how/if you will use HMIS for the count
– For residential providers that are using HMIS:
• Are staff entering the necessary data fields on all clients for count and subpopulation information?
• Is complete data available for all clients? (e.g., are exit dates being entered regularly?)
• Is using HMIS for the count and subpopulation information a feasible strategy for all participating residential providers?
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development30
3 to 4 Months – Finalize the Data Collection Approach (cont.)
• Make a final decision about how/if you will use HMIS for the count (cont.)– For residential providers that are not using HMIS:
• Can you extrapolate from your existing HMIS data to cover these programs?
• Do you need to provide manual/paper surveys for non-participating providers to collect the required data?
• Establish a regular count meeting where the HMIS lead can coordinate with the point-in-time planning team
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development31
3 to 4 Months – Finalize the Data Collection Approach (cont.)
• Communicate with and secure the participation of providers– Work with the planning group to help ensure that
every provider is reporting information via HMIS or manual/paper survey
• Develop procedures for integrating the HMIS sheltered data with other sheltered and street survey data– For example, residential providers that complete
manual/paper surveys
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development32
2 Weeks – Remind Providers
• Remind program managers that the HMIS data is being used for the point-in-time count.
• Providers should:– Conduct a final data quality check during the days
before the count– Enter information about each required data
element for each client, including subpopulation information
– Enter data on a daily basis, even if they don’t routinely do so
• Give providers a date by which they must enter the data – typically two to three days after the count
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development33
A Few Days Before – Final Data Quality Check
• 2 to 3 days before the count:– Ask providers to review their active client rosters
one final time to ensure that all residents are reflected
– Run a system-level occupancy report to conduct your own estimate of problems and follow-up with providers as needed
• 1 day before the count:– Run the missing values report to see if critical data
elements are missing on current clients
– Ask providers to collect and enter the information while clients are still accessible
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development34
The Week After – Assess Results and Follow-up
• Evaluate the completeness/accuracy of data
• De-duplicate transitional/shelter programs
• Produce draft system-wide HMIS count reports for discussion– System-occupancy report showing list of programs
represented in the chart, along with client count and bed inventory for that night
– Overall counts and subpopulation estimates– Missing value report on count fields and
subpopulation fields to inform discussion on percent error
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development35
The Week After – Assess Results and Follow-up (cont.)
• Produce draft reports on the HMIS count for each program:– The count and subpopulation numbers– The client to bed ratio– The missing value % for each field
• Share these program reports with providers; work with providers to see if they can fill in any missing data
• Seeing the results may help providers realizethe benefit of HMIS and may motivate themto work on improving their data quality
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development36
The Month(s) Following the Count – Assemble Findings
• Develop final HMIS count report– Population and Subpopulations Chart– Document the process / methodology– Document limitations/external influences (i.e.
weather; barriers to achieving 100% accuracy)
• Merge HMIS sheltered count data with other data sources– Survey or extrapolated data from non-participating
providers– Survey data on client characteristics not captured
by HMIS (if HMIS used for count only)– Unsheltered count data
• Must de-duplicate with sheltered data
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development37
Evaluate the Process
• Should HMIS be used for future PITs? – For count data? For subpopulation data?
• Develop written recommendations on year-round and pre-count activities to improve the HMIS count for next year– Focus on year-round data quality– Identify critical providers to bring into the HMIS– Improve pre-count communication strategies– Define additional count-related report needs– Improve integration of data from other sources– Address other issues / concerns
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development38
Iowa Balance of State PIT Count
• One statewide HMIS implementation– The I-COUNT Network– Primarily rural, with some urban areas
• Strong data support structure– Iowa Council on Homelessness
• Research and Analysis Standing Committee plans PIT strategy
• Point in Time (Sheltered and Unsheltered)– Combined use of automated HMIS data and
manual survey instruments
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development39
Iowa BoS PIT Count Execution
• Research/Analysis Committee develops Point In Time strategy and timeline (sets date)
• HMIS staff execute the plan – Design of HMIS network reports– Development of hard copy collection instruments– Provides state-wide trainings for both sheltered
and unsheltered counts– Receives verified HMIS reports from participating
agencies and tabulates “hard copy” reports– Produces final Point In Time report for each
continua and the “state-wide” report
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development40
First Iowa Statewide Unsheltered Count
• Utilized “known locations” methodology
• Survey instruments were cooperatively developed
• State wide trainings provided for rural local Homeless Coordinating Boards
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development41
First Iowa Statewide Unsheltered Count (cont.)
• Iowa Department of Public Safety– Organized street count participation by law
enforcement (state, county and city)
• Iowa Department of Education– Facilitated participation of school district homeless
liaisons
• HMIS staff received survey summary documents and tabulated results for PIT inclusion– Staff also provided follow-up to “call-in” missing data
and also investigate any “anomalies”
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development42
Example of Homeless Street Count Survey (Part 1)
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development43
Example of Homeless Street Count Survey (Part 2)
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development44
Sample Data Collection Form
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development45
Example Data Collection Form
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development46
Overview of PIT Data Checks
• HUD conducts several data checks on all PIT data submitted by CoCs
• CoCs can check PIT data to ensure accuracy before submitting it to HUD
• Simple checks for:– Households with dependent children
– Households without dependent children
– Ratio of unsheltered to sheltered households
– Chronically homeless subpopulation numbers
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development47
PIT Data Checks – Households with Dependent Children
# of Households # of Persons in Householdsis less than
Average # of Persons per Household is typically
greater than 2, but less than 6
• For households with dependent children make sure:
# of Persons in Households
# of Households
If = 0, then = 0
# of Persons in Households
If = 0= 0, then # of Households
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development48
PIT Data Checks – Households without Dependent Children
• For households without dependent children make sure:
Average # of Persons per Household is typically less than 2
# of Persons in Households
is less than or equal to
# of Households
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development49
PIT Data Checks – Unsheltered to Sheltered Ratios
• Calculate ratio for households with dependent children and households without dependent children
• If either ratio is greater than 5, review the data.
EXAMPLECoC reports 205 unsheltered and 20 sheltered households
without dependent childrenRatio: 205 / 20 = 10.25
10.25 > 5 Check the data
=# of UnshelteredHouseholds ÷ # of Sheltered
HouseholdsRatio
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development50
PIT Data Checks – Chronically Homeless
• To check the number of chronically homeless make sure:
# of Individuals in
Emergency Shelter
Is less than or equal to
# Sheltered Chronically
Homeless Persons
• Remember to follow the definition of chronically homeless!
# of Unsheltered Persons
Is less than or equal to
# Unsheltered Chronically
Homeless Persons
2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development51
Additional Resources
• Revised HUD documents– A Guide to Counting Unsheltered Homeless
People• http://www.hudhre.info/documents/counting_unsheltered.pdf
– A Guide to Counting Sheltered Homeless People• http://www.hudhre.info/documents/counting_sheltered.pdf
– Annual Homeless Assessment Report• http://www.hudhre.info/documents/3rdHomelessAssessmentReport.pdf
• www.hmis.info • www.hudhre.info• PIT Technical Assistance
– [email protected] – 1-877-789-2427