module 1 slide deck: child welfare data 101

60
Using Publicly Available Data to Engage IV-E Students in Research and Statistics: Instructional Modules MODULE 1 SLIDE DECK: CHILD WELFARE DATA 101

Upload: tex

Post on 09-Feb-2016

38 views

Category:

Documents


0 download

DESCRIPTION

Module 1 Slide deck: Child Welfare Data 101. Instructor Notes about this module. This module serves as an entry point for the entire course, it’s purpose is to: Help reduce student anxiety and resistance towards engaging in research and completing the course - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Module 1 Slide deck: Child Welfare Data 101

Using Publicly Available Data to Engage IV-E Students in Research and Statistics: Instructional Modules

MODULE 1 SLIDE DECK:CHILD WELFARE DATA 101

Page 2: Module 1 Slide deck: Child Welfare Data 101

2Module 1: Child Welfare Data 101 2

Instructor Notes about this module This module serves as an entry point for the

entire course, it’s purpose is to: Help reduce student anxiety and resistance towards

engaging in research and completing the course Provide an overview of California's child welfare data Introduce basic statistical concepts that are

informative, accessible, and relatively easy to generate

Increase students’ comfort level with data, thereby instilling confidence in their capacity as informed consumers of data and research

Page 3: Module 1 Slide deck: Child Welfare Data 101

3Module 1: Child Welfare Data 101 3

Agenda Introduction and purpose/goals of the course Data basics:

Overview of sources and uses of data Sampled vs. population-level data Aggregate vs. micro-level data Longitudinal vs. cross-sectional data

Introduction to accessing and using California’s child welfare data

Understanding and computing basic descriptive statistics with child welfare data (e.g. variable descriptions, measures of central tendency, computing percentages)

Page 4: Module 1 Slide deck: Child Welfare Data 101

Engaging Students in Research, possible slides for instructor use to introduce course

Opening Slides

Page 5: Module 1 Slide deck: Child Welfare Data 101

5Module 1: Child Welfare Data 101 5

The Problem MSW students rank research courses as one

of their least favorite classes in the program (anecdotal…) Coursework feels disconnected from practice Most students do not enter with strong math or

stats backgrounds – high anxiety Timeline allows only a superficial coverage of

analytic methods Need to develop “statistical literacy”

Knowledge which enables people to think for themselves, judge independently, and discriminate between good and bad information (Dewey, 1930)

Page 6: Module 1 Slide deck: Child Welfare Data 101

6Module 1: Child Welfare Data 101 6

Competing Course Models

informed consumers of data & research

junior social scientists

difficult (if not impossible) to do well even successful training is lost as not used in post-grad work

technology = data are everywhere!field is increasingly oriented around continuous improvement, outcomesstatistical literacy & critical thinking necessary for EBP and EIP

burdens agencies tasked with helping students access data

Page 7: Module 1 Slide deck: Child Welfare Data 101

7Module 1: Child Welfare Data 101 7

Why we must be skilled consumers of research… Ethical obligation to our clients to be up to date on

the most recent research and skilled in its critical appraisal Cannot only rely on researchers to tell us the

relevant results and findings Necessary for effective advocacy and practice Helps with the efficient translation of research to

practice Critical that practitioners/researchers can speak

the same “language” in order to ensure future research efforts are relevant and that findings are understood in context and translated appropriately

Page 8: Module 1 Slide deck: Child Welfare Data 101

8Module 1: Child Welfare Data 101 8

Managing by DataProvides social workers with the ability to: Compare metrics with agency mission

and practice model Connect to evidence-based practice and

desired outcomes Strategize on what work needs to be

done Focus on what is being achieved Identify what needs attention

Page 9: Module 1 Slide deck: Child Welfare Data 101

9Module 1: Child Welfare Data 101 9

Connecting Data to Practice

Observe

Explain

Strategy

Outcome

We have noted that…

And believe

it is becaus

e..

So we plan to…

Which will

result in…

HYPOTHESIS: A HIGH LEVEL CAUSE AND EFFECT STATEMENT

IN OTHER WORDS…

Slide Developed by NY OCFS

Page 10: Module 1 Slide deck: Child Welfare Data 101

1010

This Course One of the most important you will take here at X

(in my humble opinion) Focuses on practical skills, understanding data,

statistical literacy, consuming research Very connected to your current field placement –

and work post-graduation – yet helps you acquire empirical skills you may not otherwise have the opportunity to develop outside of your graduate studies

Expects that you read, ask questions, think critically, and engage with the material

Requires that you produce a relevant, readable, empirical research report based on publicly available administrative (secondary) child welfare data

Page 11: Module 1 Slide deck: Child Welfare Data 101

11Module 1: Child Welfare Data 101 11

Administrative Data

Collected during the normal course of agency operations

Tabulated/aggregate data are publicly available Full coverage of populations served Free of “reactivity”(data problems are usually

transparent) Analysis of trends over time Performance indicators Social indicators Particularly salient to social work…

Page 12: Module 1 Slide deck: Child Welfare Data 101

12Module 1: Child Welfare Data 101 12

Why a Secondary Analysis of Administrative Data? Data directly support agencies and capture information for

the clients we are working with Increasing emphasis on making these data available to

researchers Technological advances – data storage, frequent refreshes,

web-based/online access and analysis tools Non-intrusive (we work with vulnerable populations and

busy co-workers!) More transferrable to a post-graduate career in social work

Efficient, cheap, available Useful for advocacy efforts, needs assessment, proposals for

new programs (substantiate a service need) These are needed skills in both public and private agencies

Page 13: Module 1 Slide deck: Child Welfare Data 101

13Module 1: Child Welfare Data 101 13

Realities of Administrative Data Analysis

Measures are often crude Sometimes limited documentation Missing data Do you trust what has been entered? Often much more difficult to analyze

than expected…requires careful thinking May be dated Definitions may have changed over time

Page 14: Module 1 Slide deck: Child Welfare Data 101

Sources and Uses of Data

Module 1, Section 1

Page 15: Module 1 Slide deck: Child Welfare Data 101

15Module 1: Child Welfare Data 101 15

Data Sources1. Census data2. Longitudinal surveys of a subsample of

a population3. Cross sectional surveys4. Longitudinal (multi-wave) surveys of a

single sample5. Administrative data6. Multisource data systems

Page 16: Module 1 Slide deck: Child Welfare Data 101

1616

Uses of Data Descriptive

Demographic characteristics of a population, place, office, etc. Trends over time (one period compared to another) Differences/similarities between groups, counties, placement

settings, interventions, etc. Exploratory

Often conducted as pilot studies, attempt to examine feasibility issues (e.g., recruitment), preliminary data to develop fuller hypotheses and research proposals

Explanatory Analysis of the relationship between two events (or two

variables) Looking at the contributions of various factors to some outcomes

(y=a+bX) Evaluation

To evaluate social policies, programs, and interventions The evaluation process encompasses all three uses of data listed

above

Page 17: Module 1 Slide deck: Child Welfare Data 101

17Module 1: Child Welfare Data 101 17

Data Terminology (Review) Qualitative vs. Quantitative

why, how vs. who, what, where, when “All quantitative data is based upon qualitative judgments; and all

qualitative data can be described and manipulated numerically.” (Research Methods Knowledge Base)

Longitudinal vs. Cross-Sectional repeated observations over time vs. a slice

Primary vs. Secondary you collect it vs. someone else collected it

Aggregate vs. Micro-level group tabulations vs. individual units

Deidentified vs. Identified Joe Smith vs. 987334

Sample vs. Census/Population partial vs. full coverage

Page 18: Module 1 Slide deck: Child Welfare Data 101

Sample vs. Census/Population

Module1, Section 2

Page 19: Module 1 Slide deck: Child Welfare Data 101

19Module 1: Child Welfare Data 101 19

Samples vs. Population Can we select a few people or things

for observation and then apply what we observe to a much larger group? Often impractical to gather data from the whole

population, so samples are drawn (this is not relevant to the administrative data we will be using in this course)

Key is the researcher’s ability to generalize findings from the sample to the whole population

Sample=a finite part of a larger population whose properties are studied to gain information about the whole

If all members of a population were identical in all respects – would we need careful sampling procedures?

population

sample

Page 20: Module 1 Slide deck: Child Welfare Data 101

20

Sampling The act, process, or technique of selecting a suitable

sample, or a representative part of a population for determining the parameters (or characteristics) of the whole population

For a sample to provide useful information, it must reflect the same general variations as the overall population

NSCAW data = sample; CWS/CMS data = populationpopulation sample

Use characteristics/observations of sample, to draw conclusions (inferences) about the larger

population

Module 1: Child Welfare Data 101

Page 21: Module 1 Slide deck: Child Welfare Data 101

Aggregate vs. Microdata

Module 1, Section 3

Page 22: Module 1 Slide deck: Child Welfare Data 101

2222

Aggregated data

Page 23: Module 1 Slide deck: Child Welfare Data 101

2323

Micro-data (individuals as units of analysis)

Page 24: Module 1 Slide deck: Child Welfare Data 101

24Module 1: Child Welfare Data 101 24

Working with Aggregated Data…Disaggregate One of the most powerful ways to work with

data… Disaggregation involves dismantling or

separating out groups within a population to better understand the dynamics

Useful for identifying critical issues that were previously undetectedAggregate Permanency Outcomes

Race/Ethnicity

AgeCounty

Placement Type

Page 25: Module 1 Slide deck: Child Welfare Data 101

25Module 1: Child Welfare Data 101 25

The Problem with Summary Statistics:

The average human has one breast and one testicle. *

* ~Des McHale www.quotegarden.com/statistics.html

Page 26: Module 1 Slide deck: Child Welfare Data 101
Page 27: Module 1 Slide deck: Child Welfare Data 101
Page 28: Module 1 Slide deck: Child Welfare Data 101

2000 July-December First Entries California:

Percent Exited to Permanency 132 Months From Entry, by race and placement

Page 29: Module 1 Slide deck: Child Welfare Data 101

Module 1, Section 4Longitudinal vs. Cross-Sectional (Point in Time) Views of Data

Page 30: Module 1 Slide deck: Child Welfare Data 101

30Module 1: Child Welfare Data 101 30

Time Dynamics Cross-Sectional Studies

Examines a phenomenon by collecting/examining a “cross-section” of data at one time (one observation at a point in time) BIG problem: many questions we seek to answer aim to

understand causal processes that occur over time (e.g., children in foster care and mental health)

Longitudinal Studies Based on repeated observations of a given unit over

multiple points in time Trend Studies Cohort /Panel Studies

Page 31: Module 1 Slide deck: Child Welfare Data 101

31Module 1: Child Welfare Data 101 31

Longitudinal Data1988 2006Birth 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Exit

X X X X X X X X X XX X X X X X X X X X X X X

X X X X X X X X X X X X X X X X

Age

Longitudinal Analysis

Page 32: Module 1 Slide deck: Child Welfare Data 101

32Module 1: Child Welfare Data 101 32

3 Key Data Views in Child Welfare

Data

Entry

Cohorts

Exit Cohorts

Point in

Time

Page 33: Module 1 Slide deck: Child Welfare Data 101

33

January 1, 2005 January 1, 2006July 1, 2005Child 1

Child 2

Child 3

Child 4

Child 5

Child 6

Child 7

Child 8

Child 9Child 10

The “View” Matters!

Page 34: Module 1 Slide deck: Child Welfare Data 101

34

Longitudinal vs. Point in Time

Module 1: Child Welfare Data 101

Page 35: Module 1 Slide deck: Child Welfare Data 101

35Module 1: Child Welfare Data 101

Page 36: Module 1 Slide deck: Child Welfare Data 101

36Module 1: Child Welfare Data 101

Page 37: Module 1 Slide deck: Child Welfare Data 101

California’s Child Welfare Data

Module 1, Section 5

Page 38: Module 1 Slide deck: Child Welfare Data 101

38Module 1: Child Welfare Data 101 38

Why do we have these data? In 2001, the California Legislature passed the

Child Welfare System Improvement and Accountability Act (AB 636)

Designed to improve outcomes for children in the child welfare system while holding county and state agencies accountable for the outcomes achieved

The statewide accountability system went into effect January 1, 2004 It is an enhanced version of the federal oversight

system mandated by Congress to monitor states’ performance and provides the legal framework for the California Child and Family Services Reviews

Page 39: Module 1 Slide deck: Child Welfare Data 101

39Module 1: Child Welfare Data 101 39

How are these data used? The foundation for this new oversight system comes

from data obtained from the Child Welfare Services/Case Management System (CWS/CMS)

Each quarter, the state provides county child welfare agencies with county-specific data on 14 outcome measures related to safety, permanency and well-being The baseline performance data is gathered for each county

and also made available to the public Quarterly reports provide counties with quantitative data

and serve as a management tool to track performance over time

Public data is also refreshed quarterly (data available on website is 3-6 months old)

Page 40: Module 1 Slide deck: Child Welfare Data 101

4040

Where are the Data from?

Page 41: Module 1 Slide deck: Child Welfare Data 101

4141

How are the public data configured?

unique characteristics and path through the child welfare system

personal info

aggregate data

tabulations

ageracegenderallegationplacementdispositioncountyyear

Page 42: Module 1 Slide deck: Child Welfare Data 101

42Module 1: Child Welfare Data 101 42

What “type” of information is available?

first allegation of

maltreatment

allegation evaluated out

second allegation of

maltreatment

allegation substantiated

pre-placement family

maintenance services provided

child placed in out-of-home foster care

child reunified

third allegation of

maltreatment

child re-enters foster

care

Page 43: Module 1 Slide deck: Child Welfare Data 101

43

Where can I find these data?

http://cssr.berkeley.edu/ucb_childwelfare

Page 44: Module 1 Slide deck: Child Welfare Data 101

Univariate Statistics, child welfare examples

Module 1, Section 6

Page 45: Module 1 Slide deck: Child Welfare Data 101

45

Variablesvariable

categorical

nominal

ordinal

continuous

exhaustive

mutually exclusive

variability in magnitude

(quantitative in nature)

finite number of values or categories (qualitative in nature)

Page 46: Module 1 Slide deck: Child Welfare Data 101

46Module 1: Child Welfare Data 101

Continuous vs. Categorical The average foster child has 2.6 placements while in

foster care This number makes little sense because the underlying

dimension is discrete (i.e., categorical, discontinuous) 1 2 4 5 6

placements

x2.6 3

Continuous Data Discrete DataAge Days in Care Percentages / Rates

Race/Ethnicity Placement Type Referral Reason

Page 47: Module 1 Slide deck: Child Welfare Data 101

47Module 1: Child Welfare Data 101 47

Descriptive Statistics “Summary” statistics, used to describe what’s

“going on” in our data Describe a situation or condition numerically

by quantifying phenomena In California, there are far fewer children in foster care today

than was true a decade ago. In California, the number of children in foster care today is

47,729 less than was true a decade ago, translating into a 49% decline.

Frequency tables (the distribution), measures of central tendency, measures of variability (the dispersion)

Page 48: Module 1 Slide deck: Child Welfare Data 101

48

Computing a Percent

PERCENT: A proportion in relation to a whole expressed as a fraction of 100.

100totalpart100)(per percent %

100totalpart

100440290

100659.0

%9.65

100reunified # total

12m w/in reunified #

Raw Numbers (counts)

# Reunified w/in 12m# Reunified (total)

= 290= 440

What Percentage of Children reunified in 2005 were reunified within 12 months of entering care?

Page 49: Module 1 Slide deck: Child Welfare Data 101

49

Computing a Rate per 1,000RATE: A proportion in relation to a whole, can be expressed as a fraction of 100, 1000, 100,000, etc.

1000totalpart

1000363,3761,333

100000366.

7.3

1000population child #

care entered #

Raw Numbers (counts)

# Entered Care

# Child Population

= 1,333= 363,376

What was the foster care entry rate in 2005? (i.e., how many children entered care out of all possible children in the population?)

1000totalpart1000per rate

Scales for a meaningful interpretation…

Page 50: Module 1 Slide deck: Child Welfare Data 101

5050

Measures of Central TendencyMean: the average value for a range of data Median: the value of the middle item when the data

are arranged from smallest to largestMode: the value that occurs most frequently within the data

4.168

631715129744 Mean

5.102129 Median

4 Mode

7

= 9

= 9.7

12 4 15 63 7 9 4 17 4 4 7 9 12 15 17 63

Page 51: Module 1 Slide deck: Child Welfare Data 101

51

Measures of Variability

Minimum: the smallest value within the dataMaximum: the largest value within the dataRange: the overall span of the data

4 Minimum

63 Maximum

59463 Range

4 4 7 9 12 15 17 63

51

Page 52: Module 1 Slide deck: Child Welfare Data 101

52Module 1: Child Welfare Data 101 52

The Relationship between Mean and Variability Standard Deviation (represented by the

symbol σ) shows how much variation there is from the mean (average or expected value) Low standard deviation indicates that most

of the data points are close to the mean (less variation)

High standard deviation indicates that data points are spread out over a large range of values (more variation)

Page 53: Module 1 Slide deck: Child Welfare Data 101

53Module 1: Child Welfare Data 101

Standard Deviation

symbol for standard deviation

Sum the difference between each “score” (xi ) and the overall mean (m)

Square the sum

Divide by the count of observations/scores minus 1

Take the square root

Page 54: Module 1 Slide deck: Child Welfare Data 101

54Module 1: Child Welfare Data 101

Standard DeviationScores/Observations: 4 4 7 9 12 15 17 63Mean: 16.375

1. Find the distance between each value and the mean• 4-16.375 = -12.375• 4-16.375 = -12.375• 7-16.375 = -9.375• 9-16.375 = -7.375• 12-16.375 = -4.375• 15-16.375 = -1.375• 17-16.375 = 0.625• 63-16.375 = 46.625

2. Square all Values• -12.375 x -12.375 =

153.1• -12.375 x -12.375 =

153.1• -9.375 x -9.375 = 87.9• -7.375 x -7.375 = 54.4• -4.375 x -4.375 = 19.1• -1.375 x -1.375 = 1.89• 0.625 x 0.625 = 0.39• 46.625 x 46.625 =

2173.9

Page 55: Module 1 Slide deck: Child Welfare Data 101

55Module 1: Child Welfare Data 101

Standard DeviationScores/Observations: 4 4 7 9 12 15 17 63Mean: 16.375

3. Sum the squared values (Sum of Squares: SS)

153.1+153.1+87.9+54.4+ 19.1+1.89+0.39+2173.9 = 2643.9

4. Divide the SS by the count of scores minus 1 (this gives you the Variance)

= 2643.9 / 7= 377.574. Take the square root of the variance (this gives you the

standard deviation)= = 19.4

Page 56: Module 1 Slide deck: Child Welfare Data 101

56Module 1: Child Welfare Data 101 56

Percentage Points vs. Percent Change

Percentage point difference Absolute increase or decrease from one percentage value

to another (calculated by addition or subtraction) The percentage of children in foster family agency (FFA)

care increased by 12 percentage points between 1998 and 2012, from 15% to 27%

Percent change Relative change from one value to another (as a fraction

of the original amount) The proportion of children in foster family agency (FFA)

care increased by 80% between 1998 and 2012, from 15% to 27%

Page 57: Module 1 Slide deck: Child Welfare Data 101

57Module 1: Child Welfare Data 101

Percent Change

Time Period 1

Time Period 2

10 children

12 children

10011 Period2 PeriodChange %

10011.2

1000.2

20%

1001kids 10kids 12

Page 58: Module 1 Slide deck: Child Welfare Data 101

58Module 1: Child Welfare Data 101

Percent ChangeTime Period 1 Time Period 2

10% 12%

% %% %% %% %% %

% %% %% %% %% %

%%

100110%12%Change %

%20

Page 59: Module 1 Slide deck: Child Welfare Data 101

59

Percent Change Calculation

Baseline Referral Rate (time period 1):

7.50100005067.963,637,9

419,488

Comparison Referral Rate (time period 2):

3.4810000483.199,988,9

706,482

Percent Change:

1001Rate Baseline

Rate Comparison

100150.748.3

%7.4100047.0

1001)-.9526(

50.7 48.3 -4.7%

12.0 10.8 -10%

Min

or D

iffer

ence

s du

e to

Ro

undi

ng…

Page 60: Module 1 Slide deck: Child Welfare Data 101

Using Publicly Available Data to Engage IV-E Students in Research and Statistics: Instructional Modules

QUESTIONS? PLEASE CONTACT:

[email protected]@berkeley.edu