Data Management: Developing Data Governance Structures

May 6, 2013

Welcome!

Kathryn Tout, Child Trends

Ivelisse Martinez-Beck, OPRE

A

3

INQUIRE Webinar Series

Webinar 1: Overview and Application of the INQUIRE Data Tools • Completed on March 20, 2013

• Available at http://www.ResearchConnections.org

Webinar 2:Data Management: Developing Data Governance Structures

Webinar 3: Data Management: Best Practices for Producing High Quality Data • May 16, 2013

• Register at http://www.ResearchConnections.org

Purpose of Webinar #2

To illustrate the need for and benefits of building strong ECE data governance and system-wide data management policies and practices using the example of QRIS.

4

Agenda

Background on INQUIRE

Data governance for QRIS • Challenges to QRIS data quality

• State options for coordinated data systems

• Data governance’s role in producing high quality data

State Perspectives & Applications • Mississippi

• Maryland

Next steps

Questions/Discussion

5

6

The Quality Initiatives Research and Evaluation Consortium (INQUIRE)

Consortium of primarily researchers and evaluators who are working on projects related to Quality Rating and Improvement Systems (QRIS) or other quality improvement initiatives or topics

Purpose of INQUIRE • Support high quality, policy relevant research and evaluation

• Provide guidance to policymakers on evaluation strategies, new research, interpretation of research results, and implication of new research for practice

7

Through OPRE-funded projects and in state QRIS evaluations, we heard from states and from evaluators about the need for support on data.

The need for guidance on how to organize and manage the data they are collecting

The need to coordinate the efforts of the different departments and organizations collecting early care and education data

The need for better understanding of how to implement an effective early care and education data system that is part of a larger early childhood data system

Presenters Child Trends

• Sarah Friese, Senior Research Analyst

Oregon State University, College of Public Health and Human Sciences

• Bobbie Weber, Research Associate

Frank Porter Graham Child Development Institute, University of North Carolina-Chapel Hill

• Iheoma Iruka, Scientist

Mississippi

• Michael Taquino, National Strategic Planning and Analysis Research Center

• Jill Dent, Mississippi Department of Human Services

Maryland

• Lindi Budd, Maryland Department of Education

• Chris Swanson, Johns Hopkins University

• Phil Koshkin, Maryland Department of Education

8

9

What is data governance?

Data governance is the set of business processes, policies, and data management practices that provide guidance on the use of a single data set or compilation of multiple, related sets. Governance promotes systematic data usage through adherence to uniform data quality and confidentiality practices.

10

What is the importance of data governance for the field of early childhood?

• Early childhood data is often collected by different agencies, housed in different data systems, and managed using different sets of rules.

• Early childhood data governance allows policymakers and practitioners to share data that describes the population and programs and to understand the impact of interventions on children’s development and readiness for school.

11

QRIS is embedded in larger data systems

12

Glossary of Terms

DATA SET A collected set of data elements collected for one program or

purpose.

DATA SYSTEM A data system is a collection of data sets housed within

single or multiple organizations.

13

Glossary of Terms

COORDINATED DATA SYSTEM A coordinated data system is one where

multiple sub-data systems and sets are governed by a central body that provides guidance related to the policies and procedures for handling and sharing data.

INTEGRATED DATA SYSTEM An integrated data system builds on a coordinated one by also provided direct assistance in the management of individual data sets housed in different sub-data systems.

14

Challenges to QRIS Data Quality 1. States use data from data systems governed and

administered by multiple agencies and organizations.

• QRIS ratings are generated by drawing on data from a variety of sources.

• Licensing, workforce registry, and subsidy are some of the data sources that are used with QRIS.

15

Challenges to QRIS Data Quality 2. Differences in database design and practices impede

linkages to other data systems.

• Systems (workforce, licensing, etc.) use their own unique identifiers.

• Values my be overridden at time of updates.

• Linking data may be difficult and opportunity for error increased.

16

Challenges to QRIS Data Quality 3. Data practices often do not support the production of high

quality data.

• Data set and system documentation is often limited.

• Departments don’t have established procedures for ensuring data quality and confidentiality and, when they do, they may conflict with those in place in other departments.

17

Challenges to QRIS Data Quality 4. States typically lack a governance framework for ECE data

systems and management.

• Many states do have not established authorities that govern and manage the policies and practices of their QRIS.

• The policies and practices that affect data management may vary across the databases linked to the QRIS.

18

Challenges to Data Quality 5. States are designing or redesigning their QRIS data systems

and are looking for models and guidance.

• We are in a period of rapid change for standards in the design and management of QRIS data systems.

• Now is an ideal time to support states in building QRIS data management capacity.

19

Moving from Multiple Independent Databases to an Integrated QRIS Data System

• In states with QRIS, developing a quality rating for a program requires linking of data on the workforce, licensing, and facilities.

• Linking is facilitated when data is shared in a coordinated or integrated data system.

20

Options for a Coordinated or Integrated Data System Vary On Key Characteristics:

• Data Quality

• Data Availability

• Cross Agency Workflow

• Data Governance

21

Types of Coordinated or Integrated Data Systems

22

Unlinked Databases or Point Solutions

Benefits

• Least disruptive in the short run.

Drawbacks

• Will produce the lowest quality data

• Least efficient and most expensive in the long run

23

Coordinated data systems with linked customized interfaces

Benefits • Databases are linked one by one as needed.

Drawbacks • Data are not based on standards.

• Interfaces are designed ad hoc and require ongoing maintenance.

• Governance occurs at the agency level, and sharing is addressed on a case by case basis.

24

Federated, shared data system

Benefits • Data elements needed for QRIS and other purposes are

extracted from databases, mapped to standards, linked to master identifiers and stored in shared repository.

• Cross-agency governance is required for shared data, but individual databases may retain their own governance process.

Drawbacks • Redundancies still exist as the same data is entered into more

than one database. The sharing system requires maintenance.

25

Comprehensive, integrated data system

Benefits • Data silos are eliminated which also reduces the potential for

redundancies.

• Data is managed according to uniform standards so quality is high.

Drawbacks • An investment of time and resources is required including

changing data management policies and processes in agencies involved in the integrated data system.

26

Governance Essential Regardless of Data System Option Selected

• Governance is where stakeholders come together to make

decisions about what the vocabulary will be, which nationally-recognized standard will be used for its representation, and who will have permission to access the data.

• States have options for the type of coordinated data system but data governance is essential for all types.

27

Identifying Governance Body for QRIS Data System

• Executive Council—sets overall mission and goals, secures funding and resources

• Strategic Committee—develops high-level plan to achieve goals

• Tactical group—develops short-term goals and tasks

• Partners and stakeholders—provide ideas and feedback

28

Tasks of the Governance Body for the ECE Data System include:

• Produce standard data-sharing agreement.

• Develop documentation for databases in system.

• Have a policy on database updates.

• Ensure data are saved and system changes captured.

• Develop common data standards.

• Determine unique identifiers for children, workforce, & facilities.

• Train data management staff.

• Establish consistent security and back-up policies.

State Perspectives & Applications

29

STATE PERSPECTIVES STATE PERSPECTIVES & & APPLICATIONSAPPLICATIONS

MISSISSIPPIMISSISSIPPI

SIX BUILDING BLOCKS OF SIX BUILDING BLOCKS OF INTEGRATED INTEGRATED DATA DATA GOVERNANCEGOVERNANCE

The scope defines the purpose of the

integrated data system and provides the

general framework for supporting and

institutionalizing its use.

Example: Evaluate the effectiveness of the

Allies For Quality Care program. The overall

goal is to support childcare providers in

increasing the level of quality offered in their

program.

SCOPESCOPE

A data stakeholder is an individual or

organization affected by information generated

from an integrated data system and aligned

with the scope

Allies data stakeholders:

Mississippi Department of Human Services

Mississippi Department of Education

Head Start

Mississippi Department of Health

DATA STAKEHOLDERSDATA STAKEHOLDERS

Data will only be used for activities

directly related to the scope

Key factors for successful applications: Determine data availability

Data documentation through data dictionaries or

codebooks

Develop data mapping

Mechanism to access to data

• Files saved in separate environments

• Data warehouse/logical model

• Federated intersystem exchange

APPLICATIONAPPLICATION

Ability to fulfill scope of integrated data system • Secure data and access data

Components: • Create a Center of Excellence through University Partnerships

• Data, technical, and research expertise

• Legal and compliance expertise

• Formal agreements

• Policies and procedures for data lifecycle for state data clearinghouse

(data warehouse) or federated system

OPERATIONAL CAPACITYOPERATIONAL CAPACITY

DATA ACCESS DATA ACCESS -- WEBWEB--BASED PORTALBASED PORTAL

Leadership and Accountability Who owns, promotes, and oversees the system? Who is

responsible for making sure things are done right?

In Mississippi, a governing board provides a single point of

leadership and accountability, and a management board provides

technical advice. A center of excellence provides the capacity for

the system to operate.

Sustainability In Mississippi, sustainability has been established through legal

authority:

• Memoranda of Understanding

• Governor’s Executive Order

• Legislative Appropriation

LEADERSHIP & ACCOUNTABILITYLEADERSHIP & ACCOUNTABILITY

Quality Care and Education System for

Maryland’s Children

Maryland EXCELS

Excellence Counts in Early Learning and School Age Care

Where are we now?

• Fall 2011-Spring 2012 – Pilot Phase • Fall 2012-Spring 2013 - Field Test

• July 1, 2013 – Statewide Implementation

Maryland EXCELS Website http://marylandexcels.org

Maryland EXCELS Data Connectivity

Specific Maryland EXCELS Data • Program quality “Check”

level

• Data obtained; Date expired – archival by “cycles”

• All reviewers – and level of communication

• Rate of change in quality status

• Specific program supplied evidence demonstrating quality (eg. Schedule, lesson plans, parent communications)

Types of Questions We Seek to Address

• Longitudinal impact on school performance of children by early-care experience

• Correlations between quality elements (accreditation, credentialing, level of technical assistance, etc.)

• Degree of reliability among all raters

Lessons Learned

• Have a great central data architect

• Clearly identify needs, then look at data collection mechanisms

• Understand the data use from multiple perspectives

Looking Across the State Examples

The state examples provide an overview of different activities related to data governance.

The examples demonstrate the important connections that are made between data, program monitoring and policy questions of interest in the state.

45

Next Steps

Upcoming Webinar on Data Management • May 16, 2013, 2:00-3:30 EST:

Best Practices for Producing High-Quality Data

Webinar recording will be available on Research Connections

46

Acknowledgements of Contributors to INQUIRE’s Data Work Group Rick Brandon, Consultant

Missy Cochenour, AEM

Iheoma Iruka, FPG, University of North Carolina

Tabitha Isner, MN Department of Human Services

Fran Kipnis, Center for the Study of Child Care Employment at UC Berkeley

Lee Kreader, National Center for Children in Poverty

Minh Le, Office of Child Care, ACF

Lizabeth Malone, Mathematica Policy Research

Frances Majestic & Elizabeth Hoffman, Office of Head Start, ACF

Dawn Ramsburg, Office of Child Care, ACF

Bobbie Weber, Oregon State University

47

Questions and Discussion

48

Contact Information

Ivelisse Martinez-Beck, Office of Planning, Research and Evaluation, Administration for Children and Families

• Ivelisse.martinezbeck@hhs.gov

Kathryn Tout, Child Trends • ktout@childtrends.org

49