taking stock: assessing & improving early childhood learning and program quality board of the...
Post on 02-Jan-2016
214 Views
Preview:
TRANSCRIPT
Taking Stock: Taking Stock: Assessing & Improving Assessing & Improving
Early Childhood Learning Early Childhood Learning and Program Qualityand Program Quality
Board of the MassachusettsBoard of the MassachusettsDepartment of Early Education and CareDepartment of Early Education and Care
May 13, 2008May 13, 2008
Dr. Sharon Lynn KaganDr. Sharon Lynn Kagan Columbia University & Yale UniversityColumbia University & Yale University
OverviewOverview Impetus for the Work
Challenges
Proposed System Design
Differing Viewpoints on Using Child Assessment Data for Local Agency Accountability
Action Steps
Implications
Growing Importance of ECEGrowing Importance of ECE Throughout the United States and
countries around the world, Early Childhood Education is ascending to prominence.
States are investing more dollars in the field, with growth in virtually every segment of ECE, including:– Direct Services– Professional Development– Quality Rating Systems
Growing InvestmentsGrowing Investments These growing investments are
matched by calls to assess that the investments are paying off.
Are programs delivering the services they say they are delivering?
Are children achieving the outcomes desired?
Are the investments yielding appropriate returns?
Growing Emphasis on ECEGrowing Emphasis on ECE Practitioners also are interested in
results, recognizing that:– The early years come only once for a
child – The early years are critical to long term
development– Teachers are the keys to making
programs high quality– Teachers and policy makers can learn
from data about children.
Interest in AccountabilityInterest in Accountability Interest in Accountability is not
limited to ECE alone. All human service agencies are being
asked to take on a results orientation. Business and industry has provided a
lead in this area and fueled efforts toward greater efficiency and quality.
There is a prevailing ideology that accountability provides the information to make progress.
Many Factors Provide ImpetusMany Factors Provide Impetus
So there is a collision of interest that propels a focus on outcomes and accountability:– New data that attests to the importance
of the early years– New investments and the need to see if
they are making a difference.– New movement toward accountability
as an elixir for quality improvement.
Four ChallengesFour Challenges
1. Structural Challenges
2. Conceptual Challenges
3. Technical Challenges
4. Resource Challenges
Challenges: StructuralChallenges: Structural Fragmented non-system of programs
for preschool-aged children
Disjointed early childhood and public education policies
MultipleMultiple Standards and AssessmentsStandards and Assessments
Child Care Head Start State Pre-K Special Education
Program Quality
Standards
State Licensing Standards(50 states)
Quality Rating Systems (QRS) (13 states + 29 pilots)
Program Performance
Standards
State Program Standards(39 states)
IDEA regulationsState program
standards
Assessing Program
Quality
Licensing VisitsQRS Assessments
(13 + 29)
PRISM Reviews Program Monitoring(30 states)
State Program Monitoring
Standards for
Children’s Learning
Early Learning Guidelines(49 states)
Head Start Child Outcomes
Framework
Early Learning Guidelines(49 states)
3 functional goals
Child Assessments
No current requirements National Reporting
System
Pre-K Assessments(13 states)
Kg. Assessments(17 states)
States report % of children in 5
categories on 3 goals
Research/Evaluations Yes Yes Yes Yes
Kindergarten to Grade 3 Standards, Assessments, Data
Challenges: StructuralChallenges: Structural Costs, burdens, confusion of multiple
standards, assessments, & reports.
Multiple new initiatives all at once.
Pre-K–K-3 disconnects:
– Pre-K assessments aren’t transferred to schools.
– Standards, assessments, curricula aren’t aligned.
Challenges: ConceptualChallenges: Conceptual Early childhood assessment training
and practice vs. standards-based assessment and curriculum.
Early childhood reliance on program standards/data vs. expanded interest in children’s learning.
Challenges: TechnicalChallenges: Technical Need appropriate assessment tools
and methods to report on:
– Progress/status of young children in all domains of learning and development
– Young ELLs and children with disabilities
– Program quality in diverse local agencies
Challenges: ResourcesChallenges: Resources Limitations and inequities in
funding for:
– Programs
– Infrastructure
Risk that accountability efforts ignore and exacerbate inequities in resources
Task Force ChargeTask Force Charge Recommendations for a state
accountability system for early education programs for pre-kindergarten children and linkage to standards-based assessments in kindergarten-grade 3.
OriginsOrigins Conceived by The Pew Charitable Trusts
as part of their Advancing Quality Pre-K For All initiative.
Additional funding from the Foundation for Child Development and the Joyce Foundation.
Task Force convened in fall, 2005, report release fall, 2007.
Presentation reflects progress-to-date.
Dr. Sharon Lynn Kagan, Chair Dr. Eugene Garcia, Vice-Chair
– Dr. W. Steven Barnett– Ms. Barbara Bowman– Dr. Mary Beth Bruder– Dr. Lindy Buch– Dr. Maryann Santos
de Barona– Ms. Harriet Dichter
– Mr. Mark Friedman– Dr. Jacqueline Jones– Dr. Joan Lombardi– Dr. Samuel Meisels– Ms. Marsha Moore– Dr. Robert Pianta– Dr. Donald Rock
Task Force MembersTask Force Members
Framing BeliefsFraming Beliefs Accountability is here to stay.
Programs should be held to performance standards that are documented and verified.
Assessments should inform policy decisions and be tied to program enhancement efforts.
Current approaches to accountability and assessment must be reformed.
State Accountability & Improvement State Accountability & Improvement System DesignSystem Design
Pre-K-Grade 3 Alignment and Linkages
Assessment/Program Improvement Approaches
APPROACHES I CHILD POPULATION
IIPROGRAM POPULATION
III STATE PROGRAM
EVALUATION
IV LOCAL AGENCY
QUALITY
COREQUESTION
How well are all children progressing in
learning and development?
What is the quality of all early childhood
programs?
What is the quality and how well are children progressing in specific
state programs?
What is the quality in local agencies?
HOW DATA IS USED
- Oversight of state investments/initiatives - Planning new investments/initiatives - Baseline information for K-12 education planning
- Oversight of state investments/initiatives - Planning new investments/initiatives - Baseline information for K-12 education planning
- Program-wide improvement efforts - Refining standards/policies- Appropriations decisions
- Technical assistance to individual agencies.- Awarding incentives and recognition to local agencies for program improvements - Decisions on funding local agencies
Infrastructure
Early Learning &Program Quality
Standards
Program Rating & Improvement
Professional Development Data Management & Reporting
InfrastructureInfrastructure
System Infrastructure
Early Learning
&Program Quality
Standards
Program Rating &
Improvement
Professional Developmen
t
Data Manageme
nt & Reporting
InfrastructureInfrastructure Early Learning and Program
Quality Standards– Alignment between:
• Standards, assessment systems and curricula
• Standards between ages and grades
• State and federal program structures and funding streams
• Child and program standards
InfrastructureInfrastructure Program Rating & Improvement
– Assesses and reports on the quality of all forms of early education programs
– Provides technical assistance and professional development to improve quality
– May provide public recognition/incentives to reward higher levels of quality
InfrastructureInfrastructure Professional Development System
– Links informal training with formal education, provides career pathways, links education and compensation.
– Supports training on assessment administration, analysis and use.
InfrastructureInfrastructure Data Management and Reporting
– All-in-one place data on:• Children
• Programs
• Workforce
– Unified system of child identification numbers.
– Provides for quality assurance of data and assessments.
Assessment ApproachesAssessment Approaches States vary in:
– What they want to know– How they plan to use data– Available resources
States may implement one or any combination of options.
Report includes cautions/safeguards for each option.
Assessment/Improvement ApproachesAssessment/Improvement Approaches
APPROACHES Child Populatio
n
ProgramPopulatio
n
State Program
Evaluation
Local Agency Quality
COREQUESTION
How well are all young
children progressin
g in learning
and developme
nt?
What is the quality of all early
education programs?
What is the quality and
how are children
progressing in specific
state programs?
What is the quality in
local agencies?
AllAll Four Approaches Discuss:Four Approaches Discuss:
Questions that can be Answered
Data to be Collected
Designs for Data Collection
Uses of the Data
Challenges and Cautions
I. Child Population ApproachI. Child Population Approach How well are all young children
progressing in learning and development?
– Data:• Demographic data collected on representative
sample of all young children
• Comprehensive data on health, well-being, pre-school enrollment
– Designs: • At Kindergarten Entry
• At Ages 1, 3, 4
• Longitudinal ECLS State Strategy
I. Child Population ApproachI. Child Population Approach How well are all young children
progressing in learning and development?
– Uses: • Planning interagency investments/initiatives
• Legislative oversight
• Baseline information for public education
– Challenges/Cautions: • Gaining access to unrolled preschoolers
• Can’t use data to maker inferences about pre-school programs’ impacts/quality
• No causal attributions
• Expensive and not routinely done
II. Program Population II. Program Population ApproachApproach
What is the quality of services in all early childhood programs?– Data:
• Program quality• Workforce• Public investments
– Design:• Data collected from states
Program Improvement (PRS/QRS) System
II. Program Population II. Program Population ApproachApproach
What is the quality of services in all early childhood programs?– Uses:
• Planning interagency investments/initiatives
• Legislative oversight
• Baseline information for public education
– Challenges/Cautions:• Need to consider quality of programs in light of
investments in them
• Need to be sensitive to changes in program quality
• Unable to attribute causality to program quality rankings
III. State Program Evaluation III. State Program Evaluation ApproachApproach
What is the quality and how well are children progressing in specific programs?– Data:
• Sample of centers that represent the universe of programs to be studied
• Comprehensive data on children, program, and teachers
• Data aligned with program mandates and standards
– Design: • Standard Evaluation Design-program and
instrument identification, data collection and analysis
• Program Rating System Design-uses data for program information from the PRS/QRS
III. State Program Evaluation III. State Program Evaluation ApproachApproach
What is the quality and how well are children progressing in specific programs?– Uses:
• Program-wide improvement efforts• Refining standards/policies• Appropriations decisions
– Challenges/Cautions:• Attribution of single program effects due to
many children’s participation in multiple programs (e.g., lack of pure control group)
• Pre-mature evaluation inappropriate• Implementation fidelity to program design
IV. Local Agency Quality IV. Local Agency Quality ApproachApproach
What is the quality in local agencies?– Data:
• Program quality data in relation to state standards
• Observations of teaching/learning opportunities
– Design:• Use PRS/QRS data• On rotational basis
– 1/3 annually– Low performing programs
IV. Local Agency Quality IV. Local Agency Quality ApproachApproach
What is the quality in local agencies?– Uses:
• Technical assistance to individual local agencies
• Awarding incentives and public recognition
– Challenges/Cautions:• Heavy burden on state agencies
• Heavy costs to do classroom observations
• States need to develop technical assistance to address program needs/weaknesses
Pre-K—Grade 3 IntegrationPre-K—Grade 3 Integration Align standards, assessments and
reporting on:– Children’s progress– Quality of teaching/learning
opportunities
“Vertical” teams of teachers/managers to:– Review assessment information
– Enrich learning experiences and teaching strategies
Joint professional development
Part IV Part IV
Differing Viewpoints on Differing Viewpoints on Using Child Assessment Data Using Child Assessment Data
for Local Agency for Local Agency AccountabilityAccountability
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies Task Force reached an easy
consensus on: – Infrastructure components
– Four assessment/improvement approaches
– Linking of pre-kindergarten through grade 3
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies Some members said this wasn’t
enough. Needed more information:
– How well are children in local agencies learning?
– How does what they are learning compare with state standards?
– Are some local agencies unusually effective in fostering learning progress?
To do this, some members of the Task Force advocated collecting and reporting data on all children's performance and to distill it so it could be used by local agencies for planning and improvement.
They recognized that we can have children in good programs who don’t do well, and they contend that program quality is NOT a proxy for child outcomes.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies These Task Force members
recognized the huge challenges associated with this:
1. When we disaggregate data, it will be held to a higher standard.
2. The field needs better tools3. The field needs well-trained reliable
data collectors4. The field needs systematic ways of
collecting, cleaning, and analyzing data.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies Most importantly, these Task Force
members noted, we need to be very careful not to misuse data:– Not to teach to the test.– Not to use data to label or place
children.– Not to defend programs.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies The Task Force members who
support the use of agency-level child assessment data acknowledged the challenges, but contended, over time, states would benefit by using both child and program quality data.
They recommended nine crucial safeguards.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies1. Collect enriched data on the programs,
including teacher-child interactions, nature of curriculum interventions, and nature of teacher training.
2. Collect detailed data on the children, including prior ECE experience and primary language.
3. Impel the state to align its assessments to all domains in standards.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies4. Select approaches that include direct
observation.
5. Base reporting on progress.
6. Collect data at more than one point in time.
7. Stagger strategies for data collection:– One third in each of 3 years
– Priority accorded to low-quality programs
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies8. Do not, under any circumstances,
report any data on individual children.
9. Use the data collected to make program improvements.
SAFEGUARDS #8 AND #9 ARE, BY FAR, THE MOST IMPORTANT
ONES RECOMMENDED.
Differing Viewpoints on Using Differing Viewpoints on Using Child Assessment Data on Child Assessment Data on
Local AgenciesLocal Agencies
While there was consensus on the safeguards recommended, the Task Force did not reach consensus on offering this a
separate approach.
Action Steps: LegislaturesAction Steps: Legislatures
Provide adequate funding for programs and infrastructure for ongoing assessments and program improvements.
Action Steps: State AgenciesAction Steps: State Agencies
Develop a strategic plan for early childhood accountability and program improvement system.
Create a robust, positive, and rigorous culture for early childhood accountability efforts.
Enable local Pre-K–3 partnerships.
Action Steps: Federal Action Steps: Federal GovernmentGovernment
“Harmonize” information systems.
Fund research and development for better assessment tools.
Support ongoing longitudinal research on children and programs.
Action Steps: Local AgenciesAction Steps: Local Agencies
Create opportunities for teachers and managers to review assessments and enhance children’s learning opportunities.
Initiate dialogue with local school districts.
The BenefitsThe Benefits For Children: Enhanced learning
opportunities and improved outcomes
For Legislators: Better data to guide state policies and investments
For Teachers/Directors: Targeted and well-resourced professional development and program improvement efforts
For the Early Childhood Profession: Enhanced public awareness and credibility
Action Steps: Follow-UpAction Steps: Follow-Up The Pew Charitable Trusts approved
a grant to the Council of Chief State School Officers (CCSSO) to disseminate the report and work with selected states in implementing recommendations.
Project will build on CCSSO efforts to improve state education standards, assessments, and data systems for all children—preschool through high school and beyond.
Implications Implications
Acknowledge the intensity of sentiment regarding potential misuse of child assessment data.
Attend to the infrastructure and the resources to do the collection.
Think systemically.
Implications Implications Need more work on tools that are
aligned with standards and curriculum.
Need more work on hard-to-capture domains in assessment construction.
Need far more work on the considerations regarding English Language Learners and children with disabilities.
Implications Implications Need states to focus on effective
accountability systems. Such accountability systems should be
designed to meet the state’s needs. Such accountability systems should take
into consideration state resources (human, technical, and fiscal).
Every state should develop and begin implementing an ECE accountability system that provides data and respects families, teachers, children, and child development principles.
top related