evaluation 101: after school programs

Download Evaluation 101: After School Programs

Post on 20-Jan-2016

41 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Evaluation 101: After School Programs. February 1, 2007 Region 3 After School Technical Assistance Center Conference. Evaluation 101. ASES Evaluationstudent data and how to get it Prop 49 evaluation requirements in SB638 Suggested Timeline Roles and Responsibilities. ASES Evaluation. - PowerPoint PPT Presentation

TRANSCRIPT

  • Evaluation 101: After School ProgramsFebruary 1, 2007Region 3 After School Technical Assistance CenterConference

  • Evaluation 101ASES Evaluationstudent data and how to get it

    Prop 49 evaluation requirements in SB638

    Suggested Timeline

    Roles and Responsibilities

  • ASES EvaluationAnnually by October 1 funded programs must submit to CDE data on participating studentsTwo years data required: 2006/2007 follow up/program year2005/2006 baseline/prior yearData submitted to CDE in Excel fileEvaluation narrative

  • Prop 49 Evaluation SB638Programs must submit individual student data:Program attendance participationRegular school day attendanceSTAR test results

    Program effectiveness measures (1 or more from list) STAR test dataPositive behavioral change*Homework completion rates*Skill development* Additional measures to be determined by CDE* Measured by ASP teachers or regular day teachers

  • Prop 49 Evaluation SB638

    State Board of Education will define performance measures by November 2007

    Funded programs will comply with state evaluation requirements determined in collaboration with Advisory Committee

  • TermsASP: After School Program

    CHKS: California Healthy Kids Survey

    Participation: after school program attendance

    Regular attendance: regular school day attendance

    Baseline: prior year 2005/2006

    Follow up: program year 2006/2007

    Continuing students: those who participated the previous year in After School Program

  • ASP Evaluation Keys to SuccessFour distinctive types of student information neededBuild evaluation database throughout yearClear roles and responsibilities Good ongoing communicationDocument process and sources

  • Evaluation data: Four Types Needed on StudentsParticipation in after school program: number of days attended program in the school yearRegular school attendance: days enrolled and days absent in the school yearCalifornia Standards Test proficiency level math and language arts California Healthy Kids Survey questions at entry and exit from program (4 questions)

  • Sources of Data

  • Sample Excel File in Folder

  • Who Needs to be involved?ASP director and site staff

    District data managers, or evaluation department, or fiscal department with regular school attendance information on students

    District testing and curriculum staff who obtain the CST results annually and disseminate them to schools

    Staff with expertise in Excel (or database software)

  • TipsCreate a roster of key people involved in ASP evaluation with their phone numbers, faxes and email addresses.Do ASP evaluation update every 2-3 months to keep all informed on progress. Will raise ASP evaluation visibility Immediately thank via email and/or phone call anyone who gives you dataWrite it down for next year!

  • Participation Data

  • Participation DataPrior to program beginning:Decide on how to gather and record program attendance: after school program software, Excel spread sheets or Access databaseFind last years evaluation report in Excel format to get baseline data (ongoing programs)Decide on roles and responsibilities for data entryWho records attendanceWho enters data into computer

  • Participation DataRecord childs name (first and last), grade, school, Student ID Number and days attended by monthAs new children enter, record them in the data system on an ongoing basisContinuing participants, gather baseline for participation, regular school attendance and CST from prior years Excel file

  • TipsKeep track of Student ID Numbers they are the key to obtaining regular school attendance and CST results

    Get childs name as they are registered in school no nicknames. James Jones not JJ Jones or Jimmie Jones

    Dont record attendance in Word , you will need to be able to add figures and sum them up

  • Regular School Attendance

  • Regular School AttendanceData needed:Days enrolled, days absent current school year 2006/07 and prior school year 2005/06

    Where is it kept:District data systemLink by Student ID Number and school

  • Regular School AttendanceWhat you need to do:Identify where data is kept in districtContact person who can get you access to data and what you will need: initial information query in fallContact in March to obtain baseline prior year attendance for students new to programContact in June to obtain current year attendance for all participants

  • TipsDont be surprised if there is no prior year attendance for new participants student mobility means students may be new to the school in program yearCheck data for new participants in March for reasonablenessDays enrolled should be around 170 (150-181)Days absent should be between 1 and 20; rarely over 20

  • California Standards Tests

  • California Standards TestsResults available July or August following end of school year

    Math score and Language Arts Score

    Scores are 1 through 5 or Far Below Basic, Below Basic, Basic, Proficient and Advanced

    Best source for student demographics: gender, ethnicity, English Language Learner Status

  • California Standards TestsData neededProficiency level in math and language arts for program year (spring 2007) and baseline year (spring 2006)Demographics on studentsWhere it is keptComes to district testing staff July or August on CD ROM from state testing contractor

    For continuing ASP students, baseline data is on prior year evaluation report to CDE

    For new ASP students, baseline data is prior year CST results 2005/2006

  • California Standards Tests What you need to do:

    Identify test contact person in districtContact person who can get you access to test data: initial information query fallContact in March to obtain baseline prior year proficiency for students new to programSubmit file with id numbers of new students by school with data requestContact in June to remind them of upcoming summer requestSubmit file with id numbers of all participating students in summer for new test data and new demographics

  • TipsDont be surprised if there is no prior year CST results for new participants student mobility means students may be new to the school in program year Keep in touch with testing person for district, sometimes data comes in earlyDont wait until the summer, many people are not working in summer months!

  • California Healthy Kids Survey

  • California Healthy Kids SurveyTrain staff to administer survey when program begins and when students leave the program or near the close of school year

    Enter initial (pre) surveys in evaluation database as they come in.

  • Survey Questions1.In the past 30 days, how often have you wanted to go to school? (Never, one or two times, About once a week, A few times a week, Everyday)2.In the past 30 days, how often have you studied for a test? (Never, one or two times, About once a week, A few times a week, Everyday)3.In the past 30 days, how often have you felt unsafe at school? (Never, one or two times, About once a week, A few times a week, Everyday)4.In the past 30 days, how often have your parents asked you about school? (Never, one or two times, About once a week, A few times a week, Everyday)

  • TipsBe sure to gather pre surveys for students who enter the program during the year.Train staff to look for red flags if a student reports they dont feel safe, for example.Gather the post follow up surveys (same 4 questions as on entry to program) beginning about one month before the program ends.Enter into database.

  • Master timelineMonth before program starts

    Fall

    Decide on database formatSet up participation data entry systemTrain staff on California Healthy Kids Survey and attendance proceduresFind last years evaluation report Excel filesIdentify person in district with regular school attendance data and CST test scores and hold initial meeting

  • Master timeline, continuedOctober through March

    Check to see that you can generate list of students by school with id numbers and total days attended for first three months or six monthsCheck that number of initial California Healthy Kids Surveys in database match number of participantsBuild baseline for ongoing students using ASP participation, regular school attendance and CST results from previous evaluation Excel file

  • Master timeline, continuedMay of school year

    Meet with district attendance data contact; provide list of id numbers of participating students; obtain prior year attendance for new students

    Review completeness of database, look for holes on continuing students and fill them

  • Master timeline, continuedJune

    July

    AugustRequest current year attendance data for all participants, giving school list of students with ID numbers

    Request CST data and demographics from district test contact

    Finalize Excel file with Student ID, days of ASP attendance; regular school attendance; CHKS answers

  • Master timeline, finalSeptemberAdd test scores from CST to Excel filesComplete narrativeSubmit to CDE