crescitelli - edtech 505 - request for proposal

Phone: 609‐532‐6125 Fax: 609‐532‐6128 E‐mail: [email protected]

801 Shakedown Street Suite 11A San Rafael, CA 43141

Terrapin Evaluations, Inc.

Introduction Far West Laboratory for Educational and Research Development (FWL) has made a preliminary investment in the design and implementation of a training program entitled Determining Instructional Purposes. Before mov-ing forward with continued investment and possible expansion, FWL is re-questing RFPs that focus the evaluation of this program. This document is being submitted by Terrapin Evaluations, Inc. (TEI) in accordance with the RFP. The Determining Instructional Purposes Training Program Prudent, fiscal decision making is catamount to a school system’s success. Assuring that approved, educational programs are sound and effective re-quires a skill set that must be maximized for efficiency. In turn, FWL has created a program called DIP, Determining Instructional Purposes. This program is being designed for school administrators and graduate students in order to help administrators become more effective in planning school programs. The program centers on three training units, each containing modules and activities that actively engage learners in authentic administrative scenar-ios. Each unit focuses on a certain concentration – setting goals, analyz-ing problems, and deriving objectives. The units can be run concurrently or in isolation depending on the needs of the school system. Unit books can be purchased individually or by the set. The program can be run with the use of a simple coordinator. Each coordi-nator requires a short training that examines each unit, but does not re-quire a special skill set. Coordinator tasks are most organizational in na-ture. Coordinator training manuals are available for purchase.

Evaluation of Determining Instructional Purposes

A Proposal Submitted to Far West Laboratory for Educational and Research Development

by Terrapin Evaluations, Inc.

March 7, 2010

John Crescitelli

Boise State University

EdTech 505

Week 6/7 Assignment

Submitted

March 7, 2010

Request for Proposal—Determining Instructional Purposes


Evaluation Methods The DIP training program will be evaluated on several fronts. This proposal will categorize the evaluations as either formative or summative in nature. Formative assessments will focus on two target populations – the training participants and the training coordinators. The summative assess-ments will focus on program participants and school system administration. Each of the assessment methods will be discussed below. For this evaluation, it is suggested to run the training units in three-day programs during June and July to accommodate administrative staff and suggested training times. Trainings will be conducted Monday-Wednesday during the two month period allowing for examination of the assessments prior to the next training week. It is the hope to TEI that 24 administrators and graduate students will par-ticipate in each of the three training programs. Formative Assessments Assessment 1 – Training participants will be asked to take part in surveys and interviews throughout the training to assess the effectiveness of the instruction and facilitation. A Likert-type scale will be incorporated to assess the effectiveness of each training session. The surveys will also contain ar-eas for participants to make anecdotal comments about the program, identifying areas of strength and others recommended for correction. Did the participants feel that the instruction was adequate in meeting learner need? Did the participants feel that the level of instruction was in line with expec-tations? Interviews will be conducted with all participants, as the number of initial participants is relatively small. Assessment 2 – Interviews will be held with training coordinators twice per unit. The interview ques-tions will focus on two areas: The effectiveness of the training preparation and areas for revision based on live implementation. It is important to determine if the coordinator preparation is adequate for the training programs to run effectively. FWL notes that coordinators need only complete the units prior to becoming a coordinator. It is important to assess the validity of that. Summative Assessments Assessment 3 – Both Pre and Post-testing will be administered to all program participants to assess learning outcomes for each of the three project units. The pre and post tests will consist of educa-tional scenarios similar to the modules and group activities present in each instructional unit. Partici-

Before moving forward, the DIP will need to be evaluated on several fronts: • Are the three units designed and implemented properly to meet the intended learning out-

comes? • Are the Coordinator’s handbook and training adequate for proper facilitation of the DIP pro-

gram? • Will the training impact the behaviors of participants when back in the school administrative

environment?



pants will be asked to make administrative decisions based upon the information presented. The post testing will used to assess how decision making skills have changed and adapted based upon new understandings. Some scenarios may be identical in both the pre and post testing to further measure growth. The purpose is to assess if participants met the objectives as outlined in the pro-gram protocol? Were students able to make adequate progress in the training program? Assessment 4 – Interviews with administrators to assess overall satisfaction with the program as it pertains to meeting desired outcomes. This will be conducted three months following the summer training. This later date will be done to determine the impact of the training in the actual school set-ting. Administrators will be interviewed to assess the authentic impact of the training on the adminis-trative staff and the overall impact on systemic change. The goal of this assessment is to determine if administrators found participating in the training a worthwhile monetary investment. Task Schedule An outline of tasks to be completed during this evaluation can be found in the chart on page four. It highlights an order of processes, those responsible for completion, and proposed dates. Proposed Budget A thorough break down of the proposed budget and payment schedule can be found on page five. Project Personnel John Crescitelli, M.E.T., president and CEO of Terrapin Evaluations, Inc. will coordinate the imple-mentation of all assessment protocol. Mr. Crescitelli has been president of TEI for six years. He has been evaluating educational programs throughout the country for more than two decades. Mr. Cre-scitelli has helped 14 school districts restructure training and professional development protocols throughout the northeast. He leads seminars on the national circuit focusing on Administrative Leadership and Systemic Change. George Pelc, Senior Evaluation Coordinator, will be in charge of producing all evaluative tools re-quired for the assessment. Mr. Pelc has extensive knowledge in evaluation design, receiving a Mas-ters in Instructional Design from the University of Wisconsin at Madison. He has been recognized nationally by the National Education Association for his work in transforming evaluation design. As he completed the model of a comprehensive assessment for the Boston Public Schools focusing on new teacher training protocols. Betsy Ross, administrative assistant to the President, will act as the contact person for FWL employ-ees and all training participants. Ms. Ross has worked on more than two dozen projects similar to this proposal, and displays incredible organizational skills. She will be available at TEI via Facebook, Twitter, Google Video Conference, and through a secure web site designed for this RFP. The web-site will contain all pertinent information surrounding timelines, interviews, and surveys and will act as the portal for all scheduling.



Task Schedule The following chart represents the proposed organizational outline for the implementation of the DIP evaluation. It is projected with a starting date of March 15, 2010 and an ending date of January 14, 2011.

Task Responsible Deadline 1. Initial 2-day meeting. This will entail opening dialogues between FWL and TEI. All three representatives of TEI will be present. The goal is to determine areas of focus for evaluation and determine evaluation strategies.

FWL March 15, 2010

2. Videoconference between Mr. Pelc and FWL representatives. The focus of this meeting is to review the questionnaires, surveys, and interviews designed by Mr. Pelc for the evaluation.

TEI April 6, 2010

3. Final submission of evaluation design revisions to TEI by FWL for final mark-up. FWL April 21, 2010

4. Meeting with FWL to examine final drafts of questionnaires, surveys, and inter-views. Meetings will also include planning sessions for evaluation strategizing.

TEI May 14, 2010

5. Initial posting of a secure, DIP-specific website and Twitter. These will include scheduling operatives and discussion blogs.

TEI May 30, 2010

6. Full summer training session for all three units in the DIP training program. All scheduling of workshops will be coordinated through the secure web site.

FWL June 1– July 30, 2010

7. Meetings and Interviews with program coordinators to assess coordinator imple-mentation and student attitudinal surveys. TEI will examine participant surveys from each week at these designated meetings, and organizing pertinent data for assess-ment.

TEI June 04, 11, 18, 25

July 9, 16, 23, 2010

8. Meeting with FWL. At this time, FWL will provide TEI with all Pre and Post test data, all interviews and surveys. This meeting will also be used to make determina-tions about which participant will be used for the long term impact assessment in November.

TEI August 18, 2010

9. Letters will be mailed to long-range participants detailing their role in the process. They will each be given a secure access password to the DIP assessment site. This site will how each participates in ending surveys.

FWL August 20, 2010

10. Meeting with FWL to summarize the pre and post test data, the participant inter-views and surveys, and the coordinator interviews.

FWL October 23, 2010

11. Open secure portal on the DIP web site for participants to begin impact surveys. Send emails to all participants notifying them the impact study survey will be avail-able for 14 days.

FWL November 3, 2010

12. Close the survey portal on the DIP site and close all data collection to begin re-view.

FWL November 17, 2010

13. Submit final report to FWL . FWL January 14, 2011

Personnel Sub Total Total Actual Cost Breakdown Mr. Crescitelli, Project Di-

rector (45 days @ $325/day) $14,625.00

Mr. Pelc, Evaluation Coor-

dinator (45 days @ $275/day) $12,375.00

Betsy Ross, Communica-

tions Liason $3,425.00

Total $30,425.00 Travel Sub Total Total Flights

$3,250.00

March 15, May 14, Oct. 23 Hotels

$2,150.00

March 15, May 14, Oct. 23 Estimated mileage $327.00 Total $5,727.00 Communications Sub Total Total Website and portal hosting $1,200.00 Video conferencing $750.00 Fax and phone services $475.00 Total $2,425.00 Supplies/Materials Sub Total Total Payment Schedule Paper, binding materials $550.00 Printing services $745.00 Payment 1: March 28, 2010 Stationery supplies $115.00 25% due within 14 days following

initial meeting

Postage $325.00 $10,877.37 Total $1,735.00

Payment 2: May 30, 2010 Miscellaneous Sub Total Total 25% due prior to the start of train-

ing sessions

Per diem meals $1,200.00 $10,877.38 Total $1,200.00

Payment 3: October 23, 2010 Program Supplies Sub Total Total 25% due at time of summary

meeting

Coordinator handbooks $202.50 $10,877.37 Unit handbooks $1,750.00

Other $45.00 Payment 4: January 11, 2011 Total $1,997.50 25% due at time of final report

meeting

$10,877.38

Total Expenses Total Total

$43,509.50 $43,509.50



Proposed Budget

crescitelli - edtech 505 - request for proposal

Documents