the evaluation of mathematics and science partnerships program a quasi experimental design study...
TRANSCRIPT
![Page 1: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/1.jpg)
The Evaluation of Mathematics and Science Partnerships Program
A Quasi Experimental Design Study
Abdallah Bendada, Title II Directorhttp://www.dpi.wi.gov/cal/t2bgrant.html
MSP Regional ConferenceFebruary 14-15, 2008
![Page 2: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/2.jpg)
• Mathematics and Science Partnerships Grant
• Evaluation Model: Quasi-experimental design
• Sampling
• Instruments
• Outcomes
• Conclusions
• Future Studies
Program Design
What is the true impact of the Mathematics and Science Partnerships Grant on student
achievement in Wisconsin?
![Page 3: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/3.jpg)
Evaluation Model
1. Participants reaction (Surveys, logs, …)
2. Participants Learning (Assessment, tests, ..)
3. Organization support for changes (administration )
4. Participants use of new knowledge (Observation, performance)
5. Student learning outcomes (Assessment, tests, …)
All = Gold
2 to 5 = Silver
2 and 5= Copper
Anyone = Tin
![Page 4: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/4.jpg)
R O X OR O O
R is the program Group and R the Comparison
Group
Time
O refers to the testing
X is the treatment
Vertical alignment of O shows the pre- and post- testing time
Design
![Page 5: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/5.jpg)
Sampling
• Program group: 75 middle school mathematics teachers from high-need LEAs. pre-selected
• Only 69 teachers participated completed the program
• Comparison group: 75 middle school teachers from similar
LEAs. Matched to the program group.
• Only 71 teachers completed the program
• Program group: 5 students from each participating teacher randomly assigned. (325)
• Comparison group: 5 students from each participating teacher randomly assigned. (325)
![Page 6: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/6.jpg)
Instruments
1- Teachers
The Test of Mathematics Knowledge for Teaching School Mathematics also known as Learning Mathematics for Teaching (LMT) developed by Deborah Ball and Hyman Bass at the University of Michigan as an NSF project, were used to assess the teacher in both the pre- and post-test.
2- Students
The Measures of Academic Progress (MAP) developed by the Northwest Evaluation Association (NWEA) in mathematics has been used to assess student achievement .
More than 138 school districts in Wisconsin use MAPs in mathematics and reading to monitor the progress of student achievement.
![Page 7: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/7.jpg)
X Xz
SD
z
2
( )1
1xx
k X k Xr
k k
1 2
2 21 2( ) / 2
X Xd
SD SD
Statistical Formulas
![Page 8: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/8.jpg)
Score Gain Differences between Program and Comparison Groups
-2
0
2
4
6
8
10
12
14
16
1 2 3 4
Groups
Sco
re G
ain
s
Program Teachers
Comparison StudentsComparison Teachers
Program Students
![Page 9: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/9.jpg)
Program Teachers' Gains
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
0 20 40 60 80
Teacher
Z-S
core
![Page 10: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/10.jpg)
Teacher: Score Distribution
0
0.1
0.2
0.3
0.4
0.5
-3 -2 -1 0 1 2 3 4
Score
Freq
uenc
y
ProgramComparison
![Page 11: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/11.jpg)
Student Gains
-5
0
5
10
15
20
0 10 20 30 40 50 60 70 80
Student Teams
Raw
Sco
re G
ain
Program
Comparison
![Page 12: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/12.jpg)
Z-Score True Gains
-2.00-1.50-1.00-0.500.000.501.001.502.002.503.00
-2.00 -1.00 0.00 1.00 2.00 3.00 4.00 5.00
Teacher Gain
Stu
den
t Gai
n
![Page 13: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/13.jpg)
Students: Score Distribution
0.00
0.20
0.40
0.60
0.80
1.00
1.20
-2.00 -1.00 0.00 1.00 2.00 3.00
Score
Comparison
Program
![Page 14: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/14.jpg)
Raw Score
Program Group Comparison Group
Teachers Students Teachers Students
Pre Post Gain Pre Post Gain Pre Post Gain Pre Post Gain
mean 8 18 10 12 23 11 8 8 0 9 9 0
SD 1.97 2.79 1.90 3.64 1.81 3.30 1.06 1.22 1.00 1.69 1.46 1.33
min 3 12 6 6 18 3 6 4 -4 5 5 -2
Max 13 24 15 19 26 18 11 11 1 14 12 4
d 6.44 4.32 --------- --------------
Participants 67 335 67 335
Raw score Analysis
![Page 15: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/15.jpg)
Conclusions
• Positive Impact in mathematics,
• Program teacher gained more content knowledge,
• The program impact is on teachers with lower content knowledge,
• Program students achieved higher,
• The program impact is on students with lower content knowledge,
• Effective evaluation designs are necessary for impact programs, and
• Variables such as the socioeconomic group, ethnic group, and disability should be factored in the evaluation design
![Page 16: The Evaluation of Mathematics and Science Partnerships Program A Quasi Experimental Design Study Abdallah Bendada, Title II Director](https://reader030.vdocuments.net/reader030/viewer/2022032702/56649f495503460f94c6ad01/html5/thumbnails/16.jpg)
Disclaimer
The instructional practices and assessments discussed or shown
in these presentations are not intended as an endorsement by
the U.S. Department of Education