Learning from errors and Feedback: Statistics

Efficacy

Dissertation Proposal CUI Department

Simin Hall

Organization

Background Statement of the

problem Statement of

purpose Previous Research

Significance of this research

Research Design Questions of the

study Analysis Utility/Results

Background

Increase of 62% between 1976 to 2004 in enrollment in graduate programs www.NCES.ed.gov,

career development and personal enrichment UNC-Long_range_planning,

These graduate students are required to take statistics & quantitative research courses,

Their background may be far removed from the field of statistics,

They experience anxiety, What interventions are needed?

What does this intervention intend to accomplish?

To reduce anxiety in learning statistics.

Cultivate self-efficacy in graduate students in using statistics and research methods.

Environmental Scanning

The population of the study

The nature of the problem

Characteristics of adult learners

problem-centered results –oriented self-directed seek education that relates or applies

directly to their perceived needs accept responsibility for their own

learning, if learning is perceived timely and appropriate

Time/Distance are issues of concern

Nature of the problem students may experience anxiety due to

internal factors such as low efficacy perceptions in statistics or external factors such as poor instruction.

Instruction fragments theory, statistical analysis, and empirical reality.

Anxiety due to lack of background in quantitative reasoning or problem solving skills.

Statement of Purpose

Explore how students’ learning can be enhanced through instruction that builds self-efficacy.

Test the social cognitive theoretical prediction that self efficacy is enhanced by feedback that fosters problem solving skills.

Examine these predictions using a network-portfolio or a web-based feedback technology such as a wiki.

Previous Research

Wang and Lin (2006): Social Cognitive theory

Technology Liu et al. (2001): Web-based

peer review Seifert and Hutchins (1992):

errors as opportunity for instruction & learning Navigation

Mejias (2006):Network Portfolio system & practical research skills Teamwork

Network-technology Lovette et al. (2000): cognitive

theory, statistics instruction, individual practice, real-time feedback

Lin et al. (2001):Web-based peer assessment NetPeas

McKendree (1990): feedback content

Mayer, R.E. (2002, 1996) Cognitive theory; Examples & Statistics word Problems, Models for understanding

Fredrickson et al. (2005) web-supported vs lecture-based: course in statistics & research methodology for graduate students in education

Larreamendy-Joerns et al. (2005)Six on-line Statistics courses

Penn State

Features of this Research/Significance of this

Study

Monitoring errors (Recovery Strategy) Explicit Demonstration of how to provide

feedback Pattern recognition:Problems

,Mathematical-steps, Soft-ware Decision tree Map Use of Published research articles

research Control & treatment groups

Social Cognitive Theory Social learning theory explains human

behavior in terms of a continuous reciprocal interaction between cognitive, behavioral and environmental determinants (Bandura, 1986).

Students are capable to self-regulate, self-reflect, to plan alternative strategies, to learn by doing, and by modeling.

Social Cognitive Model of learning

Network Technology Allows continuous reciprocal

interaction between cognitive, behavioral and environmental.

Reduces individual cognitive loads Allows students to monitor and integrate

the information with the help of peers. interactive technology accommodates

students’ differences in regard to their information processing rate.

Feedback/Efficacy

Hypothesis: four sources of feedback impact students’ efficacy positively through their direct link to metacognitive strategies of planning, monitoring and regulating.

Planning, monitoring, and regulating are steps to problem solving strategies.

Sources of Efficacy

Theoretical Prediction According to social cognitive model of

learning, students monitor their steps and errors, they note their progress toward the solution, and progress indicators convey to students that they are capable of solving statistical problems. This reciprocity enhances self-efficacy for continued learning and impacts students’ emotion positively (Pintrich & Degroot, 1990; Pintrich & Schunk, 2002).

Research Design

Quasi-experimental design with two groups, one control and one treatment group. The control group will receive all instructions via Blackboard system Course, and the treatment group will use the Web-Based feedback system (WBF- wiki).

Intervention

(a) The teacher posts the homework assignment. (b) Each student prepares the homework and uploads it to the wiki. (c) For each assignment, the teacher assigns randomly three reviewers for each student. (d) Each student views other works and must comment on the steps of how the solution is derived. (e) Each student must revise the original assignment in line with the comments received. The final solution has a brief description from each student to summarize their initial understanding of the problem and their final understanding.

What do students achieve during the feedback

process? In completing the assignments, a student may

plan the steps to the solution, the procedures to be shown in the solution, and finally execute the plan. In reviewing peer homework, student must read, compare, or question ideas, suggest modification, or even reflect how well student’s work is compared with others. These cognitive processes involve monitoring the adequacy of steps adopted. However, if student receives a message that a step is not adequate, then the student must regulate the cognitive function and employ other alternatives.

Control Group

Students receive the same assignment as the treatment group; however, it is in the assignment format of Black Board for which students can not view each other’s works. Moreover, students can not retrieve and correct errors since they only receive feedback from the teacher on their final submissions. Students are encouraged to use the Discussion Board and emails to interact among themselves and with the teacher. They are assigned into groups to facilitate communication and learning. However, there is no grade associated with these activities.

Question-1

Is there a significant difference in the means of the two achievement grades across the groups (WFB vs. BB)? Students receive scores on assignments with peer feedback and the final is without the peer feedback. The assignment score will be compared with the final score using a repeated measure analysis. The dependent variables are the assignment and review scores combined and the final score and the independent variable is the method which is the web-based feedback system (treatment) and the Blackboard (control). The covariates are the students’ characteristics. It is hypothesized that students’ scores are higher with peer feedback.

H0: μt = μc ; Ha: μt > μc.

Question-2

Is there a significant difference in the means of pre/post of efficacy scores across the groups? The pre and post test scores of students on a questionnaire that measures students’ confidence/competence with statistics and feedback system are compared using a repeated measure analysis. The dependent variables are the pre and post efficacy scores and the independent variable is the method with students’ characteristics as the covariates.

H0: μt= μc, the null hypothesis states that the difference in efficacy scores is the same for both groups. Ha: μt> μc

Question-3 Is there a significant difference in the means of

pre/post of anxiety scores across the groups? The pre and post test scores on statistics anxiety for both control and treatment groups are analyzed using a repeated measure analysis with the test scores as within subject effect (dependent variable) and the intervention system as the between subject effect (independent variable) with students’ characteristics as covariates.

H0: μt= μc, the null hypothesis states that the difference in anxiety scores is the same for both groups. The alternative hypothesis is that the treatment group manage to reduce anxiety more than the control group. Ha: μt < μc

Question-4 How do students cluster based on the

variables in the study? In cluster analysis, there is no distinction between dependent variables and independent variables. A cluster analysis is performed to determine the grouping of students that have similar characteristics in terms of the variables measured in this study. By looking at the combination of variables in each cluster, I hope to define each cluster.

Question-5 Is there a significant difference in the

means of satisfaction between the two groups? A one-way analysis of variance with satisfaction scores as dependent variable and the method as the independent variable with students’ characteristics as covariates is analyzed.

H0: μt= μc, the null hypothesis states that the mean of satisfaction scores is the same for both groups and Ha: μt > μc.

Advantages & Disadvantages: web-based feedback

Advantages Continuous interaction Learning from modeling

and errors Planning, monitoring,

regulating nonintimidating

characteristics, such as anonymity in asking questions

Disadvantages potential losses due to

free riding, social loafing, and diffusion of responsibility

to lack focus or willingness to participate or the discipline for the web-based environment

some students experienced greater peer pressures in the peer assessment

dislike peer assessment because raters were at the same time competitors

What benefits/Who would use these results?

The prevalence of quantitative methods in social science literature (Vijverberg, 1997).

The decisions for doctoral

students to persist may vary widely, however, the peer mentoring in the area of research were identified as a predictor for retention.

Mastery of quantitative techniques opens up topics for thesis and dissertations and could potentially impact career choices.

Understanding efficacy beliefs should be helpful for teaching courses that traditionally have produced anxiety in students

This research may provide empirical evidence in regard to the effect of sources of feedback on efficacy in learning quantitative courses and thus research. Students’ satisfaction with this intervention may lead universities instructors to adopt the more advanced form of this technology, which provides instructors with tools for assessments (Wang & Lin, 2006).

Recommendation for Future Research

Researchers and engineers who construct web-based learning systems tend to overlook the reciprocal interactions between three influences in social cognitive theory.

Interactive web-based system provide scaffolding to learning. Need more empirical results to improve functions such as assessment tools.

Design better Psychometric measures of students’ learning.

Conclusion In this study, it is hypothesized that monitoring

and learning from errors on the network technology lead to successful problem solving skills. My main interest is not just limited to determine whether providing a learning assisted system promote active knowledge construction but also the effective learning strategies supported by the system is of high priority.

Finding the grouping of students who do not adapt well in the feedback system (are they not effective team players? What cognitive strategies do they lack?) are important findings for further studies to examine the effectiveness of web-based feedback system in different fields.