the effect of online discussion on daily outcomes

8/10/2019 The Effect of Online Discussion on Daily Outcomes

1/37

Running Head: Effect of Online Discussion on Outcomes 1

The Effect of Online Discussion on Daily Outcomes

Kurt C. Hargis

Southeastern Louisiana University


2/37

Effect of Online Discussion on Outcomes 2

Contents

List of Figures .................................................................................................................................. 4

Chapter One: Introduction ............................................................................................................. 5

Background to Research .............................................................................................................. 5

Chapter 3 Methodology ................................................................................................................ 13

Hypothesis ................................................................................................................................. 13

Research Design ........................................................................................................................ 13

Sample ....................................................................................................................................... 13

Instrumentation ........................................................................................................................ 14

Procedure .................................................................................................................................. 14

Data Analysis ............................................................................................................................. 15

Chapter 4: Analysis of Data ........................................................................................................... 16

Introduction ............................................................................................................................... 16

Subjects ..................................................................................................................................... 16

Discussion of Data ..................................................................................................................... 18

Reliability and Validity ............................................................................................................... 22

Discussion of Results of Chapter Tests ...................................................................................... 22

Discussion of Results of Survey ................................................................................................. 25

Chapter 5 Conclusions and Implications. ...................................................................................... 31

Introduction ............................................................................................................................... 31

Conclusions about Hypothesis .................................................................................................. 32

Conclusions about the Research Problem ................................................................................. 33

Implications for Theory ............................................................................................................. 34

Implications for Policy and Practice .......................................................................................... 34

Implications for further research .............................................................................................. 35


3/37


References ..................................................................................................................................... 36


4/37


List of Figures

Figure 1: Journaling form for Chapter 2 ........................................................................................ 17

Figure 2: Chapter 5 Journaling Example 1. .................................................................................... 19

Figure 3: Chapter 5 Journaling Example 2 ..................................................................................... 19

Figure 4: Journaling Example of Stating Frustration .................................................................... 20

Figure 5: Solution provided by fellow student to Figure 4 ............................................................ 20

Figure 6: Journaling Example of Stating Frustration 2 .................................................................. 20

Figure 7: Solution provided by fellow student to Figure 6 ............................................................. 21

Figure 8: Journaling Example of Stating Success .......................................................................... 21

Figure 9: Results from Chapter 2 Test. The reflection for this test was the Word document ....... 23

Figure 10: Chapter 5 Test on Control Structures -- Online Journaling with No Feedback ............ 24

Figure 11: Chatper 9 Test results for the Peer feedback journaling. Error! Bookmark not defined.

Figure 12: Survey Question 1 ........................................................................................................ 25

Figure 13: Survey Question 2 ........................................................................................................ 26

Figure 14: Survey Question 4 ......................................................................................................... 27




5/37


Chapter One: Introduction

Background to Research

In most classrooms, students often wonder, "What are we doing today?"

Teachers will often point to the board where they have carefully written out the day's

agenda. At the top of the agenda is often the daily outcome or objective which usually

begins with the phrase "The learner will be able to..." Yet, the question arises at the end

of each lesson, "can the student perform the written outcome?" How is the teacher to

know? What methods of data collection is the teacher using? How do students perceive

outcomes? In a computer-based classroom, are there more efficient ways of determining

how students understand the objectives? This action research proposal will look at how

students perceive and interact with class objectives in computer science classroom.

Students will evaluate their understanding of the objectives in three ways: a traditional

journal form, in a comments section of a blog only seen by the teacher, and in a

moderated comments section of a blog where the students may interact with each other.

Purpose

The purpose of this study is to determine the effect of electronic communication in

discussing and evaluating how students achieve learner outcomes in a classroom setting.

Hypothesis

It is hypothesized that there will be a statistically significant improvement by the

students who reflect electronically upon their daily desired outcomes as compared to the

students who did not reflect upon the daily desired outcomes.


6/37


7/37


well as three computer labs that can be reserved by teachers including one that can hold

multiple classrooms. DHS is on a 4x4 block for most classes (some AP classes have

combined with other classes to form an AB schedule.) The classes on the 4x4 are a

semester long.

Student Needs

In the 2013-2014 School Improvement Plan, data teams/review were considered a

priority in that we needed to use data to focus on student learning and achievement.

While our school uses extensive formal assessments in most classrooms before

determining if students are equipped to take the summative assessment, few teachers

have reported data focusing on how students perceive the outcomes. Those that did do

this did it through a traditional journal style of expressing their thoughts. None did it in

an electronic form in which students could share ideas about the days lesson.

School Goal

Destrehan High School has determined that 90% of students should score Fair or

better on each end of course exam. Courses that do not have end of course exams still

determine how well students do on those exams by focusing on effective learning

strategies that can be used cross-curriculum such as being able to read charts and graphs,

maps, and using literacy, which all courses at DHS cover.

School Ecology and School Climate

Destrehan has long had a reputation for being a great school. One of the reasons why

we have this reputation is that our leadership at both the local and the district levels has

high standards for the school. Besides having an effective administration, the school has

a full time staff development coordinator who provides teachers with constant support.


8/37


The DLT meet on a weekly basis to evaluate the status of the school in relation to the

School Improvement Plan. The literacy team creates and disseminates strategies for

improving literacy at DHS. Every other Wednesday is a late take-in for students, which

provides for professional development time as well professional learning teams (PLTs)

that focus on one subject are or one type of learning, such as AP teachers. Destrehan is

dedicated to improving the quality of learning for every student and has developed an

environment that encourages teachers to strive for that extra step in reaching their

students.

Paradigm Shift

While teachers are ready to embrace technology at school, there might be some

hesitation for teachers to do moderated online discussion, particullarly with older teachers

who do not feel as comformtable with technology as the younger teachers do. One

obstacle is time, especially for non-lab based classrooms. This would require the teacher

to set aside some time in the evening to check on the comments of their students.

Risks

There are few risks to this study. No content is being scarified to be able to do this

study. The study will be implemented at the beginning of the semester in that it will be

as if it is an established procedure for the students. Since the same material is being

taught with the same methods as the year before, the students from the year before will

become the control group.


9/37


Chapter 2 Review of Literature

The role of having strong learner outcomes to guide learning is practically

undisputed in the field of Education. Outcomes are what drive instruction, which in turn

drives assessments. Gusky (1991) states, they are the basis by which instructional

materials are selected, content is outline, instructional process are developed, and

evaluation procedures are prepared an carried out. He further writes, citing the work of

Ralph Tyler, that learning and teaching go hand in hand. Outcomes sometimes referred

to as objectives, provide this link between teaching and learning. Yet, often, in

Education, the outcomes are often looked at as a teacher task. Does the outcome match

what is being taught? Is the outcome measurable? While these are important and vital

components of a classroom, the people who are often evaluating the outcome are another

educator. In a study written by Sullivan, the author argues that the development and

proper classroom installation of effective objectives-based instructional systems will

require the combine efforts of learning specialists, curriculum and subject-matter experts,

and school administrators and teachers (1968). Should the outcome be evaluated from

the students' perspective? Ilin, Inozu, and Yumru (2007) claim that the student

perception represents a potentially powerful variable that should be taken into account.

If the students do not perceive the outcome in the same way as the teacher, then a

disconnect occurs and the outcome was not truly taught. In fact, there is evidence that

students will feel better about their learning when they realize that they did in fact learn

something (Houlden R. & Collier, 1999). Houlden and Collier further state problem-

based learning can also benefit by reflection on the learning outcome objectives after the

main learning activity (1999). In other words, when students have an active stake in


10/37


identifying with the learning objectives, they will be able to perceive how much learning

has actually occurred.

With the rise of technology, more classroom work is becoming more student

centered. This does not change the role of outcomes in a classroom. The outcome sets

the stage for learning and clues the student in learning how that outcome will be

evaluated. In a study of outcomes in multimedia instruction, researchers saw a dramatic

difference in students who were fully aware of what the outcomes were for a particular

lesson as compared to those who were not told what the outcomes were (Abbot, Siskovic,

Nogues, & Williams, 2000).

The research shows that students who are aware of the daily outcomes and reflect

upon the outcomes tend to have greater success than students who are not aware of the

outcomes. Daily agendas should be used to make sure students have understood the

days lesson and outcomes (Eccleston, 2004). Reflecting can occur by having the

students complete journals or in class discussion. However, neither of these methods

provides students a way to take home the information gleamed by other students. If they

are taking notes during the discussion, then chances are they are not partaking in the

discussion. A way to have both a written reflection and a group discussion is using

online tools for discussion. Just like in a classroom discussion, looking at who

participates can provide the teacher with feedback on who is actually learning the

outcomes. In a study by Xie, the researcher discovered that student who are not

confident of their capabilities are much less likely to engage in learning activities and

more likely to exhibit minimum effort in online discussions (2013). This provides the


11/37


12/37


complained about evaluators who provided feedback without specific suggestions for

modification (2013). Students welcomed feedback when feedback is constructive

whether it is from the instructor or a fellow student.

The research shows that students perform better when they understand the desired

outcomes for each class. Research also shows that online discussions can greatly

improve learning. If students have online discussions based on their understanding of

desired learner outcomes, then student achievement should improve.


13/37


Chapter 3 Methodology

Hypothesis

It is hypothesized that there will be a statistically significant improvement by the

students who reflect electronically upon their daily desired outcomes as compared to the

students who did not reflect upon the daily desired outcomes.

Research Design

This study will utilize a sequential, mixed methods design, which can be

illustrated as follows: Quant -> qual. The quantitative component of the study will use an

ex post facto design. The independent variable will be students enrolled in a Pre-

Computer Science course. The levels will be students enrolled in last years computer

science course compared to those enrolled in this semesters course. The dependent

variables will be the students test scores in the course. A survey will be given at the end

of the course to all students enrolled in this semesters course to determine whom they

feel about reflecting on the use of online discussions in reflecting upon the daily desired

outcomes. In addition, certain students from last years course will be given surveys to

determine if they would have benefited from online reflections.

Sample

This study will utilize convenience sampling using Pre-Computer Science

students at Destrehan High School. There will 21 students enrolled in this semesters


14/37


15/37


will provide the feedback at the beginning of the next class. The teacher will only

provide feedback if the topic completely stumps the class, but may provide additional

points to ponder. For Chapters 8 and 9, students will be allowed to choose any of the

above methods as a way of reflecting on the day's lesson.

The chapter tests will provide the quantitative feedback to determine if the reflections

are working. Surveys will also be administered to both groups to determine how they

perceived how reflecting on the outcomes influenced their learning.

Data Analysis

In order to determine whether a statistically significant difference exists, the

teacher will provide charts and graphs. Qualitative data from the reflections will be

analyzed to identify students perceptions of reflecting on daily outcomes using online

tools.


16/37


Chapter 4: Analysis of Data

Introduction

Because of an illness that debilitated the instructor, the original plan to spread the

research over six chapters was restricted to three. However, the instructor is confident

that this will provide an accurate, albeit small, sample to measure the hypothesis.

However, this research will not take the place of an intensive research study. Students

reflected on the daily lessons using three methods: traditional journaling with only one

check before the test, online journaling that was only seen by the teacher and student, and

online journaling in which peers could respond to each other. Data was collected on test

scores for the three methods as well as a survey of student perceptions on reflecting on

outcomes.

Subjects

Students who chose to enroll in Computer Science I were the subjects for this

study. 21 students enrolled the fall semester of 2014. Of those 21 students, 14 students

are male and 7 are female. Looking at the racial breakdown of the class, 17 are white, 2

students are black, 1 student is Asian, and 1 student is Hispanic. For last years control

group, 13 students were male while 9 were female. The breakdown according to race, 18

were white, 3 were black while 1 was Hispanic. Most students are enrolled in the

Advanced Studies program at Destrehan High School, but the class is open enrollment.

Seven students are enrolled that are not enrolled in the Advanced Studies Program. Two

students in the Fall 2014 class are on a non-traditional diploma track.


17/37


Figure 1: Journaling form for Chapter 2


18/37


19/37


studied Computer Science under the instructor.

Figure 2: Chapter 5 Journaling Example 1.

Figure 3: Chapter 5 Journaling Example 2

The third chapter covered the Object Oriented Programming concept of

inheritance. This is a core concept that must be understood for any student who wishes to

take AP Computer Science or who chooses to major in Computer Science in college. The

journaling was broken up into two parts. At the beginning of the day, students completed

a frustration check online. They stated what hindered them from achieving success in the


20/37


previous days lesson(Figures 4 & 6). Then, each student had to answer at least one

question (Figures 5 &7). While no instructions were given to the students who achieved

little or no success the day before, most sought out the students who were at a similar

level and attempted to answer their question. The few students who did not answer

received extra help from the instructor. Students were also encouraged to ask lingering

questions at this time, even if it was on a previous chapter. For the second part of the

activity, students wrote in an online journal that was public their success of the day

(Figure 8). They did not have to answer each other, but they were not discouraged from

providing positive feedback.

Figure 4: Journaling Example of Stating Frustration

Figure 5: Solution provided by fellow student to Figure 4

Figure 6: Journaling Example of Stating Frustration 2


21/37


Figure 7: Solution provided by fellow student to Figure 6

Figure 8: Journaling Example of Stating Success

In addition to the test results, the students answered a survey after taking the

Inheritance test. The following questions were included on the survey:

Which do you like best when reflecting on the outcomes?

For the Word Document, rank the sections in order of what part helped.o Restating outcomes

o Sharing your success

o Sharing your Furstration

o Stating why you should learn the outcomes

For the next section, evaluate how you felt about posting frustrations with no feedback from peers. (Strongly Disagree to Strongly Agree)

o I like the anonymity

o I didnt feel the teacher addressed my problem specifically just generally.

o I would have like to use this more often.

For this section, evaluate the statements based on how you felt about stating

your frustrations with your peers.

o I felt comfortaqble answering my peers

o I felt uncomfortable answering my peers

o I was able to see that I wasnt the only person having a particular problem

o I was able to answer questions for people

o

I would like to have used this more in class.

If you had to choose one of the following which would you choose?o The Word Document option only

o Frustrations with no peer feedback only

o Frustrations with peer feedback only

o Success Sharing only


22/37


23/37


Figure 9: Results from Chapter 2 Test. The reflection for this test was the Word document

The test on Control Structures (Figure 10) shows an increase in the test group

compared to the research group. Because students received feedback to anonymous

questions, most students were inclined to ask questions. Nearly every student asked

questions and quite a few asked similar questions. The instructor would incorporate

answers to these questions without singling out who asked the question or informing the

student that he was responding to a reflection question. The goal of this type of reflection

was to allow students to state problems without fear of judgment from peers. The results

for the Fall 2014 group show that they responded well to this type of journaling. It

allowed the instructor to understand what the students did not comprehend about control

structures.

80%

82%

84%

86%

88%

90%

92%

94%

96%

98%

100%

Fall 2013 Fall 2014

Average

Average


24/37


Figure 10: Chapter 5 Test on Control Structures -- Online Journaling with No Feedback

Chapter Nine had students provide feedback to each other. The feedback allowed

students to understand the project associated with the core concept of this chapter,

inheritance. The goal was for the students to better understand the concept of inheritance

by providing feedback to each other. While the students performed well on the test

(Figure 11), the result does fall short of the significant statistical difference of the

hypothesis of this action research.

Figure 11: Chapter 9 Test results for the Peer feedback journaling

80

82

84

86

88

90

92

94

96

98

100

Fall 2013 Fall 2014

Average

Average

808284868890929496

98100

Fall 2013 Fall 2014

Averages

Averages


25/37


Discussion of Results of Survey

How the students perceived the journaling was determined by a survey. Students

were asked a set of questions to determine how they felt about reflecting on each lesson.

The first question (Figure 12) examined which method they preferred. Interestingly,

none of the students preferred the posting of frustrations without feedback. Yet, this

contradicts the data that shows this type was the most successful

Figure 12: Survey Question 1


26/37


Question 2 (Figure 13) dealt with the feedback form shown in Figure 1. Students

ranked the order of what section they like the most. Students felt that sharing their

frustrations is what helped the most. None of the students ranked first stating why they

should learn the outcomes. This particular question would benefit from its own research

study.



27/37


The next question (Figure 14) focused on the online posting of frustrations

without having peer feedback. In Figure 12, no student chose this response, yet for the

Chapter 5 test showed the most drastic difference between this years class and last

years. No real trend shows here except they did not perceive that their issues with the

days lesson were being addressed. Yet, the test that coincides with this type of reflection

showed the highest scores. Perception did not meet the reality in this case.



28/37


The next question (Figure 15) had students evaluate statements based on how you

felt about sharing their frustrations with their peers. The most noticeable piece of

information is that sharing frustrations with the days lesson and outcomes allowed them

to understand they are not alone. They were able to see the problems that they have are

the problems their peers are having. It also shows that many of them felt that they could

answer their peers.



29/37


Finally, the last question asked students what method would they choose (Figure

16). Feedback with peers was the overwhelming choice. However, students split

between just feedback, and having feedback at the beginning of the class and sharing

their success at the end of the class. This goes against the data found in the test scores.

Sharing their frustrations of reaching the outcomes without peer feedback had only

4.35% support. Yet, for this method the test score was overwhelmingly higher than the

control group. Student perceptions of achieving the outcomes differed from what the

research data states.


30/37




31/37


32/37


33/37


students from both groups scoring nearly the same. Yet, for the survey, the students

overwhelming preferred the third method to other two. The method that caused the

highest scores was not chosen as the type of reflection the students preferred. While there

was not a statistically significant difference in the scores between the groups, the students

did perform at high level with the average for both classes being a B.

Conclusions about the Research Problem

Having students focus and reflect on outcomes helps students. The previous

research has shown this as well the students attitudes in this survey toward sharing

feedback with each other. However, it is difficult to determine how well it worked in this

study or in any study. In the Fall 2014 class, the experimental group, two students who

were not in traditional graduation paths were registered for the class. The control group

had all students on the traditional track. One student excelled; the other struggled on tests

but made As and Bs on all programs. In other words, this student could program, but

tests poorly. Another factor is the student who does not try on the test. One student on

the last test scored a 50. The next lowest was a 72, which is still a passing grade. When

asked after class why he scored a 50, his response was that he does not care. If that

outlier is taken out of the average, the Fall 2014 group scores higher than the 2013 group

easily (Figure 17). This particular student is a senior who will graduate regardless of his

grade in Computer Science. The researcher kept the score to keep the integrity in the

research. In addition, the health of the instructor definitely affected the results of the

study. However, the program that coincided with the final test saw the highest average

that the instructor had for that program. Nearly every student scored near a 110 on the

assignment. While this was not part of the study, the reflection with peer feedback


34/37


definitely helped the students in the completion of this program. Perhaps, reflecting on

outcomes would work better if formative assessments were used as the instrument of

measurement. While the researcher was confident in the hypothesis at the beginning of

the study, the data derived strongly suggests more refined research is needed. Lessons

learned from the application of this study would allow the researcher to conduct a more

focused, refined, and accurate study.

Figure 17: Chapter 9 Test with the student who stated he did not care removed from the

average.

Implications for Theory

The results of this study due to outside factors became a sample for the research

problem. The qualitative data shows that if students were to reflect on the daily outcome

they preferred to have feedback from peers and provide feedback to peers. With

increasing resources allowing this to be done online, students could continue the

conversations started during the reflection after school. This could provide for a new

way of studying. Instead of fighting social networking, teachers could employ its

significant power to bring group discussions into the 21stcentury.

Implications for Policy and Practice

80

85

90

95

100

Fall 2013 Fall 2014

Averages

Averages


35/37


Teachers in most school districts are required to have daily outcomes recorded

and visible for the students. Having students doing this online would help engage student

discussions in a new and exciting way. Teachers would reinforce writing skills by having

students communicate with the written word instead of verbally. However, there are

caveats. Teachers would have to constantly monitor to ensure bullying does not take

place. In addition, teachers will have to ensure that they are using district-approved

websites to comply with state law, especially if teachers are taking part in the discussion.

Implications for further research

While this study allowed the teacher and the students to look at outcomes in ways

they might not have done before, using it in such a small class with such a limited

population affected the results. This area needs more research. More tools are available

for this on a daily basis. This study would need to be applied school-wide to see how it

affects learning across the curriculum. Developing a control group might be difficult

except for using the previous year grades. Maybe comparing similar schools that have

common assessments in a district would provide interesting data.


36/37


References

Abbot, L., Siskovic, H., Nogues, V., & Williams, J. G. (2000). Learner assessment in

multimedia instruction: Considerations for the instructional designer. Society for

Information Technology & Teacher Education Internal Conference: Proceedings

of SITE 2000.1-3, pp. 1-7. San Diego: SITE.

Arend, B. (2009). Encouraging critical thinking in online threaded discussions.Journal of

Educators Online, 6(1), 1-23.

Bye, L., Smith, S., & Rallis, H. (2009). Reflection using an online discussion forum:

Impact on student learning and satisfaction. Social Work Education, 28(8), 841-

855.

Eccleston, J. (2004) And on today's agenda... Teaching Pre K-8, 34(4), 54-55.

Ertmer, P. A., & al, e. (2007). Using peer feedback to enhance the quality of stuent online

postings; An expoloratory study. 12(2), 412-433.

Guskey, T. R. The Importance of Focusing on Student Outcomes.Annual Meeting of the

North Central Association of Colleges and Schools , (pp. 1-12). Chicago.

Houlden R.: Collier, C. (1999). Learning outcome objectives: A critical tool in learner-

centered education.Journal of Continuing Education in the Health Professions,

19(4), 208-213.

Illin G, I. J. (2007). Teachers' and learner's perceptions of tasks: Objectives and

outcomes. 3(1), pp. 60-68.

Liu, E., & Lee, C. (2013). Using peer feedback to improve learning via online peer

assessment. Turkish Online Journal of Education Technlogy, 12(1), 187-199.

Skinner, E. (2009). Using community developent theory to improve student engagemen

in online discussion: A case study .ALT-J: Research in Learning Technology, 17

(2), 89-100.

Sullivan, H. J. (1968). Improving learner achievemnt through evaluation by objectives.

Inglewood, CA: Southwest Regional Educational Lab.


37/37


Xie, K. (2013). What do the numbers say? The influence of motivation and peer feedback

on students' bheaviour in online discussions.British Journal of Educational

Technology, 4(3), 288-301.

the effect of online discussion on daily outcomes

Documents