Computers in Human Behavior 32 (2014) 244–252

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

A cloud-based learning environment for developing student reflectionabilities

0747-5632/$ - see front matter � 2013 Elsevier Ltd. All rights reserved.http://dx.doi.org/10.1016/j.chb.2013.12.014

⇑ Corresponding author. Tel.: +886 2 77343347; fax: +886 2 23929449.E-mail addresses: ricky014@gmail.com (Y.-T. Lin), sophia.t04008@gmail.com

(M.-L. Wen), joum@ntnu.edu.tw (M. Jou), iebill@cc.ntnu.edu.tw (D.-W. Wu).

Yen-Ting Lin a, Ming-Lee Wen b, Min Jou a,⇑, Din-Wu Wu b

a Department of Industrial Education, National Taiwan Normal University, 162, He-ping East Road, Section 1, Taipei 10610, Taiwan, ROCb Graduate Institute of Education, Taiwan Shoufu University, 168, Nanshi Li, Madou Dist., Tainan 72153, Taiwan, ROC

a r t i c l e i n f o

Article history:Available online 10 January 2014

Keywords:Reflective learningCloud computingImproving classroom teaching

a b s t r a c t

Students learn new knowledge effectively through relevant reflection. Reflection affects how studentsinteract with learning materials. Studies have found that good reflection abilities allow students to attainbetter learning motivation, comprehension, and performance. Thus, it is important to help studentsdevelop and strengthen their reflection abilities as this can enable them to engage learning materialsin a meaningful manner. Face-to-face dialectical conversations are often used by instructors to facilitatestudent reflection. However, such conventional reflection methods are usually only usable in classroomenvironments, and could not be adopted for distance learning or after class. Cloud computing could beused to solve this issue. Instructor guidance and prompting for initiating reflection could be seamlesslydelivered to the students’ digital devices via cloud services. Thus, instructors would be able to facilitatestudent reflective activities even when outside the classroom. To achieve this objective, this study pro-posed a cloud-based reflective learning environment to assist instructors and students in developingand strengthening reflection ability during and after actual class sessions. An additional experimentwas conducted to evaluate the effectiveness of the proposed approach in an industrial course. Resultsshow that the learning environment developed by this study is able to effectively facilitate student reflec-tion abilities and enhance their learning motivation.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Enhancing student reflection is important for the teaching andlearning of new knowledge or skills because reflection affectshow instructors and students interact with learning materials theyencounter (McNamara, 2004). From instructors’ perspective, stu-dent reflection often influence their best teaching (McNamara,O’Reilly, Best, & Ozuru, 2006). Reflection also influences howinstructors plan teaching strategies for new classes in order to en-hance student learning motivation and performance. From the stu-dents’ perspectives, the lack of reflection abilities is a higher riskthat new knowledge may be built upon faulty foundations (Boyd& Fales, 1983). Additionally, psychological investigations showedthat good reflection abilities would improve learning becausememory or mental storage capacity could be developed throughassociation with pre-existing knowledge or experience (Schon,1987). Studies also indicated that good reflection abilities wouldenhance student motivation, comprehension, and performance inlearning new knowledge or skills (Boud, Keogh, & Walker, 1985;

Kemmis, 1985; Paris & Ayres, 1994). Reflection abilities may alsobe a critical component in the acquisition, processing, and applica-tion of new information within other contexts (Chen, Kinshuk, Wei,& Liu, 2010; Chen, Wei, Wu, & Uden, 2008).

The reasons stated above suggest that it is important to developand strengthen student reflection abilities to help them engagenew learning materials in a meaningful manner. To induce studentreflection in a classroom setting, instructors usually ask studentscertain questions to which the students reflect and provide feed-back (Chi, de Leeuw, Chiu, & Lavancher, 1994; Davis, 2000). How-ever, engaging students in face-to-face dialectical conversation inclassroom settings is not possible for instructors during remotelearning or after school sessions. Cloud computing and servicescould provide a solution. Prompts and activities helping to inducereflection could be seamlessly delivered to students’ digital de-vices, allowing instructors to facilitate student reflective activitieseven when outside the classroom.

In order to assist instructors in developing student reflectionabilities and to help students attain greater learning motivationand performance, a cloud-based reflective learning environmentwas proposed in this study. An experiment was also conductedin an industrial course in a Taiwanese university to investigatethe effectiveness of the proposed approach.

Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252 245

2. Literature review

2.1. Reflection

Since information explosion leads students always browsethrough on Internet, this phenomenon could affect their thinkingand further influence their learning performance. Therefore, pro-moting students’ reflection on their learning and behavior is cur-rently a major educational goal in higher education. The conceptof reflection was first proposed by John Dewey in 1933 (Dewey,1933), which is defined as repeated thinking, searching, observa-tion, and understanding toward problems, surrounding environ-ments, or causal relationships. Reflection is described as aprocess of active, persistent and careful consideration that aimsto construct individual knowledge and meaning by using personalexperiences, perceptions, and beliefs (Carver & Scheier, 1998). Theprocess would be initiated through personal experience, thinking,comprehension and awareness, so that people would be able tosurvey, explore, and evaluate issues, opinions, feelings, and behav-iors they encounter (Ward & McCotter, 2004). Reflection is also re-garded as a useful learning process that can help students expressand evaluate their attitudes and feelings, expand their learningcognition, and increase the comprehension of their own thinking(Chirema, 2007; Ladewski, Krajcik, & Palincsar, 2007). Throughindividual inquiry and socially interact with others, reflection canlead students’ thinking from surface to deep level (Moon, 2004;van den Boom, Paas, & van Merriënboer, 2007).

From a teaching perspective, Schon (1987) divided reflectioninto two major frameworks, which are reflection-on-action andreflection-in-action. Reflection which occurs after teaching or dur-ing the interval before planning and thinking would be reflection-on-action. On the other hand, reflection-in-action would be theadjustments of personal teaching methods and feedback responsesthat occur during the teaching process. From the learning perspec-tive, reflection would provide opportunities to stimulate studentsto examine knowledge they have learned (Etkina et al., 2010; Jou& Shiau, 2012).

To prompt and guide student knowledge construction,researchers recommended instructors to use different types ofquestions (Chen et al., 2008; Lee & Chen, 2009). King (1994) usedthree types of questions to help students construct individualknowledge, which are questions on memory, comprehension, andconnection. Students would gradually enhance their cognitionand comprehension as they progress from lower-level (memory)questions based on factual knowledge to higher-level (connection)questions based on reflective practices. The literature review alsoindicated that higher-level questions would facilitate reflectionand could result in better understanding and higher learning per-formances (Redfield & Rousseau, 1981; Roscoe & Chi, 2008).

2.2. Cloud computing

During the last two years, cloud computing has become anincreasingly popular phenomenon in every field. Cloud computingcurrently includes a series of hardware and software service pro-vided via the Internet (Venters & Whitley, 2012). Since applicationsand user data would be stored remotely on cloud servers, users canseamlessly access cloud services and applications by using any dig-ital device with an internet connection. In other words, cloud com-puting can enable users to access, process, share, and storeinformation via the Internet from any location or device (Lin, Fu,Zhu, & Dasmalchi, 2009). In traditional IT systems, user productiv-ity may be restricted since personal information was exclusive tospecific applications, services or devices (Sang & Sung, 2013). How-ever, these obstacles would be overcome by cloud computing since

it can provide services to users seamlessly. Therefore, cloud ser-vices would make users more efficient, help facilitate collaborationwith their peers, and give users seamless access to their informa-tion anytime and anywhere from any digital device (Marston, Li,Bandyopadhyay, Zhang, & Ghalsasi, 2011).

Cloud computing would be highly practical in education forboth instructors and students. The technology allows instructorsand students to access powerful services and massive computingresources wherever and whenever they needed, including varioususeful applications, services, and tools which are provided freelyand openly to instructors and students (Jou & Wang, 2013). Sinceone of the feature of cloud computing is enable software as a ser-vice (SaaS), rather than as a standalone program. Therefore, theinterconnectivity feature of cloud computing would allow instruc-tors to administer entire learning processes easily and conve-niently, allowing students to learn effectively (Paul, Chen, &Gloria, 2010). Additionally, cloud computing offers a potentialway to enable instructors and students to conduct formal lessonseven without a standard indoor classroom since the cloud servicesallow them to share their data with anyone, anywhere, and at any-time (Astrid, Paul, Carol, & Jordana, 2012).

2.3. Influence of web-based learning instrument on learning

Generally, the online learning environment is suitable to offeropportunities for reflection. It is useful for students to constructindividual knowledge if appropriate learning services can be ap-plied to assist them in concentrating on learning and guiding theirengagement in reflection (Lamy & Goodfellow, 1999; Yang, 2010).To date, several web-based applications or services have been pro-posed to support various classroom activities including reflectionactivities (Huang, Lin, & Cheng, 2009; Jou, Chuang, & Wu, 2010;Lin, Tan, Kinshuk, & Huang, 2010). However, most of these servicesare novel or stand-alone programs. This means that users (instruc-tors and students) have to spend additional time and efforts tofamiliarize themselves with these new tools. Users may be re-quired to install additional programs on their own devices or reg-ister as a new user, which would negatively affect the motivationfor using these services to support specific educational contexts(Lin, Lin, & Huang, 2011).

However, the age of cloud computing saw other web applica-tions such as SkyDrive, Evernote, DropBox, and Google Apps beingdeveloped. Different from traditional web pages and applications,such cloud computing applications offer SaaS to users who canuse various digital devices to apply these services openly andfreely. Users today are inundated with a myriad of web applica-tions that provide friendly user interfaces and powerful functionsin the cloud. Therefore, many instructors and students were al-ready using these cloud applications in their daily lives (Lin &Jou, 2012). These observations compelled several researches tosuggest that these modern web applications could be a new wayof engaging participants in meaningful teaching and learning activ-ities (Alexander, 2006; Hughes, 2009; Schneckenberg, Ehlers, &Adelsberger, 2011; Thompson, 2007; Wang, Woo, Quek, Yang, &Liu, 2012). Most of both instructors and students would be moremotivated in using these applications in an educational contextsince they already have the necessary technical skills (Dohn,2009). Therefore, they would only need to learn how to apply theseapplications in supporting educational activities for their classes(Pretlow & Jayroe, 2010). Previous studies also found that partici-pants who take part in a web-enhanced class outperformed thosewho underwent traditional lectures (Crook & Harrison, 2008; Ha-mann & Wilson, 2003). Effective use of web applications can alsoblur the boundaries between formal and informal learning (Ben-nett, Bishop, Dalgarno, Waycott, & Kennedy, 2012).

246 Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252

With these in mind, this study integrated Google web applica-tions as a set of teaching and learning tools within a reflectivelearning environment. The applications used are briefly describedin the following:

� Google plus: This application is a free social networking servicethat can be accessed from various digital devices. Social net-works between instructors and students can be established eas-ily using Google plus. The social networks would allow users toshare reflective thoughts, activities, and events effectively.� Google drive: This web-based office suite allows users to create

and edit documents online while collaborating with other usersin real-time. Google drive allows instructors and students toimmediately write down, view, and reply to ideas conveyedthrough any connected PC or mobile device.� Google sites: This structured wiki- and webpage-creation appli-

cation allows users to create dynamic web pages with theirpartners with the ease of writing a document. The use of Googlesites can therefore assist instructors and students in consolidat-ing and expressing their reflective thoughts.

3. Cloud-based reflective learning environment

A cloud-based reflective learning environment was developedin this study to assist instructors in administering reflective activ-ities and help students develop reflection skills. The followingparagraphs describe the development and content of the learningenvironment framework.

3.1. The framework of cloud-based reflective learning environment

Fig. 1 shows the framework of the cloud-based reflective learn-ing environment with its three main components, namely the tea-cher side, student side, and cloud service.

The core component of the framework is the cloud servicewhich includes the course website, web applications, and dat-abases. The course website was developed to consolidate teachingand learning functions for instructors and students respectively.The web applications were used to assist instructors and studentsin administering and conducting reflective learning activities. Thedatabases include a course database and a user profile database.The course database was built to store course information and re-sources, while the user profile database was utilized to store stu-dent personal information, learning profiles, and learning records.

Instructors can use the teacher side of the reflective-learningenvironment to administer course instructions and conduct reflec-tive learning activities to help student learning reflection. Studentscan log into the learning environment on the student side and par-ticipate in various reflective learning activities during or after thecourse.

3.2. The development of cloud-based reflective learning environment

The primary aim of developing a cloud-based reflective learningenvironment is to find an appropriate approach to achieve this. Thecourse website was created using ASP.NET 3.5 (C#) and the data-base was built using Microsoft SQL Server 2005. Google web appli-cations were also integrated into the learning environment inorder to facilitate participation in reflective activities.

An instructor interface based on the framework of the cloud-based reflective learning environment is provided for instructorson the teacher side of the course website. As shown in Fig. 2,instructors can gather and examine relevant information fromthe course archive to administer entire teaching and learning pro-cesses. They can also connect directly to each web application from

the user function list to administer and conduct reflective learningactivities via the course web site.

Students can log into the course website on the student side andobtain or browse course information using a student interface.They can also connect to the web applications from the user func-tion list. Moreover, they can view and participate in particularreflective learning activities during or after the course, as shownin Fig. 3.

4. Experiment

An experiment was conducted to determine the effectiveness ofthe proposed cloud-based reflective learning environment onenhancing reflective practices and learning performance in anindustrial course on product design.

4.1. Research instruments, measures and goals

To evaluate the effects of the proposed cloud-based reflectivelearning environment on reflective teaching and learning perfor-mance, various data sources were utilized, including question-naires, pre-test/post-test results, and interviews. Thequestionnaires were designed to document student learning moti-vation and learning attitude. In addition, tests designed to assessthe student reflective levels were implemented before and afterthe learning activities. Finally, the interviews were further usedto investigate participant perception towards the entire teachingand learning process.

4.1.1. Instrument of learning motivation surveyTo measure student learning motivation toward entire teaching

and learning processes, the intrinsic value scale of the learningmotivation questionnaire Motivated Strategies for Learning Ques-tionnaire (MSLQ) was adopted to explore whether students’ learn-ing motivation had improved. The intrinsic value scale of MSLQwas recommended by researchers to measure student goals andbeliefs about the importance and interest for class works, whichare highly relevant to the aim of developing the cloud-based reflec-tive learning environment. The scale of MSLQ included nine itemsbased on a seven-point Likert scale (Pintrich & De Groot, 1990).

4.1.2. Instrument of learning attitude surveyStudent learning attitude was surveyed using a learning atti-

tude questionnaire that consisted of seven items scored using afour-point rating scale. The questionnaire has been previously usedto measure student learning attitudes towards learning activitiesin other courses (Hwang & Chang, 2011; Hwang, Wu, & Chen,2012).

4.1.3. Instrument of reflection level evaluationFinally, the pre- and post-tests were designed to assess student

reflective levels in the produce design course. The tests includedmemory, comprehension, and connection questions. The pre-testcontained five multiple-choice test items (memory questions),such as ‘‘During a product design process, which one is not belongto the consideration of ergonomics?’’ and four short-answers testitems (two comprehension questions and two connection ques-tions), such as ‘‘What is different between innovation and inven-tion?’’ and ‘‘Please give an example to explain how to applyergonomics on product design.’’ that asked students the means ofdesigning an industrial product according to their preliminaryunderstanding. The post-test was similar in structure, with alsofive multiple-choice test items (memory questions) and fourshort-answer test items (two comprehension questions and twoconnection questions) that asked students the means of designing

Fig. 1. The framework of the cloud-based reflective learning environment.

Fig. 2. Screenshot of the instructor interface.

Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252 247

an industrial product specifically for a hand-held device usingknowledge acquired from the teaching activities. The maximumscore for both tests were 50 points. In addition, three trained re-search members were assigned to evaluate student answers inboth the pre- and post-tests. The inter-rater reliability of the pre-test and post-test were 0.931 and 0.902 respectively. The resultsindicated that the three research members have a concordancewith regard to the pre- and post-tests.

4.2. Experimental design, participants and procedure

To investigate the effectiveness of the proposed cloud-basedreflective learning environment, a quasi-experimental researchwas conducted on a product design course in a computer-aidedclassroom at a Taiwanese university. 70 university students (38males and 32 females) from department of industrial educationwith an average age of 21 years and a course instructor participatedin the experiment. All of the participated students were randomlyand averagely assigned to an experimental group and a controlgroup. The experimental group included 35 students who used theproposed learning environment to support their reflective learningactivities. The remaining 35 students served as the control groupand underwent similar activities as the experimental group, albeitwith the aid of a discussion board instead of web applications.

The same learning and teaching activities were implemented bythe instructor for both groups during class time. The activities in-cluded lecturing, engaging students in reflective learning activities,stimulating interaction and discussion, and consolidating thelearning activity objectives. During the class, the course instructorwould give lectures and engage students in reflective learningactivities. Moreover, the students in the experimental group wouldapply Google drive to record personal learning status and reflectivelearning results. After class, the students in the experimental groupwould use Google plus to interact with peers and the instructor tocontinuously extend the discussions and reflective learning activi-ties in the class. On the other hand, the students in the controlgroup would utilize a discussion board to carry out such activitiesafter the class. Before the end of the entire course, the instructorwould require the students of the experimental group to form se-ven groups to conduct and present a specific issue by using Googlesites. With regard to the students in the control group, the studentswould be required to use Microsoft Office Word document to con-duct and present a specific issue.

No matter the students in which group, in order to stimulatethem to think reflectively, the course instructor would especiallyuse real-life problems in the course to facilitate their discussionssince there are few guidelines in their texts for the kind of pro-blems (Brown, 1997). In other words, these problems are the kind

Fig. 3. Screenshot of the student interface.

248 Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252

of ill-defined, messy, complex problems for which reflective think-ing is needed anyway (Halpern, 1998; Kennedy, Fisher, & Ennis,1991). Students have to continuously challenge their own ideasand argue among themselves about the problems and the solutionto the problems.

Learning and teaching activities for both groups were con-ducted in five sections that had a total of 400 min as shown in Ta-ble 1. Students spent an average of 50% of the course time learningthe course materials and used the remaining time to discuss andreflect on newly learned knowledge. The scheduling was a goodbalance between instruction and student interaction, the latterhelping to verify that learning was achieved (Kong & So, 2008).

Fig. 4 shows the experimental process. All students took threepre-tests before undergoing the learning activities. The first testwas on prior knowledge of product design and the second was areflection level test. Thirdly, all students were then asked to fillout a learning motivation questionnaire before participating inthe learning activities. After going through the learning activities,all students received a post-test, a second learning motivationquestionnaire as well as a learning attitude questionnaire to re-evaluate student reflection levels and learning motivation and tosurvey student learning attitude. Interviews were also carried outafter the learning activities to investigate the participant percep-tions on the entire teaching and learning process.

5. Results

5.1. The effect of prior knowledge

Prior knowledge tests with independently developed questionswere used to determine whether knowledge level on product de-sign were the same between the experimental and control groups.To conduct the analysis, a Kolmogorov–Smirnov Z test was firstlyused to examine whether a t-test could be used in the prior knowl-edge tests. Results showed that the data satisfied the assumptionof normality. Finally, a t-test was used to determine whether therewere significant differences in prior knowledge of product designbetween the groups. Results showed that prior to the course, therewere no significant differences between the experimental and the

control groups (t = �0.081, p-value = 0.936). In other words, stu-dent abilities in both groups were statistically equivalent beforethe product design course.

5.2. Learning motivation survey

All students were asked to fill out an MSLQ questionnaire beforeand after the learning activities. The Cronbach’s alpha values of thequestionnaire items were 0.815 and 0.865 for the experimentalgroup and control group respectively. A one-way independentsample analysis of covariance (ANCOVA) was used to analyze thesurvey. In the ANCOVA, the post-test and pre-test scores of learn-ing motivation were treated as the dependent variable and covar-iate respectively. Homogeneity of the regression coefficient wastested before performing ANCOVA. Results confirmed the homoge-neity of the regression coefficient (F(1,68) = 13.628, p-va-lue = 0.611 > 0.05). Table 2 shows ANCOVA results of the learningmotivation post-test score for the two groups as well as the meanscores and standard deviations. The mean of the experimentalgroup (46.543) was higher than that of the control group(40.800), implying that students whose learning was supportedby the proposed cloud-based reflective learning environment hadhigher motivation. After eliminating the influence of the covari-ance on the dependent variable, post-test learning motivationscores between the two groups turned out to be significantly dif-ferent (F(1,67) = 13.866, p-value = 0.00 < 0.05). Therefore, the pro-posed cloud-based reflective learning environment significantlybenefited the students in terms of learning motivation.

5.3. Learning attitude survey

Students in both the experimental and control groups were re-quired to fill out a learning attitude questionnaire after participat-ing in the learning activities to survey their learning attitude. TheCronbach’s alpha value of the questionnaire items was 0.732. A t-test was performed to compare learning attitude scores betweenthe two groups. The t-test results revealed significant differencesin student attitudes between the experimental group and controlgroup. (t(34) = 5.115, p-value < 0.05). In addition, to further survey

Table 1Major teaching and learning activities in the product design course.

Unit Instruction activities Time(min)

Design theory 1. Introduction of learningenvironment (10)

80

2. Instructor presentation (30)3. Discussion (20)4. Reflection (20)

Human factors engineering 1. Review of previousinstruction (5)

80

2. Instructor presentation (35)3. Discussion (20)4. Reflection (20)

Ergonomics 1. Review of previous instruction (5) 802. Instructor presentation (35)3. Discussion (20)4. Reflection (20)

Product patent 1. Review of previous instruction (5) 802. Instructor presentation (35)3. Product patent practice (20)4. Reflection (20)

Creative design management 1. Review of previousinstruction (5)

80

2. Instructor presentation (35)3. Discussion (20)4. Reflection (20)

Fig. 4. Experimental process.

Table 2ANCOVA results of the learning motivation post-test score between the two groups.

Group Number ofstudents

Mean S.D. Adjustedmean

F(1,67) p-Value

Experimentalgroup

35 46.543 3.559 46.463 13.866 0.00*

Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252 249

the differences of the students’ learning attitudes between the twogroups, t-tests were used to analyze each questionnaire item. Thestatistical results are presented in Table 3.

More students in the experimental group reported having posi-tive attitudes towards product design learning compared to stu-dents who did not use the cloud-based reflective learningenvironment. The t-test results revealed significant differences instudent attitudes between the two groups. Students in the experi-mental group: (1) had better attitude for learning more aboutproduct design material, (2) had higher desires for product designknowledge and (3) were better motivated for actively searchinginformation about product design.

However, no significant differences in student opinions werefound on the importance of the course between the two groups.Most students in both groups felt that the product design learningwas meaningful and useful, and expressed that they would like tolearn product design well. Additionally, a vast majority of studentsin the experimental (94%) and control (90%) groups stated thatlearning about product design was important.

These results indicated that most students in both groups had apositive attitude towards the product design course. Nevertheless,students in the experimental group were significantly more posi-tive in terms of attitude towards the course compared to those inthe control group.

Control group 35 40.800 7.858 40.880Total number

of students70 43.671 6.711

Note: S.D.: standard deviation.* p < 0.05.

5.4. Pre-test/post-test evaluation of reflection level

To further evaluate student reflection levels, pre- and post-testswere administered before and after the product design learningactivities. The test-sheet consisted of five multiple-choice testitems (memory questions) and four short-answer test items (twocomprehension questions and two connection questions).

A one-way independent sample analysis of covariance (ANCO-VA) was used to analyze the tests. In the ANCOVA, the post-testand pre-test scores of reflection level were treated as the depen-dent variable and covariate respectively. Homogeneity of theregression coefficient was tested before performing ANCOVA. Re-

sults confirmed the homogeneity of the regression coefficient(F(1,68) = 0.286, p-value = 0.594 > 0.05). Table 4 shows ANCOVAresults of the reflection level post-test score for the two groupsas well as the mean scores and standard deviations. The mean ofthe experimental group (35.429) was higher than that of the con-trol group (32.143), implying that students whose learning was

Table 3Students’ attitude towards product design learning.

# Item Experimental group(Mean/S.D.)

Control group(Mean/S.D.)

t(68)

1 The product design course is valuable and worth studying for. 3.771/0.426 3.600/0.497 1.5492 It is worth learning about product design 3.829/0.382 3.714/0.458 1.1333 It is worth learning the product design course well 3.857/0.355 3.686/0.471 1.7194 It is important to learn more about product design, such as observing and learning about

industrial products3.429/0.502 3.000/0.485 3.632*

5 It is important to know about novel design and new technologies relevant to product design 3.914/0.284 3.571/0.558 3.241*

6 I will actively search for more information about product design 3.800/0.406 3.486/0.507 2.863*

7 It is important for everyone to take the product design course 3.743/0.443 3.714/0.458 0.265

Note: S.D.: standard deviation.* p < 0.05.

Table 4ANCOVA results of the reflection level post-test score between the two groups.

Group Number ofstudents

Mean S.D. Adjustedmean

F(1,67) p-Value

Experimentalgroup

35 35.429 5.736 35.378 5.173 0.026*

Control group 35 32.143 6.450 32.193Total number

of students70 33.786 6.281

Note: S.D.: standard deviation.* p < 0.05.

250 Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252

supported by the proposed cloud-based reflective learning envi-ronment had higher reflection levels. After eliminating the influ-ence of the covariance on the dependent variable, post-testreflection scores between the two groups turned out to be signifi-cantly different (F(1,67) = 5.173, p-value = 0.026 < 0.05). Therefore,the above results implied that students achieved higher reflectionlevels after using the proposed cloud-based reflective learningenvironment compared to those that did not.

5.5. Interview investigation

The course instructor and students in both groups were inter-viewed to understand their perceptions of the product designcourse and investigated the differences of the teaching and learn-ing experiences between the two groups. Instructor and studentresponses were recorded and then transcribed for all the inter-views. To clearly present the interview results, the transcript con-tents were divided into three main topics of instruction,interaction, and technology according to the interview results.

5.5.1. Instruction perspectiveThe instructor stated that students in the experimental group

were more engaged in the learning events and activities duringand after the class as they could easily and conveniently adminis-ter, share, and consolidate individual thoughts and knowledgethrough the cloud-based reflective learning environment. Addi-tionally, he was impressed that students in the experimental groupdemonstrated excellent performance in every discussion andreflection activity. He also felt that web application support usedby the experimental group could easily extend discussion andreflection activities to after school sessions. This is because theuse of the modern web applications that could enable the studentsto keep higher motivation and more positive attitude for contin-uing their learning. Thus, more time could be used to address con-cepts in detail or interact with students. In contrast, the instructorthought that only one fourth of students in the control group could

fully follow the activities after class, while the rest were too unmo-tivated to get involved.

For the student interviews, a small group of students in bothgroups preferred more direct instruction from the course instruc-tor since they thought that some of the concepts were difficult tounderstand and hindered participation in discussion and reflectionactivities. Nevertheless, majority of the students, especially thosefrom the experimental group, indicated that learning about prod-uct design with reflection activities was interesting.

5.5.2. Interaction perspectiveThe instructor observed that students in the experimental

group often gave feedback and asked questions on product designconcepts. These students also had better discussions and reflec-tions in each learning activity during and after the course. In con-trast, students in the control group carried out fewer discussionsand reflections, especially after school. Therefore, the instructornoted that adopting well-known web applications would posi-tively influence student participation and responsiveness in theirlearning.

5.5.3. Technology perspectiveThe instructor and the majority of students in the experimental

group indicated that the user interface of the cloud-based reflec-tive learning environment was clear and straightforward. Theywere already familiar with major learning tools before participat-ing in the product design course. Most students actively used thecloud-based reflective learning environment due to its conve-nience, which also motivated them to carry out reflective learningactivities and strengthen their reflection abilities. This was possiblesince the cloud-based reflective learning environment was inte-grated with well-known web applications. Most students neednot install extra applications or register as a new user for usingthe learning environment. The instructor, on the other hand,thought that reflection learning was driven by the students them-selves instead of the cloud-based reflective learning environment.He also believed that students were willing to conduct reflectionlearning during and after learning new things in a class becauseit helped improve their learning, but were often unable to do soas they lacked suitable means and tools. Therefore, the instructorfelt that the cloud-based reflective learning environment was pro-vided effective tools that helped students to interact, discuss, andshare their individual thoughts and knowledge with their instruc-tor and peers. The proposed learning environment facilitatedreflective practices and thus successfully addressed the issue ofimproving student reflection abilities.

With regard to the control group, the instructor and most stu-dents indicated that the discussion board is an older learning toolthan the web applications applied in this study. Therefore, it is notsuitable for the present student behaviors on Internet since theinformation transmission and social interactions on the discussion

Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252 251

board were poorer than those on the modern web applications.Moreover, the discussion board was not convenient to users dueto it is a standalone program that may not support various devices.However, a major advantage of the discussion board is that it had alow technical obstacle for each student to use.

6. Conclusion and discussion

This study developed a cloud-based reflective learning environ-ment capable of supporting student reflective learning activities.The environment seamlessly provides a student-centered learningcontext to help students strengthen their reflection abilities. Anexperiment was also conducted in an industrial course to evaluatethe effectiveness of the proposed approach.

The following paragraphs provide detailed discussions explain-ing how this research contributed to reflection learning, providednovel ideas and expanded upon current literatures. Further applica-tions of the cloud-based reflective learning environment, researchlimitations, and proposals for further research on the developmentof reflective learning environments were discussed as well.

6.1. Contribution of the cloud-based reflective learning environment toreflection learning

In order to promote reflection learning, it is important to moti-vate and enhance learning motivation and participation for everystudent during and after a class. Students must receive reflectivelearning support from their learning environments when studyingfor a subject or course. However, student learning aids are ofteninadequate such as learning tools, which often confounds aninstructor’s best efforts to deliver reflective instructions effectively.Conventional learning environments are often unable to provideeffective teaching and learning tools for administering reflectivelearning activities after school for instructors and students. There-fore, the major contribution of this study is the proposed cloud-based approach that provides students appropriate learning toolsto facilitate reflective motivation. In other words, the objectivesof this study are to facilitate student learning participation and fur-ther promote learning motivation and reflection during and after aclass by integrating modern web applications to the reflectivelearning environment. The improved reflection and motivationprovided by the proposed learning environment also made teach-ing more successful.

The proposed cloud-based reflective learning environment wasimplemented experimentally in a product design course at a Tai-wanese university. A series of evaluation processes that includedtests, questionnaires, and interviews were used. Results verifiedthat the proposed learning environment provided both instructorand students with effective supports. Students in the experimentalgroup exhibited higher learning motivation and reflection perfor-mance as they could obtain adequate learning supports duringand after the class. Despite the importance of reflection for stu-dents and instructors, reflective learning activities are usually notincluded in actual teaching due to limits on class duration. It is alsodifficult for instructors to induce student reflections or work withthem effectively. Therefore, this study concluded that the cloud-based reflective learning environment is able to effectively assistinstructors and students in administering and conducting reflec-tive learning activities during and after a class.

6.2. Further applications of cloud-based reflective learningenvironment for educators

From a pedagogical perspective, although the proposed cloud-based reflective learning environment was implemented in aproduct design course, the proposed approach can also be used

by educators in different educational contexts to achieve variouspedagogical objectives. Educators can use the learning environ-ment to interact with students and survey their opinions prior toa class to improve both teaching and learning. The environmentcan also be used during an instruction to conduct a formativeassessment on student learning statuses and to identify learningproblems. Finally, educators are able to use the environment toconduct post-instruction performance assessments to find outhow well the students had acquired relevant knowledge taughtduring the learning process.

6.3. Limitations and future work

The proposed approach in this study was implemented in acomputer-aided classroom. A limitation of the approach wouldbe that it could not be easily implemented in standard classroomssince not every student owned a tablet PC or a smartphone. How-ever, future progress and popularization of mobile devices wouldeventually solve this issue. Another limitation of this study wouldbe that participants may have a limited understanding in the use ofweb applications to support their learning (Ng, 2012) even if themajority of them were already familiar with the web applicationsin their daily lives. Therefore, course instructors would be requiredto provide additional information and instructions on using theseweb applications for educational purposes.

Future directions of this study would be to apply appropriatecloud-based applications to strengthen the proposed learning envi-ronment and support the instruction of different subjects and dis-ciplines. Further investigations on student learning effects mayalso utilize suitable cloud-based learning tools developed usingapplication program interface (API) to support the reflective learn-ing environment in various educational contexts.

Acknowledgement

The authors of this study gratefully acknowledge the supportprovided by the National Science Council of Taiwan, under theGrant No. NSC99-2511-S-003-034-MY3.

References

Alexander, B. (2006). Web 2.0: A new wave of innovation for teaching and learning?Educause Review, 41(2), 32–44.

Astrid, S., Paul, R., Carol, B., & Jordana, L. (2012). An observational study ofundergraduate students’ adoption of (mobile) note-taking software. Computersin Human Behavior, 28(2), 308–317.

Bennett, S., Bishop, A., Dalgarno, B., Waycott, J., & Kennedy, G. (2012). ImplementingWeb 2.0 technologies in higher education: A collective case study. Computers &Education, 59(2), 524–534.

Boud, D., Keogh, R., & Walker, D. (1985). What is reflection in learning? Reflection:Turning experience into learning. New York, NY: Nichols Publishing Company.

Boyd, E., & Fales, A. (1983). Reflective learning: The key to learning from experience.Journal of Humanistic Psychology, 23(2), 99–117.

Brown, A. (1997). Transforming schools into communities of thinking and learningabout serious matters. American Psychologist, 52, 399–413.

Carver, C. S., & Scheier, M. F. (1998). On the self-regulation of behavior. UK:Cambridge University Press.

Chen, N. S., Kinshuk Wei, C. W., & Liu, C. C. (2010). Effects of matching teachingstrategy to thinking style on learner’s quality of reflection in an online learningenvironment. Computers & Education, 56(1), 53–64.

Chen, N. S., Wei, C. W., Wu, K. T., & Uden, L. (2008). Effects of high level prompts andpeer assessment on online learners’ reflection levels. Computers & Education,52(2), 283–291.

Chi, M. T. H., de Leeuw, N., Chiu, M., & Lavancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.

Chirema, K. D. (2007). The use of reflective journals in the promotion of reflectionand learning in post-registration nursing students. Nurse Education Today, 27(3),192–202.

Crook, C., & Harrison, C. (2008). Web 2.0 technologies for learning at key stages 3 and4: Summary report. <http://dera.ioe.ac.uk/1480/1/becta_2008_web2_summary.pdf>. Retrieved 15.05.12.

Davis, E. A. (2000). Scaffolding students’ knowledge integration: Prompts forreflection in KIE. International Journal of Science Education, 20(8), 819–837.

252 Y.-T. Lin et al. / Computers in Human Behavior 32 (2014) 244–252

Dewey, J. (1933). How we think. Boston, MA: D.C. Heath and Co.Dohn, N. (2009). Web 2.0: Inherent tensions and evident challenges for education.

Computer Supported Collaborative Learning, 4, 343–363.Etkina, E., Karelina, A., Ruibal-Villasenor, M., Jordan, R., Rosengrant, D., & Hmelo-

Silver, C. (2010). Design and reflection help students develop scientific abilities:Learning in introductory physics laboratories. Journal of the Learning Sciences,19(1), 54–98.

Halpern, D. F. (1998). Teaching critical thinking for transfer across domains.Dispositions, skills, structure training, and metacognitive monitoring. AmericanPsychologist, 53, 449–455.

Hamann, K., & Wilson, B. M. (2003). Beyond search engines: Enhancing activelearning using the internet. Politics & Policy, 31, 533–553.

Huang, Y. M., Lin, Y. T., & Cheng, S. C. (2009). An adaptive testing system forsupporting versatile educational assessment. Computers & Education, 52(1),53–67.

Hughes, A. (2009). Higher education in a Web 2.0 world. JISC Report. <http://www.jisc.ac.uk/media/documents/publications/heweb20rptv1.pdf>. Retrieved22.05.12.

Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learningapproach to improving the learning attitudes and achievements of students.Computers & Education, 56(1), 1023–1031.

Hwang, G. J., Wu, P. H., & Chen, C. C. (2012). An online game approach for improvingstudents’ learning performance in web-based problem-solving activities.Computers & Education, 59(4), 1246–1256.

Jou, M., & Wang, J. J. (2013). Observations of achievement and motivation in usingcloud computing driven CAD: Comparison of college students with high schooland vocational high school backgrounds. Computers in Human Behavior, 29(2),364–369.

Jou, M., Chuang, C. P., & Wu, Y. S. (2010). Creating interactive web-basedenvironments to scaffold creative reasoning and meaningful learning: Fromphysics to products. The Turkish Online Journal of Educational Technology, 9(4),49–58.

Jou, M., & Shiau, J. K. (2012). The development of a web-based self-reflectivelearning system for technological education. The Turkish Online Journal ofEducational Technology, 11(1), 165–171.

Kemmis, S. (1985). Action research and the politics of reflection. In D. Bound, R.Keough, & D. Walker (Eds.), Reflection: Turning experience into learning. London:Kogan Page.

Kennedy, M., Fisher, M. B., & Ennis, R. H. (1991). Critical thinking: literature reviewand needed research. In L. Idol, & B. Fly Jones (Eds.), Educational values andcognitive instruction: Implications for reform (pp. 11–40). Hillsdale, NJ: LawrenceErlbaum.

King, A. (1994). Guiding knowledge construction in the classroom effect of teachingchildren how to question and how to explain. American Educational ResearchJournal, 31(2), 338–368.

Kong, S. C., & So, W. M. W. (2008). A study of building a resource-based learningenvironment with the inquiry learning approach: Knowledge of family trees.Computers & Education, 50(1), 37–60.

Ladewski, B. G., Krajcik, J. S., & Palincsar, A. S. (2007). Exploring the role of inquiry andreflection in shared sense-making in an inquiry-based science classroom: Toward atheory of distributed shared sense-making. New Orleans: Paper presented at thenational association of research in science teaching.

Lamy, M. N., & Goodfellow, R. (1999). Reflective conversation’’ in the virtuallanguage classroom. Language Learning & Technology, 2(2), 43–61.

Lee, C. Y., & Chen, M. P. (2009). A computer game as a context for non-routinemathematical problem solving: The effects of type of question prompt and levelof prior knowledge. Computers & Education, 52, 530–542.

Lin, Y. T., & Jou, M. (2012). A web application supported learning environmentenhancing classroom teaching and learning experiences. In 12th InternationalEducational Technology Conference (IETC 2012). Taoyuan, Taiwan.

Lin, Y. T., Tan, Q., Kinshuk, & Huang, Y. M. (2010). Location-based and knowledge-oriented microblogging for mobile learning – Framework, architecture, andsystem. In Proceedings of the sixth IEEE International Conference on Wireless,Mobile and Ubiquitous Technologies in Education (WMUTE 2010), Kaohsiung,Taiwan.

Lin, G., Fu, D., Zhu, J., & Dasmalchi, G. (2009). Cloud computing: It as a service. IEEEIT Professional, 12(5), 36–41.

Lin, Y. C., Lin, Y. T., & Huang, Y. M. (2011). Development of a diagnostic system usinga testing-based approach for strengthening student prior knowledge. Computers& Education, 57(2), 1557–1570.

Marston, S., Li, Z., Bandyopadhyay, S., Zhang, J., & Ghalsasi, A. (2011). Cloudcomputing – The business perspective. Decision Support Systems, 51(1),176–189.

McNamara, D. S. (2004). SERT: Self-explanation reading training. DiscourseProcesses, 38(1), 1–30.

McNamara, D. S., O’Reilly, T. P., Best, R. M., & Ozuru, Y. (2006). Improving adolescentstudent’ reading comprehension with iSTART. Journal of Educational ComputingResearch, 34(2), 147–171.

Moon, J. A. (2004). Reflection in learning and professional development. New York:Routledge.

Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education,59(3), 1065–1078.

Paris, S. G., & Ayres, L. R. (1994). Becoming reflective students and teaching. AmericanPsychological Association.

Paul, K., Chen, K. N., & Gloria, L. (2010). When cloud computing meets withSemantic Web: A new design for e-portfolio systems in the social media era.British Journal of Educational Technology, 41(6), 1018–1028.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learningcomponents of classroom academic performance. Journal of EducationalPsychology, 82(1), 33–40.

Pretlow, C., & Jayroe, T. (2010). Training in the clouds. Computers in Libraries, 30(4),18–23.

Redfield, D. L., & Rousseau, E. W. (1981). A meta-analysis of experimental researchon teacher questioning behavior. Review of Educational Research, 51(2),237–245.

Roscoe, R. D., & Chi, M. T. H. (2008). Tutor learning: The role of explaining andresponding to questions. Instructional Science, 36, 321–350.

Sang, C. P., & Sung, Y. R. (2013). An empirical investigation of end-users’ switchingtoward cloud computing: A two factor theory perspective. Computers in HumanBehavior, 29(1), 160–170.

Schneckenberg, D., Ehlers, U., & Adelsberger, H. (2011). Web 2.0 and competence-oriented design of learning—Potentials and implications for higher education.British Journal of Educational Technology, 42(5), 747–762.

Schon, D. (1987). Educating the reflective practitioner. San Francisco: Josse-Bass.Thompson, J. (2007). Is education 1.0 ready for web 2.0 students? Innovate, 3(4).van den Boom, G., Paas, F., & van Merriënboer, J. G. (2007). Effects of elicited

reflections combined with tutor or peer feedback on self-regulated learning andlearning outcomes. Learning & Instruction, 17(5), 532–548.

Venters, W., & Whitley, E. A. (2012). A critical review of cloud computing:Researching desires and realities. Journal of Information Technology, 27(3),179–197.

Wang, Q., Woo, H. L., Quek, C. L., Yang, Y., & Liu, M. (2012). Using the Facebook groupas a learning management system: An exploratory study. British Journal ofEducational Technology, 43(3), 428–438.

Ward, J. R., & McCotter, S. S. (2004). Reflection as a visible outcome for preserviceteachers. Teaching and Teacher Education, 20(3), 243–257.

Yang, Y. F. (2010). Students’ reflection on online self-correction and peer review toimprove writing. Computers & Education, 55(3), 1202–1210.