Developing Information Technology (IT) Fluency in College Students: An Investigation of Learning Environments and Learner Characteristics

Nancy B. Sardone

Georgian Court University Lakewood, NJ USA

nsardone@georgian.edu

Abstract: Using a causal-comparative research method, data from 120 undergraduate students studying computer concepts were analyzed to determine the relationship between learning environment, IT fluency, and course satisfaction. Results suggest that in learning environments based on active learning strategies, IT fluency was achieved and course satisfaction was significantly higher regardless of preferred learning style. This research can be used as conceptual model of how current college students prefer to learn IT to determine how undergraduate programs might change existing curricula to better prepare their students for the rapidly changing 21st century workplace. Keywords: Achievement, active learning, college students, constructivism, information technology fluency, learning environments

1. INTRODUCTION

The confluence of powerful technologies of computers and network connectivity has brought explosive growth to the field of IT; emerging with a focus on applied computing that includes an ability to adapt hardware and software to solve problems and process information in conjunction with accomplishing an organization’s goals. Organizational dependence on technology will demand all individuals be IT fluent upon workplace entry as the rapid deployment of technology continues to drive up the worldwide need for a skilled workforce [1-4]. Yet, in undergraduate classrooms, the teaching tends to focus on software proficiency, even though a more expansive definition of what it means to be IT fluent is needed. In addition, teaching using traditional instructional methods such as lecture is still the norm [5] despite the push for more hands-on, constructivist approaches that may engage learners regardless of their major or preferred learning style [6-9].

3. PROBLEM STATEMENT

The problem presented in this study is whether the type of learning environment where IT concepts are taught to undergraduates has a relationship to the development of IT fluency and course satisfaction. The literature suggests that if learning environments

based on constructivist learning strategies are used, IT fluency would be achieved while controlling for known predictors of academic achievement. These predictors are mathematical background, mathematical ability, cumulative grade point average, and learning styles [10-19]. As a conceptual model of how current college students prefer to learn IT, the environment in which learning occurred as well as instructional methods used was investigated and compared to determine how undergraduate programs might change existing curricula to prepare their students for the rapidly changing 21st century workplace.

2. PUROSE OF THE STUDY

The purpose of this research was to examine the relationship, if any, between traditional and constructivist learning environments to the development of IT fluency and course satisfaction in a course in which students were learning to become IT fluent under a revised definition. The revised definition of what it means to be IT fluent still encompasses software proficiency, but expands to include demonstrated knowledge of computer operations, networks, online resources, digital media, and programming. IT fluency includes content from the disciplines of computer science, computer engineering, electrical engineering, library science, business data management, and communications [20]. IT fluency, in its broader sense, is concerned with the skills, concepts, and intellectual capabilities related to data in terms of its representation, structure, organization, processing, transmission, distribution, and the technologies involved in the interactive execution of those activities-computers, networks, and software.

The study is among the few quantitative studies designed to analyze the factors influencing IT fluency in the general college undergraduate population. Although there are quantitative research studies in the field of IT-related education, they have mainly focused on use of specific visualization tools to teach algorithms and data networking and on the smaller population of computer science majors [21]. One qualitative study [22] focused on the general student population, teaching the IT concepts of input-output processing, interrupt, and BIOS, which are

similar concepts taught in the course investigated in this research study. However, that study focused more on learner perceptions of a specific instructional tool rather than on academic achievement in a given environment, which the current study intended to do.

4. THEORETICAL FRAMEWORK

Constructivism is a theory of how people learn that considers the engagement of learners in meaningful experiences. Constructivism is a recent development in cognitive psychology, influenced by the works of Bruner, Piaget and Vygotsky [23], which shifts learning from a passive transfer of information and collection of facts to active problem-solving and discovery. The type of environment that supports this learning theory is one where the instructor provides learning activities with which students formulate and test their ideas, draw conclusions and inferences, pool and convey knowledge collaboratively, and focuses on the central role that learners play in constructing knowledge [24].

The two learning environments studied were traditional and constructivist. Traditional learning environments tend to focus on the development of basic skills using didactic lecture, teacher presentations, and lecture/discussion methods [25]. This type of environment uses the transmission instruction model, based on a theory of learning that suggests students will learn facts, concepts, and understand material by absorbing the content of their teacher's explanations or by reading explanations from a textbook. Most often, lessons taught using the transmission instructional model are intended to direct the predetermined sequence of instruction, referred to as teacher-centered instruction [26].

Constructivist learning environments are interactive, collaborative, and explorative where instructors use approaches designed to stimulate thinking, motivate student involvement, and provide an opportunity to reflect on experiences [27-28]. Reflection is the willing intellectual activity of an individual to consider and explore their experiences in order to lead to a new understanding and appreciation of the fundamental nature, purpose, and essence of those experiences [29]. These learning environments are student-centered where the role and intention of the instructor is to design activities that facilitate student learning [30]. General Attributes of Learning Environments The general attributes associated with learning environments are context, construction, and collaboration. Contextual teaching strategies include instructional methods that serve as mental bridges for learning. The purpose is to model the intention of the instruction for students, thereby allowing them to

observe and reflect through the sharing of thoughts and ideas that provide for the consideration of alternate perspectives [31]. One contextual instructional method is simulation; descriptions of events or conditions and often allow the user to change variables to see the impact of that change. They include exercises, games, media, and computer animations that place learners in an artificially constructed, yet sufficiently realistic context for learning to occur [27, 30, 32]. Lecture is also a contextual instructional method and continues to be the most widely used teaching method in undergraduate classrooms [5]. Lecturing is an efficient means of instruction insofar as ability to deliver large quantities of information to large quantities of students. They are a useful method to help students read more effectively by providing an orientation to a concept’s or the author’s salient points.

Construction teaching strategies include instructional methods that serve to build knowledge through worked examples such as writing, discussing, and reflecting to include self-evaluation of progress toward conceptual understanding [30]. Collaboration teaching strategies include instructional methods that serve to develop negotiation skills by establishing peer groups [26]. When two or more peers work together on a learning activity, it eventually requires the arrival at a shared solution through social negotiation. This negotiation process can take place either on or offline, synchronously or asynchronously, without the instructor present.

The related literature provided factors known to contribute to academic achievement in IT-related courses. The predictors of academic achievement in IT-related courses emerged as math background (calculus and/or discrete math courses), math ability (SAT math score), cumulative grade point average, and learning styles. Two gaps existed in the constructivist learning literature related to IT education: the limited amount of quantitative research and the lack of examination of the impact that constructivist environments had on non-technical majors studying IT concepts.

5. METHOD

Participants Undergraduates at a mid-size university in the New York metropolitan area responded to the study survey in spring 2007. The 294 students, who previously completed a fundamental computer course received an invitation to participate via email. By the end of twelve weeks, 124 responses had been received, a 42 percent response rate. The appropriateness of the sample size was confirmed via an apriori power analysis for ANOVA using the software program

G*POWER, where a sample size of 120 was recommended [33]. Modes of Inquiry Data describing students’ experiences and scores were collected using four instruments: (i) Kolb Learning Styles Inventory; (ii) Evaluation of Teaching Effectiveness; (iii) Departmental Final Exam; and (iv) Self-Reported Learner Characteristics Questions. The first, Kolb’s Learning Styles Inventory, measured individual preferences toward learning on a twelve-item survey asking respondents to rank-order four sentence endings in a way that best described their learning style [28]. The Evaluation of Teaching Effectiveness Scale measured course satisfaction experienced by students and contained 28 items, each a seven-point response continuum representing agreement, ranging from strongly disagree to strongly agree [34]. The third, Departmental Final Exam, measured academic achievement of fundamental IT concepts [35]. Lastly, students responded to the Self-Reported Learner Characteristics Questions, reporting their mathematical background in terms of whether they completed a calculus and/or discrete math course, their mathematical score earned on the SAT exam, and their cumulative grade point average. Data Analysis Using causal-comparative research methodology, data from two non-randomized groups of 120 qualified undergraduate students (53 in the traditional environment and 67 in the constructivist environment) was used to explore the relationship between learning environment (one traditional and one taking a cognitive science approach), IT fluency, and course satisfaction as moderated by math background, grade point average, and/or learning styles. Additional data regarding instructor strategies, student perceived performance, workload, and methods of instruction were evaluated.

The statistical data techniques performed were analysis of covariance (ANCOVA), Independent t-Tests, and analysis of variance (ANOVA). Statistical analyses were performed using SPSS for Windows (v15), with a minimum alpha of .05.

6. RESULTS ANCOVA analysis confirmed the main premises of the study: (1) when learning environments based on active learning strategies are used, IT fluency is achieved regardless of an individual’s preferred learning style; and (2) when learning environments based on active learning strategies are used, course satisfaction is higher regardless of an individual’s preferred learning style. Students in the constructivist

environment had higher exam scores, although not statistically significant, than students in the traditional group. Further, students who studied in the constructivist environment were statistically more satisfied with the course than students who studied in the traditional environment.

Results of Independent Sample t-test comparing responses on the Evaluation of Teaching Effectiveness scale revealed statistical significance in the constructivist environment in the following dimensions: active learning, class organization, media use, student perceived performance, and workload. Findings revealed significance in promotion of class discussion (p < .000) based on challenging questions posed (p < .000) along with clearly defined course objectives (p = .013) and effective and interesting use of media (p = .002). Student perception of their performance (p = .031) in the constructivist environment was significantly higher than the traditional, further reporting that the assignments given in the constructivist were very challenging (p = .034). An evaluation of instructional strategies used in the two learning environments (Table 1) revealed that active learning methods of student presentations, simulations and game play, peer feedback, development of online portfolios, use of media resources, reflective writing exercises, class discussions, and group work were used with greater frequency in the constructivist environment (Table 2). These findings expose learning differences in the two environments and indicate how these differences, tied to instructional methods and materials used in college classrooms, can affect academic achievement.

7. SIGNIFICANCE OF STUDY

Both higher final exam scores and significantly higher levels of satisfaction in the constructivist environment are explained by the specific active learning techniques used in the constructivist environment, in which methods oriented toward enhancing student-centeredness and student-teacher interactions were favored. This is confirmed in the IT literature, where higher final exam scores and greater satisfaction in constructivist environments are associated with the perceived quality of the student-teacher interactions [36-40].

One interesting connection was found between students in the constructivist environment who believed that the course covered too much material (p = .008) and the assignments very challenging (p = .034) felt they learned a lot (p = .031) and were more satisfied with the course (p = .009) at a significantly higher level than students in the traditional environment. These finding indicate interest in learning IT concepts among students in the constructivist group although the students found the

course difficult. This may point to the perception that the content learned was relevant and worth the effort that students put forth. In addition, greater satisfaction in the constructivist environment albeit difficult may be the result of reflecting on their learning. There was a decisive gap between the time participants completed the course and the time when asked to participate in this study, as much as two years in some instances. The rationale for this time lapse is purposeful to reflective learning. Since the reflective instructional methods of discussion, persuasive writing assignments, and peer reviews were modeled for students in the constructivist environment, it may be that students taught in this environment were more skilled at thinking about their own learning and showed an increased complexity in their metacognitive skills when responding to the survey questions. Once given the opportunity to reflect, they realized how hard they worked in the course and the amount of work was meaningful to their current stage in life.

In addition, it is important to note that the type of assessment used to measure IT fluency (departmental final exam) was a format (single-best-answer, multiple-choice questions) in which converging and assimilating learners have a performance advantage (Kolb, 1984). Even when using such a format, students who learned IT concepts in the constructivist environment had higher IT fluency scores than those in the traditional environment, where the assimilating learning style was higher (M=.21; SD=.409) than in the constructivist environment (M=.18; SD=.386). Stealth learning is learning that happens so naturally that learners are not directly aware of its occurrence [41] and may help to explain this occurrence. Hands-on learning attracts all types of learners through deep immersion and engagement in activities and tasks. Simply put, students in the constructivist environment learned the same course content as students in the traditional environment as evidenced by same final exam administered to all students. They just took a different path that both attracted and interested them, one that immersed them in their learning.

The findings add to the ongoing discussion about the dynamic and powerful paradigm shift in the way students’ prefer to be taught. The results of recent large research studies, scholarly writings, and consumer behavior reports indicate that students like using technology tools for school tasks; consider themselves savvy and innovative users of technology; and prefer learning actively - defined as learning by doing through the use of interactive lessons, friendly competition, and trial and error. Students entering college today are a generation of consumers of

technology devices and software, who expect that they will be able to continue to use the tools of their youth in the college setting and beyond. Students in the constructivist environment used similar tools. Student-teacher interactions existed in the constructivist environment, which matched student preferences for course materials [media use; activities; discussions] and how they prefer to process them [interactively]. This explains the similarities in exam scores as students had numerous opportunities to ask questions, express ideas, and engage in challenging and open discussions in the constructivist environment.

Overall, study findings add to an understanding of higher education learning environments, student characteristics, and how IT fluency is achieved. The results of the study has implications for designing learning environments and using associated instructional methods that foster learning IT concepts in undergraduate programs. These results provide additional support to the constructivist learning theory and its execution in higher education classrooms where IT concepts are taught to non-technology majors.

8. REFERENCES

[1] A. Foster (2005). Student interest in computer science plummets. The Chronicle of Higher Education, May 27, 2005, 51(38), p. A31. [2] D. Francis (2007 July 2). The vanishing American computer programmer. The Christian Science Monitor. http://www.csmonitor.com/2007/0702/p15s01- wmgn.html [3] Klein, A. (2006). K-12 education shrinking future college graduate population in computer studies. Journal of Computing Sciences in Colleges, 21(4), 32-34. [4] U. S. Department of Labor. (1999). Futurework: Trends and challenges for work in the 21st century. A report of the U. S. Department of Labor, U. S. Government Respository. http://www.dol.gov/oasam/programs/history/herman/ reports/futurework/execsum.htm [5] FSSE. Faculty Survey of Student Engagement (2009).Indiana University. National Survey for Student Engagement. http://fsse.iub.edu/pdf/FSSE_IR_2009/FSSE09%20R eports%20%28Bac-A&S%29.pdf [6] J. Borrenson Caruso and G. (2007, September). EduCause study of undergraduate students and information technology (ECAR). http://www.educause.edu/ers0706 [7] P. Davis (1999). How undergraduates learn computer skills: Results of a survey and focus group. THE Journal, 26(9), 68-72.

[8] D. Oblinger (2003). Boomers, gen-xers and millennials: Understanding the new students. Educause, 38(4), 37-47. [9] SpeakUp. (2009). Creating our future: Students speak up about their vision for 21st century learning. National Findings on Speak Up 2009. http://www.tomorrow.org/speakup /pdfs/SU09NationalFindingsStudents&Parents.pdf [10] J. Bennedsen and M. Caspersen (2005). An investigation of potential success factors for an introductory model-driven programming course. Association of Computing Machinery, 2005 Proceedings of international workshop ICER '05, 155-163. [11]A. Chamillard and D. Karolick (1999). Using learning style data in an introductory computer science course. Association of Computing Machinery. Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science education, 31(1), 291-295. [12] A. Chamillard and R. Sward (2005). Exploring student learning: Learning styles across the curriculum. Association of Computing Machinery. Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education ITiCSE '05, 241-245. [13] J. Chenoweth (2005). Analyzing traditional factors for success in an introductory end-user technologies course. Information Technology, Learning, and Performance Journal, 22(3), 21-29. [14] A. Goold and R. Rimmer (2000). Factors affecting performance in first-year computing. Association of Computing Machinery. SIGCSE Bulletin, June 2000, 32(2), 39-43. [15] M. Hudak and D. Anderson (1990). Formal operations and learning style predict success in statistics and computer science courses. Teaching of Psychology, 17(4), 231-234. [16] S. Kruck and D. Lending (2003). Predicting academic performance in an introductory college- level IS course. Information Technology, Learning, and Performance Journal. 21(2), 9-15. [17] B. Pioro (2006). Introductory computer programming: gender, major, discrete mathematics, and calculus. Journal of Computing Sciences in Colleges, 21(5), 123-129. [18] L. Thomas, M. Ratcliffe, J. Woodsbury and E. Jarman (2002). Learning styles and performance in the introductory programming sequence. Association of Computing Machinery, Technical Symposium on Computer Science Education, Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education, 33-7. [19] B. Wilson and S. Shrock (2001). Contributing to success in an introductory computer science course: A study of twelve factors. Association of Computing

Machinery, SIGCSE Bulletin , Proceedings of the thirty-second SIGCSE Technical Symposium on Computer Science Education, 33(1), 184-188. [20] NRC: National Research Council. Committee on Information Technology Literacy. (1999). Being fluent with information technology. Computer Science and Telecommunications Board (CSTB) Washington, DC: National Academies Press. [21] T. Naps, G. Rößling, G., V. Almstrum, W. Dann, R. Fleischer, C. Hundhausen, A. Korhonen, L. Malmi, M. McNally, S. Rodger and J. Velázquez-Iturbide. (2002). ITiCSEworking group report: Exploring the role of visualization and engagement in computer science education. Association of Computing Machinery. SIGCSE Bulletin, 35(2), 131-152. [22] K. Li and K. Zhang. (2004). Student-centered, technology-rich learning tool to teaching and learning the internal computer operation concepts. Proceedings of the 2nd International Conference on Information technology for Application, pp. 409-414. [23] D. Kauchak and P. Eggen (2003). Learning and teaching: Research-based methods. Boston: Allyn and Bacon. [24] S. Smaldino, J. Russell, R. Heinich and M. Molenda (2005). Instructional technology and media for learning. Upper Saddle River: Prentice Hall. [25] NCREL. (1995). North Central Regional Educational Laboratory. Creating high-achieving learning environments. http://www.ncrel.org/sdrs/areas/issues/educatrs/leadrs hp/le400.htm [26] C. Maddux, D. Johnson, and J. Willis (2001). Educational computing: Learning with tomorrow’s technologies. Boston: Allyn & Bacon. [27] H. Foyle (Ed.) (1995). Interactive learning in the higher education classroom. Washington, DC: National Education Association. [28] D. Kolb (1984). Experiential learning: Experience as the sources of learning and development. Upper Saddle River: Prentice Hall. [29] C. Argyris and D. Schön (1974). Theory in practice. San Francisco: Jossey-Bass. [30] J. Bransford, A. Brown, and R. Cocking (Eds.). (2000). How people learn: brain, mind, experience, and school. Expanded ed.: National Research Council (U.S.) Committee on Developments in the Sciences, National Research Council (U.S.) Committee on Learning Research in Education. Washington, D.C.: National Academy Press. [31] J. Michael and H. Modell (2003). Active learning in secondary and college science classrooms. Mahwah, NJ: Lawrence Erlbaum. [32] G. Smith and E. Escott (2004). Using animations to support teaching of general computing concepts. Association of Computing Machinery, Proceedings of the Sixth Australian Computing

Education Conference 2004, Dunedin, New Zealand, 270-275. [33] F. Faul and E. Erdfelder (1992). G*POWER: Apriori, post-hoc, and compromise power analyses for MS-DOS [Computer Program]. Bonn, FRG: Bonn University, Dept. of Psychology. [34] M. Serva and M. Fuller (1999). The role of media use and active learning in higher education: the development of an instrument to determine the dimensions of teaching. Proceeding of the 20th International Conference on Information Systems, Atlanta, GA: Association for Information Systems, 386-399. [35] DSST: Dantes Subject Standardized Test. (2005). Thomson Prometric. http://www.getcollegecredit.com/index.html# [36] C. Colbeck, A. Cabrera and P. Terenzini, (2001). Learning professional confidence: linking teaching practices, students’ self-perception, and gender. The Review of Higher Education, 24(2), 173-191. [37] S. Demetriadis, E. Triantfillou, and A. Pombortsis (2003). A phenomenographic study of students' attitudes toward the use of multiple media for learning. Association of Computing Machinery. Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education, 35(3), 183-187. [38] G. Gonzales (2006). A systematic approach to active and cooperative learning in CS1 and its effects on CS2. Association of Computing Machinery, Technical Symposium on Computer Science Education, Proceedings of the 37th SIGCSE symposium on Computer Science Education, 133-7. [39] A. Lui, R. Kwan, M. Poon and Y. Cheung (2004). Saving weak programming students: applying constructivism in a first programming course. Association of Computing Machinery SIGCSE Bulletin, 36(2), 72-76. [40] K. Whittington (2004). Infusing active learning into introductory programming courses. Journal of Computing Sciences in Colleges, 19(5), 249-259. [41] R. Borja (2007). Video games trickle from rec rooms to classrooms. Education Week, 26(14), 10-11.

Table 1.Course Syllabi General Topic

Area Specific Topic Area

Computer Organization and Hardware

Processing Components, Primary Storage, Peripherals, Architectures, Data Representation

Systems Software

Operating Systems, Utilities, User Interfaces

Application Software

Word Processing, Desktop Publishing, Spreadsheets, Multimedia

Communications and Networks

World Wide Web, Personal Communications, Network Access, Network Architecture, Data Communications

History and Social Impact

History, Social Issues, Safety and Security, Careers

Table 2.Analysis of Strategies Used

Constructivist Traditional

Instructor 1 2 3 4

Contextual Strategies

Virtual Games & Simulations

X X

Lecture > 20 min

X X

Construction Strategies

Peer Feedback X X Online Portfolio Development

X X

Research Paper

X X

Class Discussions

X X

Reflective Journals

X X

Case Study X X Collaboration Strategies

Group Work X X Assessment Strategies Student Presentations

X X

Quizzes X 5 X 8 Final Exam X X X X