computer-assisted mathematics instruction for community college students

Int. J. Man-Machine Studies (1973) 5, 385-395

Computer-Assisted Mathematics Instruction for Community College Students

W. P. OLIVIER

The Ontario Institute for Studies in Education, Canada

(Received 24 January 1972)

The Individualization Project of the Ontario Institute for Studies in Education (0ISE) and several Colleges of Applied Arts and Technology (CAATS) jointly are developing a curriculum for upgrading mathematics skills. The project uses the computer facilities of the OISE and the National Research Council of Canada for on-line diagnosis and instruction of mathematics skills. The Computer- Assisted Learning system software was designed at the OISE and implemented with a view toward independence from any specific computer; therefore all system programs are coded in widely standardized languages. The authoring language, CAN-4 which is interpreted by the system programs, allows the curriculum designer and student a large degree of flexibility in the instructional process.

The mathematics curriculum material is designed as a set of generative modules keyed to behavioral objectives prerequisite for first year CAATS students. The project's goal is to provide the student with the most efficient means of instruction suited to his individual needs. Advanced assessment techniques requiring an on- line computer system provide valid decisions which can reduce the student's time required for mastery of the curriculum. Reports on the performance of each student and course are used by the project staff to improve the system and the teaching/learning process.

Introduction

I t is the ph i losophy of the Ind iv idua l iza t ion Project tha t the computer , wi th sophis t ica ted ins t ruct ional and testing techniques current ly available, can minimize the amoun t o f in fo rmat ion necessary for effective instruct ion. This min imal in format ion t ransfer thus increases the efficiency o f the learning process. The project repor ted herein gives indicat ions o f the benefits o f computer -ass is tance for a specific cur r icu lum area wi th techniques and facilities avai lable today.

The Individualization Project

The compute r is vi tal to the a p p r o a c h used in this project , because i t is the only genera l -purpose con t ro l mechanism which is capable o f the t remen- dous informat ion-process ing burden demanded by the indiv idual iza t ion o f

385

386 w.P. OLIVIER

instruction. Unfortunately, more is known about the designing of computer systems than about the ways of maximizing learning, retention and transfer, while minimizing the student's time spent learning. Research sub-projects, however, are using the computer-collected data to help answer thesel instructional-theoretic questions.

The Individualization Project has a past history of work in the CAI system design field and has designed CAI systems for small general-purpose timesharing computers and for dedicated multi-processor systems Capable of processing hundreds of students concurrently. The computer systems (Churchill, Naess & Olivier, 1969) have software for interpreting a special- purpose course authoring language called CAN- 4, registering students to use courses, checking language syntax of CAI programs and producing reports on student and curricular performance. Another component of the project assesses the suitability of available computer-student interface devices and designs such hardware when none is available commercially. Research projects are also being conducted on individual learning differences en- countered by the individualization of instruction. Last, but not least, a large- scale mathematics curriculum project is under development and showing extremely good results during the field trim phase.

The benefits of working in such an environment with the cross-fertilization of ideas is enormous. In almost all cases, the system software and hardware are modified to enhance the utility of the curriculum software (courseware), rather than the usual situation of restricting the courseware development.

The major focus of this paper is to report on the development of a CAI curriculum for community college mathematics in Ontario. This sub-project is currently the major focus of the Individualization Project; therefore, the other components will be mentioned only briefly, where they contribute to the mathematics sub-project.

Needs and Goals

Over several years and from several sources (Oliver, 1969; Crawford, 1968) it has been reported that a large proportion of students entering the community colleges are deficient in mathematical skills. A mathematics deficiency means that students lack the knowledge of mathematics necessary to pursue, effectively, their professional courses. The mathematical deficiencies are not limited to a few specific topics in mathematics, but cover a broad range and vary greatly from individual to individual. As a result of the demonstrated societal and institutional needs for a mathematics curriculum catering to individual differences, the OISE and several community colleges began development work with the following goals: (a) to fill the need for high-

COMPUTER-ASSISTED MATHEMATICS INSTRUCTION 387

quality, original, individualized deficiency diagnosis of mathematical skills at the community college level of instruction; (b) to provide individualized remedial instruction in these skills; and (c) to do so at a cost which is acceptable to the agencies providing the students with this instruction.

Dean's Policy Group review specify tasks

allocate resources

Project Committee

specify skills a uthor courses

r u I

I F review by

project committee

J release for field test after final

review

I report J completed tasks

Editorial Board review modify

approve

i -4- - , I t + I I

Instructional Programming Group code

I test debug

release for review by project committee

FIG. 1. Project structure.

Project Structure

Although the OISE initiated the project, it is the users, the community colleges, who determine the interim goals and means for attaining these goals. Figure 1 shows the project structure. The Deans' Policy Committee is responsible for committing college resources to the project. The committee is composed of one member from the OISE and one dean from each of the participating colleges. The Project Committee is composed of OISE staff,

388 w.P. OLIVIER

and teaching staff from the colleges associated with the project. This committee creates the mathematics skills specifications, designs tests to measure these skills and writes instructional sequences to teach the students. The Editorial Board members, consisting of one course author from each college and one representative from the OISE, review, modify, and eventually approve all work done by the Project Committee. The Instructional Pro- gramming Group of the OISE implements all approved materials on the computer facility.

\

• Sudbury N o r t L ~ ' ~ 1 BaY ~ QUEBEC

/ / .

Peterborough / / L / ~ - /'~.//" BellevilJe ~

T o r o ~ o ~ ' ~ ~ , . . . . t YORK

Buffalo • Rochester • London

FIG. 2. Location of co-operating agencies.

In June 1971, Canadore College at North Bay, and Algonquin College in Ottawa joined the OISE and Seneca College in the project. At this time a one-month workshop was held for the teaching masters of the community colleges in computer-assisted instructional philosophy, methods and implementation techniques. The course authoring language, CAN- 4 was introduced but only to show the teachers how the language implements their materials. The authors were not involved with the actual coding of materials. Staff from Fanshawe and George Brown Colleges also attended the seminars. Currently, instructional materials are being authored and used by the aforementioned participants with the addition of Sir Sandford Fleming, Conestoga, George


Brown and Loyalist Colleges. Two Toronto high schools are also currently using the curriculum and recommending modifications suited to their use. In addition to the preparatory mathematics curriculum which has been developed, curricula for first-year business and technology mathematics are under development.

Teaching masters, still carrying a part-time teaching load in their own school, were used, rather than recruiting courseware "experts". It was felt that too often CAI courseware is developed without involvement of the intended user. Externally imposed curricula more likely than not would soon fall into disuse. It must be remembered that a CAI curriculum is not like a textbook and cannot be skimmed easily to ascertain how the author(s) treat certain topics.

Figure 2 shows the geographic distribution of the currently participating agencies which comprise the project's effort. It can be seen that the project encompasses a triangular area from North Bay to Guelph to Ottawa.

Computer Facility

All of the colleges are connected over leased telephone lines to a small- scale, general-purpose timesharing computer which runs a line-concentrator program as one of its higher priority jobs. The line concentrator at the OISE is connected to the National Research Council's (NRC) large-scale timesharing computer by a single, high-speed, leased telephone line (McLean, 1972). Most courseware development is conducted on a medium-scale, general-purpose timesharing system at the OISE, and when the curriculum is debugged, the materials are transferred to the NRC's machine for field trials. The student performance, data recording and reporting facilities implemented on the NRC computer, assist in making further refinements to the curriculum during the field trials.

Computer as Curriculum Administrator Given an existing set of defined mathematical skills each student is pre-

tested on a subset of these skills to determine his deficiencies and strengths. Rather than follow traditional testing methods, three computer-dependent techniques were chosen.

COMPUTER-GENERATED ITEMS

Instead of storing a large set of specific test items related to a required mathematical skill, only one model problem is stored. This model problem is parameterized so that it can produce a random sample of test items, drills or examples, within the determined limits. Eac~ model problem also has the

390 w.P. OLIVIER

procedures for most common mistakes made by students. There is one model problem for each skill, and it is believed that by using the generative technique for creating specific items the computer storage requirements have been reduced greatly over previous techniques. Of course, computer genera- tion of items also relieves the instructor of this chore while expediting the task of coding and debugging items.

SEQUENTIAL TESTING

A second technique uses a sequential sampling procedure developed by Wald (1947) for industrial applications. The traditional decision procedure

o i .

to 2

f /

I 2 3 4 5 6 7 8 9 I0 Item number

FIG. 3. Illustration of individual performance differences.

draws a sample of a predetermined size and computes statistics on this total sample. Wald's procedure samples items at random until a dichotomous decision can be reached. This sequential procedure can reduce the required sample size by 50~o. The implication is that the same information as to whether a student has a skill or not can be obtained with fewer test items and time. The Wald's Sequential Probability Ratio Test is parameterized so that a student making frequent errors can take only two test questions, while a student making no errors may take a t least four test questions. Figure 3 shows graphically the operation of the sequential test model when the probability of decision errors (a =0.2, ~ =0.1) are fixed and the proportion of errors allowed for the decisions of mastery and non-mastery are P<0-15 and P > 0.52, respectively.

The vertical axis shows cumulative errors and the horizontal axis shows the total number of test items administered. The two sloping lines indicate the points above which a student demonstrates non-mastery on the test, and


the area below which the student passes. The area between the lines defines the "no decision" region. As a student takes more test items, his point is displaced to the right. Any errors also cause an upwards displacement. Since it is possible to stay in the "no decision" region by making occasional errors, the testing proCedure is truncated after a limited number of trials, and the "decision" region, which will result in a minimal decision error is taken.

The two facilities, generative techniques associated with the test items, and the sequential testing, would be difficult, if not impossible, without a computer. The only storage requirement is one sub-program with approximately 15 lines of coding, to control the sequential testing, and one sub-program of approximately 20 lines of coding for each model problem.

MATHEMATICS SKILL HIERARCHY

The third facet of the computer-dependent technique is hierarchical branching. Most mathematics is considered hierarchical in nature; that is, certain

TABLE 1 List of mathematics units

(1) Fundamental Operations with Signed Integers (2) Factoring of Integers (3) Equivalent Common Fractions and Reduction to Lowest Terms (4) Handling of Signs and Common Fractions (5) Complex Fractions (6) Fundamentals of Decimal Notation (7) Operations with Decimal Numbers (8) Percentages (9) Laws of Exponents

(10) Algebraic Substitution (11) Addition of Monomials (12) Multiplications with Monomials and Polynomials (13) Division with Monomials and Polynomials (14) Monomial Common Factors (15) Multiplication of Polynomials (16) Special Products (17) Factoring of Special Products (18) Simple Linear Equations in One Unknown (19) Ratio, Proportion and Variation (20) Systems of Linear Equations (21) Quadratic Equations (22) Scientific Notation (23) Significant Digits (24) Logarithms (25) Graphing

392 w.P . OLIVIER

skills are prerequisite to higher level skills. If a validated hierarchy is used as a testing structure then not all skills need be tested. For instance, if a student, reliably, can find the product of two factors, each composed of two or more digits, then it may be inferred that the student has the skills of "multiplication facts" and "columnwise" addition. Such inferences, if valid, let one bypass testing lower prerequisite skills if the higher level skills are passed. By using the hierarchical structure, a great saving in testing time can result. The combination of the hierarchical branching and sequential testing techniques greatly reduces the student's time involved in taking tests. This repre- sents a saving of more than 50 ~o over the next best currently available testing techniques.

Objective 6.1. Divide a fraction by a fraction. The question takes the form of the expression

a c

b "d ' w h e r e a=ke, b=k.-b l, d=k--1, e e [3, 5, 7, 11], and k e [4, 6, 8, 10].

FI6. 4. A sample skill specification.

In order to create empirically, and to validate subsequently a hierarchy of relations between mathematics skills, it was first necessary to test a large population of students in all the prerequisite skills. OISE staff members created a "paper-and-pencil" test designed to simulate the computer's administration of diagnostic test materials. This test was administered at Seneca College, Kenner Collegiate in Peterboro and Northern Secondary School in Toronto. The students involved in this testing were from grade levels 9-13, because a wide distribution of skill profiles was needed. Teaching masters also analysed the skill relationships to construct a hierarchy. The empirical and analytic data were thus cross-validating. Information received from this first cycle of testing and analysis was sufficient to construct and implement the first test hierarchy in September 1971.

The units which are included in the prerequisite mathematics are shown in Table 1. For example, Fig. 4 shows the skill specification for objective 6.1, which is one of the objectives of the complex fractions section. The student does not see the problem in the form shown in Fig. 4, but rather, he sees specific values for the variables a, b, c and d. The algebraic and set notation shown under the problem's format, are the rules used in generating the problems. The sets for the variables c and k are positive integers selected at random. The variables a, b and d are thus transforms of the random variables k and c.

C O M P U T E R - A S S I S T E D M A T H E M A T I C S I N S T R U C T I O N 393

t~

> Q to . _

~. ~ a ) ~ rr".~ ~ u.,

0

o

0

o e~

ud

394 w.P. OLIVIER

INSTRUCTIONAL STRATEGY

The instruction is, typically, a small amount of explanatory text followed by a few examples generated from the model problem for the skill. Following the examples the student takes a drill similar to the test, but now he is given feedback appropriate to his response. Comprehensive answer analysis in the model problems is invoked to see if his answer, if wrong, could have resulted from applying an anticipated incorrect procedure to the problem. If the student is making a common type of mistake he is made aware of this, and if it recurs he is taken to the objective lower in the hierarchy which:teaches this skill. The latter is a review procedure for students who need it. While taking the drill the sequential pattern of the student's response is continuously analysed by Wald's procedure to see if a mastery or non-mastery decision can be reached.

Figure 5 shows the method of sequencing a student through the entire curriculum. This figure indicates that a student does not receive instruction on a skill until he masters the prerequisite skills. The first level of instruction for each skill is designed to pass about 70 ~ of the students. More comprehensive instruction is given to the remaining 30 ~o of the students who do not achieve mastery on the first level of instruction. A small percentage of students not achieving through the computer-administered instruction are referred to their instructor for assistance. Each instructor has a report showing the specific skill wherein the student is having difficulty, and he can assist the student directly.

TABLE 2 Evaluation study by Seneca College of applied arts

and technology

Group

Nature of instruction Conventional CAI Date Fall, 1971 Fall, 1971 Student contact hours 40 6-28

• Number of students registered 41 27 Number of students who transferred

to another class 7 1 Number who passed 7 16

(17 ~) (59 ~) 10 9

incomplete* 17 1

Number who failed

Number who dropped out

*These students did not complete the material and continued into the spring of 1972.


Evaluation

A preliminary evaluation of the project was conducted internally by Seneca staff. Table 2 shows the tabulation of these results. These results, of course, should not be taken as definitive, nor do they, in any sense, "prove" the benefits of this type of approach. The results from Seneca should be taken only as a preliminary indication of the lower dropout rates, near absence of failures and drastically reduced time for the CAI group. Similar results were found at the other colleges.

Currently, the OISE is collecting performance data from all of the users and conducting a large-scale evaluation study.

References CHURCHILL, S., NAESS, L. ~ OL1VIER, W. P. (1971). CAN- 4 an advanced author

language for CAI, computer-based testing and psychological experimentation: PDP-9 implementation. Behaviour Research Methods and Instrumentation, 3(2), 95.

CRAWFORD, D. H. (Ed.) (1968). Ontario Mathematics Gazette, 7(1). McLEAN, R. S. (1972). A model of a centralized CAI system. Paper presented to

Canadian Symposium on Instructional Technology, University of Calgary, Calgary, Alberta, May 24-26.

OLIVER, G. L. (1969). Mathematics instruction. Part 21 A review and synthesis of needed curriculum research for the design of computer-assisted vocational instruction. Toronto: OISE, Department of Curriculum.

WALD, A. (1947). Sequential Analysis. New York: John Wiley & Sons.

computer-assisted mathematics instruction for community college students

Documents