evaluation of program outcomes through the final

Journal of Engineering Science and Technology Special Issue on UKM Teaching and Learning Congress 2013, June (2015) 92 - 99 © School of Engineering, Taylor’s University

92

Peer-review under responsibility of Universiti Kebangsaan Malaysia

EVALUATION OF PROGRAM OUTCOMES THROUGH THE FINAL YEAR PROJECT

MASLI IRWAN ROSLI1,*, NURINA ANUAR

1,

SITI ROZAIMAH SHEIKH ABDULLAH1,2

, ABU BAKAR MOHAMAD1

1Department of Chemical and Process Engineering, 2Centre for Engineering Education Research, Faculty of Engineering and Built

Environment, Universiti Kebangsaan Malaysia, 43600 UKM Bangi,

Selangor Darul Ehsan, Malaysia

*Corresponding Author: [email protected]

Abstract

The Program Outcomes (PO) of the Department of Chemical and Process

Engineering (JKKP) is designed to ensure that the students enrolled in the

department attain the objectives set by the Program Educational Objectives

(PEO). Thus, evaluation was conducted on student achievement vis-à-vis the

PO program to assess the extent to which the PO was achieved and to identify opportunities for quality improvement. As the method adopted in this study, PO

achievement was evaluated through the student accomplishment in one of the

core courses and in a final course in JKKP, namely, the Final Year Project

(FYP) for the academic year (AY) 2012–2013. Students in both programs at

JKKP - the Chemical Engineering Program (KK) and the Biochemical

Engineering Program (KB) - were assessed for their performance in FYP. Their PO achievement was analyzed based on their acquired FYP performance. Both

programs generally achieved excellent POs, and higher PO achievements were

obtained mostly by students who had a higher cumulative grade point average

(CGPA). Opportunities of improving the learning process based on this PO

evaluation are proposed in this article to enhance the quality of learning in

future AYs and to ensure that the PEO of JKKP is achieved.

Keywords: Program outcomes; Final year project; Chemical engineering.

1. Introduction

An outcomes-based education (OBE) has been implemented in the Faculty of

Engineering and Built Environment (FKAB) of the Universiti Kebangsaan Malaysia

Evaluation of Program Outcomes through Final Year Project 93

Journal of Engineering Science and Technology Special Issue 1 6/2015

Abbreviations

AY Academic Year

CGPA Cumulative Grade Point Average

FKAB Faculty of Engineering and Built Environment

FYP Final Year Project

JKKP Department of Chemical and Process Engineering

KB Biochemical Engineering Program

KK Chemical Engineering Program

OBE Outcomes-Based Education

PEO Program Educational Objectives

PO Program Outcomes

since the academic year (AY) 2005–2006 to ensure that the process of teaching

and learning is effective [1]. The Department of Chemical and Process Engineering

(JKKP) has employed the OBE approaches, and the Program Outcomes (PO) has

therefore been designed for and applied in both programs of JKKP: the Chemical

Engineering Program (KK) and Biochemical Engineering Program (KB). PO refers

to a knowledge or skill that students should have mastered upon completing their

studies in these programs. The present study primarily aims to enable sustainable

quality improvement that can be implemented in both the KK and KB programs by

evaluating PO achievements. Abdullah et al. (2013) applied a planner of research

activities as a means of continuous quality improvement on the implementation of

the research project. Their findings showed positive effects based on the student

feedback [2]. As such, the present study was conducted to identify students’

achievements based on the designed POs. The weaknesses identified could be used

as an opportunity for improvement and to further reinforce the reputation of JKKP

in producing quality engineers.

Previous studies have evaluated PO achievement through several methods,

such as conducting surveys among selected graduates [3, 4] and assessing

particular POs through student presentations [5, 6]. Studies on PO achievement

have been conducted using more efficient online questionnaires among alumni

[7]. The evaluation of PO achievement has also been conducted in stages, such as

in surveys for students of integrated projects [8]. Evaluation has also been

conducted by assessing student performance in the final examination [9] and final

year projects (FYP) [10]. Table 1 shows a list of the POs implemented in JKKP

for the AY 2012–2013. The POs listed in Table 1 are evaluated in the present

study, except for POs 5, 6, and 10, which are irrelevant to the course outcomes of

FYP in JKKP.

2. Methodology

In this study, PO achievement was evaluated by acquiring the detailed marks

obtained by the students in their FYP. The marks of their FYP were distributed

based on the relevant PO of the FYP: POs 1, 2, 3, 4, 7, 8, 9, 11, and 12. The

evaluation forms utilized in the FYP were composed and mapped out to facilitate

the evaluation of PO achievement. All marks in the FYP—progress presentation,

viva, and thesis marks—were acquired for the PO evaluation. For the AY 2012–

94 M.I. Rosli et al.


2013, KK consists of 35 students, and KB of 22, thereby resulting in a total of 57

students who participated in the study.

Table 1. List of program outcomes in JKKP.

PO Description

PO1 To apply knowledge of mathematics, science, chemical/biochemical

engineering fundamentals and an engineering specialisation to the

solution of complex engineering problems.

PO2 To identify, formulate, research literature and analyse complex

chemical/biochemical engineering problems reaching substantiated

conclusions using first principles of mathematics, natural sciences

and engineering sciences.

PO3 To design solutions for complex chemical/biochemical engineering

problems and design systems, components or processes that meet

specified needs with appropriate consideration for public health and

safety, cultural, societal, and environmental considerations.

PO4 To conduct investigation into complex chemical/biochemical

engineering problems using research based knowledge and research

methods including design of experiments, analysis and interpretation

of data, and synthesis of information to provide valid conclusions.

PO5 To create, select and apply appropriate techniques, resources, and

modern engineering and IT tools, including prediction and

modelling, to complex chemical/biochemical engineering activities,

with an understanding of the limitations.

PO6 To apply reasoning informed by contextual knowledge to assess

societal, health, safety, legal and cultural issues and the consequent

responsibilities relevant to professional engineering practice.

PO7 To understand the impact of professional engineering solutions in

societal and environmental contexts and demonstrate knowledge of

and need for sustainable development.

PO8 To apply ethical principles and commit to professional ethics and

responsibilities and norms of engineering practice.

PO9 To communicate effectively on complex chemical/biochemical

engineering activities with the engineering community and with

society at large, such as being able to comprehend and write

effective reports and design documentation, make effective

presentations, and give and receive clear instructions.

PO10 To function effectively as an individual, and as a member or leader

in diverse teams and in multi-disciplinary settings

PO11 To recognise the need for, and have the preparation and ability to

engage in independent and life-long learning in the broadest context

of technological change.

PO12 Demonstrate knowledge and understanding of chemical/biochemical

engineering and management principles and apply these to one’s

own work, as a member and leader in a team, to manage projects and

in multidisciplinary environments.



This study compares the average performance of the KK program and that of the

KB program for each PO. This study also focuses on the relationship between the

PO achievements and the general performance of the students based on their

cumulative grade point average (CGPA). The CGPA performance of the students

was divided into three groups: Group 1 with a CGPA equal to or more than 3.50,

Group 2 with a CGPA of less than 3.50 and equal to or greater than 3.00, and Group

3 with a CGPA of less than 3.00. The achieved POs were plotted and compared

with the CGPA performance for the analysis of their comparison and relationship.

3. Result and Discussion

Figure 1 compares the performance of the students’ achieved PO in KK and that

in KB. The achieved POs for both programs are generally excellent given that the

rating exceeds 75%, which is equivalent to a grade of A– to A in the current FYP

grading system. The PO achievements in KK are also slightly better than those in

KB with a standard error of less than 1.7% (Fig. 1). However, the comparative

performance of the POs in both programs in the previous AY 2010–2011 shows

that KB students have a slightly higher achievement than the KK students [10].

Thus, the performance of the students varies and depends on the particular batch.

The continuous evaluation of PO achievement is necessary for a better

understanding of the trends of PO achievement in the two programs.

In this study, the average PO achievement for KK and KB is 84% with a

standard deviation of less than 7.0 and 79% with a standard deviation of less than

7.7, respectively. The achieved PO for KK is approximately 83%–87%, whereas

that for KB is between 76%–82%. The PO achievements are consistent with the

results of the average CGPA of the students in both programs. The average CGPA

of KK students is 3.45, whereas that of KB students is 3:37, which is slightly

lower than that of the former. Thus, the PO achievement results are in line with

the average CGPA of the students in KK and KB.

The lowest PO achieved can be observed in PO 2 for both programs, and the

highest in PO 12 for both programs. The prevalence of this same pattern in both

programs demonstrates the need to improve the ability of students to accomplish

PO 2, namely, to identify, formulate, research literature and analyze complex

chemical/biochemical engineering problems to reach substantiated conclusions

based on the basic principles of mathematics, natural sciences, and engineering

sciences. In the FYP, PO 2 achievement is measured by evaluating the ability of

students to conduct a literature review, particularly in analyzing previous studies

related to research and providing comments and exchanging ideas on the literature

review. An improvement on the explanations provided to students for the methods

of conducting literature reviews can be proposed to the FYP coordinator. Such

measure will better prepare the students before they conduct a literature review.

For example, the coordinator can collaborate with the library in organizing a

workshop on how to conduct and search information in the library and journal

database. Such an activity will enhance the general performance of the students,

particularly in achieving PO 2. Enhancements must also be implemented to

increase the achievement of POs involved in FYP. Efforts should be undertaken

to bridge the gap between the KK and KB programs. For example, the evaluation

of PO can be improved by organizing and assessing a poster competition for the

FYP. This activity may also help the distribution of the PO assessment, to which a



percentage of marks is contributed by the poster competition. Furthermore, the

involvement of people from related industries who can support the examination of

the students’ performance may be beneficial in reducing the gap between the PO

achievement of the KK program and that of the KB program.

Fig. 1. Comparison of PO achievements between the program

of Chemical Engineering and Biochemical Engineering.

Figures 2 and 3 show the distribution of PO achievements based on the CGPA

groups. Students were grouped based on their CGPA results. The first group

consists of excellent students with a CGPA equal to or more than 3.50. The

second group consists of good students with a CGPA equal to or more than 3.00

and less than 3.50. The third group consists of intermediate students with a CGPA

of less than 3.00. Figure 1 shows the data distribution in KK of the first group

with a standard deviation of less than 7.1 and standard error of less than 1.8%, the

second group with a standard deviation of less than 8.2 and standard error of less

than 1.9%, and the third group with a standard deviation of less than 7.1 and

standard error of less 5.0%. Figure 2 shows the distribution in KB of the first

group with a standard deviation of less than 6.7 and standard error of less than

2.5%, the second group with a standard deviation of less than 7.3 and standard

error of less than 2.0%, and the third group with a standard deviation of less than

0.0 and standard error of 0.0. The zero values of the standard deviations and

errors in KB’s third group result from the total number of students in the group,

which is only one.

Overall, no clear pattern can be inferred from the findings on the students of

KK (Fig. 2). However, for the students of KK, achieving the best grade does not

necessarily mean exerting the same best performance in PO achievement. For

example, the highest achievement in POs 3 and 12 was obtained by students in

the second group (good students) and not by those in the first group (excellent

students). These results may be attributed to PO 3, which is generally related to

the students’ ability to design experimental methods and identify tools and

experimental equipment, whereas PO 12 is generally related to project planning



and management. The same pattern is observed among students of KB (Fig. 3),

indicating that the highest PO 12 achieved was obtained by the group of

intermediate students and not by the excellent and good students. Although at this

stage of the study the relationship between the achieved PO, particularly POs 3

and 12, and groups of good and intermediate students remains unclear, the present

results call for a further study on students’ PO achievements in the coming AYs

to clarify the relationship between PO achievement and CGPA performance.

Fig. 2. PO achievements based on CGPA groups in Chemical Engineering.

Fig. 3. PO achievements based on CGPA groups in Biochemical Engineering.

In addition, a comparison of the PO achievements of the different groups of

students in KK shows that PO achievements are approximately the same for both



groups of excellent and good students. Meanwhile, the PO achievements of

intermediate students are slightly lower than those of excellent and good students.

Therefore, no significant inconsistency in PO achievement is noted across the groups

of the students in KK. Meanwhile, among the students in KB, the PO achievements of

excellent students are slightly higher than those of good students. However,

significant differences are observed between the PO achievements the group of

excellent and good students and those of intermediate students. Hence, some

developments should be undertaken, particularly for the students in KB to minimize

the gap of PO achievements across groups.

Nonetheless, the PO achievements generally follow the CGPA performance of

the students: the highest PO achievements are attained by the group of students who

scores better CGPA. However, this observation requires further evaluations of PO in

future AYs.

4. Conclusions

This study found that the students of JKKP in both KK and KB programs achieved

excellent POs, exceeding the 75% mark. KK students obtained better PO than KB

students, which is consistent with their CGPA achievements. As expected, PO

achievements were associated with the students’ CGPA performance, with the

higher POs obtained by students with higher CGPA. However, POs 3 and 12

exhibited interesting patterns, in which higher PO achievements were acquired by

students who did not excel in terms of CGPA. Nevertheless, such patterns may be

too early and limited to generate conclusions. Hence, further studies that evaluate

the POs for the next AYs are necessary to derive solid conclusions. The

improvement of teaching and learning activities is vital to the enhancement of PO

achievements, particularly PO 2, which is indicated as the lowest. The enhancement

of teaching and learning may similarly reduce the inconsistency in the POs across

the groups of excellent, good, and intermediate students.

Acknowledgment

The authors wish to express appreciation to the Universiti Kebangsaan Malaysia for

supports and funding these research activities through PTS-2013-020 and PTS-

2014-034.

References

1. Nor, M.J.M.; Hamzah, N.; Basri, H.; and Badaruzzaman, W.H.W. (2006).

Pembelajaran berasaskan hasil: Prinsip dan cabaran. Pascasidang Seminar

Pengajaran dan Pembelajaran 2005, 54-62.

2. Othman, N.T.A.; Misnon, R.; Abdullah, S.R.S.; Kofli, N.T.; Kamarudin,

S.K.; and Mohamad, A.B. (2011). Assessment of programme outcomes

through exit survey of chemical/biochemical engineering students. Procedia

– Social and Behavioral Sciences, 18, 39-48.

3. Abdullah, S.R.S.; Anuar, N.; Rosli, M.I.; Kofli, N.T.; and Rahman, N.A.

(2013). Improvement in the implementation of undergraduate research

project. Procedia – Social and Behavioral Sciences, 102, 141-147.



4. Mutalib, A.A.; Maulud, K.N.A.; Yusuff, N.I.M.; Rahmat, R.A.A.; and Chik,

Z. (2012). The achievement of program outcomes from the perspective of

CSED, UKM final year students: Part II-Session 2006/2007. Procedia –

Social and Behavioral Sciences, 60, 350-357.

5. Ismail, M.; Yaakob, Z.; Markom, M.; Nordin, D.; Talib, M.Z.M.; Rosli, M.I.;

and Jahim, J.M. (2012). Tahap kemahiran penggunaan perisian

bermatlamatkan amalan kejuruteraan professional dan pembelajaran

sepanjang hayat pelajar tahun dua Sesi 2011/2012 secara ukuran PO6 dan

PO9 melalui pembentangan projek bersepadu. Prosiding Seminar Pendidikan

Kejuruteraan & Alam Bina, 1-8.

6. Ismail, M.; Yaakob, Z.; Markom, M.; Nordin, D.; Talib, M.Z.M.; Rosli, M.I.;

and Jahim, J.M. (2012). Tahap komunikasi dan kerjasaama berkumpulan

pelajar tahun dua Sesi 2011/2012 secara ukuran PO7 dan PO8 melalui

pembentangan projek bersepadu. Prosiding Seminar Pendidikan

Kejuruteraan & Alam Bina, 93-99.

7. Khamis, N.K.; Ihsan, A.K.A.M.; Abdullah, S.; Ramli, N.F.L.; and Sajuri, Z.

(2012). Soal selidik pencapaian objektif program dan hasil pembelajaran

program alumni Jabatan Kejuruteraan Mekanik dan Bahan melalui kaedah atas

talian. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, 148-156.

8. Kamarudin, S.K.; Abdullah, S.R.S.; Kofli, N.T.; Rahman, N.A.; Tasirin,

S.M.; Jahim, J.M.; and Rahman, R.A. (2012). Pengukuran hasil pembelajaran

program untuk projek bersepadu melalui soal selidik pelajar Jabatan

Kejuruteraan Kimia dan Proses Tahun III Sesi 2010/2011. Prosiding Seminar

Pendidikan Kejuruteraan & Alam Bina, 192-203.

9. Arshad, I.; Razali, S.F.M.; and Mohamad, Z.S. (2011). Pengukuran hasil

pembelajaran program melalui peperiksaan akhir bagi Program Kejuruteraan

Awam dan Struktur. Prosiding Seminar Pendidikan Kejuruteraan & Alam

Bina, 81-86.

10. Anuar, N.; Abdullah, S.R.S.; and Mohamad, A.B. (2012). The measurement

of program outcomes through research project in the Department of Chemical

and Process Engineering, UKM. Procedia – Social and Behavioral Sciences,

60, 124-129.

evaluation of program outcomes through the final

Documents