FACULTY OF ELECTRICAL ENGINEERING UNIVERSITI TEKNIKAL MALAYSIA MELAKA

FINAL YEAR PROJECT

REPORT 2 (FYP 2)

ANALYSIS AND DEVELOPMENT OF A CONTROL STRATEGY FOR ROBOTIC WHEELCHAIR CONTROLLED USING SINGLE CHANNEL EEG HEADSET

Mohamad Amirul Bin Ariffin @ Aripin

Bachelor of Electrical Engineering

(Control, Instrumentation & Automation)

June 2015

“I hereby declare that I have read through this report entitle “Analysis And

Development Of A Control Strategy For Robotic Wheelchair Controlled Using Single

Channel EEG Headset” and found that it has comply the partial fulfillment for

awarding the degree of Bachelor of Electrical Engineering (Control,

Instrumentation and Automation)”

Signature :

Supervisor name :

Date :

a

…………………………………….

Tarmizi Bin Ahmad Izzuddin

…………………………………….

24/6/2015

…………………………………….

ANALYSIS AND DEVELOPMENT OF A CONTROL STRATEGY FOR ROBOTIC

WHEELCHAIR CONTROLLED USING SINGLE CHANNEL EEG HEADSET

MOHAMAD AMIRUL BIN ARIFFIN @ ARIPIN

A report submitted in partial fulfillment of the requirements for the degree of

Bachelor in Electrical Engineering (Control, Instrumentation & Automation)

Faculty of Electrical Engineering

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

2015

I declare that this report entitle “Analysis And Development Of A Control Strategy For Robotic

Wheelchair Controlled Using Single Channel Eeg Headset” is the outcome of my own study

except as cited in references. The report has not been accepted for any degree is not

concurrently submitted in application of ano ther degree.

Signature :

Name :

Date :

a

…………………………………….

Mohamad Amirul Bin Ariffin @ Aripin

…………………………………….

24/6/2015

…………………………………….

i

ACKNOWLEDGEMENT

Alhamdulilah, I am grateful to Allah s.w.t. for the good health and wellbeing that

were necessary for me to complete this thesis.

First of all I would like to express my grateful to Mr. Tarmizi Bin Ahmad Izzuddin for

guiding me in this study. I would also give my gratefulness, especially to my family for their

assistance, advices and inspiration. Without their continued support and interest, this project

would not have been same as presented here. Then, I would also like to express my appreciation

to Ahmad Suhaizan Bin Pandak and fa.mily, Hafiy Syazwan bin Zainoddin and his Master’s

research colleagues and of course to the entire parties that is involved in this project. They

have given their full commitment without asking for any rewards.

Besides that, I would like to compliment my special thanks to Universiti Teknikal

Malaysia Melaka (UTeM) especially to Fakulti Kejuruteraan Elektrik (FKE) and my fellow

friends under the same laboratory which is Rehabilitation Engineering and Assistive Technology

(REAT) lab for giving me an opportunity to apply the engineering understanding and develop

skills of electrical field in this project.

ii

ABSTRACT

Nowadays, with modern days of new generation many improvement and new innovative

of machine, system and devices have been made. These developments also take account of in

refining in quality life of people especially in medical. Biomedical signal lately have been a hot

topic for researchers, as many journals and books related to it have been publish. In this paper,

the control strategy to help damaged motor patient using BCI on basis of EEG signal was used.

BCI is a technology that obtain user thought to control a machine or device. This technology has

regained ability for quadriplegic or in other words a person that lost capability of his four limbs

to move by himself again. Within the past years, many researchers have come out with a new

method and investigation to develop a machine that can fulfill the objective for quadriplegic

patient to move again. Besides that, due to the development of bio-medical and healthcare

application, there are several ways that can be used to extract signal from brain. One of them is

by using Electroencephalography (EEG). This research is carried out in order to detect the brain

signal to controlling the movement of the wheelchair. A group of 5 healthy people will be chosen

in order to determine performances of the machine during dynamic focusing activity are to focus

on a stimulus. From the result that been collected during experiment, neural network

configurations will be implemented to classify the signal. Data collected will be extracted and

will be used to set a threshold for the machine to active. As a conclusion, a good neural network

configuration and a decent method of extracting EEG signal will lead to give a command to

control robotic wheelchair.

iii

ABSTRAK

Generasi moden hari ini telah melakukan pelbagai inovasi dan penambahbaikan terhadap

mesin, sistem dan peralatan. Transformasi ini termasuklah penambahbaikan kepada kualiti dalam

pelbagai bidang terutamanya bidang perubatan. Isyarat bio-perubatan muktahir ini telah menjadi

topik hangat dalam kalangan penyelidik. Ini kerana terdapat banyak jurnal dan buku yang

berkaitan dengannya telah diterbitkan. Projek yang dihasilkan ini, strategi kawalan akan

membantu pesakit yang lumpuh dengan menggunakan Otak-Komputer Penghubung (BCI)

berpandukan isyarat EEG yang akan dikumpul. BCI adalah teknologi yang digunapakai untuk

mendapatkan signal (isyarat) daripada otak untuk mengendalikan mesin atau peranti. Teknologi

ini akan mengembalikan kemampuan pesakit untuk bergerak sendiri. Dalam tahun kebelakangan

ini, ramai penyelidik telah menghasilkan kaedah baru untuk membangunkan mesin yang dapat

memenuhi objektif untuk pesakit lumpuh bergerak semula. Melalui pembangunan aplikasi bio-

perubatan dan penjagaan kesihatan, terdapat beberapa kaedah boleh digunakan untuk mengambil

isyarat dari otak. Salah satunya melalui penggunaan Electroencephalography. Kajian ini

dilakukan untuk mengesan isyarat otak yang akan digunapakai bagi mengawal pergerakan kerusi

roda. Sekumpulan 5 orang yang sihat dipilih untuk menentukan keupayaan mesin. Aktiviti fokus

adalah aktiviti untuk memberi tumpuan kepada bahan rangsangan yang akan diberikan bertujuan

untuk menghasilkan isyarat ‘tumpuan’ yang tinggi. Data yang dikumpul akan dikeluarkan dan

digunapakai untuk menetapkan had keaktifan mesin. Sebagai kesimpulan, konfigurasi rangkaian

neural yang baik dan kaedah yang cekap untuk mendapatkan isyarat EEG akan menghasilkan

kawalan yang baik untuk menggerakkan kerusi roda robot.

iv

TABLE OF CONTENTS

CHAPTER TITTLE PAGE

ACKNOWLEDGEMENT i

ABSTRACT ii

ABSTRAK iii

TABLE OF CONTENTS iv

LIST OF FIGURES vii

LIST OF TABLES x

LIST OF APPENDICES xi

LIST OF ABBREVIATION xii

LIST OF SYMBOL xiii

1 INTRODUCTION 1

1.1 Research Background 1

1.2 Motivation 2

1.3 Problem Statement 3

1.4 Objectives 3

1.5 Scope of Work 4

1.6 Expected Results 4

1.7 Report Structure 5

v

2 LITERATURE REVIEW 6

2.1 Theory and Basic Principles 6

2.1.1 Introduction 6

2.1.2 Robotic Wheelchair 7

2.1.3 Electroencephalography 8

2.1.4 Brain-Computer Interfaces 9

2.2 Review of Related Study 10

2.3 Summary and Discussion of the Review 12

3 RESEARCH METHODOLOGY 15

3.1 Summary of the Methods Used In Related Works 15

3.2 Methodology of Collecting Data 18

3.2.1 Control Strategy 19

3.2.2 Neurosky Mindwave Headset 22

3.2.3 Scalp preparation 24

3.2.4 Placement of electrodes 25

3.2.5 Before Experiment 26

3.2.6 Experimental Procedure 27

3.2.7 After Experiment 32

3.2.8 Type of Analysis 32

3.3 Project Gantt chart 33

vi

4 RESULTS 34

4.1 EEG Analysis 34

4.2 Features Extraction 37

4.3 Neural Network Analysis 38

4.4 Feed Forward Back Propagation with Scaled

Gradient Conjugate Training Function 38

4.5 Pattern Recognition and Classification Result 40

4.5.1 Matrix Confusion 41

4.5.2 Mean Square Error (MSE) & Percent Error 44

4.5.3 Receiver Operating Characteristics (ROC) 47

5 CONCLUSION 50

5.1 Conclusion 50

5.2 Recommendation 52

REFERENCES 53

APPENDICES 57

vii

LIST OF FIGURES

FIGURE TITLE PAGE

2.1 Wheelchair system setup 7

2.2 Cross section of the cerebrum through the sensory motor region 8

with the motor and sensory sequence indicated

2.3 Basic structure of BCI 9

2.4 A person wearing an EEG-cap and navigating the Bremen

Autonomous Wheelchair Rolland. 10

3.1 Flow of the project 18

3.2 Overall data process 19

3.3 An interval at certain time is taken on Normalized EEG signal 19

3.4 Scaled view of normalized data of 1 second 20

3.5 Fast-Fourier Transform was carried out at the selected time

Interval 20

3.6 Diagram of neural network configuration 21

3.7 NeuroSky Mindwave headset 22

3.8 Alcohol swabs 24

3.9 Neurosky Mindwave headset placement on head 25

3.10 (a) Track was setup for the experiment 27

3.10 (b) Top-view of the track 27

viii

FIGURE TITLE PAGE

3.11 (a) Wheelchair joystick to control the movement 29

3.11 (b) Speed set to level 3 29

3.12 (a) Unwanted data recording 30

3.12 (b) Good data recording 30

3.13 Overall experiment setup 31

4.1 Normalized EEG signal 34

4.2 (a) Frequency spectrum of Normalized EEG signal

(Delta, Theta & Alpha) 36

4.2 (b) Frequency spectrum of Normalized EEG signal

(Beta & Gamma) 36

4.3 Illustration of the ranges of EEG signal ranged from

0.1-80 Hz 37

4.4 Matrix confusion result from Pattern Recognition tools 42

in MALTAB software

4.5 Graph of Matrix Confusion Performance 43

4.6 Example of Mean Square Error graph at epoch 15 45

4.7 Graph of Mean Square Error versus No. of Hidden Neurons 46

4.8 Graph of Percentage Error Versus No. of Hidden Neurons 46

4.9 ROC curve of 50 hidden neurons 48

ix

FIGURE TITLE PAGE

4.10 ROC curve of 500 hidden neurons 48

4.11 ROC curve of 5000 hidden neurons 49

x

LIST OF TABLE

NO. TITLE PAGE

2.1 Review of previous related works 12

3.1 Review of the methodology of related works 15

3.2 Neurosky Mindwave headset specifications and hardware overview 23

3.3 Time for subject to move and stop the wheelchair 28

3.4 Gantt chart table 33

4.1 Features of EEG signal 37

4.2 Example of Input data arrangement from average of each wave 39

4.3 Example of output data arrangement 39

4.4 Confusion Matrix arrangements 41

4.5 Final result of Confusion Matrix 43

4.6 Final result of Mean Square (MSE) and Percentage Error 45

xi

LIST OF APPENDICES

APPENDICES TITLE PAGE

A Overview of the BCI where the user selects the destination 57

on the TV

B Correct position to place electrode i.e. on left frontal lobe

(Broca area) 57

xii

LIST OF ABBREVIATION

BCI - Brain Computer Interface

ECG - Electrocardiogram

EEG - Electroencephalography

EMG - Electromyography

FP - False Positive

FFBPNN - Feed Forward Back Propagation Neural Network

FFT - Fast-Fourier Transform

Hz - Hertz

LED - Light Emitting Diode

MATLAB - Matrix Laboratory

ms - Millisecond

MSE - Mean Square Error

REAT - Rehabilitation Engineering and Assistive Technology

ROC - Receiver Operating Characteristics

SSVEP - Steady- State Visual Evokes Potentials

TP - True Positive

UTeM - Universiti Teknikal Malaysia Melaka

uV - microvolt

xiii

LIST OF SYMBOL

α - Alpha

β - Beta

δ - Delta

γ - Gamma

θ - Theta

1

CHAPTER 1

INTRODUCTION

This section gives data in regards to basis of the study, issue explanations, and goals of

the study. This part additionally clarifies the extent of the study. The arrangement of how the

study will be led likewise introduced in this section.

1.1 Research Background

Currently, the use of bio-signals such as EMG , EEG and ECG to help the life of elderly

people and disable are rising. People who are severely immobilized, quadriplegics in

particular, may not be capable to easily control an electric wheelchair and are thus confined to

a push-chair, depend on external support [1]. However, researcher has developed tools to use

EEG as human command to control machines. At the same time, many ways have been

develop to help this kind of patient to recover their ability to move. One of the ways to regain

the ability is by using robotic wheelchair. This project introduces a robotic wheelchair

controlled by using single channel EEG headset to control the movement of stop and forward

and gaze tracking algorithm to control the direction of the wheelchair. It means the purpose of

this project is to develop an effective robotic wheelchair using single channel EEG headset

2

which can be used without being implanted in the user body. The significance of this research

had been supported by many academic publications. One of the academic papers [2] written

by Khalil Ullah and co.

1.2 Motivation

Nowadays, people suffer from quadriplegic are assumed to be ended their life alone on bed

without an ability to move by theirselves again. Recently, several companies have invented

equipment’s that can be used to obtained brain activity that can convert desired in thought into

actions in real life. EEG, ECG and EMG are some of the well-known technique to collect data

from patients. All of these equipment’s functions are wave and resonance based. Simple, user-

friendly, effective, and accurate are the reason these equipment’s is having a broad intensively

used in determining and analyzing of human illness. The numbers of people with the

quadriplegic difficulties are growing not only in Malaysia but all over the world. Based on that

fact, the mounting demand and awareness of people suffer of quadriplegic from a health care

provider are also increasing. Basically quadriplegic problem may due to several factors such

as accidents, ageing and other influences. This problem believes due to injury of neck that cost

all motor sensory to be impaired. In order to identify the brain signal, Neurosky Mindwave

headset was used to detect the electrical signal produced by the brain and from the collected

data [3]. After that, we can apply a neural network control technique to classify the person

intention that has a quadriplegic to move. This research carried out in order to gain the

knowledge about EEG and experience on how EEG data can be used in applications. Besides

that, this research also will identify the significant activities that will help quadriplegic patient

to regain moving ability by their own self and how to apply EEG signal for other purpose.

Although lack of knowledge and information, by doing research, it can motivate students and

other researchers to study and explore more problems related to any EEG signals.

3

1.3 Problem Statement

Quadriplegic are the patient that loss the control to move the human motor functions. They

suffer this disorder regularly due to neck injury that leads to limb paralysis. Paraplegic is an

illness that quite similar with quadriplegic but paraplegic only cost the patient to lost the

movement the organs on waist and below. While, quadriplegic lost the ability to control

organs starting from neck to below.

However, this patient have still have the ability of their upper organ such as hearing ears,

seeing with eyes and moving their mouth. This is includes brain that can control the whole

body activity. Brain activity also preserved for quadriplegic patients. Thus, detection of EEG

signal is possible. By applying BCI concept this signal can be obtained by placing a correct

spot of electrode on subject head. By using EEG signal that can be turned into command a

quadriplegic person can make their desired action into real life just by imagine in their heads.

By taking advantages of robotic wheelchair facility, quadriplegic patient can regain

their movement by their own selves with assistance of single channel EEG headset as a

medium between robotic wheelchair and controller on the wheelchair. Robotic wheelchair

basically consists of motor and controller, in certain cases includes sensor for safety of the

user.

1.4 Objectives

The main objectives for this project are:

i) To implement neural network control technique in order to move and stop robotic

wheelchair.

ii) To analyze and evaluate which is the best EEG signal pattern for moving and stopping

the wheelchair.

4

1.5 Scope Of Work

In order to achieve the objectives of this research, there are several scopes as listed. To

determine the EEG signal, there are several criteria need to be considered. Firstly, the

hardware that will be used as intermediate between quadriplegic intention and robotic

wheelchair is Neurosky Mindwave headset. This headset type use non-invasive BCI method to

obtain EEG signal. Apart from that, this headset also a single channel EEG headset. The EEG

signal will classify, validate and tested by random subject that will be selected. This project’s

subject selections are 5 healthy people. Selections of participant are chosen by random among

student in UTeM. This is because demanding and complex procedures are needed to get

participants that are quadriplegic person. Next, to identify the electrical signal yield by the

brain, subject will going to complete the experiments in a controlled surroundings. This is to

ensure no mental distractions, wandering thoughts, lack of focus, or anxiety that may lower

the performance of the machine.

1.6 Expected Results

Based, from the objectives some of the funding listed below are supposed to be

obtained at the end of the project.

1. Brain signal from Neurosky Mindwave Headset to be used for robotic wheelchair

command.

2. Neural network configuration that consist of brain signal as input and moving and

stopping the robotic wheelchair as output of the system.

3. A suitable method for subjects to focus in order to avoid subjects from suffering

fatigue.

5

1.7 Report Structure

This report comprises of five sections which are Chapter 1: Introduction, Chapter 2:

Literature Review, Chapter 3: Methodology, Chapter 4: Result and Chapter 5: Conclusion.

Chapter 1 is Introduction, which review about the undertaking study framework,

objectives, problem statement and extent of the study. Part 2 is Literature review which is

assessment the past undertaking and all material concept use in this study. Chapter 3 is about

Methodology where procedure of equipment and programming utilized in this project

procedure. Chapter 4 will consists of the discussion of findings of the experiment. Lastly

Chapter 5, the conclusion section which is the part that concludes all the results obtained in

Chapter 4 and recommendation for future research.

6

CHAPTER 2

LITERATURE REVIEW

This part displays the literature review of the analysis and improvement of a control

strategy for robotic wheelchair controlled using single channel EEG headset.

2.1 Theory And Basic Principles

2.1.1 Introduction

This chapter concentrated on the factual and also theoretical aspects of the project. It is

relating to the fundamental of controlling robotic wheelchair using Neurosky Mindwave

headset. In order to obtain complete and accurate information, in-depth research should be

done especially through reading articles and journals.

7

2.1.2 Robotic Wheelchair

Wheelchair is a four wheeled transportation used by people that unable to walk due to

accidents of illness. An external support often needed for those people that severely

immobilized, quadriplegics in particular, rely on help of others to move but researchers have

developed tools that translate the patient desired into command to control machine. Robotic

wheelchair is a wheelchair that moved by motor instead of an assist by someone or by the

patient own hands. A robotic wheelchair is usually are motorized that stimulated by the patient

intention. EEG signal was used to deliver the patient intended location to move. In other

words, this is an effort to utilize brain signal to control motorized devices. A personal

computer is attached to the back of the seat to plays the part as the intelligent controller for

this system [4]. Figure 2.1 shows the previous design of wheelchair setup [5].

Figure 2.1: Wheelchair system setup [5].