final year report fyp

THE DEVELOPMENT OF A PROGRAMMABLE INPUT EMULATOR

FOR PHYSIOLOGICAL SIGNALS

Edmond Mitchell

Degree Stream : Electronic EngineeringProject Supervisor : Dr Tomás Ward

A Final Year Project Report submitted in partial fulfilment of the requirements of the National

University of Ireland Maynooth for the degree of Batchelor of Engineering

April 2009

Department of Electronic EngineeringNational University of Ireland

Maynooth

Final Year Project Report

ABSTRACT

GlovePie is a software application that allows users to emulate inputs in Windows. However it lacks

the capacity to use physiological signals as inputs. This project remedied this by replicating

GlovePie for use with physiological signals. Physiological programmable input emulator (PhysPIE)

will promote the development of assistive technology systems as it facilitates mappings between

physiological signals and user input. Examples of possible input signals include Alpha waves

emitted from the brain, breathing patterns, kinetic signals from the Wii© Balance board or the

flexing of arm muscles. C# was chosen as the programming language that the vast majority of the

project would be written in. Common computer input controls were emulated such as the mouse and

keyboard for outputs for the users inputted physiological signals. This allows real-time acquisition

and processing of physiological signals to create events which can be mapped on to standard GUI

operating system user-input commands. A core component was the incorporation of a scripting

feature in the application which allows a user to easily select standard signal processing operations

from which to generate events.

document.doc ii Edmond Mitchell


TABLE OF CONTENTS

ABSTRACT…………………………………………….………….………………………………..ii

1 Introduction..................................................................................................................................1

1.1 Academic background..........................................................................................................1

1.1.1 GlovePie.......................................................................................................................1

2 Windows programming................................................................................................................2

2.1 Hooks....................................................................................................................................2

2.2 Events...................................................................................................................................2

2.3 Threading..............................................................................................................................2

2.4 Saving/loading......................................................................................................................3

3 C#.................................................................................................................................................3

3.1 C# programming...................................................................................................................3

3.2 C# debugging.......................................................................................................................3

3.3 Using C# to debug LUA.......................................................................................................3

3.4 C# Class’s. Ease of use (e.g sendinput)................................................................................3

4 Scripting.......................................................................................................................................4

4.1 LUA......................................................................................................................................4

4.2 Python etc.............................................................................................................................4

4.3 MatLab.................................................................................................................................4

4.4 Reflection.............................................................................................................................4

4.5 LUA functions......................................................................................................................4

5 Arduino..............................................................................................................4

5.1 Arduino programming...............................................................................4

5.2 Arduino circuits.........................................................................................4

5.3 Arduino no registers, delimiters.................................................................4

6 Serial Transmission............................................................................................4

6.1 In C#...........................................................................................................4

6.2 Buffering....................................................................................................4

6.3 Digital Vs Analog......................................................................................5

6.4 10 bit ADC.................................................................................................5

6.5 BAUD rate.................................................................................................5

7 DLLs..................................................................................................................6

7.1 DLL used...................................................................................................6

7.2 APIs............................................................................................................6

document.doc iii Edmond Mitchell


7.3 Zedgraph....................................................................................................6

7.4 Graphing –Dynamic.............................................................................................................6

8 Buffering......................................................................................................................................6

8.1 Circular Buffers....................................................................................................................6

9 Bandpass filter..............................................................................................................................7

10 Glovepie...................................................................................................................................7

11 Key ChallengEs........................................................................................................................7

11.1 Multiple Wii Devices...........................................................................................................7

11.2 Realtime Graph..........................................................................................7

11.3 Connecting to serial port............................................................................7

11.4 Storing Serial Data.....................................................................................7

11.5 Hooking Keys............................................................................................7

11.6.........................................................................................................................7

document.doc iv Edmond Mitchell


1 INTRODUCTION

1.1 Academic background

This project lies in the region of assistive technology. This is a broad term which includes assistive,

adaptive, and rehabilitative devices for people with disabilities. Assistive technology allows users to

gain more independence by letting them complete tasks that they were formally unable to or had

great difficulty accomplishing. Many prominent technologies have started of in this area and gone

on to more global use. Short Message Service (SMS) was initially designed for quick long range

communication for people with hearing impairments but has gone on to have a user base of over 2.5

billioni.

1.1.1 GlovePie

Glove Programmable Input Emulator (GlovePie) is a windows application that allows users to

emulate a variety of inputs using a multitude of devices. For instance it is possible to emulate a

mouse with the Nintendo Wii Controller [Figure 1]. This is done by writing scripts in GlovePie and

having GlovePie process them in real-time. Basic knowledge of programming is required to write

these scripts and demo scripts are widely available through out the internet ii. GlovePIE allows

scripts are to be saved and loaded. The current project aims to replicate the functionality of

GlovePIE for physiological signals. See Appendix A for an example of a GlovePie script.

Figure 1 – Use of Wiimote with a Laptop and GlovePie

document.doc 1 Edmond Mitchell


2 WINDOWS PROGRAMMING

2.1 Events

In DOS and earlier operating systems code was executed line after line in a predictable fashion via

function calls or until a list of tasks has been accomplished. This is not the case with modern

Graphical User Interface (GUI) operating systems. Windows is an event-driven operating system. In

C# an event is defined as a member that enables an object or class to provide notifications iii. In a

piece of software the change of monitored properties of an object may result in an action occurring.

An example of this would be the starting of a new process when a button is clicked. In PhysPIE an

event would occur when the start script button is clicked. The event that would occur is the

compiling and the running of the script

2.2 Software Development

For this project graphical elements in Windows needed to be controlled quickly and easily. To do

this in “C” it would take hundreds of lines of code. This would not be an efficient use of time since

all these functions have been written and complied in the .NET framework.

The .NET framework is a piece of software technology that is available with several

Microsoft Windows operating systems. It includes a large library of pre-coded solutions to common

programming problems. It also contains a virtual machine that manages the execution of programs

written specifically for the framework. The pre-coded solutions that form the framework's Base

Class Library cover a large range of programming needs in a number of areas, including user

interface, data access, database connectivity, cryptography, web application development, numeric

algorithms, and network communications. The class library is used by programmers, who combine

it with their own code to produce applications. Some languages that use this framework are Visual

basic, Visual C++, Visual C# and Visual Basic.

2.2.1 C# programming

C# was chosen as this projects primary programming language due to its object orientated

nature and its widespread use when using the .NET framework. Microsoft Visual C# 2008 Edition

was utilized to compile the C# code. This application is extremely responsive as it shows your

program compiled and updates this preview in real-time. It allows easy manipulation of graphical

user interfaces (GUI’s) by listing common features that can be added to a GUI in a Toolbox.


Edmond Mitchell, 20/03/09,

Insert picture of app


Examples of these features include buttons, checkboxes, labels and combo boxes. It

generates code for these features and links them to your own code.

2.2.2 C# debugging

Microsoft Visual C# 2008 edition debug features were used extensively through out the lifetime of

this project. Tools include breakpoints, watches and the output window.

Breakpoints deliberately stop execution at any statement of code. When C# encounters a

break point while executing code, execution is halted at the break statement, prior to it being

executed. Break points enable you to query or change the value of variables at a specific instance in

time, and they let you step through code execution one line at a time.

The Watch Window allows developers to watch certain expressions, in certain modules

and/or procedures, and then stop code execution when it changes. It also allows developers to view

variables current value when a breakpoint has been reached.

The Output window is used by C# to display various status messages and build errors. This

references what line the compiler encountered an error and the error type. This is exceptionally

helpful in correcting syntax errors.

2.2.3 C# Class’s. Ease of use (e.g sendinput)

2.2.4 Saving/loading

2.3 Threading

Threading in computer programming is where two or more instructions are executed concurrently.

This allows programs to perform multiple operations at once which speed up the program. The way

threads are implemented differs from one operating system to another but the concept is always the

same. A process, in the simplest terms, is an executing program. One or more threads run in the

context of the process. A thread is the basic unit to which the operating system allocates processor

time.

On a single processor multithreading is generally achieved by time division multiplexing.

This means that the processor switches between threads quickly enough that the user cannot

distinguish it. However on a multi-core processor different threads are actually run on different

cores at the same time. While processors may have these abilities, it is up to the operating system



and the program to take advantage of them. Certain higher level languages 1(for

example Pythoniv) may implement a software lock which restricts processes running threads on

separate cores. Windows runs threads by use of a scheduler. This scheduler runs

threads based on a priority queue. Windows runs a get message loop to continually check and

add threads to the schedulers queue. It also gives each thread a maximum runtime in order to

prevent deadlock2. In C# threading is relatively simple compared to some other

languages (E.g. C). It uses the libraries found in

Threading in C# enables you to perform concurrent processing so you can do more than one

operation at a time. For example, you can use threading to monitor input from the user, perform

background tasks, and handle simultaneous streams of input. The “System.Threading” namespace

provides classes and interfaces that support multithreaded programming and enable you to easily

perform tasks such as creating and starting new threads, synchronizing multiple threads, suspending

threads, and aborting threads.

To incorporate threading in C#, all that needs to be done is to create a function to be executed

outside the main thread and point a new Thread object at it. The following code example creates a

new thread in a C# application:

Thread t = new Thread(another_function); t.Start(); //t.Abort(); t.Resume(); t.Suspend();

This code will create a new thread that leads to a function called another function. To run this

thread all is needed is to run the Start() property. By using this method is easy to use this types

other properties such as Abort, Resume and Suspend.

I need threading in my program in order to make sure it can constantly monitor the feature

receiver while still carrying out events at the same time. In order for this program to succeed, the

program will need to be able to create new threads and abort successfully.

1 A high-level programming language is a programming language with strong abstraction from the details of the

computer2 Deadlock is a situation in which two processes sharing the same resource are effectively preventing each other from

accessing the resource, resulting in both programs ceasing to function


using System;using System.Threading;


2.4 Hooks

3 SCRIPTING

In this project in order to give the user the most possible functionality the use of an embedded

scripting language is required. A script language is a programming language that already has a

useful set of components written in other languages. They differ from system programming

language such as C, Fortran and C++ in the way they were designed. System programming

languages were designed for building data structures and algorithms from scratch. In contrast,

scripting languages are designed for gluing: they assume the existence of a set of powerful

components and are intended primarily for connecting components togetherv.

The advantages of scripting languagesvi

Rapid development: Due to the need for less code to be written development and debug

time is reduced.

Integration with existing technologies: They are built around component technologies like

.NET and Java.

Easy to learn and use: Only basic computer programming knowledge is required.

Dynamic code: Scripting language code can be generated and executed at runtime, an

advanced feature useful and necessary in some applications.

The disadvantages of scripting languages

They are not all-inclusive. They assume the presence of a system programming language.

They are not conducive to best practices in software engineering and code structuring, such

as object orientation and component-based development.

They are usually not “general purpose” but tuned toward a specific application, such as web

development (PHP).



3.1 Scripting in this project

Scripting is a main component in this project. It allows a user to easily select standard signal

processing operations from which to generate events Scripting lets the user manipulate inputs and

outputs more easily and with more capability. The Code below shows how a user can quickly check

if the incoming data exceeds a defined tolerance and performs a GUI action if required. For the

example below if the incoming data exceeds 500 then the mouse will move 5 pixels to the right.

tolerance = 500

dataA = GetChannelADataLUA()

if (dataA > tolerance)

then

MouseFunctionsLUA("MoveX",5)

end

This type of scripting will allow users to code their desired events without needing to write complex

functions to take control of GUI inputs.

3.2 LUA

3.3 Python etc

3.4 MatLab

3.5 Reflection

3.6 LUA functions

3.6.1 Using C# to debug LUA

LUA was integrated into PhysPIE by use of a dynamic link library (DLL). This DLL only included

a simple light weight compiler. This compiler lacks any debugging features. It merely throws an

exception when it encounters an error in the scripting code.



To debug LUA when it was initially embedded a c# function was used to

type the equivalent code and test that it performed identical to the LUA scripts. Eventually a limited

output window was added to LUA that referenced the line that it encountered an error at.

4 ARDUINO

4.1 Arduino programming

4.2 Arduino circuits

4.3 Arduino no registers, delimiters

5 SERIAL TRANSMISSION

5.1 In C#

5.2 Digital Vs Analog

5.3 10 bit ADC

5.4 BAUD rate



6 DLLS

6.1 DLL used

6.2 APIs

6.3 Zedgraph

6.4 Graphing –Dynamic

7 BUFFERING

In computing, a data buffer referrers to a selection of memory that is used to temporally hold data

while it is being transferred from two different locations. Typically it is needed when the rate of

flow of data differs between devices.

7.1 Double Buffer

There are two main types of buffers in software. The first is a double buffer. Double buffering

works by using two buffers. When one is being read the other is still being filled with data. Once

the first buffer has had all its data read the second buffer is full and is available to have its data read

while the original buffer is being refilled. This type of buffering is able to better combat the two

main problems, buffer under-run and buffer overflow when only a single buffer is used. Buffer

under-run occurs when data is being read in faster than data is being fed in. Buffer overflow

however is where the data is being fed in faster than is can be read.

7.2 Circular Buffers

i The UK’s definitive text related information source - http://www.text.it – LAST ACCESSED 18/12/08ii http://www.wiili.org/index.php/GlovePIE:GlovePIE_Scriptsiii The C# Programming Language By Anders Hejlsberg, Scott Wiltamuth, Peter Golde 2003iv Threading limitations - http://docs.python.org/library/thread.html LAST ACCESSED 18/12/08v Scripting - http://home.pacbell.net/ouster/scripting.html#925015 LAST ACCESSED 18/12/08viAdvantages of scripting -- http://www.builderau.com.au/program/java/soa/Java-trends-Scripting-languages/

0,339024620,320274773,00.htm LAST ACCESSED 18/12/08



The second main type of buffering is circular buffering [Figure 5]. Here a

fixed amount of buffers are placed in sequence. Visual the diagram shows that the buffer has no real

end. This type of buffer is more lends itself to data streams. In circular buffers data is added until

the buffer is full. Once the buffer is full the section with the oldest data is overwritten. In reality

circular buffers require three pointers, one to the actual buffer in memory, one to point to the start of

valid data and one to point to the end of valid data

Figure 2 – A ring showing, conceptually, a circular buffer vii

Buffering is required in this project due to the varying rates of possible inputs for the

program. Here the buffering uses the first in, first out (FIFO) method.

Buffering will need to be implemented in this project in case the rate of data flow changes

from the DAQ to feature receiver or from the feature receiver to the GUI.



8 BANDPASS FILTER

9 KEY CHALLENGES

9.1 Multiple Wii Devices

9.2 Realtime Graph

9.3 Connecting to serial port

9.4 Storing Serial Data

9.5 Hooking Keys

9.6 Embedding the scripting language

vii Circular buffer image - http://www.citengineering.com/LuaVIEW/dl_pics/circular_buffer.png LAST ACCESSED

18/12/08


final year report fyp

Documents

system programming language

output window

operating system

programming language

assistive technology

scripting language

4 graphing dynamic

debug lua