Voice Controlled Surgical Assistant

ECE 7995

Dupindar ghotra, Muhammad Syed, Sam li, Sophia lei

Outline

•Significance

•Quick recap

•Overview of system

•In-depth analysis

•Testing and Validation

•Limitations

•Competitive technology

•Future modifications and goals

•Timeline and budget

Significance

• Why is there a need for this kind of robot?

Machine does the repetitive tasks

Nurse for patient care

Efficiency

Cost• Any similar work?

Penelope - Columbia University

Tumor in forearm

Significance

• Leads to answer another question

Would this be accepted by medical people?

• Successful field study has already been performed

• What is the need for carrying on something that is already done?

Recap – Objectives

Objective1: Develop a robot as an independent assistant

Objective2: Voice integrated system;

delivers and retrieves tools

Objective3: Safe, efficient, accurate & cost-effective

Overview of the system

Working

Software

W1: Voice recognition

W2: Evaluation through pc

W3: Respective transmission to microprocessor

W4: Evaluation by microprocessor

W5: Final command to the robot

Hardware

Working and Challenges

• W1: Voice Recognition How is it being done? First thought – Build a voice recognition circuit

Instead - Microsoft speech SDK Cost and effectiveness

Libraries and tools to develop programs in Windows

Working and Challenges

• Problems related to recognizing voice

- Male / Female

- Pronunciation Can be reduced/eliminated by training the system more

- Loudness

microphones designed specifically for voice recognition

- Surrounding noise

Working and Challanges

Voce input taken. What do we do now?• W2: Evaluation through pc

Initial Approach –Have words typed in notepad Text file as input

Difficulty – Unable to find a method to have text file as input in real-time

Visual Basic

• W3: Respective transmission to microprocessor Validation of data being sent at this step

Working and Challanges

• W4&5: Evaluation of code received by microprocessor

Validation: storing the received data in a cleared memory space

Knowing what the expected Hex value to be received is

Comparing the stored values to expected numbers

Final Command sent to the robot’s arm

Microcontroller & Programming

• Freescale microcontroller chip (HCS12 family)

• serial communication interfaces (SCIs) channels (RS232)

• Forth programming languageflexibility and efficiency

• SwiftX is a interactive cross-compiler for embedded microprocessors and microcontrollers.

Working and Challanges

ASCIIVoice Chr Dec

Up 1 49 Down 2 50scissor a 97knife b 98

Pulse-Width Modulator (PWM)

Servo Direction & Operation Angle

: po.s 3500 PWMDTY2 ! 1700 PWMDTY0 !

2400 PWMDTY4 ! WAIT 1000 MS

4450 PWMDTY2 ! M.on

SCI1.Available 1 >= if \ has data in buffer 0 tr.sci1 $65 = if (S1-Key) drop

(S1-Key) CASE

53 OF po.1 ENDOF54 OF po.s ENDOF55 OF po.c ENDOF

ENDCASE

Robotic Arm

• Standard size servo motor– Rotate up to180 degrees– Offer a 417oz./in. (5v) of

torque

• 3 Degree of freedom

Aluminum Tubing

Electromagnet as a Gripper

• Using a LED to indicate the Electromagnet ON/OFF

Holding Value: 2 lbVolts: DC 12 Max.Watts: 1.4Amps: 0.12Net Weight: 2.1 oz.

Using a MOSFET as a Switch

Testing and Validation

• Rather simple method for testing and validating the system

• Different tools were asked for

• Validated by observing the right action being performed - right tool taken / put back

Timeline

Budget

Current Limitations

• Motors

Torque

Range of motion• Electromagnet

Weight limitation• Voice recognition

surrounding noise• Further testing and analyzing of system

various tools used in surgeries

Limitations and Questions Considered

• Open-loop system!!• No feedback taken from the motors• What if the tool drops because of some reason on its

way?

• What If someone hits the tray carrying tools and they

move from their position?

Limitations and Questions considered

• What if the tool used in a surgery is non-magnetic !#?

• How does the system know what tool is being placed back? Can it recognize it?

Future Goals - Improvements

• Objective1 – A thorough research on what different types

of tools (size, shape, weight) are used in surgeries

• Objective2 – Have continuous feedback to the system

Future Goals - Improvements

• Objective3 – A better gripper

• Objective4 – Integrate with image guided system

• Objective5 – A track of tools

what it started with &

what it is ending with

System Analysis after modifications

• System is getting a feedback letting it know whether tool

is still in gripper

• Be able to grab non-magnetic tools

• Would know what tools are available to it, where each

one is and what tool is being used at a certain time

New Timeline

May – July 2006

• Objective1: Research on different types of tools

• Objective2: continuous feedback to the system

• Objective3: better gripper

August – November 2006

• Objective4: Image guidance

• Objective5: Track of tools

Questions??