autonomous targeting sentry(ats) group 12
TRANSCRIPT
Ethan King - Electrical Engineer
Daniel O’Hara – Computer Engineer
Stephen Rodriguez – Computer Engineer
James Van Gostein – Computer Engineer
Autonomous Targeting Sentry
GROUP
12
ATS
PROJECT OVERVIEW AND MOTIVATION
Prototype an autonomous turret to defend high priority areas
Detect and track targets using computer vision, determine threat and eliminate
To reduce the amount of human interaction in combat situations
OBJECTIVES
• Lightweight
• Cost efficient
• Capable of detecting, prioritizing and tracking targets autonomously
• Track targets moving at high speeds
• Manual control override
• Facial recognition
• Determine when a target is no longer a threat
• Alert target before firing
• Adjustable fire rate
SPECIFICATIONS
Description Constraint
Accuracy Rating > 70%
Number of Tracked Targets >= 3
Response Time < 1.50 seconds
Lateral Angle 140 degrees
Pitch Angle 90 degrees
Video Stream Resolution 1280 x 720 pixels
Weight < 30 lbs
Dimensions < 2’ x 2’ x 4’
Operation Time 3600 seconds
HARDWARE BLOCK DIAGRAM
Master
MCU
Yaw
Servo
Pitch
Servo
Trigger
Servo
Alarm
Cap
Touch
LEDs Laser
RS232
to I2C
Laptop
Servo
Controls
MICROCONTROLLER SELECTION
MSP430 Model G2231 G2203 F247
Operating Voltage 1.8 V – 3.6 V 1.8 V – 3.6 V 1.8 V – 3.6 V
GPIO Pins 14 16 48
Capture & Compares 2 6 10
Communication SPI, I2C SPI, I2C, UART SPI, I2C, UART
Dev Board Cost $4.30 $4.30 $75.00
Initialize
Bus
Data
MAIN MCU – MSP430G2231
Decode Data
State Flag
Transmit Pitch
Transmit Yaw
Set Trigger
Set Laser
Set Alarm
Transmit to PC
If Error
If Error
LED1 ON
LED2 ON
If Error LED3 ON
MAIN MCU – MSP430G2231
opCode (Hex) Function Method Description
0x00 to 0x8C setYawAngle(char opCode) Set yawAngle = opCode
0x90 to 0xEA setPitchAngle(char opCode) Set pitchAngle = opCode – 0x90
0xEC to 0xEF setTriggerMode(char opCode) Set trigger to off, auto, burst, single
0x8D retYawAngle(char yawAngle) Return yawAngle to PC
0xEB retPitchAngle(char PitchAngle) Return pitchAngle to PC
0xF0 to 0xF1 setAlarm(char opCode) Turn alarm off or on
0xF2 to 0xF3 setLaser(char opCode) Turn laser off or on
0xF4 to 0xF7 setLED(char opCode) Turn LEDs 1, 2, 3 off or on
0xFA to 0xFF reserved reserved
MAIN MCU – MSP430G2231
Return Type Function Description
void toBuffer(char data) Send data to the buffer
void encodeData(void) Pull data from the buffer and encode according
to specifications and push onto buffer
void decodeData(void) Pull data from the buffer and decode according
to specifications
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
void receive(void) Receive data from I2Cbus and push onto the
buffer
Initialize
CAP TOUCH MCU – MSP430G2452
Calibrate Cap
Read Cap
Set Flag
Touched Up
Touched Right
Touched Down
Touched Left
Touched Middle
pitchAngle++
yawAngle++
pitchAngle--
yawAngle--
Toggle Trigger
Encode
Buffer
Transmit
CAP TOUCH MCU – MSP430G2452
Return Type Function Description
void calibrateSensor(void) Measure capacitance of the air and store into
an array
char readTouch(void) Measures capacitance using TI’s API and return
value corresponding to up, right, etc
void toBuffer(char data) Send data to the buffer
void encodeData(void) Pull data from the buffer and encode according
to specifications and push onto buffer
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
SERVO MOTOR MCU – MSP430G2203
Initialize
Bus
Data
Determine Axis
Buffer
Apply PWM
Receive Angles
SERVO MOTOR FUNCTIONS
Return Type Function Description
Void findAxis(void) Determine the axis of received angle
void setPosition(void) Pull data from buffer and apply appropriate
PWM
void toBuffer(char data) Send data to the buffer
void receive(void) Receive data from I2Cbus and push onto the
buffer
SERVO MOTOR SPECIFICATIONS
Function Pitch Yaw Trigger
Model HS-805 HS-815 HS-81
Op. Voltage 4.80 V to 6.00 V 4.80 V to 6.00 V 4.80 V to 6.00 V
Torque 343 oz-in 343 oz-in 41.7 oz-in
Speed 0.14 S/60° 0.14 S/60° 0.09 S/60°
Current 830 mA 830 mA 280 mA
Dimensions 2.26” x 1.18” x 2.59” 2.26” x 1.18” x 2.59” 1.16” x 0.47” x 1.17”
Max Angle 90 degrees 140 degrees 90 degrees
Cost $39.99 $44.99 $17.99
RS232 I2C MCU – MSP430G2231
Initialize
Bus
Data
PC
Data
Transmit to Bus
Transmit to PC
RS232 I2C MCU – MSP430G2231
Return Type Function Description
void toBuffer(char data) Send data to the buffer
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
void receive(void) Receive data from I2Cbus and push onto the
buffer
void transmitPC(char
address)
Pull data from the buffer and transmit to another
to PC
void receivePC(void) Receive data from PC and push onto the buffer
AUDIBLE ALARM
Turbo series TMC-86-530-W
Mounting Panel Mounted
Operating mode Loud – Continuous
Operating Voltage 5-30 Vdc
Operating Frequency 2900 ± 250 Hz
Operating current 2mA at 5 Vdc
POWER SOURCE
12V/1.3Ah Sealed Lead Acid Battery
Model GH1213
Dimensions 3.80 “x 2.05” x 1.89”
Capacity 1300 mAh
Weight 1.3 lbs
Rechargeable Yes
Price $15.99
HARDWARE BLOCK DIAGRAM
Master
MCU
Alarm
3.3 Volt
Reg.
5.0 Volt
Reg
Cap
Touch
LEDs Laser
RS232
to I2C
Laptop
Power Supply
Yaw
Servo
Pitch
Servo
Trigger
Servo
Servo
Controls
PAINTBALL GUN
• AirTech E-Matrix electro pneumatic version.
• Adjustable max rate of fire through DIP switches located on PCB in the grip of gun.
• Nitrogen tank.
• Electronic Hopper.
BASE AND MOUNT
Sketch by Jon Waldmann
• 1280 x 720 pixels
• $30
• USB 2.0 port
• Automatic lighting adjustments
• Fixed position above the turret
LOGITECH
C310 HD WEBCAM
SOFTWARE FLOW DIAGRAM
Web Camera
cvtColor
goodFeaturesToTrack
cornerSubPix
calcOpticalFlowPyrLK
createTargets
setTargetThreat
trackTargets
chooseTarget
Fire
OPENCV
• Open source computer vision libraries.
• Using version 2.3.1 (newest released version)
• Maintained by Intel.
• Many useful functions for detecting and tracking objects and/or motion.
DETECTION AND TRACKING OPTIONS
• Iterative Lucas-Kanade Method with Pyramids
• Gunnar-Farneback
• Block Matching Method
• Horn-Schunck Algorithm
• Blob Tracking
• Color Tracking
• Facial Tracking
ITERATIVE LUCAS-KANADE METHOD
WITH PYRAMIDS
• Calculates optical flow between 2 images
• Given a set of points from a previous image, find those points in the new
image
• Assumes that neighbors of a pixel move in a similar fashion to that pixel itself
• Pyramids break into sub images. From there, the sub images are analyzed to
detect large motions
TARGET CREATION AND THREAT LEVEL
CALCULATION
• Keep a history of the movement of each pixel from the optical flow
calculation.
• Based on all similarly moving points in a neighboring area, create a bounding
box around each target.
• Place each target into an array of targets.
• Calculate threat level of each target based on size, velocity, and color.
• Sort target array based on threat level.
DECISION MAKING For no current target:
• Get target
• Check threat level
• Sound Warning
• Fire
For current target:
• Compare current and new target
• Choose highest priority
• If new target, warn
• Fire
MANUAL CONTROL
• Windows form application using Visual Studios 2010.
• The application will contain the video feed from the webcam and buttons to
perform the following actions by a user via on-board laptop.
• Rotate and fire the gun.
• Sound the alarm and play custom audio files.
• Rate of fire selection.
• Emergency stop button.
MANUAL CONTROL FUNCTIONS Return Type Function Name Method Description
Void rotateRight() Rotates the turret right by a set degree.
Void rotateLeft() Rotates the turret left by a set degree.
Void panUP() Tilts the turret upwards.
Void panDown() Tilts the turret downwards.
Void Fire() Fires the turret.
Void alertWarning()
Plays an audio file to warn intruders that they are
entering a dangerous zone and that they should
proceed to leave.
Void alertEngaged() Plays an audio file to warn intruders that they are about to be fired upon by ATS.
Void setFireRate(intfirerate) sets the firing rate of the turret 0 = automatic, 1 = burst, 2 = single shot.
MANUAL CONTROL – PROTOTYPE GUI
FACIAL RECOGNITION
• Security system to gain access to the manual controls of the turret.
• Scans a frame in an attempt to detect a human face using Haar-like features.
- Recognizable human facial features such as the dark areas under the eyes or cheek
color. Classifiers can also be trained to detect movement of the face such as
blinking, raising eyebrows, and flared nostrils.
• Once a face is detected the software will attempt to match the face to a face found in our
face database.
DIVISION OF LABOR
Daniel O’Hara
• Motion Tracking
• Threat Calculations
James Van Gostein
• Manual Control GUI
• Base and Mount
Ethan King
• Embedded Software
• Servo Motors
Stephen Rodriguez
• Power
• Facial Recognition
PROGRESS
10%
33%
42%
75%
75%
85%
90%
System Testing
Integration
Software
Parts Testing
Parts Aquired
Design
Research
BUDGET Part Cost Acquisition
Servo Motors $102.97 Purchased
MSP-EXP430 LaunchPad $4.30 Donated
MSP430 Capacitive Touch Booster Pack $10.00 Donated
Logitec C310 HD Webcam $29.99 Purchased
GH1213 Lead Acid Battery $15.99 To be purchased
Turbo series TMC-86-530-W Alarm $9.64 To be purchased
HK – E03358 Laser Pointer $14.99 To be purchased
PCB $33.00 To be purchased
Mounting Mats $100.00 As needed
E-Matrix, Hopper, Nitrogen tank. $150.00 Donated
Paintballs $29.99 Donated
Extra Electrical Components $30.00 As needed
Total retail cost $530.87
Total cost to group $336.58
STILL TO DO
• PCB
• Construct base and gun mounting.
• Create formula for converting pixel coordinates to angles.
• Interface laptop with embedded hardware.
• Creating targets, and threat level
• Facial recognition log-in
ISSUES AND MILESTONES
• Converting pixel coordinates to angles.
• Communication from PC to I2C bus.
• Construction of the base and gun mount.
Milestone Dates
Order PCB February 24th
Completion of Software March 9th
Completion of Base March 9th
Integration March 16th
System Testing March 23rd
Complete working prototype April 1st
QUESTIONS?