Multidisciplinary Senior Design ConferenceKate Gleason College of Engineering

Rochester Institute of TechnologyRochester, New York 14623

Project Number: P18001

WI!: A GLOVE BASED WEARABLE INTERFACE

Chris AtrasComputer Engineering

Natasha AmadasunBiomedical Engineering

Adrita ArefinComputer Engineering

Corey BarrowsIndustrial Engineering

Joseph DiPassioElectrical Engineering

Zachary HankinsonMechanical Engineering

ABSTRACT The aim of Project 18001 was to provide the primary customer, smartphone users, and RIT, with a

convenient, reliable device supported by qualitative proof that a wearable, glove based interface can adequately replicate response speeds and normal functionality of a mobile phone as current phones require present and in-depth interaction. This prevents the user from be able to detach from the interface efficiently. The device must be sleek, aesthetically pleasing, durable, accurate, lightweight, simple to use, affordable, and non-distractive. The team utilized silicones, force sensitive resistors (FSRs), a custom printed circuit board, and durable case to carry out the requirements of the project. In addition, the team conducted various tests to provide qualitative results to determine if the prototype satisfies the customer and engineering requirements.

INTRODUCTION Currently, phones require both tactile and visual interaction. The user is unable to detach from the interface

efficiently and continue proper use. Typing with one hand is a possibility with phones, but can result in more errors due to an attempt to multitask. This leads to more distraction and takes a user’s full attention to properly respond to text-based messages. A portable, easy to use device that reduces distraction and does not cause any restriction of movement or normal hand functions needed to be developed that would competently mimic regular phone typing purposes. This is a Phase 2 project following Phase 1, P17001.

The wearable, glove controller interface the project produced aimed to replace typical phone interfaces by using sensors, accelerometers, and other methods to track finger motion and hand positions to provide input to a device via Bluetooth. The goals were to make it portable, fully enclosed within housing, easy to use, and not be a distraction to the user or bystanders. Though most devices currently require two hands to effectively operate, this interface should be operable with one hand using little focus. Other prototypes researched have not gained much traction due to issues with weight, size, connectivity, reliability, durability, and lack of comfort. For example, the Keyglove is a full, fabric glove, with protruding wires and is relatively expensive in comparison to other prototypes. This method did not achieve wide success mainly due to lack of convenience, bulkiness, unreliability and overall cost for a product with several issues. A more widely known device, the Tapstrap, did gain traction in technology circles. It is small in size, relatively inexpensive, and was not incredibly uncomfortable. Though this device was more popular, it requires a hard surface such as a table to function properly, therefore, it would not be usable in every environment. P17001’s prototype was bulky, had unreliable sizing for various hand sizes, and is not accurate

Copyright © 2018 Rochester Institute of Technology

Proceedings of the Multidisciplinary Senior Design Conference Page 2

and repeatable with respect to its ability to properly ascribe finger motions to letters and text functions. P18001’s goal was to create a smaller, more efficient, more reliable, and easier to use device.

METHODOLOGY

Customer Requirements:The device was required to provide the user a comfortable, protected, lightweight, and non-distracting

interface. It was to properly and accurately provide a typing scheme that replicated output from a smartphone. The device should be chargeable with a standard charging cable. The device should be fully adjustable in sizing. The device should be completely wireless and connect to the phone via Bluetooth. Engineering Requirements:

The Engineering Requirements were directly gathered and mapped from Customer Requirements. They were broken up into five categories; Convenience (CV), Comfort (CF), Durability (D), Functionality (F), and Safety (F). They can be found in the following table:

Table 1- List of Engineering Requirements

Selection Criteria:In order to determine the most feasible path to a prototype, after morphological charts were created,

selection criteria were decided upon. These were then paired with possible research, experiments, and brainstorming ideas to determine possible answers. They were as follows:

1. Ease of learning curve2. Fits within budget3. Weight of the device4. Size of device5. Number of gestures available6. Ease of actuation7. Speed of message delivery8. Distraction Level9. Steps to power on10. Can be completed in MSD1 and MSD2

PCB Thought Process:The Esp chip was chosen as it included both a microprocessor along with Bluetooth capability and an RF

antenna board already attached. The main circuitry was based on the Dev kit for the ESP-Wroom-32 which was used for testing. This kit proved useful, so the board was based on the necessary components from the Dev kit. The charging circuit was based on the chips used in the charger available from Adafruit that is used to the battery chosen

Project P18001

Proceedings of the Multi-Disciplinary Senior Design Conference Page 3

for this project. The regulator setup was found in the Dev kit for the ESP module and was verified with the datasheet for the regulator chosen. An smt RGB LED was chosen to keep the number of through hole components to a minimum. The adxl335 was used for this design since testing of this device proved promising. The void in the two ground planes was placed based on the recommendations from the ESP device in order to reduce interference with the Bluetooth/Wi-Fi connections. The board was decided to be two layers in order to keep the price for the board down and to make routing easier. However, the ESP recommended that the board be four layers for optimal performance. The chip would still function on a two layer board, so it was decided that this would not change overall functioning.Typing Scheme:

The typing scheme should be as simple as possible. It was decided that the finger movements should be limited to one finger and combinations of two fingers right next to each other. Also, the hand positions would be limited to a natural range of motion with 5 to 6 different positions. This removes the possibility of unnatural finger combinations and hand positions.

RESULTS AND DISCUSSION Testing:Mechanical

Testing was performed on the mechanical system to ensure fulfillment of engineering requirements. The mass of the device and height of the device were measured to be 140 grams and 33 mm, respectively. To test adjustability, the device was adjusted to be worn by a 5th percentile female hand and a 95th percentile male hand. Notes were taken on the level of comfort/discomfort experienced by the wearers. Both test subjects also wore the device for an extended period of time, and took notes on tasks that were affected by wearing the device, as well as how long the device could be worn comfortably. It was noted that the device was overall comfortable, but if the straps were too tight there was pressure on the knuckles and finger webbing that was deemed as uncomfortable. A dummy prototype of the same mass as the working prototype underwent a drop test to ensure both the device case and PCB would be durable enough to undergo drops from up to 6 ft. The force on the device was calculated based on the acceleration of the device as it dropped, to ensure the PCB components would not delaminate.Electrical The device was tested for a multitude of different parameters to both ensure meeting the expectations laid out in our engineering requirements and to improve the electrical system in this iteration of this project. An area of primary concern is the battery life of the device. For the battery component, we chose a low-profile, 350 milliampere hour battery rated for 3.7 to 5 Volt operation. In order to test the battery life, two experiments were run. The first was an idle test, whereby the battery was fully charged and allowed to power the selected microcontroller and its peripherals without the controller entering sleep mode. An outboard timer was used to observe how long the system stayed powered in this state. The timer observed an estimated 12 hours and 26 minutes of battery life in idle operation. A second battery test involved a simulation of a more realistic use case, whereby the vibration motor and LED would periodically turn on and flash. The same outboard timer was used and observed an estimated battery life of LOOKUP hours. In addition to battery life testing, Bluetooth connectivity testing was vital to the overall usability of our device. In order to test the reliability of Bluetooth connectivity, certain parameters were observed. In order to test for distance, the device was turned on and configured for pairing. Using a phone, the device was paired, and using a tape measure and leaving the device stationary, the distance traveled until the pairing was disconnected was recorded with a tape measure. This was done for numerous trials and resulted in an average connection distance of LOOKUP. Bluetooth connection reliability was also tested as part of the scope of this project. An outboard timer was configured to capture the moment the phone and the device became unpaired while they were sitting a constant, negligible distance from each other. During an observation period of up to 2 hours, the connection never broke, leading to a conclusion that the ESP32 antenna is robust and reliable for our purposes.Electrical-Software Testing The key testing done for the software portion of this project was to ensure efficient timing and memory management, using the previous iteration of the project as a benchmark. By using a hashtable to store the different operations, instead of an array based look-up table, the dependence the software had on global variables dropped from around 2700 bytes to 25 bytes. On the ESP32, this meant that the hashmap code used only 1% of the total space allocated on the ESP32 for global variables, whereas the benchmark code used 133% of the total global variable space (in other words, the benchmark code could not be compiled for our selected controller). The total memory for all functions dropped from 13,000 bytes to 1000 bytes, and a mandatory delay of 300 milliseconds per operation was removed from the benchmark code. To test the validity of these results, the total bytes used were read from the compiler, and the delay removal was verified by the insertion of timing statements within the code, which were printed to the serial monitor within the Arduino development environment.

Copyright © 2018 Rochester Institute of Technology

Proceedings of the Multidisciplinary Senior Design Conference Page 4

Electro-Mechanical Testing The device was tested with a spring scale to determine how much force was necessary to actuate the sensors. This information was used to calibrate the sensitivity of the sensors. A number of users tested the device, going through the entire set of available characters, recording characters that were unable to be performed.Description of the Prototype:Mechanical system

The final prototype can be seen in Figure 1 below. The case was originally designed to fit a 5th percentile female hand, and was made slightly larger from there to accommodate the finger strap size.

Figure 1: Final Prototype on a 95th Percentile Male Hand

The system has two large straps extending from the silicone base of the device, which secure the device to the hand via the wrist and palm. Four finger straps extend out of the case to connect to the fingers via adjustable finger loops. The length of the straps can be adjusted by taking the top cover of the case off, loosening the clamps which hold the finger straps on, pulling back the excess of the straps, and retightening the metal clamps. The Top cover of the case is secured with two flat-head size 10-32 bolts. The power button, charging port, and LED indicator are all at the rear of the device, next to the wrist

The construction of the physical system consists of three main parts: The Silicone Base, The PCB Case, and the Finger Straps. The Silicone Base has adjustable watch straps to attach to the wrist and hand, and is the main mechanism for attaching the device to the hand. The base was cut out of 1/16th inch thick silicone by hand using an Xacto knife. For final product manufacturing, the base can be made using a mold. The Silicone was glued together using Loctite SuperflexTM Clear RTV Silicone Adhesive Sealant (referred to simply as Loctite or adhesive for the remainder of this paper). The PCB Case consisting of a top cover (Figure 2) and a bottom cover (Figure 3) is made of 3D-Printed PLA Plastic, and protects the PCB and other internal components of the device. For final product manufacturing, a draft can be added to build the case using injection molding processes. The case secures the PCB in place with size 10-32 Bolts and nuts, and is held closed by size 10-32 bolts. The case is adhered to the Silicone base with Loctite, and has metal clasps to secure the finger straps. The finger straps (Figure 4) are adjustable both to finger diameter and length. There’s a small clasp to hold the loop around the finger (excess can be cut off for personal wear), and the length can be adjusted by loosening the metal clamps on the case and pulling the straps tight after putting the loops on.

Project P18001

Proceedings of the Multi-Disciplinary Senior Design Conference Page 5

Figure 2 - Top Cover

Figure 3 - Bottom Cover

Figure 4 - Finger Loop Straps

Software/Electrical: The electrical system was designed for power efficiency and total space taken by the components to make it compatible with the mechanical subsystem. The overall system can be described as a series of actuators, one for each finger excluding the thumb, that toggle on and off with a finger clench. This activity is then monitored by a microcontroller, who polls a general purpose input pin to see which fingers are active at any given time. The microcontroller also polls a 3-axis accelerometer to determine which position the hand is in when the certain fingers

Copyright © 2018 Rochester Institute of Technology

Proceedings of the Multidisciplinary Senior Design Conference Page 6

are clenched. Using this information, the controller then converts the combination of the accelerometer data and the finger data into a command word. This command word determines overall character to be sent to the phone. The finished electrical schematic can be seen in Figure 5 and the PCB layout of this schematic can be seen in Figure 6.

Figure 5: Schematic of WI!

Figure 6: PCB Layout of WI!

On the lower level, we can examine each individual component individually. A 350 milliampere hour battery was selected due to its form factor relative to its overall charge capacity. For the finger actuator, a force sensitive resistor (FSR) was used inside a voltage divider. As the finger clenched, the FSR was squeezed by the straps mentioned in the mechanical design portion of this report. This changed the effective resistance of the component, and therefore the overall voltage at it’s tail node. This voltage was monitored by the microcontroller as described above. Once the voltage passed a certain threshold, the microcontroller would register that the finger was indeed clenched. For the microcontroller, the ESP32 was selected for multiple reasons. It had a built in bluetooth low energy (BLE) antenna, had plug-and-play capability when it came to porting existing code written for the project, and it had a small form factor compared to other controller-bluetooth combinations. An accelerometer from the ADXL family was selected because of its prevalence in RIT’s electrical engineering curriculum. The controller was programmed using a combination of the arduino development environment and open source software available on github based on the ESP32 project. This software will be referenced in the LOOKUP. For the software design, the main priority was to optimize timing and memory efficiency. Each combination of hand location and fingers actuated corresponded with one character to be sent via bluetooth

Project P18001

Proceedings of the Multi-Disciplinary Senior Design Conference Page 7

connection to the paired device. In order to be efficient, a command word structure was formed, whereby each command word corresponds to one character in the ASCII alphabet. Each command word is comprised of 8 bytes. The following figure provides an overview of the command word structure:

Figure 7 The first, and most significant, bit is the enable bit. If the device is off or not in active mode, this bit will always be an active ‘1,’ meaning no transmission will be carried out. If the device is on, paired, and ready to send data, this enable bit will be driven low. The next three bits, bits 7-4, are the hand position bits. There are six possible hand positions, which are determined by the current state of the accelerometer. Positions 1-6 correspond to binary numbers 001 - 110. The combinations of 000 and 111 are never used for bits 6-4. The final 4 bits, 3-0, are the finger bits. If the first finger is actuated, and no other fingers are actuated, bits 3-0 will be “1000”. If the first and third fingers are actuated, and the second and fourth fingers are relaxed, bits 3-0 will be “1010” and so on and so forth. Each command word therefore contains all the data from the physical input structure. Using a hash table, whereby each ASCII character is indexed by its corresponding command word, once a command word is generated it takes only one look-up operation to move the desired ASCII character into the bluetooth pipeline. This is a significant improvement over the search algorithm needed to look-up a desired ASCII character in the previous iteration of this project.Budget/BoM:

In order to keep our product competitive with similar products in its field the cost was a factor that was considered throughout the process. The final cost of fully producing one of the prototypes designed here is around $147. This compared to the $200 for the keyglove, and $180 for the gest. However much of this cost can be reduced with the purchasing of supplies in bulk including the silicone material used for the straps and many of the components used on the PCB. The breakdown of the cost for the prototype can be seen in Figure 8 below.

Figure 8: BoM CONCLUSIONS AND RECOMMENDATIONS

Conclusions:The final fully integrated system may not have been completely operational, but progress was made when

compared to last year’s prototype. Many issues along the way lead to this finish but the main issue that halted progress was the printed circuit board. Design flaws and timing delays concerning the creation and ordering of the PCB lead to the overall delay of this project. Even with this major setback progress was still made to show that this product could not only function as expected but could fit in a much smaller housing than in previous projects. Recommendations:

Copyright © 2018 Rochester Institute of Technology

Proceedings of the Multidisciplinary Senior Design Conference Page 8

1. Consider switching the ESP chip for Feather Device from Adafruit.

2. Consider making the PCB double sided to make the WI! Smaller.

3. Consider switching the FSRs with small tactile buttons.

○ If you use a similar mechanism the tactile buttons will likely wear out the silicone quickly, but it’s worth looking into

○ Converting to tactile buttons opens up a lot of space on the PCB

4. Look into different materials (silicone isn’t breathable, a little irritative)

5. Consider increasing the hardness of the silicone

6. Improved watch buckles

7. Reduce size of bolts to reduce overall size (be careful to make sure they’re strong enough

8. Look into alternate clamping mechanisms that clamp each strap individually

9. Ease of adjustment needs to be improved

10. Create method to diffuse LED light out of case (currently just a hole)

11. Make the PCB four layers to allow for the components to be closer together.

12. Look into making the PCB flexible if that increases comfort or spacing of components.

ACKNOWLEDGMENTS

Team P18001 would like to thank the RIT MSD Department for their sponsorship, our guide, Kenneth Mihalyov, and Willow Baker, for her feedback and for proposing this project. We would also like to thank Engineering House for the use of their 3D printer.

Project P18001