Haptics in military applications

Lauri Immonen

University of Tampere Department of Computer Sciences Computer Science / Int. Technology Seminar paper: Haptics in military applications December 2008

i

University of Tampere Department of Computer Sciences Computer Science / Interactive Technology Lauri Immonen: Haptics in Military Applications Seminar paper, 14 pages, 4 index and reference pages December 2008

Haptic applications are used versatilely in the military field. Haptics can be used to enhance the users’ immersive feeling when performing training in virtual reality environments. Better performance in real life situations can be expected with the help of realistic training environments. Haptics can also be used to communicate or navigate in different surroundings. Haptic navigation and communication methods can provide an alternative to traditional methods. In this paper I have a look at different military fields and applications. I present example applications, which are being used in those fields. The applications are discussed and conclusions are made. Haptics, despite of being a relatively new topic in military field, has a small but certain ground already and will be more important in the future.

Key words and terms: Military; communication; haptic cue; tactile signal; tactile display; vibrotactile; tactors; virtual reality; surgery simulation

ii

Contents 1. Introduction ............................................................................................................... 1 2. Fields of applications................................................................................................. 1

2.1. Medical ............................................................................................................ 1 2.2. Communications and navigation...................................................................... 2 2.3. Combat ............................................................................................................. 3

3. Application examples ................................................................................................ 3 3.1. Medical field: A Haptic-Enabled Simulator for Cricothyroidotomy............... 4 3.2. Communication field: Comparison of Army Hand and Arm Signals to a Covert Tactile Communication System in a Dynamic Environment ........................ 5 3.3. Communication, navigation and combat fields: Tactile Displays and Detectability of Vibrotactile Patterns as Combat Assault Maneuvers are Being Performed .................................................................................................................. 7 3.4. Navigation and combat fields: The design and deployment of a spatialized vibrotactile feedback system ..................................................................................... 9 3.5. Combat field: A tactile cue for firearms and other trigger-activated devices 12

4. Discussion ............................................................................................................... 13 5. Conclusions ............................................................................................................. 14 References .....................................................................................................................15

1

1. Introduction Military operations and actions include often many different areas; in one large scale military operation involved could be strategic, logistical, medical, communication, combat and maintenance areas. Traditionally training in military has been training in theory and then practicing the learned theories in real life environments or in environments, which imitate real environments, such as obstacle courses. As technical and electronic products and applications are developing, there is a need also in military field to research and introduce new technologies, which could help in training and operating in different military areas. The using of new technologies can bring along advantages that can lead to improved performance and reduced risks. Virtual reality based training can have a positive effect in real life performances for military personnel. The virtual environments do not have to be entirely realistic to provide useful training, which affects the performance in real situations. However bad and unrealistic haptic feedback can cause total failure of the haptic experience, so a certain level of quality must be maintained even if total realism is not required. Haptic feedback in these training scenarios may play an important role in virtual reality training. [Jiang et al., 2005] The use of haptics has potential to enhance the feeling of immersion in training whether the training is performed in virtual reality simulators or real life training situations in e.g. an obstacle course. The use of haptic feedback can improve performance and sense of presence. [Fowlkes et al.] In this report I will have a look at haptics in military applications. To be more precise, I have divided the field of focus into categories: I shall have a look at medical, communicational and navigational and combat applications.

2. Fields of applications

2.1. Medical

In medical situations realistic practising and training is often difficult. Current practise methods are using animals, cadavers and dummies that are made of plastic. There are problems with these methods, such as that animals do not have the correct anatomy, cadavers do not respond physiologically correctly and dummies lack the full range of anatomical variations. There can also be ethical questions with using of animals and

2

cadavers. [Liu et al., 2005] Those could be such as if humans are entitled to use animals as test subjects or if corpses should be buried instead of being operated on. Haptic actions are often used in open surgery, because visual action is not possible. The surgeon depends on haptic feedback when identifying e.g. bloode vessels under other tissues [Hu et al.].

With the help of computer simulations with haptics it is possible to provide alternative training options to replace those earlier mentioned possibilities. It is possible to provide precise representations of human physiology. Different patient types can be simulated. One important function for training purposes is that computer-based simulators can save the data and thus measure the performance of the training person. [Liu et al., 2005] When the performance can be monitored, it is possible to give more accurate feedback about a person’s learning.

To my opinion, the combination of visual and haptic feedback can be very valuable when training e.g. field surgery operations. The operating conditions in a battlefield can be poor; there is lots of dirt and blood on the operating area so the sense of touch is emphasized compared to the sense of sight. If it is possible to learn, how an operation feels, it should be practised as much as possible. However, it is essential that the haptic feedback is realistic; learned unrealistic or wrong sensations from the haptic device can lead even to death in real surgery situations.

2.2. Communications and navigation

Communication between individuals usually happens by speech, but it can happen also by other means. One very common means of communication in the army is communicating by using hand signals. There are reasons for using hand signals; at battlefield the hearing of soldiers may be damaged temporarily or even permanently because of loud noises or there may be situations where speaking or shouting is not an option. An example of that kind of situation could be entering a house or area as silently as possible. However, hand signals may not be useful in all situations; there may be obstacles blocking the view, dust or smoke in the air that prevents sight or it could be too dark to see. Even if there would not be any obstacles to using hand signals, finding an alternative way to communicate could allow soldiers’ visual channel to remain free and allow them to maintain longer distances between each other.

3

The challenge for soldiers on field is to maintain and monitor visual and auditory information channels, and at the same time having enough awareness of the environment and the situation the soldier is in. The soldiers can focus more at different obstacles and environment awareness when receiving tactile signals than when receiving visual hand signals. [Krausman and White, 2006; Pettitt et al., 2006] Security issues must be taken into account; the soldiers must get reliable information via the haptic applications. The technology that delivers the information must be robust and work in extreme conditions. Even if the technology works fine, it is as important to make sure, that the messages sent are clear to be understood in difficult conditions. Reliability of the information source is essential; in case of a combat situation, if the enemy could feed wrong information to the soldiers’ tactile displays, the results could be disastrous. If the tactile signals are not reliable enough for some reasons, it could be well worth having a look if tactile signals could be used as attention cues; the tactile information would just deliver a message about more specific information somewhere else.

2.3. Combat

Safety is also in military world an important issue, so improving safety should also be considered when thinking about the possibilities of haptics in training environments. In combat situations so called friendly fire is always a risk, and in training situations the learners can be inexperienced with handling of weapons. Training with real weapons always poses a risk of a serious accident; alternative training methods are welcome if similar results can be achieved with them. In Chapter 3 I will present an example of a haptic vest, which is used in close-quarters combat training. As another example I will present a rather simple invention, which aims to improve the safety of handguns. I think this invention could be used easily in military training with assault rifles and other weapons.

3. Application examples In this chapter I want to present some of the numerous possibilities, in which haptics can be applied in the military field. Not all examples are of military applications, but are of such nature that they could easily be used also for military purposes. I will

4

present examples about medical and communicational applications. One important military point of view is combat, so I will have a look at combat applications too.

3.1. Medical field: A Haptic-Enabled Simulator for Cricothyroidotomy

Liu, Bhasin and Bowyer describe in their article [2005] their efforts to build a computer-based cricothyroidotomy simulator to address the problems mentioned in chapter 1.1. Open cricothyroidotomy is an operation, in which the patient’s airways are blocked and an alternative breathing method has to be accomplished. This could be the case in a severe facial wound caused by a gunshot or an explosion, or certain gas injuries.

In this system, users have 3D shutter glasses and a Phantom haptic device, so users can see and feel what happens on virtual environment. The authors claim that this is the most natural interface for open cricothyroidotomy. The software components in the system include the graphical user interface, the patient model and key surgical skill steps.

Figure 1. Graphical user interface [1] Figure 2. A virtual patient model [1]

The skill steps are steps, which require a specific action to be performed. There are four of them in the simulator. The first step, landmark identification, is a primarily a tactile task. The student uses the Phantom device to locate the place, where the incision will be made. In the second step, making the incision, the user can feel the sensation of cutting with a scalpel. In the third step, incision enlargement, haptic feedback constrains the instrument to penetrate skin at a single point. The final step, intubation, simulates the resistance of inserting a tube in the incision.

The authors claim that the preliminary comments of evaluators of the system have been favourable. The accuracy of the haptic feedback was commented positively. I think that

5

the system is promising; however I would like to see the Phantom device to be replaced by a haptic glove. This would bring more realism to the system, and also give more realistic haptic feedback. In my opinion, realistic haptic feedback is essential in this kind of simulators, because in real life situations operations like cricothyroidotomy require often operating by mostly touch. Visibility can be limited because of lack of lights or dirt and blood.

3.2. Communication field: Comparison of Army Hand and Arm Signals to a Covert Tactile Communication System in a Dynamic Environment

Pettitt et al. conducted a study [2006], in which soldiers’ abilities to interpret and respond to tactile commands was evaluated. The study was conducted with infantry soldiers while they were carrying out a combat patrol simulation on an obstacle course. Tactile signals and visual hand signals were sent to the soldiers, and their response times and accuracy of signal interpretation was documented. According to the authors, with proper implementation the using of tactile displays could reduce interference with the soldiers’ visual and auditory channels and improve their overall performance. The objective of the study was to find out, if soldiers are able to interpret and response to tactile commands as efficiently as they can to hand signals in a dynamic environment. They also wanted to find out, if the use of a tactile system hinders the ability of soldiers to complete an obstacle course. The soldiers were given commands while completing an obstacle course. The commands were tactile signals, hand and arm signals from a leader in front of the soldier and hand and arm signals from a leader behind the soldier. The soldiers were voluntary participants from the Infantry Training Brigade, Fort Benning, Georgia. All tasks in the study were a normal part of a soldier’s life. They were wearing their normal combat uniform and a device to simulate an M4 assault rifle.

The tactile systems in this study consist of tactile displays and receiver units. The display consists of eight tactile actuators, which are attached to a belt that is worn around the waist of a soldier. The tactors can be activated individually, sequentially or in groups to create sensations to replace standard army hand and arm signals. The four signals that were used are shown in Table 1 and the tactile display is shown in Figure 3.

6

Signal Hand signal Tactile signal Attention Hand raised straight forward at eye

level, then waving from side to side

A sequenced side-to-side activation of front tactors

Halt Hand raised straight up Four tactors simultaneously actuated

Move out Head facing the direction, then a

motion with arm from behind neck

towards the direction

A sequenced back-to-front activation of tactors,

creating movement around each

side of the body

Rally Waving the arm straight up with a

circular motion.

A sequenced activation of all tactors, creating a

circular motion around the body

Table 1. Hand and tactile signals. [Pettitt et al., 2005]

Figure 3. Tactile display. [Pettitt et al., 2005] The obstacle course simulated the following actions: patrolling, crawling, climbing and firing. Once the obstacle course was started, the soldiers would follow their team leader from a distance of around ten meters until the end of the course. A squad leader would follow behind the soldier. Before entering the obstacle course, the soldiers were trained to recognize the tactile signals with 100% accuracy.

The soldiers on the course could receive signals at any point from the team or squad leader or from the tactile display. When receiving a tactile signal, the soldiers told the meaning of the signal to a data collector, who was nearby. The study was conducted during daytime at an obstacle course, which made it much easier to notice and interpret given hand signals correctly. In a realistic environment there would have been lots of vegetation and landscape obstacles.

7

The authors found that the tactile signals were intuitive and easy to understand for the soldiers. The soldiers were able to interpret and response faster to tactile signals than to signals given by hand or arm. The study conditions were favourable to traditional signals, so the authors believe that in a real situation the difference could be even greater. One benefit of a tactile system would also be that with the system, commands to a whole group can be given at the same time instead of the message going from a soldier to soldier. The authors claim that in a real situation the system would perform even better than in the experiment, but I do have my doubts about that. In a real combat situation there is the presence of danger, which cannot be simulated. Extreme situations can be life threatening and people could behave very differently in a real situation. With a hand signal, the soldier is able to quickly communicate a signal back, for example showing that they did not get the first signal. I think this system could perform rather well in a quiet stealth mission, but not that well in a real combat situation.

3.3. Communication, navigation and combat fields: Tactile Displays and Detectability of Vibrotactile Patterns as Combat Assault Maneuvers are Being Performed

Another study about tactile displays is presented by Krausman and White [2006], in which they examine the detectability of vibrotactile patterns as combat assault manoeuvres were performed. They say that one issue, which has not yet been looked at thoroughly, is how different types of tactile displays affect to recognition on tactile messages especially considering the tasks that soldiers perform during combat operations. They wanted to find out those things in their study.

The other objectives of Krausman and White’s study were to find out how wearing protective body armour affects the recognition of tactile messages and if different tactile patterns have the same recognition rates.

The participants of the study were ten officers from Fort Stewart, Georgia. Like in the study from Pettitt et al. there was an obstacle course, which the participants carried out. The manoeuvres were similar; they included running, jumping, balancing, windows climbing and crawling.

8

Figure 4. Different manoeuvres at the obstacle course. [Krausman and White, 2006] The system consisted from a tactile display and a receiver unit. There were two different tactile displays; one like in Pettitt et al. used in their study, and a display that consisted of an 4x4 array of tactors. The belt display was worn around the participants’ lower abdomen and the array display on the lower back.

Figure 5. Array and belt displays. [Krausman and White, 2006] The participants received different tactile patterns while performing on the obstacle course, signals lasting for about two seconds. There were six patterns for each display type. The experiment began after the participants reached 100% accuracy in recognizing the tactile patterns.

9

Figure 6. The patterns in array and belt displays. [Krausman and White, 2006]

The experiment reveals that the type of tactile display did not affect the detection and recognition of the tactile patterns. Wearing the protective body armour did not have an effect either. The results show that when the participants were high crawling (see middle of fig. 4), only 63% of the tactile patterns were detected. When manoeuvring the tires, 92% were detected and with the windows 88% of patterns were detected. The differences between the six different patterns were small.

It seems clear, that more work is needed to find out, how to improve tactile pattern detection rates. The studies were performed in good conditions at an obstacle course; in real battlefield conditions with an enormous amount of stress, exhaustion and anxiety the results could drop much more. Since errors in detecting and identifying commands can cause as much as deaths, it is important to carry on research until the results show reliable detection rates.

3.4. Navigation and combat fields: The design and deployment of a spatialized vibrotactile feedback system

Lindeman et al. describe in their article [2004] a system, which deliveres haptic cues for use in immersive virtual reality environments. The system consists of a body-worn garnment that has a large number of vibration units attached to it, and a device that supports the delivery of vibrotactile cues.

10

The motivation for the authors is to design and implement a system for delivering vibrotactile cues to the whole body for use in real world and virtual environment applications. The problem is to find a solution to provide enough coverage of the body to impart collision information to the wearer in order to improve the realism of the experience. The authors have come up with a solution, in which they have positioned tactors at different points of the body with a vest, and control the vibration intensity with a computer using wireless connection. There are a few challenges with the design of a tactile vest. Those are that the vest has to keep the tactors fairly tight against the body, and that the vest should fit as well to different sized users. The authors have faced those problems by designing a tight fitting stretch neoprene vest with some hook-and-loop fasteners to keep the bits of the vest in place. The tactors are placed at locations with a higher probability to get in contact with virtual objects. The authors have also kept in mind that the users will be wearing a military protective vest and other gear during a virtual reality environment session. The placement of the tactors can be seen in figure 5.

Figure 5. A tactile vest with numbers marking the placement of tactors. [Lindeman et al., 2004]

11

The authors are a part of a team working to create a fully immersive simulator for training small-unit tactics, techniques and procedures for close-quarters battle. This task requires soldiers to often operate inside buildings. The environments in buildings vary greatly, it can be dark and noisy, there can be smoke and debris, but it can also be a quiet environment requiring the use of stealth and silent actions. The soldiers have to efficiently operate and possibly use combat actions in these conditions. The conditions are physically and mentally demanding and the authors aim to simulate that as accurately as possible. The users of the system wear in addition a military protective vest and a head mounted virtual reality display. An optical tracking system is used to track the user’s head, torso, waist, arms, legs and a weapon. The users may get in contact with virtual environment objects for several reasons: They may look at something else while moving, their view may be limited or they choose to touch a virtual object. The goal of the system is to handle these situations as realistically as possible to improve the user’s immersive feeling.

Figure 6. Using the virtual reality system. [Lindeman et al., 2004] There are challenges that need to be solved in order to the system being really realistic. The users have said that it is difficult to know where their body is in relation to the environment and they may get stuck on objects. Figure 7 shows an example of a situation like this. The left side shows the user’s view, which reveals no obstacles.

12

However, on the right side picture we can see that actually the user’s shoulder is colliding with the doorframe.

Figure 7. A user is stuck in a doorway. [Lindeman et al., 2004] The system described in the article does look promising. The situations, in which the soldiers would be, are often dangerous and very demanding, so the more realistic training environments can be provided, the better and more efficient the results will be. The system could be developed in the direction that the users would be fully covered with tactors, and even more realistic feedback would be possible. Also, among the virtual environment, some smell effects like smoke could be quite easily used to add realism to the simulation. I do think that virtual reality training is and will be an important factor in military and other demanding situations training. However virtual environments are yet not so realistic that the main part of training could be done in simulators. Real life environments and real situations training are still in my opinion the best training methods, but virtual reality training definitely does add a useful alternative training option to traditional training methods.

3.5. Combat field: A tactile cue for firearms and other trigger-activated devices

Not all haptic inventions are complicated or expensive and difficult to implement. William McMoore has invented [2005] a tactile cue for trigger-activated firearms or other devices. A tactile stimulus, either a groove or rising is placed on the firearm in the place where the user’s trigger finger should rest. The idea of the invention is to make trigger-activated weapons safer. McMoore says that it may take time to learn to keep the trigger finger in a safe position; not on the trigger. This claim especially applies to people who have little or no experience with trigger-activated firearms.

13

Figure 7. Trigger finger in a shooting position. [McMoore, 2005] The physical tactile cue (No. 22 in figure 5) reminds the users to keep their finger off the trigger until they intend to shoot. The invention is not meant only for military purposes, but for all kinds of trigger-activated devices and uses. A similar tactile cue is used in keyboards; the letters F and J and the number 5 have a similar tactile cue. I do think that this invention could easily be used in military training. It would be inexpensive to implement and hopefully improve the learning of how to properly handle a weapon. Keeping in mind, that firearms are deadly and accidents may and will happen, even this kind of simple invention could save a life at some point.

4. Discussion After getting familiar with the examples I chose to present in this paper and learning generally about haptics in military applications, I am sure that not only there is a place for haptic applications in the military world but haptic applications will become more common as the techniques evolve. It is already clear, that virtual reality training has positive effects to real life performance. When haptics are added to that, it is possible to create even more realistic simulations and improve the performance even more. It has to be kept in mind though, that unrealistic haptic effects can ruin the whole experience and make the training rather useless. Realistic effects definitely improve the simulations; I do expect fast improvement in the realism of the effects as computer power grows strongly. With the help of realistic effects and costs getting less expensive, I expect especially the medical field to benefit from using of haptics. Surgeons can train much more in quite

14

realistic environments and thus get a certain level of experience and routine, which helps in real life situations even though the situation differs significantly from a training situation. Having said that, I do think that even the best virtual practice cannot completely replace real life experiences and the lessons learned from those situations. Especially in military field there are often dangerous, even deadly situations, and performing in extreme conditions differs dramatically from safe training environments. The realism of haptic effects is still not everything. Vibrotactile signals can be used in communications and navigation, with success as the examples show. There is yet lots of work to do, because detecting haptic signals is not that simple as we can see in Chapter 3.3. However I could easily see haptic signals to be used as kind of awareness cues, to inform soldiers that there is a message for you, observe your squad or team leader. Haptic cues could be very well used as attention signals, and with more research, they could be used to give more accurate and detailed instructions to soldiers. In certain situations, like silent stealth missions, tactile signals could prove valuable. As the use of haptics, especially in the combat field, is a fairly new practice, I think that it should be taken at first slowly but surely into wider use. There will be, like with all new things, resistance to new concepts. It would be a shame if new technology would be taken into use too fast and perhaps fail with some applications because of that; leading to people abandoning that application. If new technology is taken into use bit by bit, people get used to it faster and benefit from it faster, too.

5. Conclusions As technology evolves and computer power grows, haptic devices and effects evolve and get more realistic. This makes it possible to use realistic training environments and provide training possibilities in environments, in which earlier it has been impossible or difficult to work in. The more realistic training can be provided, the more efficient learning results can be expected even though virtual training will not replace training in real environments. Haptic signals can be used to communicate and navigate. This field requires research about how to deliver signals as reliably as possible. It should be looked in if is it possible at all to give orders by haptic signals. Perhaps haptic signals should be used as cues about other information, as kind of attention grabbers, or haptic signals are good for certain kind of messages. Be that as it may, the use of haptics in military applications is looking promising as long as more thorough research will be done.

15

References [1] Liu, Bhasin and Bower: A Haptic-Enabled Simulator for Cricothyroidotomy. Studies in Health Technology and Informatics, vol. 111, pp. 308-313. IOS Press, 2005. http://simcen.org/pdf/liu%20mmvr%202005%20paper.pdf

[2] Krausman and White: Tactile Displays and Detectability of Vibrotactile Patterns as Combat Assault Maneuvers are Being Performed. U.S Army Research Laboratory, Technical Reports. 2006. http://www.arl.army.mil/arlreports/2006/ARL-TR-3998.pdf

[3] Pettitt, Redden and Carstens: Comparison of Army Hand and Arm Signals to a Covert Tactile Communication System in a Dynamic Environment. U.S Army Research Laboratory, Technical Reports. 2006. http://www.arl.army.mil/arlreports/2006/ARL-TR-3838.pdf [4] Jiang, Girotra, Cutkosky and Ullrich: Reducing Error Rates with Low-Cost Haptic Feedback in Virtual Reality-Based Training Applications. Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. http://bdml.stanford.edu/twiki/pub/Haptics/ImmersionProject/STTR_paperV7.doc [5] Fowlkes, Durlach, Drexler, Daly, Alberdeston and Metevier: Optimizing Haptics Perceptions for Advanced Army Training Systems: Impacts on Performance. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.1953 [6] Hu, Chang, Tardella, English and Pratt: Effectiveness of Haptic Feedback in Open Surgery Simulation and Training Systems. http://www.energid.com/74059600241626145924740331/Link.htm [7] McMoore: Tactile Trigger Finger Safety Cue for Firearm or Other Trigger-Activated Device. United States Patent No. 6862829. 2005. [8] Lindeman, Page, Yanagida and Sibert: Towards Full-Body Haptic Feedback: The Design and Deployment of a Spatialized Vibrotactile Feedback System. Proc. Of ACM Virtual Reality Software and Technology (VRST), Hong Kong, Nov. 2004. http://web.cs.wpi.edu/~gogo/papers/Lindeman_vrst2004.pdf