Online Submission ID: 0

Identifying Inexpensive Off-the-Shelf Laser Pointers forMulti-User Interaction on Large Scale Displays

Category: Research

ABSTRACT

We present a method for identifying inexpensive, off-the-shelf laserpointers in a multi-user interaction environment on large-scale dis-plays. We identify a laser pointer’s personality, a measure of itsoutput in a particular context. Our method requires a set of inex-pensive and unmodified green lasers, a large screen, a projector,and a camera with an infrared (IR) filter. The camera detects theIR spillover from the green laser beam, while ignoring color in-formation projected onto the screen. During a calibration phase, aradial histogram of each laser’s IR spillover (datapoints taken overseveral frames) are used to represent the laser’s personality. Usingthis signature, our system is able to identify the spots of a specificlaser, allowing multiple users to simultaneously interact in the en-vironment. In addition, we present a series of applications that takeadvantage of tracked and identified laser pointers to demonstrate theusefulness of large-scale, multi-user interactions and the feasibilityof providing such a system inexpensively and without the hassle ofadditional laser pointer modifications.

Index Terms: H.5.3 [Information Interfaces and Presentation]:Group and Organization Interfaces—Collaborative Computing;

1 INTRODUCTION

Multi-user, large-scale interfaces, in which individual users areidentified and tracked, present a challenging and worthwhile designproblem. In applications in which efficient interactivity betweenusers is important, laser pointer devices are preferable to stationarypointer devices, such as mice [15]. In most cases, the challenge ofidentifying individual users is tackled by physically modifying laserpointers, an effective yet often-times costly and time-consumingsolution. To circumvent these drawbacks, we present the idea ofdescribing a laser by its personality, a measure of the shape and in-tensity of a laser pointer’s leaked infrared light, which allows us totrack and identify an off-the-shelf laser pointer among a group ofothers for the purpose of multi-user collaborative applications. Wealso present a series of applications that take advantage of the laserpersonality system to exhibit group problem solving, data manipu-lation, and data exploration.

Our contributions are:• The laser pointer personality, the signature shape and inten-

sity of a laser point’s infrared light leak.• The laser pointer personality system used to identify an off-

the-shelf laser pointer• Several multi-user applications that demonstrate the utility of

the personality system

1.1 Related WorkSingle laser point detection is accomplished in two main ways:brightness filtering and infrared filtering. Olsen and Nelson detecta laser spot on a large display screen with a two pass system [13].The first pass detects the brightest red spot in the image, while thesecond (which only occurs if no obvious laser spot is found) passesover the screen again, employing a convolution filter, concentratingon the area where the last spot was detected. Oh and Stuerzlingerallow for multiple laser points by applying a threshold to the bright-ness field of the image [12]. The same technique is applied by Davisand Chen in their LumiPoint system [5]. However, depending on

Figure 1: False color renderings of the IR spill from 6 inexpensivegreen laser pointers. The pattern of this spill is relatively constant,allowing us to track and identify the lasers over time. The patternof spill from the laser varies most with distance of the laser to thescreen. The top row of images were collected with the laser 15 feetfrom the screen, the middle row was 10 feet from the screen, andthe bottom row. The second row was 5 feet from the screen.

the brightness of the laser spot with respect to the rest of the imagecan create trouble, as it is context sensitive and simple thresholdsmay confuse the laser points and very bright sections of the image.Ahlborn, et al. [1] present a system using multiple camera views,in which the background image is filtered out, leaving only laserpoints for detection. IR filtering is employed by work by Qin, etal. [16], Angelini, et al. [2], and Cheng, et al. [4].

In addition to laser spotting, one challenge in multi-user laserpointer systems is pointer identification. One method is to dynami-cally change the number of lasers present in the system at any givenpoint in time, and to track an IDed laser spot across frames withpredictive measures, such as the Kalman filter [12, 5, 4]. Anothermethod involves the use of time division multiplexing, or the ap-plication of a laser blinking pattern, to identify a particular laser.Given n lasers, a system can force exactly one of the lasers off eachframe. With cameras with high frame rates, this produces an un-noticeable effect with regard to the temporal coherence of the laserspot on the screen, but the system can identify the laser by its blink-ing pattern. This method is employed by Vogt, et al. [19, 18], aswell as Pavlovych and Stuerzlinger [14].

Laser point detection, both single- and multi-user, has a varietyof applications, most of which revolve around user collaborationand interaction. Francisco de la O Chavez, et al., present a systemwhereby users can operate the electronic devices and appliances intheir homes with a laser pointer [6]. The system uses a single cam-era, and the user sets up “active zones”, or groups of pixels in thecamera view around each appliance. When a laser spot is detectedin the active zone, the appliance is activated. Qin, et al., presenta system in which a special laser pointer is used to project severalbeams, and the orientation of the beams indicates the angle of rota-tion along the beam axis of the laser, thus allowing for gestures andobject manipulation that involves rotation [16]. Bi, et al. presentthe uPen, a laser pointer outfitted with right- and left-click buttons,designed to mimic computer mouse functionality [3]. Shizuki, etal. [17] present a series of gestures used with a laser pointer, and aseries of applications using them. The gestures are edge-crossinggestures, and the applications presented include a presentation ap-

1

Online Submission ID: 0

plication using gestures to move slides forward, and a picture view-ing application where gestures manipulate the images.

To our knowledge, no work has been published regarding theidentification of off-the-shelf laser pointers in multi-user applica-tions. Also, the domain of applications that have been explored formulti-user laser pointer interaction is rather small, either due to lackof need or due to the necessity of engineering laser pointers to makethe interfaces effective.

2 LASER POINTER PERSONALITIES

2.1 System Details

To detect the current position of each laser pointer dot, we use a1280 x 960 pixel monochrome, 33fps video camera. An infrared(IR) pass filter in front of the camera blocks all visible light (fromthe projector), so we can robustly detect the bright points of IR lightfrom the laser. We specifically use green laser pointers because thegreen light is produced indirectly from an infrared laser diode, andsome of the infrared light remains for our detection. Most inexpen-sive green lasers do not include an IR filter to block this light.

We begin with a simple calibration step to determine the pixelto pixel correspondence between our camera and the 1920 x 1080projector and projection surface, and to collect intensity data on alllasers in the system. Our system has been tested on screens as tallas 18’. We simply shine a laser at a 3x4 grid of calibration points.The first geometric calibration allows us to identify multiple laserssimultaneously pointed at the screen. For non-erratic laser motions,we can easily track these points over time to allow users to performactions within an application, for example moving pieces around ina jigsaw puzzle (Figure 5) or circling nodes in a graph visualization(Figure 7).

When tracking multiple lasers simultaneously, we use the Kuhn-Munkres, a.k.a. Hungarian Algorithm [8, 10, 20] to match thelasers from frame to frame. This method produces a pairing that ef-ficiently minimizes the sum of the distances between the positionsof each laser between the two frames. We also considered using aKalman filter [7] to track smoothly moving laser dots, but our earlyexperiments indicated this was complicated to tune for the acceler-ations of the laser dots at corners or tight turns and ultimately notnecessary.

2.2 Laser Spot Processing

In addition to the centroid of the laser spot we also extract the in-tensity and size of the detected blob to calibrate laser intensity datafor identification. Inexpensive lasers exhibit a range of brightnessand focus, as shown in Figure 1 and this signature is fairly con-sistent for each device (once the laser has warmed up for about 15seconds, and as long as the batteries are reasonably fresh). We callthis signature the laser’s personality. During the calibration phase,we capture several frames’ worth of this intensity data at each ofthe calibration points for each laser. We examine the blob of lightand calculate a radial histogram of the intensity values of the blob,In practice, 20 bins (representing a radius of 20 pixels) is sufficientto capture the uniqueness of a laser spot’s shape. This intensityhistogram acts as the laser’s personality. Note that the calibrationcan be performed simultaneously for many lasers (with 1 personper laser), and takes less than a minute. The laser personality inten-sity calibration could also be performed passively and continuously,which we plan to explore in future work. Sample laser personalitydata is presented in Figure 2.

2.3 Identifying Laser Spots: Matching Personalities

Once calibration is complete, the system is able to match any laserspot on the projection surface with one of the calibrated lasers.When a laser spot is detected, its personality is calculated, andmatched with that of one of the known lasers.

Figure 2: This figure shows 30 personality measurements for eachof 6 lasers from a single calibration screen location. A single datapoint (a laser personality measurement) is represented by a radialhistogram of intensities arranged by distance from the centroid ofthe laser spot, and each line represents one such data point. Eachlaser is represented by a different color. The intensity measurementsrange from 0 to 255 (y-axis), and the distances from the centroid ofthe laser spot range from 0 to 20 (x-axis). As an example, the bluelaser’s spot reaches its peak intense approximately 6 pixels from itscenter.

For efficiency, we utilize two passes to process the camera im-age. In a first coarse pass, we examine every nth pixel in the cameraimage and all pixels greater than a pre-set intensity threshold con-tinue to the second pass. In the second pass, a generous windowaround each seed pixel is examined. We collect all nearby pixelsabove the threshold and extract the largest connected component.The centroid of this component is set as the laser spot position.We then compute the histogram of pixel intensities shown in Fig-ure 2. The final task is to match the detected histogram to the li-brary of known lasers. We do this matching by calculating the sumof squared differences between the detected histogram and the his-togram of each each known laser. We then assign the laser ID asthe minimum of these sums.

It is important to note that the apparent laser intensity varies spa-tially for each laser due to a number of additional variables, includ-ing: distance from laser to screen, distance from screen to camera,and camera vignetting. We can easily normalize for these varia-tions. First, we normalize for the camera position by averaging allof the intensity data for all of the lasers collected at each of the cal-ibration grid points, and normalize the input by dividing it by thespatial average. We use barycentric coordinates and interpolationto normalize laser points between calibration grid locations. Cam-era position normalization is especially crucial when the camera isplaced at an extreme angle to the screen, and thus experiences sig-nificant perspective distortion.

Just as the camera position relative to the screen is important, thelaser spots also exhibit perspective distortion when they are used atextreme angle to the screen. We can similarly normalize for thisdistortion, assuming that the intensity calibration data for each laseris collected from a position near that laser’s use location.

2.4 Temporal Coherence and Simultaneous Use

For the tests presented in this paper and companion video we do notleverage temporal coherence in our detection. Because the cameraimage and laser output contain some noise, the quality of the per-sonality measurements and identification would benefit from aver-

2

Online Submission ID: 0

Figure 3: A diagram of the testing space. ’X’s are marked in arcs atintervals of 5’ from the center of the projection surface, separated by22.5◦ arcs. Five locations (A through E) are marked on the diagramfor reference.

aging the output of multiple frames known to be the same laser fromthe tracking algorithm in Section 2.1.

When multiple lasers are simultaneously detected on the screen,we can leverage this information to disambiguate lasers withsomewhat similar histogram personalities. We employ the Kuhn-Munkres algorithm to assign unique labels to all detected points;that is, no two lasers will be assigned the same ID, even if they bothselect the same ID as their first choice.

3 ACCURACY TESTING

We performed several tests to judge various notions of accuracy ofthe laser personality system. The tests were performed using sixlasers (given IDs 1-6) in the 19’ x 23’ space shown in Figure 3. Thespace was divided into a series of testing locations, marked with an’X’ in the diagram, at discretized 22.5◦ arcs with radii of 5’, 10’,and 15’ from the center of the projection surface.

Two sets of calibration data were collected. For the first cali-bration (referred to from now on as calibration C1), all six laserswere calibrated from the 15’ mark along the line perpendicular tothe projection surface (point A in Figure 3). For the second set ofdata (calibration C2), all six lasers were calibrated simultaneouslyfrom different points along the 10’ arc (point B resides on this arc).The accuracy of a laser was defined as the percentage of frames inwhich the laser was identified correctly out of all detection frames.These results are presented in Table 1.

The minimum number of frames considered for each test is re-ported in Table 1 along the bottom row. Five accuracy tests wererun in total, with users stationary, walking in an arc, walking alonga straight line, walking along a path, and using all lasers simulta-neously. For the tests, we count the number of times the laser iscorrectly identified as the primary choice, and the number of timesthe correct laser is the secondary choice (the laser intensity his-togram with the next smallest sum of squared differences). Thesetwo values comprise the two columns in Table 1 for each test, withthe primary ID on the left and the secondary on the right.

3.1 Description of TestsThe first four tests all used calibration data C1, while the final testused calibration data C2.

Stationary Test The purpose of this test was to judge how welllasers could be matched to the calibrated data. Data was recorded

from the position at which the calibration C1 was collected, positionA, and the tester did not move from that location. Each laser wasshone on the screen for at least 521 frames (roughly 17.75 seconds),making sure to point at both the middle and each of the four cornersof the screen. Each laser was correctly identified in at least 90% ofthe frames taken, and each laser was either identified correctly or asthe secondary laser at least 96% of the time.

Arc Walking Test The purpose of this test was to judge howmuch side-to-side movement affected the overall accuracy of theidentification system, because the shape of the laser spot is skewedwhen seen from an angle, changing the laser’s personality. Datawas recorded while users walked along the 10’ arc of data collectionpoints, passing right-to-left through point B. The laser was kept asclose to the center of the screen as possible to avoid affecting thelaser spot in additional ways. Each of these tests lasted at least347 frames (roughly 11.5 seconds). Four of the lasers performedwell (with at least 93% accuracy), while lasers 3 and 4 maintained70% accuracy. Each laser was identified as either its first or secondchoice laser ID in at least 93% of the frames, except laser 3.

Straight Line Walking Test The purpose of this test was tojudge how much distance from the screen affected the overall accu-racy of the system. Changing the distance to the projection surfacechanges the radius and intensity of the laser spot, and thus affectsthe histogram of the laser’s personality. For this test, the user stoodas far from the screen as possible (approximately 20’) and walkedforward to the 5’ mark, perpendicular to the screen. The laser wasonce again kept as close as possible to the center of the screen. Eachof these tests lasted at least 230 frames (roughly 7 seconds). The re-sults for this test were significantly poorer than other tests, as threeof them scored below 60% (laser 3 scored below 40%).

Path Walking Test The purpose of this test was to provide anoverall averaging of the previous two tests, and to mimic move-ment expected by users in real-world environments. The path takenfollowed the letters marking room positions in Figure 3 from posi-tion A to B, C, D, E, and back to A. The laser was again kept asclose as possible to the center of the screen. Each of the tests lastedat least 645 frames (roughly 19.5 seconds). In general, the lasersperformed as expected, an average somewhere between the arc andthe straight line tests. Only laser 1 fell below 60% accuracy, whilenone of the other lasers beside laser 1 had more than 15% failure tobe the first or second choice histogram personality.

All Lasers Simultaneous Test The purpose of the final test wasthe assess how accurately the laser identification system performedwhen several lasers were on the screen at once. In this test, usersstood at each of 6 marks along the 10’ arc (the positions from whichcalibration C2 data was collected) and all shone lasers at the screensimultaneously. The laser spots followed a path that touched allfour corners as well as the middle of the projection surface. Twolasers, 3 and 6, were confused with one another in approximately15% of the frames, but each of the other 4 lasers were at least 99%accurate.

3.2 Results and Discussion

Overall, our laser identification system is effective and useful for avariety of applications. It is overwhelmingly effective when lasersremain in the general area from which their calibration data is col-lected while the system is in use. It is rare that a laser is mislabeledin this instance. Many multi-user applications do not require usersmove around the room while the system is in use, such as edu-cation applications in a classroom environment, or games playedcollectively by an audience. However, when moving from placeto place, the shape of the laser spot can change dramatically, andso the personality can as well. As is to be expected, this is mostextreme when moving a different distance from the projection sur-face, as the spot’s intensity signature changes and its radius growsand shrinks dramatically, making histogram comparison consider-

3

Online Submission ID: 0

Laser Single Position Arc Movement Line Movement Walking Path All Lasers1 100.00% 100.00% 96.54% 98.27% 53.88% 74.14% 51.13% 68.49% 99.79% 100.00%2 100.00% 100.00% 100.00% 100.00% 90.43% 95.21% 78.65% 86.25% 99.07% 100.00%3 95.85% 100.00% 70.96% 73.48% 35.14% 48.65% 60.50% 96.10% 83.49% 98.75%4 92.79% 100.00% 77.40% 100.00% 80.41% 100.00% 83.02% 100.00% 99.61% 99.78%5 99.67% 99.67% 100.00% 100.00% 57.99% 92.57% 82.95% 94.26% 99.80% 99.92%6 90.33% 96.03% 93.89% 95.91% 72.56% 79.70% 91.81% 94.86% 84.08% 99.90%

Min. Frames 521 347 230 645 968

Table 1: Results from accuracy tests. Five tests were performed in all, and for each test and for each laser two percentages are reported: thepercentage of frames in which it was correctly identified as its primary ID (left column), and the percentage of frames in which it was identifiedas the secondary ID (right column). The minimum number of frames collected for each test is reported along the bottom row.

ably more difficult.These tests bring to light two significant shortcomings of our

identification system. The first is that position is important whencomparing a laser spot to a given set of laser personalities, espe-cially distance from the projection surface. The farther one movesfrom the spot of calibration, the less accurate the laser identificationbecomes. However, even in tests where the distance to the screenchanged, most of the lasers performed well. The system’s secondshortcoming is that a laser’s personality changes over the time ofits use. Battery-powered laser pointers change the range of theiroutput intensities as their batteries drain and the lasers themselveswarm up. Therefore, it is often the case that, over the course ofits use, a laser’s personality will shift away from the data collectedduring the calibration phase. Both of these shortcomings can be ad-dressed by the idea of continuous calibration, discussed in Section5.1.

4 APPLICATIONS

We have implemented a series of five applications that effectivelytake advantage of multi-user interfaces for visualization, education,and problem-solving. The common thread among each of the ap-plications is its use of input from several different users to achievea common goal, e.g. exploration of a data visualization or solving apuzzle. Our applications are a multi-user painting program, a puz-zle solving program, a terrain and hydrography data visualizationtool, a graph visualization tool, and an infrastructure map visual-ization tool. In each application, laser points are identified usingtheir personalities, as described in section 2.

4.1 Goal-Oriented Group-Based Problem SolvingAn aim of our multi-user interface is creating an environment thatsimplifies goal-oriented group-based problem solving. To demon-strate this idea, we present two applications that use the group dy-namic to collectively reach a goal. The first is our paint program,which allows any number of users to paint on the screen using thelaser pointer as a brush, seen in Figure 4. Colors are selected froma tablet of choices on the right side of the screen, and each usercan select his own. The ID of the laser that has selected a color isdisplayed in the color button, telling the user which color he hascurrently selected. Each pointer is identified when it is detected onthe painting surface portion of the screen, and the stroke gesturemade by the pointer creates a curve along the trajectory of the ges-ture in the laser pointer’s chosen color. There are also buttons onthe interface to increase and decrease line width, clear the paintingsurface, and undo the last action performed by a particular laser.Identifiable lasers are important for the paint application, as it al-lows each user to manipulate the same canvas while each drawingin his own color, making collaborative images easy to create.

To demonstrate a goal-oriented user interface, we present a puz-zle solving program, which reads in an image and breaks it up into n equally-sized rectangular textured tiles, randomizes them, anddisplays the new order on the screen, seen in Figure 5. The common

Figure 4: A visualization of the multi-user paint program demonstrat-ing two-user collaboration. Notice that laser #1 has currently beenassigned the color white and is filling in the cloud, while laser #2 hasrecently selected the “undo” function which removed its color assign-ment.

goal of the users of this application is the reassemble the originalimage by clicking and dragging the tiles. Each tile consists of a cen-tral space (the majority of the tile) which displays that tile’s portionof the image, and a faded border. The user must hover the laserpointer over a tile for approximately a second before it is selected,which is indicated by the faded border being filled in with the user’slaser’s assigned color (each laser pointer is assigned its own uniquecolor as it enters the system). When two tiles the system knows tobe adjacent are placed in the proper order, the adjacent borders fillin, completing that portion of the image.

In our informal play with this application we found that usershighly enjoyed the game and used verbal communication in addi-tion to the joint editing to more quickly solve the puzzle. Userscommented that each program’s use of the laser pointer as an in-teraction device is intuitive, and users in general became comfort-able with the interface with very little instruction. However, wedid observe some trouble-makers who sneakily and intentionallydisassembled the puzzle. Since the application can identify laserpointers, it can track the productivity and positive contribution ofeach user. If specific users repeatedly make destructive actions, thisuser’s privilege in the system can be decreased and their actionsignored.

4.2 Hydrography VisualizationData manipulation in a multi-user environment can be a powerfulteaching tool. To this end, we have implemented a visualization

4

Online Submission ID: 0

Figure 5: A visualization of the puzzle application, in which five users(each represented by his own color) attempt to solve a 7x5 piecepuzzle. The five images represent five points in time during the solv-ing of the puzzle. Each border between two tiles that should not beadjacent is greyed out (top left image), but once a tile is placed nextto its proper neighbor, the border fills in (middle image). Notice howthe group has worked to assemble various pieces of the puzzle indi-vidually before finally moving them to the correct areas of the board.

tool for terrain hydrography analysis, seen in Figure 6. The goalof the application is to allow for the exploration of terrain hydrog-raphy data by multiple users through a laser pointer interface. Theapplication consists of a data view and a graphical user interfaceside-by-side, in which modes are selected in the GUI through dwell-selection. In addition to selection of modes, the laser pointers arealso used to interact directly with the 3D terrain data and cameraview.

The application takes as input a height field (a scalar field ofheight values arranged on a regular grid), which represents the ter-rain to be visualized. The terrain is drawn in the data view sec-tion of the display, color coded according to scalar value (height,in this case) and normalized to fit the range of maximum to mini-mum values of the data. The user selects a mode using buttons onthe right side of the screen. These modes include camera positionmodes (Rotate, Translate, Zoom), visualization modes (Watershedand Channel Network), and interaction modes (Add Rain, LowerLand, Raise Land). Because each user can have any mode activeat any given point in time, multiple updates to the data and visu-alization can occur simultaneously. When a button is selected, theapplication identifies the laser pointer that made the selection andassociates it with the corresponding mode. For example, if laser #3selects the Rotate mode, a small “3” appears on the Rotate buttonand laser #3 can now rotate the camera through a click-and-drag

Figure 6: A visualization of our terrain hydrography exploration tool,which takes in as input a height field and visualizes it. Notice thechannel highlighted by the top left image, and water flow down thechannel in the top right and middle images. While user #1 adds rainto the terrain, user #3 begins to raise the land, damming the channel(lower left image), until the hydrography information has been up-dated and the water must flow around the newly formed dam (bottomright image).

motion in the data view window.Upon read in of the data (and every time it is altered by one of

the interaction modes), the hydrography information of the terrainis determined using both a variation of the least-cost flow routingmethod by Metz, et al. [9], and the flow accumulation method pre-sented by O’Callaghan and Mark [11]. The least-cost flow routingis fast enough in its implementation to allow the hydrography in-formation to be updated in interactive time (approximately 15 to 20FPS). This is important, as interaction with the data and the cameraview should not impede exploration of the data by other users.

The camera position modes allow users to rotate, translate, andzoom the camera, changing the view of the data, as described above.Watershed and Channel Network modes both visualize hydrogra-phy information on the terrain surface. While in watershed mode, alaser point detected on the surface of the data highlights the wa-tershed of the corresponding space on the data. When ChannelNetwork mode is active, the channels of the terrain are visualized.Finally, interaction modes allow the users to manipulate the data.Raise and Lower Terrain modes allow the user to raise and lowerthe surface data by selecting a space on the data. The data is manip-ulated according to a radial Gaussian filter, centered at the space se-lected by the laser. This, in essence, grows hills and craters. WhenAdd Rain mode is selected, the user can select a space on the ter-

5

Online Submission ID: 0

Figure 7: A visualization of our graph exploration application, in whicha database with connectivity information is read in, displayed, and ex-plored by users. In this instance, the data used is a taxonomic treeof animals. In the top row, the node “Mammals” is circled and ex-panded, visualizing the nodes from the next level down the sub-tree.The third level of this sub-tree is visualized in the center image (withpictures of each of the animals found in the database). In the bottomleft image, you can see one user dragging a node while another usercollapses one level of the tree, with the result of this operation seenin the bottom right image.

rain, and rain drops fall onto the surface in a radial pattern centeredat the selected space. The rain then flows, according to the internalhydrography information, down hill and off the edge of the terrain.These modes are demonstrated in Figure 6.

The multi-user nature of the application allows some users to bemanipulating the data itself, while others explore the repercussionsof this manipulation (by creating rain drops and watching wherethey flow and highlighting various watersheds along the path of theflowing water), all while others manipulate the camera, perhaps fol-lowing the path of the water as it flows between mountains and offthe edge of the terrain. Users have commented informally that theability to manipulate the surface data while watching rainwater flowacross the terrain was particularly useful, as it demonstrated howmuch hydrography can change with slight alterations to the terrain.

4.3 Spatial Network VisualizationIn addition to data visualization and manipulation, data explorationin a group setting is an important application of our multi-user laserinterface. To demonstrate this, we present a graph visualization toolthat allows users to explore data organized by nodes and edges, seenin Figure 7. Our application takes as input node and connectivityinformation. Laser points which dwell on a node for a period oftime “grab” the node, indicated by the laser trail turning green, andcan drag the node to a new location. The graph will rearrange itself

based on a mass-spring simulation, minimizing the energy in thesystem. If there is a notion of “expansion” that can be applied to thegraph (such as with a tree) then the interface supports expansion andcollapse operations. Our example dataset is a taxonomic databaseof animals, organized in a tree structure, so expansion in inherentin the data. To expand a node (such as the “Mammals” node, inour example), a circle motion is made around the node, and thenext level of the subtree is exposed. To collapse a node (whichreverses the expand operation), a squiggle or cross-out gesture ismade. These operations can be seen in Figure 7.

Our final application is a visualization, exploration, and edit-ing tool for infrastructure data shown in Figure 8. The data areorganized as a spatial graph of interconnected nodes and arcs inseveral different infrastructure systems: electric power, telecom-munications, transportation, etc. The tool is used to visualize thecomplex network along with vulnerabilities during hurricane andflooding scenarios (e.g., where are ambulances re-routed if a localhospital is flooded).

Representatives from the different infrastructure systems canmeet in the Emergency Response Organization control room anddiscuss interdependencies between the networks. It is importantthat the interface be exible, highly interactive, and intuitive with avery minimal learning curve, since in-depth training of these inter-faces will not be practical in disaster response situations. The largeformat projection system with laser pen interface is particularly ap-plicable to this situation. Tracking the identities of the users andtheir input to the system is also critical.

Users may select and edit nodes and edges of the data, and eachindividual user’s edits are saved, not only for logging purposes, butto save the state of each user, as similar to the hydrography visual-ization tool. This application demonstrates the potential advantagesof a multi-user system in a real-world, fast-paced, problem-solvingenvironment.

5 CONCLUSION

Multi-user interaction on large-scale displays is a powerful collab-orative tool with several applications. In this paper, we have pre-sented a method for identifying off-the-shelf laser pointers in aninexpensive and simple manner by calibrating each laser pointer’sIR spillover’s intensity histogram, called the laser’s personality, al-lowing for closest-neighbor matching of data points. The systemrequires only a calibration step to set up, and once it is completemultiple users can interact with interfaces in a large-scale environ-ment. Applications tailored to multi-user collaborate problem solv-ing efforts are presented in this paper that take advantage of oursystem’s ability to identify laser points to explore data, manipulatedata, and solve problems in a group environment. Our method isinexpensive, accurate, simple, and scalable to large screen displays.

5.1 Limitations and Future Work

While the system works well when users do not change how far theyare from the project surface (sufficient for many applications), thereare times when this is not enough. One clear extension to this workis the introduction of continuous calibration. First, we will omitthe special calibration step, and instead generate initial calibrationfrom early incidental usage, and continue to dynamically update alaser’s data as it is used. As new data is added to the system, olderdata will be replaced, and so the calibration data will change as thelasers’ personalities do. This continuous update of the calibrationdata will allow for less confusion between laser spots as the laserpoints warm up and change from continuous usage.

Our applications will also be improved. One such improve-ment would be a move to a multiple-screen multiple-camera for-mat. Laser strokes will need to be “passed off” accurately from onecamera’s view to the other’s, and each viewing system will need to

6

Online Submission ID: 0

Figure 8: A visualization of our infrastructure data exploration and editing tool that demonstrates two users working interacting with the map’sdata. The GUI on the left allows users to select which nodes are visible, as well as select actions to take (bottom left corner) between point,select, pan, and draw. Selecting a node enlarges it to display the data found within, while panning moves the camera around the map. Drawallows the user to make annotations to the visualization, for the benefit of other users.

be calibrated independently (unless dynamic calibration is imple-mented). We will also extend our applications to more fully takeadvantage of identifiable laser points. For instance, we will create auser feedback system for the puzzle application, in which users whoare contributing will be rewarded while those who are destructivemay face a penalty to their piece-moving privileges. This wouldmimic real-world collaborative reward systems.

REFERENCES

[1] B. A. Ahlborn, D. Thompson, O. Kreylos, B. Hamann, and O. G.Staadt. A practical system for laser pointer interaction on large dis-plays. In Proceedings of the ACM symposium on Virtual reality soft-ware and technology, VRST ’05, pages 106–109, New York, NY,USA, 2005. ACM.

[2] L. Angelini, M. Caon, S. Carrino, O. A. Khaled, and E. Mugellini.Multi-user pointing and gesture interaction for large screen using in-frared emitters and accelerometers. In HCI (2)’11, pages 185–193,2011.

[3] X. Bi, Y. Shi, X. Chen, and P. Xiang. uPen: laser-based, personalized,multi-user interaction on large displays. In MULTIMEDIA ’05: Pro-ceedings of the 13th annual ACM international conference on Multi-media, pages 1049–1050, New York, NY, USA, 2005. ACM Press.

[4] K. Cheng and K. Pulo. Direct interaction with large-scale display sys-tems using infrared laser tracking devices. In Proceedings of the Asia-Pacific symposium on Information visualisation - Volume 24, APVis’03, pages 67–74, Darlinghurst, Australia, Australia, 2003. AustralianComputer Society, Inc.

[5] J. Davis and X. Chen. Lumipoint: Multi-user laser-based interactionon large tiled displays. Displays, 23, 2000.

[6] F. de la O Chavez, F. Fernandez de Vega, G. Olague, andJ. Llano Montero. An independent and non-intrusive laser pointerenvironment control device system. In Proceedings of the 5th interna-tional conference on Pervasive services, ICPS ’08, pages 37–46, NewYork, NY, USA, 2008. ACM.

[7] R. Kalman. A new approach to linear filtering and prediction prob-lems. Journal of Basic Engineering, 82(1):35–45, 1960.

[8] H. W. Kuhn. The hungarian method for the assignment problem.Naval Research Logistics Quarterly, 2:83–97, 1955.

[9] M. Metz, H. Mitasova, and R. S. Harmon. Efficient extractionof drainage networks from massive, radar-based elevation modelswith least cost path search. Hydrology and Earth System Sciences,15(2):667–678, 2011.

[10] J. Munkres. Algorithms for the assignment and transportation prob-lems. Journal of the Society for Industrial and Applied Mathematics,5(1):32–38, March 1957.

[11] J. F. O’Callaghan and D. M. Mark. The extraction of drainage net-works from digital elevation data. Computer Vision, Graphics, andImage Processing, 28(3):323 – 344, 1984.

[12] J.-Y. Oh and W. Stuerzlinger. Laser pointers as collaborative pointingdevices. In Graphics Interface, 2002.

[13] D. R. Olsen and T. S. Nielsen. Laser pointer interaction. In CHI, pages17–22, 2001.

[14] A. Pavlovych and W. Stuerzlinger. Laser pointers as interaction de-vices for collaborative pervasive computing. Technical report, Centerfor Parallel Computing, 2004.

[15] A. Pavlovych and W. Stuerzlinger. Effect of screen configuration andinteraction devices in shared display groupware. In Proceeding ofthe 3rd ACM international workshop on Human-centered computing,HCC ’08, pages 49–56, New York, NY, USA, 2008. ACM.

[16] Y. Qin, Y. Shi, H. Jiang, and C. Yu. Structured laser pointer: enablingwrist-rolling movements as a new interactive dimension. In Proceed-ings of the International Conference on Advanced Visual Interfaces,AVI ’10, pages 163–166, New York, NY, USA, 2010. ACM.

[17] B. Shizuki, T. Hisamatsu, S. Takahashi, and J. Tanaka. Laser pointerinteraction techniques using peripheral areas of screens. In Proceed-ings of the working conference on Advanced visual interfaces, AVI’06, pages 95–98, New York, NY, USA, 2006. ACM.

[18] F. Vogt, J. Wong, S. Fels, and D. Cavens. Tracking multiple laserpointers for large screen interaction. In Ext. Abstracts UIST, pages95–96, 2003.

[19] F. Vogt, J. Wong, B. A. Po, R. Argue, S. S. Fels, and K. S. Booth. Ex-ploring collaboration with group pointer interaction. In Proceedingsof the Computer Graphics International, pages 636–639, Washington,DC, USA, 2004. IEEE Computer Society.

[20] J. Weaver. Kuhn-munkres (hungarian) algorithm in c++, 2010.

7