Bridging the Last Gap: LedTX -Optical Data Transmission ofSensor Data for Web-Services

Philipp M. SchollESS, TU Darmstadt.Hochschulstr. 1064283 Darmstadtscholl@ess.tu-darmstadt.de

Nagihan KucukyildizESS, TU Darmstadt.Hochschulstr. 1064283 Darmstadtnagi@ess.tu-darmstadt.de

Kristof Van LaerhovenESS, TU Darmstadt.Hochschulstr. 1064283 Darmstadtkristof@ess.tu-darmstadt.de

Permission to make digital or hard copies of part or all of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for third-partycomponents of this work must be honored. For all other uses, contact theowner/author(s). Copyright is held by the author/owner(s).UbiComp’13 Adjunct, September 8–12, 2013, Zurich, Switzerland.ACM 978-1-4503-2215-7/13/09.

http://dx.doi.org/10.1145/2494091.2494104

AbstractData transmission from small-scale data loggers such ashuman activity recognition sensors is an inherent system’sdesign challenge. Interfaces based on USB or Bluetoothstill require platform-dependent code on the retrievalcomputer system, and therefore require a largemaintenance effort. In this paper, we present LedTX, asystem that is able to transmit wirelessly through LEDsand the camera included in most user’s hardware. Thissystem runs completely in modern browsers and presents auni- directional, platform-independent communicationchannel. We illustrate this system on the UbiLighter, aninstrumented lighter that tracks ones smoking behaviour.

Author Keywordsoptical, wireless, data transmission

ACM Classification KeywordsB.4.1 [Hardware]: Input/Output and DataCommunications; J.3 [Computer Applications]: Health

IntroductionLong-term monitoring of human subjects has found anumber of interesting applications in recent years.Electronic diaries, via ambulatory assessment, for patientsundergoing psychological treatments [8], gesture controlfor mobile phones [1] or a multitude of other applications

Session: Poster, Demo, & Video Presentations UbiComp’13, September 8–12, 2013, Zurich, Switzerland

47

[7] are just a few examples of the monitoring needs. All ofthose systems need to communicate with a PC applicationthat in turn analyses the recorded data to give additionalinformation about the monitored activities. Where delayedaccess to this data is a possibility, a USB connection (orother wired system) is typically used instead of a wirelessconnection. Both wired and wireless transmissionshowever share the problem of requiring some software onthe PC that retrieves this data from the device. Adesigner of such a monitoring system then faces thechallenge of making sure that the retrieval software workscross-platform on multiple Operating Systems, an effortnot to be underestimated, especially for successiveiterations of research prototypes.

In this paper we introduce a system that works completelyin an HTML5-compliant webbrowser, which allows totransmit data from LEDs via a webcam, nowadays oftenfound in laptops, smart-Phones and other computingaccessories. We show the applicability of this approach onthe example of the UbiLighter, an instrumented lighterthat records the smoking behaviour of its users. The datawill be sent to a webbrowser to generate visual feedbackof the user’s smoking behaviour. This drastically reducesthe amount of time spent in developing a retrieval andanalysis software and bridges the last gap to retrieve thegathered data in a platform-independent way. Furthermorethis enables an almost instant feedback for people usingan easily deployable system for in-field, long-term studies.

Related WorkData transmission with visible light and a commoditycamera has provided the means for remote identificationand localization of sensors attached to factory machineryin [3]. The camera integrated into a tablet computer hasbeen used to provide a “see-through” image of the scene

augmented with the sensor values, in which sensors havebeen identified through active optical markers. Theoptical identifier has been encoded by a Hammingsequence transmitted at 50Hz.

The µPart configuration tool, presented in [2], follows thesame motivation as this paper - computer software forcustom devices present a large effort to be maintained.The configuration of the µPart, a miniatureuni-directional wireless sensor node, was achieved byexploiting the included light sensor to transmit data via ablinking area on a LCD screen. Reconfiguration couldtherefore be done with a single platform-independentprogram running in a webbrowser.

Bi-directional optical data-transfer using only LEDs hasbeen shown in [5]. An RS-232 like protocol has beenemployed. More recently data transmission via themodulation of headlights in indoor environments has beenshown to achieve data-rates of up to 100Mbps [6].

System Design and ImplementationIn this section we describe the system design fortransmitting data from the UbiLighter. The data we wantto transmit are timestamps of the times when the lighterhas been used to light up a cigarette. This is achieved byblinking the two included LEDs on the UbiLighter andanalysing the video stream recorded by a webcam on aPC. The LEDs blinking patterns contain the differentiallycompressed timestamps.

This compression is necessary because of the lowachievable throughput of this solution. The timestampsare transmitted differentially, i.e., after a primingtimestamp has been send, only the difference to the lasttransmitted timestamp in multiples of 6-minutes istransmitted. This 6-minute interval is assumed to be the

Session: Poster, Demo, & Video Presentations UbiComp’13, September 8–12, 2013, Zurich, Switzerland

48

minimum difference between smoking incidents andencoded as an unsigned 8-Bit integer.

An overview of the communication setup can be seen inFig. 1. A commodity webcam records a video stream ofthe LEDs embedded in the UbiLighter. Since this simplexoptical channel will be subject to transmission errors, wefollowed [3] to guard the transmission with theforward-error correcting Hamming code. The recordedvideo frames are split into images which contain only thecolor of the two LEDs, and a transmitted bit is detectedwith the help of a Blob detector. If there is a blob of thespecific color a ‘1’ has been transmitted and its absencewill detected as a ‘0’. These signals are the passed into ademodulator for the Hamming encoded data. All of thesecomponents are implemented in Javascript and besides acompliant webbrowser, no additional software needs to beinstalled on the user’s PC.

ColorFilter

BlobDetector

SignalDemodulator

RGBimage

greenimage

redimage

green LEDsignal

red LEDsignal

bitstream

Figure 1: Data reception pipeline, the webcam records alive-stream of the red/green LED on the UbiLighter, which ispassed into a color filter and split into a red/green picture. Ablob detector finds the LED in each video-frame and passesthat information into the signal demodulator which returns thebitstream.

Theoretical Transmission BandwidthThe achievable transmission bandwidth largely depends onthe capture frame rate of the used webcam. Typicallyframe rates of λ = 30fps are possible, but the samplingprocess is limited by the Nyquist-Shannon theorem to asample rate of λ

2 . The number of usable LEDs is another

important factor, in our case k = 2 bits can be transferredin parallel. Furthermore the rate of the error coding hasalso an influence on the amount of transmittable bits, wedeliberately chose a (7,4)-hamming code, which results inthe rate R = 0.571. Putting these factors together ourtheoretical transmission bandwidth can be calculated asR ∗ k ∗ λ

2 = 17.13bps. Alternatively a CRC-8 could beused, which would only allow to check for an error but hasthe advantage of a higher encoding rate (R = 8

9 = 0.88).A CRC-8 based transmission would allow for atransmission rate of up to 26.4bps.

Javascript-Based Data ReceptionThe implementation of the data reception pipeline (Fig.1) is based on Javascript and the getUserMedia [4]standard for accessing the webcam. It is executable onmost current webbrowsers without any additional softwareinstallation.

The data transmission is started by accessing the receptionwebsite and pushing the button on the UbiLighter threetimes in quick succession. The UbiLighter is then facedtowards the webcam and a synchronization message issent repetitively for a 10-second interval. This messagecontains identification information and the number of tobe transmitted bytes. Afterwards the data/timestamps aretransmitted. A successful transmission is acknowledged bythe user pushing the button. Bits are transmitted byturning the LEDs on and off (amplitude modulation) witha frequency of half the sampling rate of the webcam.

Each video frame on the reception part will first becolor-filtered, in order to get a camera image of eachLED. A blob detector, which extracts the area with mostco-located pixels, is then used to retrieve the status of theLED. If no blob can be detected a ‘0’ has been read, if ablob was detected a ‘1’ has been transmitted. This

Session: Poster, Demo, & Video Presentations UbiComp’13, September 8–12, 2013, Zurich, Switzerland

49

Figure 2: The system in use, the UbiLighter is sending data to the Javascript reception pipeline via the LEDs and webcam.

information will enter the signal demodulation, where thebitstream is checked with either the Hamming code or theCRC-8 check. The decoded information has then beensuccessfully transmitted from the device to the PC and isdisplayed to the user on the visited webpage. After thatthis data can be uploaded to some remote server forfurther analysis or directly analysed in the Javascriptprogram.

ConclusionWe have presented LedTX1, a system that enablesuni-directional wireless data transfer from monitoringsensor devices. Albeit our solution does not offer a greatbandwidth, it offers an installation-free, platform-independent way of data transmission that can be easilyconnected to webservices, by running completely in awebbrowser. For future work the bandwidth can beincreased by using additional LEDs, or encode informationin colors generated by RGB LEDs or miniature displays.

1Code available at http://www.github.com/pscholl/ledTX.

AcknowledgementsThe authors would like to acknowledge funding by theDFG/GRK1362 - Cooperative, Adaptive and ResponsiveMonitoring in Mixed Mode Environments.

References[1] Amma, C., Georgi, M., and Schultz, T. Airwriting: Hands-Free Mobile

Text Input by Spotting and Continuous Recognition of 3d-SpaceHandwriting with Inertial Sensors. ISWC (June 2012).

[2] Beigl, M., Krohn, A., Riedel, T., Zimmer, T., Decker, C., and Isomura, M.The uPart experience: Building a wireless sensor network. In IPSN (2006).

[3] Berning, M., Riedel, T., Karl, D., Schandinat, F., Beigl, M., and Fantana,N. Augmented Service in the Factory of the Future. In INSS, vol. 54(2012).

[4] Burnett, D. C., Bergkvist, A., Jennings, C., and Narayanan, A. MediaCapture and Streams, 2011.

[5] Dietz, P., Yerazunis, W., and Leigh, D. Very Low-Cost Sensing andCommunication Using Bidirectional LEDs. In UbiComp 2003: UbiquitousComputing (2003).

[6] Komine, T., and Nakagawa, M. Fundamental Analysis for Visible-LightCommunication System using LED Lights. Consumer Electronics, IEEETransactions (2004).

[7] Lockhart, J. W., Pulickal, T., and Weiss, G. M. Applications of mobileactivity recognition. UbiComp (2012).

[8] Shiffman, S., Stone, A. a., and Hufford, M. R. Ecological MomentaryAssessment. Annual Review of Clinical Psychology 4, 1 (Apr. 2008), 1–32.

Session: Poster, Demo, & Video Presentations UbiComp’13, September 8–12, 2013, Zurich, Switzerland

50