magcontroller: a system for touchless mobile devices...
TRANSCRIPT
MagController: A System for Touchless MobileDevices Control Using the Magnetic Field
Heba AbdelnasserComputer and Sys. Eng. Department
Alexandria UniversityAlexandria, Egypt
Ayman KhalafallahComputer and Sys. Eng. Department
Alexandria UniversityAlexandria, Egypt
Moustafa YousefDepartment of Computer Science and Eng.
Alexandria University and E-JUSTAlexandria, Egypt
Abstract—A lot of improvements have been proposed in mobilecomputing devices, especially the wearable devices, in recentyears. However, the input techniques remain a key challenge.One of the main methods for interacting with small screen devicestoday is the inflexible and error-prone touch-screen. In this paper,we propose MagController: a magnetic-based input technique thatoffers users wireless, unpowered, and high precision input forcomputing devices.
The core idea is to leverage the triaxial magnetometer em-bedded in standard devices to accurately recognize a magnetdifferent control actions. MagController also provides a numberof modules that allow it to cope with background magneticnoise, heterogeneous devices, different magnet shapes, sizes, andstrengths, as well as changes in magnet orientations.
Our implementation of MagController on Android phones withextensive evaluation in wide range of environmental conditionsscenarios shows that it can detect the magnet orientation withaccuracy of more than 98%. In addition, MagController canrecognize the magnetic actions with accuracy of 97% in differentmagnet’s orientations relative to the device. This accuracy isrobust with different phones and magnets, highlighting MagCon-troller’s promise as an input technique for computing devices.
I. INTRODUCTION
Nowadays, touch-screen input is the primary interactionmodality for smart devices which requires a display, no matterhow small the device is. Touchscreen interfaces for smalldisplay devices have several limitations: limited screen sizecausing user inconvenience and entry errors, occlusion of thekeys by the user’s fingers, and security issues that can leakusers’ passwords and other critical information [1]. Further-more, the traditional touchscreen does not work in severalcases, e.g. when it is raining, the user is wearing gloves, or theuser’s hand is dirty. More recently, smart glasses with limitedor no touch input are starting to emerge commercially. Theprimary input to these systems is an audio input. But speechcontrol is still associated with disadvantages, e.g. reliabilityproblems in noisy environments and it can’t happen in private.
There are many other techniques that have been proposedto replace the touch-screen keyboard, motivated by freeing theuser from specialized powered devices and leveraging naturaland contextually relevant human movements. The Xbox Kinectis an example of a commercially available input sensor thatenables gesture-based interaction using depth sensing andcomputer vision. The commercial success of these kinds of
devices has spurred interest in developing new user interfacesthat remove the need for a traditional input techniques andopened the door for IoT systems [2]. Other techniques includevision-based [3], acoustic-based [4], and wireless signals-based [5, 6]. Nevertheless, these techniques suffer from oc-clusion, interference from other moving humans, surroundingnoises, and/or high energy consumption.
In this work, we present MagController, a magnetic-basedinput technique that offers users wireless, unpowered, and highprecision input for mobile devices. The magnetic field is amore promising technique as it is transparent to human motionand the magnetometer consumes order of magnitude lessenergy compared, e.g., to the camera or WiFi. The user utilizesan off-the-shelf magnet and performs contextually relevanthuman movements actions to control different applications.MagController leverages the triaxial magnetometer embeddedin the standard devices to accurately recognize the differentmagnet control actions.
There are several challenges, however, that need to beaddressed to realize MagController. These include: (a) iden-tifying different actions to be able to control objects in3D, (b) how to separate the target magnet effect from theother environmental magnetic sources, where the embeddedmagnetometer inside the mobile device can only provide theaggregated magnetic field of the Earth geomagnetic field, thetarget magnet magnetic field, and the background magneticfield of the other sources in the environment. How to make thesystem independent from the magnet’s strength, shape, size,and the deployment orientation. To address these challenges,MagController defines six magnetic actions to control objectsin 3D space. It also leverages different signal processingtechniques that allow it be independent from the environmentaleffect, and to detect and recognize the different magneticactions. In addition, the system uses a preamble that isperformed by the user to make the system independent fromthe orientation and the used magnet’s specifications.
The main contributions presented in this paper are:• The design of MagController, a machine learning
magnetic-based input technique that offers users wireless,unpowered, and high precision input for computing de-vices.
• MagController provides a number of modules that allow
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
978-1-5386-3227-7/18/$31.00 ©2018 IEEE 300
it to cope with background magnetic noise, heterogeneousdevices, different magnet shapes, sizes, and strengths, aswell as changes in magnet orientations.
• Evaluating MagController performance, implemented onAndroid devices, with different attributes in differentsituations using different hardware.
The remainder of this paper is organized as follows: Sec-tion II discusses the related work. Section III presents thenecessary background to understand MagController. Section Vpresents the architecture and operation of MagController.Details of our implementation and extensive evaluation areshared in Section VI. Finally, Section VII conclude the paper.
II. RELATED WORK
The human body is transparent to static and low-frequencymagnetic fields, which makes magnetic tracking a promisingtechnique in a number of human-computer interaction applica-tions. A number of systems have been proposed for the contin-uous tracking of a magnet location based on nonlinear modelsof the relation between the magnetic field and distance [7].However, such systems usually require multiple external high-accuracy magnets and/or assume that the magnet diameter isvery small relative to the distance between the magnet andthe magnetometer sensor. They also restrict the magnet oncertain orientation. MagController, on the other hand, can beoperate with magnets with any size and does not restrict themagnet’s orientation. Similarly, MagPen [8] uses a speciallydesigned hardware stylus that combines a magnet with capaci-tive sensing to extend the functionality of the current styluses.Abracadabra [9] is a magnetically-driven input approach forsmart watches. It senses the movement of a magnet placed onthe fingertip to determine its relative position from the watch.This information is used to control a cursor, make selections,or issue basic gestures. Abracadabra, though, depends ona high-accuracy external magnetometer sensor to avoid thenoises inside the mobile device. Similarly, Nenya [10] employsan external magnetometer to track two specific motions ofring: twisting around the user finger and sliding along thefinger. Both Abracadabra and Nenya have been evaluated witha specific phone and magnet. The MagiWrite [11] systemuses a machine learning approach to detect specific patternsof the magnetometer motion. Specifically, MagiWrite detectsthe 10 digits. However, these systems require using specialexternal magnetometer sensors, that can provide more robustreadings and are not affected by several kinds of environmentalmagnetic noises, such as hard-iron and soft-iron noises [12].They also put restrictions on the magnet’s orientation whilemoving.
MagController, in contrast, based on a previous work [13],uses standard phone sensors, which have higher noise, and canwork with any phone or magnet without any restrictions onmagnet’s size, shape, strength, or orientation as we show andquantify in the evaluation section.
to-downto-up to-right to-left to-sky to-ground
Fig. 1: MagController actions.
III. BACKGROUND
To better understand MagController, we provide a briefoverview on geomagnetic field basics and magnetometer op-eration on smartphones.
The triaxial magnetometer is a standard component of mostcurrent mobile devices. The sensor detects the cumulativemagnetic field generated by multiple sources in the environ-ment on each of its three sensor axes. The total magneticfield vector at any location (B) is superposition of: the Earth’sgeomagnetic field (BEarth), the magnetic field from the en-vironment, which includes background magnetic field fromthe ferromagnetic materials (Bbackground), and the magneticfield strength from the MagController’s magnet (Bmagnet ).Therefore: B = BEarth +Bbackground +Bmagnet .
The magnetic field measured by the phone’s magnetometerat the same location, Bp, can be obtained from B while takinginto account the phone’s three rotation angles: yaw (ψ), pitch(θ ), and roll (φ ):
Bp = Rx(φ)Ry(θ)Rz(ψ)B (1)
where Rx(φ), Ry(θ), and Rz(ψ) are corresponding rotationmatrices. However, due to environment noise, hard-iron (H)and soft-iron (S) effects impact phone readings of B [12] asfollows:
Bp = S Rx(φ)Ry(θ)Rz(ψ)B+H (2)
Where the hard-iron effect H is an offset vector and the soft-iron effect S is a matrix. The effect of such noise on themagnetometer readings is to make the locus of the magneticreadings when the phone is rotated at a fixed point an ellipsoidrather than a sphere.
IV. MagController ACTIONS INVESTIGATION
There are many basic questions that come in mind includ-ing: how to identify the different magnetic actions? and willthe relative different situations between the device and magnetaffect the translated actions?
Different Actions Identifying: MagController identifies sixdifferent actions to be able to control an object in an 3Dspace (Figure 1). The six actions are: to-right, to-left, to-up, to-down, to-sky, and to-ground. Figure 2 (the six actions patterns)shows that the differet actions have different patterns. we usea machine learning technique to recognize MagController’sdifferent actions.
Magnet Orientation Discretization: Figure 3 shows plotsof ‘to-right’ action in different angles. All angles are in
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
301
-150
-100
-50
0
50
100
150
200
250
300
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(a) ‘to-right’ action.
-600
-400
-200
0
200
400
600
800
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(b) ‘to-left’ action.
-1500
-1000
-500
0
500
1000
1500
2000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(c) ‘to-sky’ action.
-400
-300
-200
-100
0
100
200
300
400
500
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(d) ‘to-up’ action.
-1000
-500
0
500
1000
1500
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(e) ‘to-down’ action.
-600
-400
-200
0
200
400
600
800
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(f) ‘to-ground’ action.
Fig. 2: MagController actions patterns.
-800
-600
-400
-200
0
200
400
600
800
1000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(a) Angle = -10.
-1000
-500
0
500
1000
1500
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(b) Angle = 0
-1500
-1000
-500
0
500
1000
1500
2000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(c) Angle = 10
-200
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(d) Angle = 80.
-500
0
500
1000
1500
2000
2500
3000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(e) Angle = 90
-500
0
500
1000
1500
2000
2500
3000
3500
4000
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(f) Angle = 100
Fig. 3: ‘to-right’ action with different angles in xy-plane. Theindicated angles are the angle between the magnet’s north andthe sensor’s x-axis in different situations.
the x-y plane. The figure shows that the close angles havesimilar patterns. Based on this observation and to simplify themagnet’s orientation determination process, we discretize thecontinuous 3D space into 6 orientations, shown in Figure 4:magnet’s north in the positive and negative directions of thedevice’s x-axis, y-axis, and z-axis. In total, MagControllerincludes 6 actions in 6 orientations, resulting for 36 differentpatterns. The details of how MagController detects, segments,and recognizes these patterns are explained in the next Section.
V. THE MagController SYSTEM
The system architecture is shown in Figure 5. MagCon-troller contains two main phases: offline phase and onlinephase. The magnetic field readings first enter silence removalstep in both phases that seeks to make the system independentfrom different surrounding environmental variables. In theoffline phase, the system is trained once in lifetime withthe defined magnetic actions. The trained actions include apreamble (action to define the magnet orientation) and thesix controlling actions defined by the system. In the onlinephase, MagController first determines the magnet’s orientationrelative to the device. Then the performed actions are detected,segmented, and finally recognized.
X
Y
Z
N
N
N
N
N
N
(0,0,1) (0,1,0) (1,0,0)(0,0,1) (0,1,0)
(-1,0,0) (0,-1,0) (0,0,-1)
Fig. 4: Magnet’s different orientations relative to the device.
Silence
Removal
Action
Trained Classifier
Fig. 5: MagController architecture.
A. Silence Removal
The silence removal module makes the system independentfrom the environmental noises. Since the Earth’s magneticfield (BEarth) varies at different locations, and the backgroundmagnetic field strength (Bbackground) and the hard-iron offset(H component) dynamically change, it is important for Mag-Controller to compensate for these changes to obtain a robustmagnetic action identification. To do that, whenever the sys-tem’s environment is changed, we remove the effect of thesethree varying sources of magnetic field by collecting sampleswithout the existence of the MagController’s magnet for afew seconds (5 sec). Subtracting the average of this “silenceperiod” from the magnetometer’s readings leads to magneticactions patterns that are independent from the system locationon Earth as well as the background magnetic interference.More formally, from Equation 2, the measured magnetic fieldafter silence removal becomes:
B′p = Bp−Bsilence = SRx(φ)Ry(θ)Rz(ψ)Bmagnet (3)
where
Bsilence = S Rx(φ)Ry(θ)Rz(ψ)(BEarth +Bbackground)+H (4)
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
302
-40
-30
-20
-10
0
10
20
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(a) Car magnet pattern.
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(b) Car magnet pattern after nor-malization.
-300
-200
-100
0
100
200
300
400
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(c) Ring magnet pattern.
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70 80 90 100
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
(d) Ring magnet pattern afternormalization.
Fig. 6: Different magnets with different strengths patterns.The user performs ‘to-up’ action example while the magnet’ssouth is moving on the direction of the device embeddedmagnetometer’s z-axis.
B. Offline phase
This section discusses how MagController is trainedthrough the offline phase process. In the offline phase, Mag-Controller builds a once-in-lifetime training data for thepreamble in different magnet orientations and for the differentcontrol actions. At the beginning, the input signal is manuallysegmented, and then a normalization process is applied in thesegmented action. Finally, the segmented actions are entered toa classifier for training. In the rest of this section the differentoffline modules are explained in detail.
1) Normalization: Figures 6a and 6c show the ‘to-up’action pattern performed with two different magnets withdifferent strengths. As shown in the figures, the differentmagnets have similar pattern (across the different magnetaxises). However, the patterns have different peaks magnitudesthat are in proportion to the magnets’ strengths. MagControllerapplies normalization to be independent of different magnetswith different strengths.
The normalization is applied by obtaining the maximumvalue of the x-, y-, z-axis and the magnitude of the total streamin the action pattern. Then the four streams are divided by themaximum value. Therefore:
Bpnorm =K.Bmagnet
K.max(Bx,By,Bz,BT )(5)
where K = S Rx(φ)Ry(θ)Rz(ψ)Equation 5 shows that the soft-iron noise effect is removed
by applying the normalization. To this end, the hard- and soft-iron noises are removed in the silence removal and normaliza-tion steps. Therefor we do not need the standard magnetometercalibration process required in normal to remove hard- andsoft-noise. The signal is also normalized over time using
-3000
-2000
-1000
0
1000
2000
3000
0 50 100 150 200 250
Mag
net
ic f
iled
(M
icro
Tes
la)
Sample
BxByBzBT
var var var
Fig. 7: Preamble detection. Var is the variance of all samplesin the 4 streams.
over/down sampling, so that all the segmented actions willhave the same number of samples.
The next step is to feed the segmented normalized annotatedactions to a classifier to be trained.
2) Classifier Training: As shown in Section IV, one actionhas different patterns in different relative orientations betweenthe magnet and the device. MagController determines the mag-net orientation first using a special preamble (Section V-C1),then it recognizes the performed actions.
As shown in the previous sections, the silence removaland normalization processes make the system independent ofthe environmental noises, the used device, and the magnet’sstrength and orientation. So the training data collection processis done only once in the lifetime using a master magnet anda master device.
One classifier is trained for the preamble in the six prede-fined orientations and six classifiers are trained for the controlactions, one classifier for each relative magnet orientation.The used features are the magnetic field three axises and totalmagnitude.
C. Online phase
In the online phase, at the beginning the user performsthe preamble to determine the magnet’s orientation, then thecommunication channel starts between the user and the device.The entered actions are then automatically segmented and fedto the action recognition module to be classified.
1) Preamble Detection: The preamble contains three mo-tions: the user moves the magnet along the three x,y, and zaxises near to the device in a sequential order. In the Evalua-tion section we show that using such preamble, MagControllercan achieve a high accuracy for orientation determination. Thefirst step in online phase is to detect that the preamble isperformed by the user. In Figure 7, the magnetic field signalvariance of the preamble duration is high surrounded by twodurations of low variance. Using this observation, we apply asimple thresholding method on the signal variance to detect thepreamble, which is then recognized to determine the magnet’sorientation.
2) Magnet’s Orientation Determination: The orientationdetermination module works on determining the orientation ofthe magnet relative to the mobile phone. After the preambleis detected, the system uses the trained classifier in the offlinephase to tag the communication session between the magnet
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
303
-2000
-1500
-1000
-500
0
500
1000
1500
2000
2500
0 10 20 30 40 50 60 70 80
Sample
BxByBzB
Mag
net
ic f
iled
(M
icro
Tes
la)
T
(a) Peak detection.
-2000
-1500
-1000
-500
0
500
1000
1500
2000
2500
0 10 20 30 40 50 60 70 80
Sample
BxByBzBT
Mag
net
ic f
iled
(M
icro
Tes
la)
Action start finding
window (W) Action End finding
window (W)
(b) Limits determination.
Fig. 8: Action segmentation. ‘To-down’ action example whilethe magnet’s south is moving on the direction of the deviceembedded magnetometer’s z-axis.
and the device with one orientation of the predefined sixorientations. All the detected control actions in this sessionafter that are tagged with the recognized orientation.
3) Action Detection and Segmentation: After the userperforms the preamble, she performs the control actions tocontrol the target application. The system starts to search forthe performed control actions after detecting the preamble.Detection and segmentation processes are applied on the inputdata to extract the action segment from it. This processincludes mainly two steps: peak detection and limits detection(shown in Figure 8). The two steps are applied on the totalmagnetic field stream |BT |= |Bx +By +Bz| after applying thesilence removal. Peak detection process detects the positivepeaks p that are greater than a threshold (θ ) in the input streamas BT is always positive. Then the action segment’s start andend limits are determined around the detected peak.
To make action detection adapt to dynamic environments,devices, and magnets, we use a dynamic threshold rather thana fixed one. The threshold should be determined to includethe true peaks only and exclude the others false peaks. Thethreshold (θ ) is taken as a percentage (τ) of the average mag-nitude range of the performed calibration preamble’s peaks,as the peaks magnitudes depend on the magnet’s strength. Wequantify the effect of (τ) on performance with the differenttechniques in the evaluation section.
4) Action Recognition: Once the action boundary is deter-mined, the action pattern is entered to the action recognitionmodule to be recognized using the trained classifier in theoffline phase. One of the trained six classifiers is used in thismodule based on the determined magnet relative orientation.
VI. EVALUATION
This section presents our evaluation methodology of theMagController system. We start with describing the exper-imental setup, followed by studying the effect of differentparameters on the preamble and control actions detection andrecognition accuracies.
A. Experimental Setup
We extensively evaluate MagController in different envi-ronments, over different days and times, and with different
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 0
0.1
0.2
0.3
0.4
0.5
Acc
ura
cy
FP
/ F
N
Standard deviation threshold
Acc. FP FN
Fig. 9: Preamble detection ac-curacy.
0
0.2
0.4
0.6
0.8
1
One
Two
Three
Acc
ura
cy
Number of preamble motions
Fig. 10: Orientation recognitionaccuracy.
surrounding humans activities. We experiment with three An-droid phones: a Sony Z2, a Samsung S7, and a Nexus 4. Weutilized different magnets with different shapes and strengths:a ring magnet (865 µT ), a cube magnet (971 µT ), and amagnet attached to a car toy (114 µT ).
The experiments have been conducted in several situations,in the different six magnet’s relative orientations as describedin Section IV, using different magnets and devices, anddifferent environmental conditions with existing of severalelectromagnetic filed sources (TVs, laptops, PCs, and mobilephones with moving people) in the same room; at least oneelectromagnetic field source in distance of less than one meterfrom the user existed while performing each experiment. Foreach situation, each action has been performed 50 timesfor training and 50 times for testing. The training data issegmented and labeled manually.
B. Preamble Detection Accuracy
This experiment measures the performance of MagCon-troller’s first step in the online phase, which is detecting theuser performed preamble to start the communication channelbetween her and the device. Figure 9 shows the effect ofusing different standard deviation thresholds on the preambledetection accuracy. We observe that, as expected, increasingthe threshold reduces the false positive rate while increasingthe false negative rate. From the figure, threshold value of 900leads to accuracy of 100% for the preamble detection.
C. Magnet’s Orientation Determination Accuracy
We evaluate the magnet’s orientation detection module whenthe preamble consists of one, two, and three motions (themagnet is moved along one, two, or three axises of the device),as shown in Figure 10. The figure shows that, as expected,the magnet’s orientation determination with three motionspreamble has better accuracy (98.33%).
D. Action Detection
Figure 11 shows the effect of the detection threshold ratio(τ) on action detection accuracy, false positive rate, and falsenegative rate. We observe that, as expected, increasing thedetection threshold ratio (τ) reduces the false positive ratewhile increasing the false negative rate. As shown in the figure,τ value of 14% leads to accuracy of 98% for action detection.
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
304
0.5
0.6
0.7
0.8
0.9
1
1% 12%13%14%15%16%17%0
0.1
0.2
0.3
0.4
0.5
Acc
ura
cy
FP
/ F
N
Ratio threshold ( )τ
Acc. FP FN
Fig. 11: Action detection Ac-curacy using different detec-tion thresholds.
0
0.2
0.4
0.6
0.8
1
(1,0,0)
(0,1,0)
(0,0,1)
(-1,0,0)
(0,-1,0)
(0,0,-1)
Overall
Acc
ura
cy
Magnet and device relative angle
Fig. 12: MagController’s ac-tion recognition accuracy indifferent orientations.
0
0.2
0.4
0.6
0.8
1
Actions Preamble
Act
ion
Rec
og
nit
ion
Acc
ura
cy
Without normalization
With normalization
(a) Using the same devices.
0
0.2
0.4
0.6
0.8
1
cube/ring cube/toy ring/cube
Act
ion
Rec
og
nit
ion
Acc
ura
cy
Without normalization
With normalization
(b) Using different magnets. Eachpin label has this form: trainingmagnet/testing magnet.
Fig. 13: Effect of signal normalization on the action recogni-tion accuracy.
E. Effect of Different Parameters on Action Recognition
This section quantifies the impact of the different parameterson the action recognition accuracy of our system.
1) Effect of Different Magnet Orientations: Figure 12shows the predefined six different orientation of the magnetrelative to the device. The figure shows an overall accuracy ofabout 97% at all six relative magnet’s orientations.
2) Effect of Normalization: Figure 13 shows the effectof the signal normalization on the recognition accuracy. Theused magnet and device in Figure 13a are the same for thetraining and testing datasets. It shows that applying normal-ization causes small improvement effect on the accuracies ofaction recognition and preamble recognition. On the otherhand, the used magnets in Figure 13b are different for thetraining and testing datasets. The figure shows the actionrecognition accuracies of different combinations of magnetsused in training/testing: car/ring, cube/toy, and ring/cude. Itshows that applying normalization causes a big improvementeffect on the accuracies. This is intuitive as different magnetswith different strengths cause similar patterns with differentpeaks magnitudes as discussed in Section V-B1.
VII. CONCLUSION
Through this paper, MagController system is presented forproviding wireless, unpowered, and high precision input formobile devices. Additionally, it works with heterogeneous de-vices and magnets with different shapes, sizes, and strengths.MagController depends on the device’s embedded magne-tometer sensor readings. It leverages calibration steps and apreamble to make the system dynamically adapted with the
environmental and hardware changes. It employs a two stepsclassifier: first the system recognize the preamble to determinethe orientation, then recognize the performed control actions.
Evaluation of MagController on different Android devicesin various realistic scenarios shows that it can achieve actiondetection accuracy of about 98% and control action recogni-tion accuracy of 97%. It can detect a preamble is performedwith accuracy reaches to 100% to start the communicationsession between the user and device. In addition, it canrecognize the preamble with accuracy of about 98%.
REFERENCES
[1] H. Wang, T. T.-T. Lai, and R. Roy Choudhury, “Mole:Motion leaks through smartwatch sensors,” in MobileComputing and Networking. ACM, 2015.
[2] M. Elhamshary and M. Youssef, “Towards ubiquitousindoor spatial awareness on a worldwide scale,” TheSIGSPATIAL Special, 2017.
[3] D. Balzarotti, M. Cova, and G. Vigna, “Clearshot: Eaves-dropping on keyboard input from video,” in Security andPrivacy SP 2008. IEEE Symposium on, 2008.
[4] T. Zhu, Q. Ma, S. Zhang, and Y. Liu, “Context-freeattacks using keyboard acoustic emanations,” in ACMSIGSAC Conference on Computer and CommunicationsSecurity, 2014.
[5] K. Ali, A. X. Liu, W. Wang, and M. Shahzad, “Keystrokerecognition using wifi signals,” in Mobile Computing andNetworking. ACM, 2015.
[6] H. Abdelnasser, M. Youssef, and K. A. Harras, “Wigest:A ubiquitous wifi-based gesture recognition system,” inINFOCOM. IEEE, 2015.
[7] X. Han, H. Seki, and M. Hikizu, “Wearable handwritinginput device using magnetic field,” in SICE. IEEE, 2007.
[8] S. Hwang, A. Bianchi, M. Ahn, and K. Wohn, “Magpen:magnetically driven pen interactions on and around con-ventional smartphones,” in Human-computer interactionwith mobile devices and services. ACM, 2013.
[9] C. Harrison and S. E. Hudson, “Abracadabra: wireless,high-precision, and unpowered finger input for very smallmobile devices,” ACM symposium on User interfacesoftware and technology, 2009.
[10] D. Ashbrook, P. Baudisch, and S. White, “Nenya: subtleand eyes-free mobile input with a magnetically-trackedfinger ring,” SIGCHI, 2011.
[11] H. Ketabdar, M. Roshandel, and K. A. Yuksel, “Magi-write: towards touchless digit entry using 3d spacearound mobile devices,” Human computer interactionwith mobile devices and services, 2010.
[12] T. Ozyagcilar, “Calibrating an ecompass in the pres-ence of hard and soft-iron interference,” Freescale Semi-conductor Ltd, 2012.
[13] H. Abdelnasser, M. Youssef, and K. A. Harras, “Mag-board: Magnetic-based ubiquitous homomorphic off-the-shelf keyboard,” in SECON. IEEE, 2016.
PerIoT'18 - Second International Workshop on Mobile and Pervasive Internet of Things
305