Indoor Localization Accuracy Estimationfrom Fingerprint Data
Artyom Nikitin 1
Christos Laoudias2 Georgios Chatzimilioudis2
Panagiotis Karras3 Demetrios Zeinalipour-Yazti2,4
1Skoltech, 143026 Moscow, Russia
2University of Cyprus, 1678 Nicosia, Cyprus
3Aalborg University, 9220 Aalborg, Denmark
4Max-Planck-Institut fur Informatik, 66123 Saarbrucken, Germany
Outline
1 Motivation
2 Background
3 Our Solution
4 Experiments
5 Conclusions
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 2/31
Motivation: Indoor Localization
Indoor Navigation Services spread widely.
Applications: localization, marketing, warehouse optimization,guides, games, etc.
Different sources of data: cellular, Wi-Fi, BT, magnetic fieldof the Earth, light, sound, etc.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 3/31
Motivation: Indoor Localization
Indoor Navigation Services spread widely.
Applications: localization, marketing, warehouse optimization,guides, games, etc.
Different sources of data: cellular, Wi-Fi, BT, magnetic fieldof the Earth, light, sound, etc.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 3/31
Motivation: Accuracy Estimation
Important to estimate the accuracy of localization.
Online: important for the end-user (Google Maps, CONE).
Offline: important for the service provider.
Provide quality guarantees.Perform decision making.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 4/31
Motivation: Accuracy Estimation
Important to estimate the accuracy of localization.
Online: important for the end-user (Google Maps, CONE).
Offline: important for the service provider.
Provide quality guarantees.Perform decision making.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 4/31
Motivation: Accuracy Estimation
Important to estimate the accuracy of localization.
Online: important for the end-user (Google Maps, CONE).
Offline: important for the service provider.Provide quality guarantees.Perform decision making.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 4/31
Outline
1 Motivation
2 Background
3 Our Solution
4 Experiments
5 Conclusions
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 5/31
Background: Localization Approaches
Modeling
Known APs positions
Known data model, e.g., Path Loss: L = 10n log10(d) + C
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 6/31
Background: Localization Approaches
Modeling + Fingerprinting
Known APs positions
Known data model, e.g., Path Loss: L = 10n log10(d) + C
Known pre-collected fingerprints (position + readings)
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 6/31
Background: Localization Approaches
Fingerprinting
Known APs positions
Known data model, e.g., Path Loss: L = 10n log10(d) + C
Known pre-collected fingerprints (position + readings)
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 6/31
Background: Accuracy Estimation
Existing solutions
Heuristics: e.g., fingerprint density, cluster & merge, etc.
+ Do not require models− No theoretical guarantees
Theoretical: e.g., use Cramer-Rao Lower Bound (CRLB)
+ Provide theoretical guarantees− Model is required
Our goal:
+ No model required
+ Provide guarantees via CRLB
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 7/31
Background: Accuracy Estimation
Common theoretical approach for offline accuracy estimation:
1 Measurements are random, e.g., Gaussian.
2 From the known information estimate the likelihood, i.e., theprobability p(m|r) of measuring m at r.
3 From the likelihood calculate Cramer-Rao Lower Bound(CRLB) on the variance of any unbiased estimator of r.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 8/31
Background: Accuracy Estimation
Common theoretical approach for offline accuracy estimation:
1 Measurements are random, e.g., Gaussian.2 From the known information estimate the likelihood, i.e., the
probability p(m|r) of measuring m at r.
3 From the likelihood calculate Cramer-Rao Lower Bound(CRLB) on the variance of any unbiased estimator of r.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 8/31
Background: Accuracy Estimation
Common theoretical approach for offline accuracy estimation:
1 Measurements are random, e.g., Gaussian.2 From the known information estimate the likelihood, i.e., the
probability p(m|r) of measuring m at r.3 From the likelihood calculate Cramer-Rao Lower Bound
(CRLB) on the variance of any unbiased estimator of r.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 8/31
Background: Accuracy Estimation
Common theoretical approach for offline accuracy estimation:
1 Measurements are random, e.g., Gaussian.2 From the known information estimate the likelihood, i.e., the
probability p(m|r) of measuring m at r.3 From the likelihood calculate Cramer-Rao Lower Bound
(CRLB) on the variance of any unbiased estimator of r.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 8/31
Background: Accuracy Estimation
Common theoretical approach for offline accuracy estimation:
1 Measurements are random, e.g., Gaussian.2 From the known information estimate the likelihood, i.e., the
probability p(m|r) of measuring m at r.3 From the likelihood calculate Cramer-Rao Lower Bound
(CRLB) on the variance of any unbiased estimator of r.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 8/31
Background: Accuracy Estimation
How to find the likelihood?
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 9/31
Background: Accuracy Estimation
Modeling. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + CModel parameters, e.g., n = 2,C = 20 log10
4πλ (FSPL)
Position xAP of the APNoise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 10/31
Background: Accuracy Estimation
Modeling. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ (FSPL)
Position xAP of the AP
Noise
1 Predict using model
2 Estimate noise
3 Compare to measurements
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 10/31
Background: Accuracy Estimation
Modeling. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ (FSPL)
Position xAP of the AP
Noise
1 Predict using model
2 Estimate noise
3 Compare to measurements
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 10/31
Background: Accuracy Estimation
Modeling. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ (FSPL)
Position xAP of the AP
Noise
1 Predict using model
2 Estimate noise
3 Compare to measurements
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 10/31
Background: Accuracy Estimation
Modeling + Fingerprinting. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + CModel parameters, e.g., n = 2,C = 20 log10
4πλ
Position xAP of the APNoise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 11/31
Background: Accuracy Estimation
Modeling + Fingerprinting. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ
Position xAP of the AP
Noise
1 Assume parametric model
2 Get fingerprints
3 Estimate parameters
4 Estimate noise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 11/31
Background: Accuracy Estimation
Modeling + Fingerprinting. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + CModel parameters, e.g., n = 2,C = 20 log10
4πλ
Position xAP of the APNoise
1 Assume parametric model
2 Get fingerprints
3 Estimate parameters
4 Estimate noise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 11/31
Background: Accuracy Estimation
Modeling + Fingerprinting. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ
Position xAP of the AP
Noise
1 Assume parametric model
2 Get fingerprints
3 Estimate parameters
4 Estimate noise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 11/31
Background: Accuracy Estimation
Modeling + Fingerprinting. We know:
Model, e.g., Path Loss: L = 10n log10(|x − xAP |) + C
Model parameters, e.g., n = 2,C = 20 log104πλ
Position xAP of the AP
Noise
1 Assume parametric model
2 Get fingerprints
3 Estimate parameters
4 Estimate noise
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 11/31
Background: Accuracy Estimation
Pure Fingerprinting
No model provided.
Data is too complex, e.g., ambient magnetic field:vector field = direction + magnitude;predictable outdoors;perturbed indoors by metal constructions and electricalequipment.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 12/31
Background: Accuracy Estimation
Pure Fingerprinting
No model provided.Data is too complex, e.g., ambient magnetic field:
vector field = direction + magnitude;predictable outdoors;perturbed indoors by metal constructions and electricalequipment.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 12/31
Background: Accuracy Estimation
Pure Fingerprinting
No model provided.Data is too complex, e.g., ambient magnetic field:
vector field = direction + magnitude;predictable outdoors;perturbed indoors by metal constructions and electricalequipment.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 12/31
Outline
1 Motivation
2 Background
3 Our Solution
4 Experiments
5 Conclusions
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 13/31
Our solution: Goal
Pure fingerprinting approach
Arbitrary data sources
FM = {(ri ,mi ) : i = 1,N, ri ∈ Rdr , mi ∈ Rdm}mi - dm-dimensional vector of measurements at location ri .
Given the FM, assign to any location a navigability score.
Visualize navigability scores to assist INS deployer.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 14/31
Our solution: ACCES Framework
ACCES framework
1 Interpolation:FM + Gaussian Process Regression (GPR) ⇒ likelihood
2 CRLB:Likelihood + CRLB ⇒ lower bound on localization error
3 Lower bound on localization error ⇒ navigability score
Theoretical bound on localization errorAssume that behaves similar to real error
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 15/31
Our Solution: Interpolation
Gaussian Process Regression:
Input: fingerprint map of measurements
Output: Gaussian likelihood p(m|r) of measuring m at r(prediction + uncertainty)
Intuition:measurements are Gaussian random variablesspatial correlation: close are correlated, far are not
Properties:models arbitrary noisy datacaptures FM’s spatial sparsity
Nuances:parameters tuning is required (kernel, length scale, etc.)assume normality condition (does not directly work for NLOS)directly applicable only to scalar data ⇒ assume independencecomputationally expensive ⇒ clustering
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 16/31
Our Solution: Interpolation
Gaussian Process Regression:
Input: fingerprint map of measurements
Output: Gaussian likelihood p(m|r) of measuring m at r(prediction + uncertainty)
Intuition:measurements are Gaussian random variablesspatial correlation: close are correlated, far are not
Properties:models arbitrary noisy datacaptures FM’s spatial sparsity
Nuances:parameters tuning is required (kernel, length scale, etc.)assume normality condition (does not directly work for NLOS)directly applicable only to scalar data ⇒ assume independencecomputationally expensive ⇒ clustering
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 16/31
Our Solution: Interpolation
Gaussian Process Regression:
Input: fingerprint map of measurements
Output: Gaussian likelihood p(m|r) of measuring m at r(prediction + uncertainty)
Intuition:measurements are Gaussian random variablesspatial correlation: close are correlated, far are not
Properties:models arbitrary noisy datacaptures FM’s spatial sparsity
Nuances:parameters tuning is required (kernel, length scale, etc.)assume normality condition (does not directly work for NLOS)directly applicable only to scalar data ⇒ assume independencecomputationally expensive ⇒ clustering
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 16/31
Our Solution: Interpolation
Gaussian Process Regression (1-D example)
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 16/31
Our Solution: CRLB
Cramer-Rao Lower Bound:
Input: likelihood p(m|r) of measuring m at r
Output: smallest RMSE achievable by any unbiasedestimator of r
Intuition:likelihood carries information about the distributiondistribution does not vary locally ⇒ degradation
Properties:theoreticaleasily found for unbiased estimators
Nuances:an underestimation of the real error ⇒ we care aboutqualitative behavioranalytical representation depends on GPR parameters ⇒ weinvolve numerical methods
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 17/31
Our Solution: CRLB
Cramer-Rao Lower Bound:
Input: likelihood p(m|r) of measuring m at r
Output: smallest RMSE achievable by any unbiasedestimator of r
Intuition:likelihood carries information about the distributiondistribution does not vary locally ⇒ degradation
Properties:theoreticaleasily found for unbiased estimators
Nuances:an underestimation of the real error ⇒ we care aboutqualitative behavioranalytical representation depends on GPR parameters ⇒ weinvolve numerical methods
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 17/31
Our Solution: CRLB
Cramer-Rao Lower Bound:
Input: likelihood p(m|r) of measuring m at r
Output: smallest RMSE achievable by any unbiasedestimator of r
Intuition:likelihood carries information about the distributiondistribution does not vary locally ⇒ degradation
Properties:theoreticaleasily found for unbiased estimators
Nuances:an underestimation of the real error ⇒ we care aboutqualitative behavioranalytical representation depends on GPR parameters ⇒ weinvolve numerical methods
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 17/31
Our Solution: CRLB
CRLB: error of any unbiased location estimator is bounded as
RMSE ≥√tr(I−1(r)),
where I(r) is a Fisher Information Matrix :
I(r) = −E(∂2 log p(m|r)
∂ri∂rj
)
From GPR:m|r ∼ N (µ(r),Σ(r))
Thus,
I(r) =1
2
dm∑k=1
[(σ2k+µ2k)H(σ−2k )+H(µ2kσ
−2k )−2µkH(µkσ
−2k )+2H(log σk)
]
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 18/31
Our Solution: CRLB
CRLB: error of any unbiased location estimator is bounded as
RMSE ≥√tr(I−1(r)),
where I(r) is a Fisher Information Matrix :
I(r) = −E(∂2 log p(m|r)
∂ri∂rj
)From GPR:
m|r ∼ N (µ(r),Σ(r))
Thus,
I(r) =1
2
dm∑k=1
[(σ2k+µ2k)H(σ−2k )+H(µ2kσ
−2k )−2µkH(µkσ
−2k )+2H(log σk)
]
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 18/31
Outline
1 Motivation
2 Background
3 Our Solution
4 Experiments
5 Conclusions
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 19/31
Experiments: Data
UJIIndoorLoc-Mag database
8 corridors over 260 m2 lab
40,159 discrete captures
Magnetometer readings
Measurements along thecorridors ⇒ 1-D data
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 20/31
Experiments: Algorithms
Real accuracy: RMSE via WkNN
Naıve approach: Fingerprint Spatial Sparsity Indicator:
FSSI (r) = mini∈1,N
‖r − ri‖
considers only distance between measurements
ACCES: our solution
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 21/31
Experiments: Algorithms
Real accuracy: RMSE via WkNN
Naıve approach: Fingerprint Spatial Sparsity Indicator:
FSSI (r) = mini∈1,N
‖r − ri‖
considers only distance between measurements
ACCES: our solution
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 21/31
Experiments: Algorithms
Real accuracy: RMSE via WkNN
Naıve approach: Fingerprint Spatial Sparsity Indicator:
FSSI (r) = mini∈1,N
‖r − ri‖
considers only distance between measurements
ACCES: our solution
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 21/31
Experiments: Metrics
DQRelSim(X ,Y ) - behavior similarity of sequences X = Xi ,Y = Yi for 1-D case.
Construction:Difference Quotient ⇒ DQ(X ) and DQ(Y )DTW ⇒ optimally warped DQ(X )′ and DQ(Y )′ fromNormalization
Values:Similar: 1, if X = Y + constDissimilar: 0, if either X or Y is constantOpposite: −1, if X = −Y + const
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 22/31
Experiments: Settings
DQRelSim(ACCES ,RMSE ) vs DQRelSim(FSSI ,RMSE )
1 “Cut” scenario:Contiguous sequence of measurements is removed⇔ fingerprints were not collected
2 “Flat” scenario:Contiguous sequence of measurements is made constant⇔ low signal variability
3 “Sparse” scenario:Measurements are removed uniformly⇔ different frequency of fingerprint collection
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 23/31
Experiments: Evaluation
“Cut” scenario: magnetic field magnitude
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 24/31
Experiments: Evaluation
“Cut” scenario: similarity of RMSE, ACCES, FSSI
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 24/31
Experiments: Evaluation
“Flat” scenario: magnetic field magnitude
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 25/31
Experiments: Evaluation
“Flat” scenario: similarity of RMSE, ACCES, FSSI
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 25/31
Experiments: Evaluation
“Sparse” scenario: behaviour of RMSE, ACCES
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 26/31
Outline
1 Motivation
2 Background
3 Our Solution
4 Experiments
5 Conclusions
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 27/31
Conclusions
Summary:
ACCES provides offline accuracy estimations and FMassessment.
Does not consider the origin of the data.
Applicable to pure fingerprinting.
Shows reasonable correspondence to the real localization errorbehaviour.
Future work:
Extensive experimental study with other data.
Comparison to online accuracy estimation algorithms.
Adding support for arbitrary models.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 28/31
Conclusions
Summary:
ACCES provides offline accuracy estimations and FMassessment.
Does not consider the origin of the data.
Applicable to pure fingerprinting.
Shows reasonable correspondence to the real localization errorbehaviour.
Future work:
Extensive experimental study with other data.
Comparison to online accuracy estimation algorithms.
Adding support for arbitrary models.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 28/31
Advertisement: Anyplace
Anyplace Indoor Information Service
Wi-Fi + IMU
Optimize data usage
3 awards
Crowdsource-based
Open-source
anyplace.cs.ucy.ac.cy
github.com/dmsl/anyplace
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 29/31
Advertisement: Anyplace
Anyplace Indoor Information Service
Wi-Fi + IMU
Optimize data usage
3 awards
Crowdsource-based
Open-source
anyplace.cs.ucy.ac.cy
github.com/dmsl/anyplace
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 29/31
Advertisement: Anyplace
Anyplace Indoor Information Service
Wi-Fi + IMU
Optimize data usage
3 awards
Crowdsource-based
Open-source
anyplace.cs.ucy.ac.cy
github.com/dmsl/anyplace
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 29/31
Advertisement: Anyplace
Anyplace Indoor Information Service
Wi-Fi + IMU
Optimize data usage
3 awards
Crowdsource-based
Open-source
anyplace.cs.ucy.ac.cy
github.com/dmsl/anyplace
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 29/31
Demo
Thank You!
Come to see our demoyesterday!
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 30/31
Conclusions
Summary:
ACCES provides offline accuracy estimations and FMassessment.
Does not consider the origin of the data.
Applicable to pure fingerprinting.
Shows reasonable correspondence to the real localization errorbehaviour.
Future work:
Extensive experimental study with other data.
Comparison to online accuracy estimation algorithms.
Adding support for arbitrary models.
IEEE MDM 2017 | Nikitin, Laoudias, Chatzimilioudis, Karras, Zeinalipour 31/31