an efficient bit vector approach to semantics-based machine perception in resource-constrained...
DESCRIPTION
Presentation at ISWC 2012, Boston, USA, Nov. 14, 2012TRANSCRIPT
![Page 1: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/1.jpg)
1
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled ComputingWright State University, Dayton, OH, USA
![Page 2: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/2.jpg)
2
The Patient of the FutureMIT Technology Review, 2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
![Page 3: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/3.jpg)
3
… and do it efficiently and at scale
What if we could automate this sense making ability?
![Page 4: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/4.jpg)
4
Sensing is a key enabler of the Internet of Things
BUT, how do we make sense of the resulting avalanche
of sensor data?
50 Billion Things by 2020 (Cisco)
![Page 5: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/5.jpg)
5
People are good at making sense of sensory input
What can we learn from cognitive models of perception?• The key ingredient is prior knowledge
![Page 6: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/6.jpg)
6* based on Neisser’s cognitive model of perception
ObserveProperty
PerceiveFeature
Explanation
Discrimination
1
2
Perception Cycle*
Translating low-level signals into high-level knowledge
Focusing attention on those aspects of the environment that provide useful information
Prior Knowledge
![Page 7: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/7.jpg)
7
To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web
![Page 8: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/8.jpg)
8
The Web is becoming a global knowledge base
![Page 9: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/9.jpg)
9
Prior knowledge on the Web
W3C Semantic Sensor Network (SSN)
OntologyBi-partite Graph
![Page 10: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/10.jpg)
10
Prior knowledge on the Web
W3C Semantic Sensor Network (SSN)
OntologyBi-partite Graph
![Page 11: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/11.jpg)
11
ObserveProperty
PerceiveFeature
Explanation1
Translating low-level signals into high-level knowledge
Explanation
Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building
![Page 12: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/12.jpg)
12
Explanation
Inference to the best explanation• In general, explanation is an abductive
problem; and hard to compute
Finding the sweet spot between abduction and OWL• Single-feature assumption* enables use
of OWL-DL deductive reasoner
* An explanation must be a single feature which accounts for
all observed properties
Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building
![Page 13: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/13.jpg)
13
Explanation
Explanatory Feature: a feature that explains the set of observed properties
ExplanatoryFeature ≡ ssn:isPropertyOf∃ —.{p1} … ssn:isPropertyOf⊓ ⊓ ∃ —.{pn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Observed Property Explanatory Feature
![Page 14: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/14.jpg)
14
Discrimination is the act of finding those properties that, if observed, would help distinguish between multiple explanatory features
ObserveProperty
PerceiveFeature
Explanation
Discrimination2
Focusing attention on those aspects of the environment that provide useful information
Discrimination
![Page 15: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/15.jpg)
15
Discrimination
Expected Property: would be explained by every explanatory feature
ExpectedProperty ≡ ssn:isPropertyOf.{f∃ 1} … ssn:isPropertyOf.{f⊓ ⊓ ∃ n}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Expected Property Explanatory Feature
![Page 16: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/16.jpg)
16
Discrimination
Not Applicable Property: would not be explained by any explanatory featureNotApplicableProperty ≡ ¬ ssn:isPropertyOf.{f∃ 1} … ¬ ssn:isPropertyOf.{f⊓ ⊓ ∃ n}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Not Applicable Property Explanatory Feature
![Page 17: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/17.jpg)
17
Discrimination
Discriminating Property: is neither expected nor not-applicable
DiscriminatingProperty ≡ ¬ExpectedProperty ¬NotApplicableProperty⊓
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Discriminating Property
Explanatory Feature
![Page 18: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/18.jpg)
18
Through physical monitoring and analysis, our cellphones could act as an early warning system to detect serious health conditions
canary in a coal mine
Our Motivation
kHealth: knowledge-enabled healthcare
![Page 19: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/19.jpg)
19
How do we implement machine perception efficiently on aresource-constrained device?
Use of OWL reasoner is resource intensive (especially on resource-constrained devices), in terms of both memory and time
• Runs out of resources with prior knowledge >> 15 nodes
• Asymptotic complexity: O(n3)
![Page 20: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/20.jpg)
20
intelligence at the edge
Approach 1: Send all sensor observations to the cloud for processing
Approach 2: downscale semantic processing so that each device is capable of machine perception
![Page 21: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/21.jpg)
21
Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge and execute semantic reasoning
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
![Page 22: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/22.jpg)
22
lift
lower
Lifting and lowering knowledge
Translate prior knowledge, observations, and explanations between SW and bit vector representation
![Page 23: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/23.jpg)
23
bp 1
cs 0
pa 1
HN HM PE
bp 1 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 1
HN HM PE
1 1 1
1 1 0
AND =>
AND
1 1 1
ObservedProperty Prior Knowledge
PreviousExplanatory Feature
CurrentExplanatory Feature
=>
Explanation: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties.
![Page 24: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/24.jpg)
24
bp 1
cs 0
pa 1
HN HM PE
bp 0 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 0
HN HM PE
0 1 0AND => 0 1 0
ObservedProperty Prior Knowledge
PreviousExplanatory Feature
CurrentExplanatory Feature
=
Discrimination: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those features that discriminate between the explanatory features
bp 0
cs 0
pa 0
Discriminating
Property
1 1 0
… expected?
0 0 0… not-applicable?ZERO Bit Vector
0 1 0
=
=> FALSE
=> FALSE
1
Is the propertydiscriminating?
![Page 25: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/25.jpg)
25
O(n3) < x < O(n4) O(n)
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes• Time reduced from minutes to milliseconds• Complexity growth reduced from polynomial to
linear
Evaluation on a mobile device
![Page 26: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/26.jpg)
26
2 Prior knowledge is the key to perceptionUsing SW technologies, machine perception can be formalized and integrated with prior knowledge on the Web
3 Intelligence at the edgeBy downscaling semantic inference, machine
perception can execute efficiently on resource-constrained devices
3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory signals into high-level knowledge useful for decision making
![Page 27: An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices](https://reader035.vdocuments.net/reader035/viewer/2022081602/554b9857b4c9052d448b4b4e/html5/thumbnails/27.jpg)
27
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
0101100011010011110010101100011011011010110001101001111001010110001101011000110100111
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
Thank you.
Cory Henson, Krishnaprasad Thirunarayan, Amit ShethKno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing