smart cities and data analytics: challenges and opportunities
TRANSCRIPT
![Page 1: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/1.jpg)
Smart Cities and Data Analytics: Challenges and Opportunities
1
Payam BarnaghiInstitute for Communication Systems (ICS)/5G Innovation Centre University of SurreyGuildford, United Kingdom
Workshop on Smart City: Applications and ServicesBudva, MontenegroOctober 2015
![Page 2: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/2.jpg)
2IBM Mainframe 360, source Wikipedia
![Page 3: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/3.jpg)
Apollo 11 Command Module (1965) had 64 kilobytes of memory operated at 0.043MHz.
An iPhone 5s has a CPU running at speeds of up to 1.3GHzand has 512MB to 1GB of memory
Cray-1 (1975) produced 80 million Floating point operations per second (FLOPS)10 years later, Cray-2 produced 1.9G FLOPS
An iPhone 5s produces 76.8 GFLOPS – nearly a thousand times more
Cray-2 used 200-kilowatt power
Source: Nick T., PhoneArena.com, 2014
![Page 4: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/4.jpg)
Computing Power
4
−Smaller size−More Powerful−More memory and more storage
−"Moore's law" over the history of computing, the number of transistors in a dense integrated circuit has doubled approximately every two years.
![Page 5: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/5.jpg)
Cyber-Physical-Social Data
5P. Barnaghi et al., "Digital Technology Adoption in the Smart Built Environment", IET Sector Technical Briefing, The Institution of Engineering and Technology (IET), I. Borthwick (editor), March 2015.
![Page 6: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/6.jpg)
Internet of Things: The story so far
RFID based solutions Wireless Sensor and
Actuator networks, solutions for
communication technologies,
energy efficiency, routing, …
Smart Devices/Web-enabled
Apps/Services, initial products,
vertical applications, early concepts and
demos, …
Motion sensor
Motion sensor
ECG sensor
Physical-Cyber-Social Systems, Linked-data,
semantics,More products, more
heterogeneity, solutions for control and
monitoring, …
Future: Cloud, Big (IoT) Data Analytics, Interoperability, Enhanced Cellular/Wireless
Com. for IoT, Real-world operational use-cases and
Industry and B2B services/applications,
more Standards… P. Barnaghi, A. Sheth, "Internet of Things: the story so far", IEEE IoT Newsletter, September
2014.
6
![Page 7: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/7.jpg)
7
“Each single data item is important.”
“Relying merely on data from sources that are unevenly distributed, without considering background information or social context, can lead to imbalanced interpretations and decisions.”?
![Page 8: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/8.jpg)
Data- Challenges
− Multi-modal and heterogeneous− Noisy and incomplete− Time and location dependent − Dynamic and varies in quality − Crowed sourced data can be unreliable − Requires (near-) real-time analysis− Privacy and security are important issues− Data can be biased- we need to know our data!
8
![Page 9: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/9.jpg)
Data Lifecycle
9Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment: Challenges and opportunities of data driven systems for building, community and city-scale applications, http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm
![Page 10: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/10.jpg)
10
“The ultimate goal is transforming the raw data to insights and actionable knowledge and/or creating effective representation forms for machines and also human users and creating automation.”
This usually requires data from multiple sources, (near-) real time analytics and visualisation and/or semantic representations.
![Page 11: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/11.jpg)
11
“Data will come from various source and from different platforms and various systems.”
This requires an ecosystem of IoT systems with several backend support components (e.g. pub/sub, storage, discovery, and access services). Semantic interoperability is also a key requirement.
![Page 12: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/12.jpg)
Device/Data interoperability
12The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.
![Page 13: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/13.jpg)
Search on the Internet/Web in the early days
1313
![Page 14: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/14.jpg)
Accessing IoT data
14
“ The internet/web norm (for now) is often to use an interface to search for the data; the search engines are usually information locators – return the link to the information; IoT data access is more opportunistic and context aware”.
The IoT requires context-aware and opportunistic push mechanism, dynamic device/resource associations and (software-defined) data routing networks.
![Page 15: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/15.jpg)
IoT environments are usually dynamic and (near-) real-time
15
Off-line Data analytics
Data analytics in dynamic environments
Image sources: ABC Australia and 2dolphins.com
![Page 16: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/16.jpg)
What type of problems we expect to solve using the IoT and data analytics solutions?
![Page 17: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/17.jpg)
17Source LAT Times, http://documents.latimes.com/la-2013/
A smart City exampleFuture cities: A view from 1998
![Page 18: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/18.jpg)
18Source: http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/
Source: wikipedia
Back to the Future: 2013
![Page 19: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/19.jpg)
Common problems
19Source: http://www.me.undp.org &
Guildford, Surrey
![Page 20: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/20.jpg)
20
![Page 21: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/21.jpg)
Applications and potentials
− Analysis of thousands of traffic, pollution, weather, congestion, public transport, waste and event sensory data to provide better transport and city management.
− Converting smart meter readings to information that can help prediction and balance of power consumption in a city.
− Monitoring elderly homes, personal and public healthcare applications.
− Event and incident analysis and prediction using (near) real-time data collected by citizen and device sensors.
− Turning social media data (e.g. Tweets) related to city issues into event and sentiment analysis.
− Any many more… 21
![Page 22: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/22.jpg)
EU FP7 CityPulse Project
22
![Page 23: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/23.jpg)
23
CityPulse Consortium
Industrial SIE (Austria,
Romania),ERIC
SME AI,
HigherEducation
UNIS, NUIG,UASO, WSU
City BR, AA
Partners:
Duration: 36 months (2014-2017)
![Page 24: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/24.jpg)
24
AnalyticsToolbox
Context-awareDecision Support,
Visualisation
Knowledge-based
Stream Processing
Real-TimeMonitoring &
Testing
Accuracy & Trust
Modelling
SemanticIntegration
On Demand Data
Federation
OpenReferenceData Sets
Real-TimeIoT InformationExtraction
IoT StreamProcessing
Federation ofHeterogenousData Streams
Design-Time Run-Time Testing
Exposure APIs
![Page 25: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/25.jpg)
Designing for real world problems
![Page 26: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/26.jpg)
101 Smart City scenarios
26http://www.ict-citypulse.eu/scenarios/
Dr Mirko PresserAlexandra Institute Denmark
![Page 27: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/27.jpg)
27
Data Visualisation
![Page 28: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/28.jpg)
28
Event Visualisation
![Page 29: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/29.jpg)
CityPulse demo
29
![Page 30: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/30.jpg)
Data abstraction
30F. Ganz, P. Barnaghi, F. Carrez, "Information Abstraction for Heterogeneous Real World Internet Data", IEEE Sensors Journal, 2013.
![Page 31: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/31.jpg)
Adaptable and dynamic learning methods
http://kat.ee.surrey.ac.uk/
![Page 32: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/32.jpg)
Correlation analysis
32
![Page 33: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/33.jpg)
Analysing social streams
33With
![Page 34: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/34.jpg)
City event extraction from social streams
34
Tweets from a city POS Tagging
Hybrid NER+ Event term extraction
Geohashing
Temporal Estimation
Impact Assessment
Event Aggregatio
nOSM
LocationsSCRIBE
ontology
511.org hierarchy
City Event ExtractionCity Event Annotation
P. Anantharam, P. Barnaghi, K. Thirunarayan, A.P. Sheth, "Extracting City Traffic Events from Social Streams", ACM Trans. on Intelligent Systems and Technology, 2015.
Collaboration with Kno.e.sis, Wright State University
![Page 35: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/35.jpg)
Geohashing
35
0.6 miles
Max-lat
Min-lat
Min-long
Max-long
0.38 miles
37.7545166015625, -122.40966796875
37.7490234375, -122.40966796875
37.7545166015625, -122.420654296875
37.7490234375, -122.420654296875
437.74933, -122.4106711
Hierarchical spatial structure of geohash for representing locations with variable precision.
Here the location string is 5H34
0 1 2 3 4 5 67 8 9 B C D EF G H I J K L
0 172 3 4
5 6 8 9
0 1 2 3 4
5 6 7
0 1 23 4 5
6 7 8
![Page 36: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/36.jpg)
Social media analysis
36
City Infrastructure
Tweets from a city
P. Anantharam, P. Barnaghi, K. Thirunarayan, A. Sheth, "Extracting city events from social streams,“, ACM Transactions on TICS, 2014.
![Page 37: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/37.jpg)
Social media analysis (deep learning – under construction)
37
http://iot.ee.surrey.ac.uk/citypulse-social/
![Page 38: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/38.jpg)
Accumulated and connected knowledge?
38Image courtesy: IEEE Spectrum
![Page 39: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/39.jpg)
Reference Datasets
39http://iot.ee.surrey.ac.uk:8080/datasets.html
![Page 40: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/40.jpg)
Importance of Complementary Data
40
![Page 41: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/41.jpg)
Users in control or losing control?
41
Image source: Julian Walker, Flicker
![Page 42: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/42.jpg)
Data Analytics solutions for IoT data
− Great opportunities and many applications;− Enhanced and (near-) real-time insights;− Supporting more automated decision making and
in-depth analysis of events and occurrences by combining various sources of data;
− Providing more and better information to citizens;− …
42
![Page 43: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/43.jpg)
However…
− We need to know our data and its context (density, quality, reliability, …)
− Open Data (there needs to be more real-time data)
− Complementary data − Citizens in control − Transparency and data management issues
(privacy, security, trust, …)− Reliability and dependability of the systems
43
![Page 44: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/44.jpg)
In conclusion
− IoT data analytics is different from common big data analytics.
− Data collection in the IoT comes at the cost of bandwidth, network, energy and other resources.
− Data collection, delivery and processing is also depended on multiple layers of the network.
− We need more resource-aware data analytics methods and cross-layer optimisations.
− The solutions should work across different systems and multiple platforms (Ecosystem of systems).
− Data sources are more than physical (sensory) observation.− The IoT requires integration and processing of physical-cyber-
social data.− The extracted insights and information should be converted
to a feedback and/or actionable information. 44
![Page 45: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/45.jpg)
IET sector briefing report
45
Available at: http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm
![Page 46: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/46.jpg)
CityPulse stakeholder report
46http://www.ict-citypulse.eu/page/sites/default/files/citypulse_annual_report.pdf
![Page 47: Smart Cities and Data Analytics: Challenges and Opportunities](https://reader036.vdocuments.net/reader036/viewer/2022062904/587703e61a28ab890e8b4ef9/html5/thumbnails/47.jpg)
Other challenges and topics that I didn't talk about
Security
Privacy
Trust, resilience and reliability
Noise and incomplete data
Cloud and distributed computing
Networks, test-beds and mobility
Mobile computing
Applications and use-case scenarios
47