shared data infrastructures from smart cities to education
TRANSCRIPT
![Page 1: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/1.jpg)
Shared data infrastructures: From Smart Cities to Education
Mathieu d’Aquin (@mdaquin)Data Science Group (@DataScienceGr)
KMi, The Open Universitydatahub.technology
![Page 2: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/2.jpg)
A city?
![Page 3: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/3.jpg)
A city?
A place
![Page 4: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/4.jpg)
A city?
A place
A bunch of people (and organisations)
![Page 5: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/5.jpg)
A city?
A place
A bunch of people (and organisations)
live / reside there
![Page 6: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/6.jpg)
A city?
A place
Power infrastructure
A bunch of people (and organisations)
![Page 7: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/7.jpg)
A city?
A place
Power infrastructure
Water infrastructure
A bunch of people (and organisations)
![Page 8: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/8.jpg)
A city?
A place
Power infrastructure
Water infrastructure
Transport infrastructure
A bunch of people (and organisations)
![Page 9: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/9.jpg)
A city?
A place
Power infrastructure
Water infrastructure
Transport infrastructure
A bunch of people (and organisations)
use
![Page 10: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/10.jpg)
A city?
A place
Power infrastructure
Water infrastructure
Transport infrastructure
A bunch of people (and organisations)
useunderstand???
![Page 11: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/11.jpg)
A city?
A place
Power infrastructure
Water infrastructure
Transport infrastructure
A bunch of people (and organisations)
useunderstand???
effective
with???
![Page 12: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/12.jpg)
A smart city?
A place
Power infrastructure
Water infrastructure
Transport infrastructure
Data infrastructure
A bunch of people (and organisations)
![Page 13: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/13.jpg)
mksmart.org
![Page 14: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/14.jpg)
Making Milton Keynes a Smart City
![Page 15: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/15.jpg)
Making Milton Keynes a Smart City
![Page 16: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/16.jpg)
![Page 17: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/17.jpg)
Challenges
Transport
Water
Energy
By 2026, transport demand in MK is estimated to grow by 60%, with engineering solutions only likely to meet half of this
MK is in a water stressed area and it is projected that climate change may reduce regional water availability by 29 million litres water per day by 2025.
MK needs to transition to being a low energy city to support sustainable economic growth. The 2013 core strategy of MK Council aims to achieve a 22% reduction in CO2 emissions per capita from a 2005 base by 2020.
![Page 18: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/18.jpg)
![Page 19: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/19.jpg)
![Page 20: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/20.jpg)
data
apps
![Page 21: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/21.jpg)
apps
![Page 22: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/22.jpg)
apps
![Page 23: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/23.jpg)
![Page 24: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/24.jpg)
![Page 25: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/25.jpg)
![Page 26: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/26.jpg)
![Page 27: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/27.jpg)
A data infrastructure
Because addressing vertically every route from data to application in isolation is crazy!
Challenges:
- Data heterogeneity: The content of the data is not the same
- Data diversity: The context and conditions under which the data is available are not the same
![Page 28: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/28.jpg)
Data heterogeneity
3 types of data in a city:
- Highly temporal data- Highly spatial data- Others...
![Page 29: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/29.jpg)
Answer the question: “what do we know about x?” where x is a place, organisation, bus route, roundabout, etc.
Data source 1
Data source 2
Data source 3
Data source 4
Data source 5
Data source 6
Big City Warehouse
access
ETL process
ETL process
ETL process
ETL process
ETL process
ETL process
Typical integration approach
![Page 30: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/30.jpg)
Answer the question: “what do we know about x?” where x is a place, organisation, bus route, roundabout, etc.
Data source 1
Data source 2
Data source 3
Data source 4
Data source 5
Data source 6
Big City Warehouse
access
ETL process
ETL process
ETL process
ETL process
ETL process
ETL process
Typical integration approach
Hard to maintain and keep running at scale (i.e. as number of datasets grow)
![Page 31: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/31.jpg)
Taking a Linked Data approach
![Page 32: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/32.jpg)
Taking a Linked Data approach
![Page 33: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/33.jpg)
Answer the question: “what do we know about x?” where x is a place, organisation, bus route, roundabout, etc.
Data source 1
Data source 2
Data source 3
Data source 4
Data source 5
Data source 6
Query template Query template Query template Query template Query template Query template
access access access
![Page 34: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/34.jpg)
Answer the question: “what do we know about x?” where x is a place, organisation, bus route, roundabout, etc.
Data source 1
Data source 2
Data source 3
Data source 4
Data source 5
Data source 6
Query template Query template Query template Query template Query template Query template
access access accessCan be added and maintained in isolation from each other
![Page 35: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/35.jpg)
Answer the question: “what do we know about x?” where x is a place, organisation, bus route, roundabout, etc.
Data source 1
Data source 2
Data source 3
Data source 4
Data source 5
Data source 6
Query template Query template Query template Query template Query template Query template
access access accessCan be added and maintained in isolation from each other
Virtual (i.e. never fully materialised) - no need for maintenance
![Page 36: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/36.jpg)
Result
An “Entity-API” for things in the city, integrating hundreds of datasets and providing thousands of data endpoints, each providing integrated information about a bus stop, an area, a restaurant, a school, a roundabout, etc.
![Page 37: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/37.jpg)
Result - Top MK - card playing with city data
![Page 38: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/38.jpg)
Result - MK Insight - City data portal
![Page 39: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/39.jpg)
Data Diversity
The eskimo language has 255 different words for “visiting linguist”
What’s the point of integrating data if we have to go through
every bit of it to check if it can be used?
![Page 40: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/40.jpg)
Example
Smart meter data
Anonymisation analysisAnon data Modelprediction/
recommendation
Results
![Page 41: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/41.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
analysisAnon data
Anon data
Modelprediction/
recommendation
Results
![Page 42: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/42.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
Weather data
Location data
Electricity tariff data
analysisAnon data
Anon data
Modelprediction/
recommendation
Results
![Page 43: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/43.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
Weather data
Location data
Electricity tariff data
analysisAnon data
Anon data
Modelprediction/
recommendation
Data prot.
Corp lic. 1
Results
![Page 44: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/44.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
Weather data
Location data
Electricity tariff data
analysisAnon data
Anon data
Modelprediction/
recommendation
Data prot.
Corp lic. 1
Corp lic. 2
Data prot.
Data prot.
Results
User T&C
OGL
Corp lic. 3
![Page 45: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/45.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
Weather data
Location data
Electricity tariff data
analysisAnon data
Anon data
Modelprediction/
recommendation
Data prot.
Corp lic. 1
Corp lic. 2
Data prot.
Data prot.
Results
User T&C
OGL
Corp lic. 3
?
![Page 46: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/46.jpg)
Example
Smart meter data
Anonymisation
Solar panel monitoring
Anonymisation
Weather data
Location data
Electricity tariff data
analysisAnon data
Anon data
Modelprediction/
recommendation
Data prot.
Corp lic. 1
Corp lic. 2
Data prot.
Data prot.
Results
User T&C
OGL
Corp lic. 3
?
?
?
?
![Page 47: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/47.jpg)
Semantic approach
Explicit, machine readable representation of data policies and licences...
… as well as of the data flows through which they are processed
![Page 48: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/48.jpg)
Handled through a sophisticated data cataloguing process
![Page 49: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/49.jpg)
Handled through a sophisticated data cataloguing process
![Page 50: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/50.jpg)
Result
datahub.mksmart.org
~ 400 registered users~ 700 datasets
![Page 51: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/51.jpg)
Result - Applications
![Page 52: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/52.jpg)
Result - Reusable components
datahub.technology
Open source data cataloguing components that integrate with CMS system
“Entity API” framework for data integration
High Speed processing components
Data portal framework
![Page 53: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/53.jpg)
What about education?
![Page 54: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/54.jpg)
Learner
Platform
Analytics
VLE | Website | LibraryAssessment |
Enrollment
School/University
Prediction Drop out
BI
Planning
Recommendation
Sentiment AnalysisCollective
Intelligence Behaviour Analysis
Collaboration
Community Support
AFEL - Analytics for Everyday Learning (afel-project.eu)
![Page 55: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/55.jpg)
Learner
Platform
Analytics
VLE | Website | LibraryAssessment |
Enrollment
School/University
Prediction Drop out
BI
Planning
Recommendation
Sentiment AnalysisCollective
Intelligence Behaviour Analysis
Collaboration
Community Support
AFEL - Analytics for Everyday Learning (afel-project.eu)
Same challenges…… same solutions
![Page 56: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/56.jpg)
Pointers and next steps
Deploying to other cities in the UK and Europe, as well as for research projects and other types of organisations.
Dealing with data quality and accuracy - automatic checking based on a cross comparison with other datasets.
Reusable, parameterizable services for analytics - building a catalogue of pipelines and models.
![Page 57: Shared data infrastructures from smart cities to education](https://reader031.vdocuments.net/reader031/viewer/2022030216/5887da0b1a28abfb678b54a1/html5/thumbnails/57.jpg)
Thank you!
@mdaquin@DataScienceGr
http://datahub.technology