Download - Geographical aspects of Big Data
Presentatie 2 juni 2014
IM Themadag te GeoFort (Herwijnen) Erik van der Zee (Geodan)
Drs. Erik van der Zee ◦ Physical Geographer and Business Economist
◦ Senior Consultant at Geodan (www.geodan.nl)
◦ PhD Candidate “Added value of location in the Internet of Things and Smart Cities”
◦ Subject matter expert in the Geonovum pilot Making Sense for Society (#labms4s)
E-mail [email protected]
Twitter @erikvanderzee
What is Big Data?
Spatial Big Data ◦ Big Vector Data (examples)
◦ Big Raster Data (examples)
Big Data Analysis
Spatial Analysis of Big Data ◦ Big Vector Data Analysis (real-time spatial analytics)
◦ Big Raster Data Analysis
Spatial Big Data Visualization (3DJS)
Spatial Big Data Tools and Technologies
Wat is Big Data?
Haseeb Budhani, 2008 “a blanket term for any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications”
Gartner “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”
Big Vector Data
2D3D4D datasets (XYZ+Time) “Grote” Geodatasets (BGT/BAG) Lidar Pointclouds ◦ AHN2 (aerial) ◦ Laserscanners op autos (360°)
Real-time Event Streams ◦ Sensors measurements ◦ Actuator Control information ◦ Social Media APIs (twitter, etc.) ◦ “Pasjes en poortjes” (scanners bvb OV checkins)
…
“Internet of Things”
Real-time sensor and control data (XYZT)
Sensor-Actuator Networks (Nationaal Datawarehouse Wegverkeergegevens)
Real-time data van tienduizenden sensoren elke seconde
Content can be Tekst (Twitter/Whatsapp), Photos (Instagram/Flickr), videos (YouTube), sounds (Soundcloud), …etcetera
Georeferenced Social Media Content
Big Raster Data
Steeds hogere resoluties
Steeds hogere update frequenties (Dynamic HR satellite photos and video)
Steeds meer “banden” (hyperspectral)
Overal land / op water/onder water / in de lucht
Voorbeelden ◦ Luchtfoto’s (5cm resolutie…)
◦ Gigapans
◦ Videobeelden (PTZcams, drones, cars, bodycams)
◦ 360° Panoramas
◦ Multispectral raster data (o.a. satellite imagery)
Real-time Raster data
T=16:00 T=17:00 T=18:00 T=19:00
T=20:00 T=21:00 T=22:00 T=23:00
Hyperspectral data cubes (High Resolution Multispectral Rasters ) = Big Data (petabytes of data)
Hyperspectral data cubes created yearmonthweekday…
Zeer hoge resolutie images die grote gebieden afdekken
http://www.gigapan.com/galleries/11203/gigapans/152220 (demo)
E.g. Google Streetview
Cycomedia Cyclorama/aquarama
DARPA’s big eye: ARGUS-IS 1.8-gigapixel camera for air surveillance ◦ clear images of objects as small as 15 centimeters
from an altitude of six kilometers ◦ One gigapixel is equal to 1,000 megapixels. For
comparison: Modern professional digital cameras have a resolution of about 20 megapixels
Also City Wide Video Surveillance… http://youtu.be/6VkKeM-OK6g?t=8m6s
Past – Present - Future
Types of analysis (Past – Present – Future…) Naald in enorme (BIG) hooiberg vinden (bvb 1 specifiek nummerbord) Soms Trends en patronen destilleren uit berg informatie
Big Vector Data Analysis
(Real-time) analyse van vector data (Event Steam Processing)
Sensing Analysis Act(uat)ing raw events meaningful
events
Data creation
Controlling / alerting /
notification / routing of
objects and people
Data usage
Big Raster Data Analysis
Analyse van rasterdata m.b.v. beeldherkenningsalgoritmen
Meer en meer informatie kan real-time worden afgeleid uit images and videos (creates new derived data that can be analysed in event processing engine) ◦ Face recognition (ov Rotterdam)
◦ Number plates (ANPR)
◦ Traffic signs (Google)
◦ ...
Gezichtsherkenning
Project Ground Truth Google Feature extractie van informatie uit panorama images t.b.v. map Generation and validation
http://youtu.be/FsbLEtS0uls?t=3m52s ◦ Verkeersborden ◦ Rijrichtingen ◦ Bedrijfslogo’s ◦ Straatnamen ◦ Huisnummers ◦ …
Automated Number Plate Recognition (ANPR)
Spatial Big Data Visualization
Trend van visualiseren van BI (big) Data in kaarten (bvb o.b.v. postcode / adres / woonplaats / wijk) naar Ruimtelijke Analyse (spatial analysis) van gegeorefereerde BI Data
Big Data needs New User Interfaces
Nieuwe visualisatie mogelijkheden Voorbeelden http://d3js.org/
Spatial Big Data Tooling
Spatial Extensions voor NoSQL Databases ◦ Neo4J Spatial Extensions (graph database)
◦ MonetDB Spatial Extensions (column store)
◦ Spatial Hadoop http://spatialhadoop.cs.umn.edu/
Spatial Event Stream Processing engines ◦ Esri GeoEvent Processor
◦ Oracle Spatial CEP Engine
◦ Microsoft Stream Insight…
Vragen?
