the beauty of mapping big data

The Beauty of Mapping Big Data

Stoimen Popov

R&D Lead, Product Innovation Team, HERE IoT| Dec 05, 2016

HERE is the Open Location Platform company

• Provides mapping services and location intelligence across the automotive, enterprise and internet industries• Employs 7,000 people in 56 countries• Produces maps for every country on earth• Enables four out of five in-car navigation systems in North America and Europe• Enables mobile, web and enterprise solutions for global industry leaders

Agenda

01. Data Lens

01.1 Cloud Storage

01.2 Rest API & Query Language

01.3 JS API

02. Server-Side Clustering

03. Data Lens Heat Maps

03.1 Averaged Value

03.2 Alpha Mask by Density

03.3 Value Based Heat Map

01

Data Lens

Data Lens is a cloud-based API toolkit for developing sophisticated visualizations of geographically referenced data, accessible in a web browser.

Data is queried via a REST API from the Data Lens cloud, and transformed and aggregated to provide input to the JavaScript API, which renders the visualization.

Cloud Storage

• Data storage

• Integration with HERE account

• Data enrichment

Data Lens REST API

• Authentication

• Data Upload

• Datasets

• Queries

• Query Language

• Access Management

• Data Reprojection

• ...

Data Lens Query Language

• JSON formatted queries

• Similar to prepared statements in SQL for later execution

• Only dataset owners can create queries for a dataset

• Protect sensitive datasets

Presentation title | Month 00, 20169 © 2016 HERE | HERE Internal Use Only

Data Lens

Query Language

Data Lens JS API

• Data Lens JavaScript API is a module of HERE Maps API for JavaScript and connects it to the Data Lens REST API

• Provides data-driven styling of data on a map

• Solves non-trivial tasks like tiling, caching and rasterizing data


Data Lens Heat MapDataset with 11M points of data about taxi rides in NY City


Data Lens Street ShapesThe data is geo-enriched to street geometry in NY City


Data Lens ZIP ShapesThe data is geo-enriched to ZIP code

boundaries in NY City


Data Lens BuildingsThe data is geo-enriched to building geometries

in NY City

02

Server-Side Clustering

How to deal with BIG DATA on the server?

Data Tiling & Grouping in Pixel Space

• Data tiling reduces the amount of data received by the client

• Data Lens groups the data points per tile pixel

03

Data Lens Heat Maps

• Averaged value

• Alpha mask by density

• Value-based heat map


Averaged heat map


Averaged heat map alone, and

with an applied alpha mask

Big Data can be visualized in many ways …

Heat Maps Server-Side Clustering Hybrid Clustering

(Server-side and client-side)

https://developer.here.com

Docs & API Reference Tech Examples Industry Examples

Develop

Code Examples

Data Lens APIs

• Detailed Story

• JS/HTML Code

• Query Definitions

• Styles & UI

Develop

Code Examples

Data Lens APIs

• Data-Driven Styling

• Server-Side Clustering

• Hybrid Clustering

• …and more!

Develop

Data Lens

• Getting Started

• Tutorial

• Developer Guide

• API Reference

the beauty of mapping big data

Engineering