EO technologies

The cause and solution to

the big data questionCiaran Kirk, IMGS

We deliver innovative spatial solutions for the desktop, web & mobile

Built on our partner’s technology

Designed to meet the challenges of Government, Mapping Agencies

and Utility & Communications Customers

Our customers include OSI, OSNI (LPS), ESB, Eircom, NIE, BT, Irish

Naval Service and every Local Authority in Ireland

BIG DATA CAUSES

Data Deluge

Geospatial data volumes

are growing at an ever

increasing rate…

Faster than processing and

storage costs are falling

Data Deluge

Greater Resolution, Greater Coverage, Greater Frequency

in 2020CONNECTED SENSORS

Billion

Sensors ‘Sense’ and Collect

Raw Content

Space

Air

Ground

Data Deluge

New Data Sources

Data Deluge

New Data Sources

3D Buildings from UAV

Real 3D Object

using tridicon

every day

,500,000,000,000,000,00

0

bytes

…that will generate

2.5 EXABYTES2or

THOUSANDQUINTILLION QUADRILLION MILLIONTRILLION BILLION

Economic value of geospatial data

could reach $700 billion/year by 2020 (McKinsey Global Institute)

Too Much Data – Not Enough

Information

Rising User Demand

Every day, new users are

demanding imagery, across

all applications, on a variety

of devices

Summary of Market Drivers

Increasing need to leverage huge amounts of

geospatial data

Dynamic discovery of dispersed information

Need for quicker access

Multi-purpose data

BIG DATA SOLUTIONS

20

Data

Compression

Data

Management

Data Compression –

ECW TechnologyENCODING

SPEED

DECODING

SPEED

IMAGE

QUALITY

FILESIZE

TIME

SAVED

REDUCED

STORAGE

COSTS

ENHANCED

USER

PERFORMANCE

REDUCED

DATA

MANAGEMENT

The Power of ECW

ERDAS APOLLO

A comprehensive data management, analysis and delivery system

enabling an organization to describe, catalog, search, discover,

process and securely disseminate massive volumes of data.

COMPRESSION

Quick Facts

Compressed to a

single ECW file70,000

employees

985 Gb ECW

image

38 TbOf imagery

Organize and manage

geospatial data for entire

country of Germany, at

20cm GSD

RWE Deutschland AG, a

leading company in the

utility industry in Europe

370,000Image files

Case Study: Big Data Made Small

The World’s Largest

Geospatial Image?

A single aerial image covering

Germany @ 20cm GSD

3,210,000 px by 4,340,000px

Big Data Made Small

38,000gb Uncompressed

50,000gb with image pyramids

370,000 source files

1 ECW file

875gb ECW Compressed

Enhanced Compression Wavelet (ECW)

Compresses bulky imagery files

into manageable sizes while preserving

their visual quality

1000 Gb

400Gb

50 Gb

1300

Gb

Uncompressed

Original

Numerically

Lossless

Compression

ECW Visually

Lossless

Compression

ECW image compression:

• Instant storage savings

• Faster performance

• Full visual quality

Uncompressed

Original & Pyramids

$ cost per month

$ 4,700

ECW

$ 82

$ 6,200

$ 4,600

Origin

al w

ith p

yra

mid

s

… p

lus tile

-cache 1

7 levels

… p

lus tile

-cache 1

9 levels

Amazon S3 cloud storage

cost comparison

• 98% lower costs using ECW

• >$4.6k monthly saving

• Up to $73k annual saving

* Data generated using the Amazon S3 Cloud Calculator

ECW Protocol (ECWP)

world’s fastest streamingof ECW and JPEG2000 imagery

Delivers the

DATA MANAGEMENT

Solving the “Bounding Box”

problemGiven a bounding box in space, and a given time period, discover and access

all available, relevant & authorized information within that area.

Geospatial Data Types

Maps, Imagery, Features, Terrain, Place

Names, Buildings, Infrastructure, Roads,

Political Boundaries, Hydrographic,

Geodetic, etc.Location References in

Structured Data

Relational Databases, Travel Itineraries,

Financial Transactions, Corporate Data,

Personnel Records, Statistical Data, etc.

Sensor Data

EO, Spectral, Radar,

LiDAR, Infrared, FMV, in

situ, GPS, etc.

Access from Any Device

Desktop, Laptop, PDA, Wireless,

Smartphone

Location References in

Unstructured Data

News Reports, Publications, Manifests,

Internet, World Wide Web, Audio, Video, etc.

Wizard-driven Workflow to

Catalog and Publish

Search any file system visible

to the server for data.

Set schedulers to repeat the

search on a given schedule

as a one-time event or on a

recurring basis.

Process and service-enable

the datasets.

Set security to control access.

Restrict by user, resolution

and location.

Why treat LAS like a Raster

(Conversion to grid)Why treat LAS like raster?

Easy viewing

Easy distribution

Because LiDAR is so dense and

semi-regular it can be gridded quickly

and automatically

Grid automatically

Available via

WMS, WCS and WPS

Download or CZS

Raster

LiDAR

Clip, Zip and Ship LAS

CZS LAS-formatted point cloud data

Select clip area

Select classifications

Filter by return value

Output to LAS, IMG or TIFF

Clipped output LAS

Region

to subset

Streaming Point Clouds

In Summary

Organizations increasingly face the challenge of managing massive, and increasing, amounts of geospatial data. As the sheer volume and variety of geospatial data grows, the need for effective data management grows with it.

Government, defence, and private organisations derive great economic and strategic benefit from well maintained geospatial data

Dealing effectively with geospatial big data requires a multi-faceted approach, addressing factors storage footprint, data validation and assurance, comprehensive metadata models, and distribution via wide variety of protocols to ensure interoperability in a heterogeneous GIS environment

Thank You