siva ravada - nloug · •gps locations for each train collected throughout the day •each...

Siva Ravada Director of Development

Oracle Spatial Technologies

<Insert Picture Here>

Agenda

• Spatial on Exadata

• Introduction to Exadata

• Spatial operations on Exadata

• Customer Examples

• 3D Data Support

• OBIEE and MapViewer

• How maps are integrated into OBIEE

• Oracle Spatial Technology for Cloud

• Q & A

What Is the Oracle Exadata Database

Machine?

• Oracle SUN hardware uniquely engineered to work together with Oracle database software

• Key features: • Database Grid – Up to 128 Intel cores connected by 40 Gb/second

InfiniBand fabric, for massive parallel query processing.

• Raw Disk – Up to 336 TB of uncompressed storage (high performance or high capacity)

• Memory – Up to 2 TB

• Exadata Hybrid Columnar Compression (EHCC) – Query and archive modes available. 10x-30x compression.

• Storage Servers – Up to 14 storage servers (168 Intel cores) that can perform massive parallel smart scans. Smart scans offloads SQL predicate filtering to the raw data blocks. Results in much less data transferred, and dramatically improved performance.

• Storage flash cache – Up to 5.3 TB with I/O resource management

4

12 Xeon cores

2.93 GHz

12 Xeon cores

2.93 GHz

2 TB storage

12 Xeon cores, 2.26 GHz

2 TB storage

12 Xeon cores, 2.26 GHz

2 TB storage

12 Xeon cores, 2.26 GHz

RAC, OLAP

Partitioning

Compression

Exadata DB Machine X2-2 Quarter Rack

Exadata X2-2 Quarter Rack Diagram

Oracle Spatial Focused on Parallelizing

Spatial Computations

• US Rail • Bulk nearest neighbor queries to find closest track,

and project reported train positions onto tracks

• Validate home appraisals for a Government Sponsored Enterprise (GSE) • Find all the parcels touching parcels to validate

appraisals

• Satellite Imagery Provider • Find all the useful portions of cloud covered

imagery • 850,000 strip images

• 58,000,000 cloud cover geometries

Spatial on Exadata: Overview

• Parallelizing spatial queries against partitioned tables

• Parallelizing massive spatial computations with Create Table As

Select (CTAS)

• Parallelizing spatial operators

• SDO_JOIN and parallelizing spatial functions

Parallelizing Spatial Queries Against

Partitioned Tables

Parallel Spatial Query Against

Partitioned Tables

• Partition pruning occurs first

• If a spatial operator’s query window spans multiple

partitions, partitions are spatially searched in

parallel

• True for all spatial operators

Parallel Spatial Query Against

Partitioned Tables – Example

• Example:

• A re-insurance company maintains portfolios for

hundreds of insurance companies

• Which companies will be (or are) affected by the

projected path of a hurricane

• 36 million rows, 64 partitions, each with about 571,000

rows

• 50 seconds serial

• 1.28 seconds parallel (on a ½ rack Exadata V1

database machine)….. 39 times faster

Parallelizing Spatial Operators

US Rail Application

• Requirement • GPS locations for each train collected throughout the day

• Each location has other attributes (time, speed, and more)

• GPS locations have a degree of error, so they don’t always fall on a track

• Bulk nearest neighbor queries to find closest track, and project reported train positions onto tracks

• This information is used for: • Tracking trains

• Analysis for maintenance, ensure engineers are within parameters, etc…

Parallel and CTAS With Spatial Operators

• Spatial operators can parallelize with CTAS

when multiple candidates feed the second argument

• For example, a GPS records 45,158,800 train positions

For each train position:

• Find the closest track to the train (with SDO_NN)

• Then calculate the position on the track closest to the train

• See SQL on next slide…

Linear Scalabiltiy to process 45,158,800

Train Positions (One SQL statement)

ALTER SESSION FORCE PARALLEL ddl PARALLEL 72;

ALTER SESSION FORCE PARALLEL query PARALLEL 72;

CREATE TABLE results NOLOGGING PARALLEL

AS SELECT /*+ ordered index (b tracks_spatial_idx) */

a.locomotive_id,

sdo_lrs.find_measure (b.track_geom, a.locomotive_pos) measure

FROM locomotives a,

tracks b

WHERE sdo_nn (b.track_geom, a.locomotive_pos, 'sdo_num_res=1') = 'TRUE';

• X2-2 Half Rack, 34.75 hours serially vs. 41.1 minutes parallel

• X2-2 Half Rack (48 cores) over 47x faster … Linear scalability

Summary of Rail Application Results

(Performance and Compression)

• Results compare serial and parallel “Exadata” runs

• Serial runs on other hardware are slower than Exadata

• On Exadata:

• 34.75 hours serially vs. 41.1 minutes in parallel

• X2-2 Half Rack (48 database cores) - 47x faster

• X2-2 Full Rack (96 database cores) about 94x faster

• EHCC (Exadata Hybrid Columnar Compression)

• Can compress point geometries

• 12.03 GB vs 1.125 GB EHCC Archive High Compression

• 10.69X compression

The SDO_JOIN Operation

Parallelizing Spatial Functions

• Validate home appraisals for a Government

Sponsored Enterprise (GSE)

• Find all the parcels touching parcels to validate

appraisals

• Satellite Imagery Provider

• Find all the useful portions of cloud covered

imagery

• 850,000 strip images

• 58,000,000 cloud cover geometries

SDO_JOIN – Spatial Cross Product

SELECT /*+ ordered */ b.risk_zone_id,

c.parcel_id

FROM TABLE (SDO_JOIN ('RISK_ZONES', 'GEOM',

' PARCELS', 'GEOM',

'mask=anyinteract')) a,

risk_zones b,

parcels c

WHERE a.rowid1 = b.rowid

AND a.rowid2 = c.rowid;

• Effective way to compare all geometries in one layer

to all geometries in another (or most to most)

• Leverages spatial index for both spatial layers

• Can be orders of magnitude faster

SDO_JOIN – Strategy For Parallelization

CREATE TABLE result1 NOLOGGING AS

SELECT a.rowid1 AS risk_zones_rowid,

a.rowid2 AS parcels_rowid

FROM TABLE ( SDO_JOIN ('RISK_ZONES', 'GEOM',

' PARCELS', 'GEOM') );

• First SDO_JOIN, primary filter only

SDO_JOIN – Strategy For Parallelization

ALTER SESSION FORCE PARALLEL ddl PARALLEL 72;

ALTER SESSION FORCE PARALLEL query PARALLEL 72;

CREATE TABLE result2 NOLOGGING PARALLEL AS

SELECT /*+ ordered use_nl (a,b) use_nl (a,c) */

sdo_geom.relate (b.geom, 'DETERMINE', c.geom, .05) relation,

b.risk_zone_id, c.parcel_id

FROM result1 a,

risk_zones b, parcels c

WHERE a.risk_zones_rowid = b.rowid

AND a.parcels_rowid = c.rowid;

• Then parallelize spatial function

• For this example, call sdo_geom.relate on each pair.

Strategy – First SDO_JOIN, then

SDO_GEOM.RELATE in Parallel

•SDO_JOIN –

• Very effectively utilizes the spatial index of high risk zones

and parcel polygons

• Returns rowid pairs that likely intersect

•SDO_GEOM.RELATE with DETERMINE mask, in

parallel:

• Given two polygons, returns their

reationship

• Intersection is expensive

Strategy – First SDO_JOIN, then

SDO_GEOM.RELATE in Parallel

• Validate home appraisals for a Government Sponsored

Enterprise (GSE)

• Find all the parcels touching parcels to validate appraisals

• Processed 2,018,429 parcels serially in 38.25 minutes

• Exadata ½ RAC (48 cores) – 45x faster - 50 seconds

• X2-2 Full RAC (96 cores) about 90x faster

• X2-8 (128 cores) even faster

• Satellite imagery requirement from previous slide

• 45 days to 4 days (non-Exadata with hardware constraints)

• From 4 days to hours or minutes on Exadata

<Insert Picture Here>

Oracle Spatial

3D Data Model

Oracle Confidential

Simulation & Virtual Worlds

3D Urban Modeling Energy Exploration

Architecture & Engineering Simulation, Gaming, VR

3D Functionality in Oracle Spatial 11g

• SDO_GEOMETRY (3D)

• SDO_TIN

• SDO_POINT_CLOUD

3D

CO

OR

DIN

AT

E S

YS

TE

MS

Types Building Models,..

Surface

Modeling

Scene,

Object Modeling

Efficient

Storage

Query

Analysis

Data Types SDO_GEOMETRY

• Points

• Lines

• Simple Surfaces • All points of a surface lie in a 3D plane

• Composite surfaces • Has one or more connected simple surfaces

• Simple surfaces cannot cross

• Simple Solids

• Composed of closed surfaces

• Can have interior surfaces

• Composite Solids

• Consists of n connected simple solids

• Representation might be easier

• Multi-points, -lines, -surfaces, -solids

• Multi-solid is different from composite solid

Data Types

• SDO_PC

• Point clouds

• Used to capture as built

• SDO_TIN

• Used to model terrain

City GML and Visualization Models

• Schemas for modeling 3D data for different

information models

• City GML

• Building Information Model (BIM)

• Utilities to publish as CityGML

• Provide a standard Visualization Schema

Themes

• Combine

• Geometry

• Texture

• Materials,

color maps, …

• Attributes

• Declare schema in metadata

Sample Theme Geometry

create table TEXTURES (

id number,

texture blob,

CONSTRAINT textures_pk PRIMARY KEY(id));

create table road_tiles_zoom4 (

id number,

geom sdo_geometry,

texture_id number,

texture_coords sdo_ordinate_array,

gen_id number,

CONSTRAINT road_tiles4_pk PRIMARY KEY(id),

CONSTRAINT ROAD_TILES4_FK FOREIGN KEY(texture_id) REFERENCES TEXTURES(id)

deferrable initially deferred,

CONSTRAINT ROAD_TILES4_GEN_FK FOREIGN KEY(gen_id) REFERENCES

ROAD_TILES_ZOOM3(id)

deferrable initially deferred);

Sample Theme Texture

create table TEXTURES (

id number,

texture blob,

CONSTRAINT textures_pk PRIMARY KEY(id));

create table road_tiles_zoom4 (

id number,

geom sdo_geometry,

texture_id number,

texture_coords sdo_ordinate_array,

gen_id number,

CONSTRAINT road_tiles4_pk PRIMARY KEY(id),

CONSTRAINT ROAD_TILES4_FK FOREIGN KEY(texture_id) REFERENCES TEXTURES(id)

deferrable initially deferred,

CONSTRAINT ROAD_TILES4_GEN_FK FOREIGN KEY(gen_id) REFERENCES

ROAD_TILES_ZOOM3(id)

deferrable initially deferred);

Sample Theme Generalization

create table TEXTURES (

id number,

texture blob,

CONSTRAINT textures_pk PRIMARY KEY(id));

create table road_tiles_zoom4 (

id number,

geom sdo_geometry,

texture_id number,

texture_coords sdo_ordinate_array,

gen_id number,

CONSTRAINT road_tiles4_pk PRIMARY KEY(id),

CONSTRAINT ROAD_TILES4_FK FOREIGN KEY(texture_id) REFERENCES TEXTURES(id)

deferrable initially deferred,

CONSTRAINT ROAD_TILES4_GEN_FK FOREIGN KEY(gen_id) REFERENCES

ROAD_TILES_ZOOM3(id)

deferrable initially deferred);

Sample Theme Attributes (BIM, CityGML, …)

create table TEXTURES (

id number,

texture blob,

CONSTRAINT textures_pk PRIMARY KEY(id));

create table road_tiles_zoom4 (

id number,

geom sdo_geometry,

texture_id number,

texture_coords sdo_ordinate_array,

gen_id number,

… Attribute columns: name, address, population, etc …

CONSTRAINT road_tiles4_pk PRIMARY KEY(id),

CONSTRAINT ROAD_TILES4_FK FOREIGN KEY(texture_id) REFERENCES TEXTURES(id)

deferrable initially deferred,

CONSTRAINT ROAD_TILES4_GEN_FK FOREIGN KEY(gen_id) REFERENCES

ROAD_TILES_ZOOM3(id)

deferrable initially deferred);

Metadata (Theme) insert into USER_SDO_3DTHEMES (

name,

description,

base_table,

theme_column,

style_column,

theme_type,

definition)

values (

'Road Tile 4 Theme',

'Road Tile 4 Theme',

'ROAD_TILES_ZOOM4',

'GEOM',

' ',

'SDO_GEOMETRY',

'<Theme3d srid="4327">

<LOD>

<ThemeLOD>4</ThemeLOD>

<Generalization>

<ForeignKey>ROAD_TILES4_GEN_FK</ForeignKey>

<TargetTheme>Road Tile 3 Theme</TargetTheme>

<ActivationDistance uom_id="9001">3200000</ActivationDistance>

</Generalization>

</LOD>

<DefaultStyle>

<Texture>

<TextureBLOB>

<ForeignKey>ROAD_TILES4_FK</ForeignKey>

<ValueColumn>TEXTURE</ValueColumn>

</TextureBLOB>

<TextureCoordinates>

<ValueColumn>TEXTURE_COORDS</ValueColumn>

</TextureCoordinates>

</Texture>

</DefaultStyle>

<hidden_info>

<field column="name" name="Name"/>

<field column="address" name="Address"/>

</hidden_info>

</Theme3d>');

create table TEXTURES (

id number,

texture blob,

CONSTRAINT textures_pk PRIMARY KEY(id));

create table road_tiles_zoom4 (

id number,

geom sdo_geometry,

texture_id number,

texture_coords sdo_ordinate_array,

gen_id number,

CONSTRAINT road_tiles4_pk PRIMARY KEY(id),

CONSTRAINT ROAD_TILES4_FK FOREIGN KEY(texture_id) REFERENCES TEXTURES(id)

deferrable initially deferred,

CONSTRAINT ROAD_TILES4_GEN_FK FOREIGN KEY(gen_id) REFERENCES ROAD_TILES_ZOOM3(id)

deferrable initially deferred);

Metadata (Scene)

insert into USER_SDO_SCENES (

name,

description,

definition)

values (

'Road Tile 3 Scene',

'Road Tile 3 Scene',

'<scene srid="4327">

<theme display="true" pickable="false">Road Tile 2 Theme</theme>

<theme display="true" pickable="false">Berlin Theme LOD1</theme>

<theme display="true" pickable="false">Ground Theme</theme>

<theme display="false" pickable="true">Pickable Features Theme</theme>

</scene>');

Metadata (Viewframe)

insert into user_sdo_viewframes (

name,

description,

scene_name,

definition)

values (

'Sat Tile 3 Viewpoint',

'Sat Tile 3 Viewpoint',

'Sat Tile 3 Scene',

'<ViewFrame3d>

<SceneName>Sat Tile 3 Scene</SceneName>

<ViewPoint3d srid="4327">

<Eye x="-75" y="45" z="12000000"/>

<Center x="0" y="0" z="-6378137"/>

<Up x="0" y="1" z="0"/>

</ViewPoint3d>

<DefaultStyle>String</DefaultStyle>

</ViewFrame3d>');

OBIEE and MapViewer

<Insert Picture Here>

Oracle Business

Intelligence Enterprise

Edition 11g

Part of Oracle Fusion Middleware

Infrastructure & Management

Database

Middleware

Applications

When are Map views useful

• Visualizing data related to geographic locations

• Showing or detecting spatial relationships and

patterns

• Showing lots of data in a relatively small area

• Drilling down from a (map) overview to a detailed

report, chart, or graph

Why Spatial Map Visualizations? Ideal For Master-Detail Analysis

© 2010, NAVTEQ

Spatial Visualization: Map

views in OBIEE 11g

Oracle BI Enterprise Edition &

Location Integration Map views: Thematic Map-based Visualizations

• Thematic Mapping in OBIEE 11g

• First-class view in Answers Point-and-click interface

Creation of Map Views

Interactions are UI-driven

State management

Dashboard Prompts

Print & Export

Agents

Integration with

Presentation Server stack

Oracle Fusion Middleware

MapViewer

Support for thematic overlays

Master-detail linking

Map Metadata Management

Via Presentation Server Administration

Creating Map views

• (At Least) One Column related to location (e.g.

country code, state name)

• (At Least) One Measure

• Default map format created

• Provided…

• Edit Map View to add / edit / delete formats

Map View Formats

• Color Fill (choropleth)

• Percentile, Value,

Continuous binning

• Dashboard user run-time

slider

• Graphs – Bar, Pie

• Adjustable graph size

• Series by second dimension

• Bubble (variable sized)

• Min-Max size specification

• Color specification

• Variable Shape

• Circle, Triangle, Diamond

• Customizable

• Image

• Imported via MapViewer

• More can be added from

MapBuilder

• Custom Point Layer

• Uses Lat / Long

• Does not require a Layer

Def

OBIEE 11g Map view feature

• New view type in 11g

• Supports common use cases

• And interactions: drilling, master-detail linking

Master-Detail linking: Country at a glance on

the map, details for a state in charts

A geographic dimension usually has a

well known hierarchy, e.g. country,

region, state, county

Spatial Technology for the Cloud

• Spatial components are built to run on premise and in the cloud

• Applications built on these components only need to change the URL to switch from on premise to cloud implementations

• Examples • J2EE Geocoder

• Routing Engine

• MapViewer

• Web services (WLS, CS-W, WMS)

• Network Data Model

• More components being built in this web services model

• 3D data server

• City GML server

Find out more...

oracle.com/database/spatial.html

oracle.com/technology/products/spatial

oracle.com/technology/products/spatial/htdocs/pro_oracle_spatial.html

A Q &