think huddle and neustar performance gains with 12c -...
TRANSCRIPT
OVERVIEW • “Big Data” and its impact on Spatial performance • Testing configurations • Use cases / techniques for improving performance • Performance improvement data
CHALLENGES / OPPORTUNITIES • With 11g, performance sometimes limited interactions with
Big Data • Making improvements was time-consuming
SOLUTIONS • Oracle Database 12c Enterprise Edition
• Spatial Option with GeoRaster, Network Data Model • Partitioning
• Oracle Fusion Middleware 11.1.1.7 • MapViewer
• Oracle Business Intelligence Enterprise Edition
RESULTS • Achieving at least 2x performance improvement – in
some cases up to 300x improvement • Only small configuration changes are needed • Time to develop and maintain these solutions is reduced
Think Huddle and Neustar Performance Gains with 12c /
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000
11g 12c 12c SVA
6.5x faster performance 11m 53s vs 1hr 17m 26s
May 2014 Oracle Spatial Summit
May 21, 2014 Walter E. Washington Convention Center
Washington, DC USA
Steve Pierce CEO
Nick Salem Distinguished Engineer
Performance Gains with Spatial & Graph Using Oracle Database 12c
• Big Data: What’s the big deal? • Testing configurations • Use cases and performance data on:
– Geometry aggregation – Point in polygon – Joins
• Q&A
Program Agenda
Think Huddle Corporate Overview
• Our company – Oracle Spatial and MapViewer – Big Data and BI for Spatial – Fortune 500 clients
• Steve Pierce – Chair – Oracle Spatial SIG – 15 years Oracle
What’s the Big Deal?
• Data growing – Big Data systems for spatial processing
• Fundamental changes in 12c – Faster vector algorithms – Better caching – Improved memory management
• 12c works for Big Data – (and maintains data integrity – … and keeps data secure – … and supports relational integrity – … and your users can access it)
BIG DATA
Test Environment
Environment Oracle 11g Oracle 12c
OS Oracle Linux 6.5 Oracle Linux 6.5
Processor E5 Xeon 2.9Ghz x 2
E5 Xeon 2.9Ghz x 2
Disk Intel SSD s Intel SSD s
Memory 32GB 32GB
Huge Pages Yes Yes
SGA/PGA 8GB/2GB 8GB/2GB
Source data Census / Generated
Census / Generated
• 1 Server – 2 Oracle Databases
• Trusted company serving the internet, telco, information services, retail, media and advertising industries
• Publicly held - $902M+ Company • 1,600+ employees worldwide – HQ’d in Sterling, VA • Extensive data assets (4 billion global phone numbers,
2.8 billion global IP addresses, 6.4 million global domain names and 13.1 US business listings)
• 20 billion requests a day (DNS, text messages, phone)
Neustar Corporate Overview Real Intelligence. Better Decisions.
Neustar’s ElementOne Platform Geospatial Analytics in the Cloud
• Highly flexible and high performance mapping
• Rich market and trade area capability • Road network / drive time analysis • Heat themes • GeoRaster images • Aerials / terrain map overlays • Scalable and high performance data
retrieval • Centralized secure data repository
• Oracle VM Server 3.0 (Xen version 4.1 PVM) • Created 2 identical VMs with Oracle Linux 6.5
– Installed Oracle 11g on one and 12c on the other
• Database configuration – 8G of SGA / 2G of PGA – 4 x 8 Core CPUs (Intel Xeon 2.70GHz)
• Loaded core ElementOne database on each • Compare 11g to 12c w/ & w/o spatial vector acceleration
Neustar’s ElementOne Platform 11g vs 12c Test Benchmark Environment
Use Case – Geometry Aggregation
Aggregation of census tracts into larger geometries based on neighbor relationships
INTERACT UNION
• Each Census tract in Virginia – Aggregate Union of Neighbors
• SDO_ANY_INTERACT + SDO_AGGR_UNION
– 2,000 Aggregations – Tracts have 7 neighbors = 14,000 tracts – 280 tracts processed per second 0
50
100
150
200
250
300
350
11g 12c 12c SVA
Elapsed Time (s)
Use Case – Geometry Aggregation
Neustar’s ElementOne on 12c SDO_AGGR_UNION – CA ZIPs
• Use case consisted of aggregating all ZIP Codes in California using plain SDO_AGGR_UNION and with mod/group by functions
• Achieved up to 300x faster performance w\ SDO_AGGR_UNION and 40x faster with mod/group by method in12c with SVA enabled vs 11g
• Reduced elapsed time from 1 hour 54 seconds to just 12 seconds
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000
11g 12c 12c SVA
Elapsed Time In Seconds
Using SDO AGGR UNION
SDO AGGR UNION + Mod/Group
Neustar’s ElementOne on 12c SDO_AGGR_SET_UNION – CA ZIPs
0 20 40 60 80
100 120 140 160 180 200
11g 12c 12c SVA
14x faster performance 12s vs 2m 51s
SDO AGGR SET UNION
Neustar’s ElementOne on 12c SDO_TOUCH – CA ZIPs touch BGs
• For all California ZIP codes, retrieve the block groups that they touch
0
50
100
150
200
250
11g 12c 12c SVA
8x faster performance 25s vs 3m 23s
SDO_TOUCH
Neustar’s ElementOne on 12c Spatial Relationships
• Determine spatial relationships and interactions – Inside, touch, anyinteract, overlap, disjoint
• SDO_UTIL and SDO_GEOM for various geography creation and manipulation
• Shape intersection, XOR, union, difference, point at bearing, radian, distance, convex hull, polygon smoothing and more
• ElementOne Trade Areas (customer threshold, non-overlapping radii + component geos, donuts, square + hex grids, drive time, heat regions)
• Thematic overlays and extents
Neustar’s ElementOne on 12c SDO_WITHIN_DISTANCE Benchmarks
• For 544 super regional shopping centers, retrieve all businesses with a 10 mile radius
0 100 200 300 400 500 600 700 800 900
1,000
11g 12c 12c SVA
6x faster 2m 17s vs 14m 32s
SDO WITHIN DISTANCE
Neustar’s ElementOne on 12c SDO_GEOM.RELATE – CA ZIPs & BGs
• Determine spatial topological relationship for California ZIP codes and surrounding block groups
0 50
100 150 200 250 300 350 400 450
11g 12c 12c SVA
3x faster performance 2m 16s vs 7m 2s
SDO_GEOM RELATE (mask = determine)
Neustar’s ElementOne on 12c SDO_GEOM.DISTANCE Benchmarks
• Calculate distances between a single site and 112,610 surrounding sites
0
50
100
150
200
250
11g 12c 12c SVA
2.6x faster performance 1m 13s vs 3m 11s
SDO_GEOM DISTANCE
Neustar’s ElementOne on 12c SDO_INSIDE – Businesses inside NY BGs
• Retrieve all businesses inside New York state block groups
0 20 40 60 80
100 120 140 160 180 200
11g 12c 12c SVA
2.1x faster performance 1m 22s vs 2m 53s
SDO_INSIDE
Use Case – Point in Polygon
Point in polygon calculations for finding houses within an area of interest
What’s in there?
Use Case – Point in Polygon 11g
• In 11g there were ways to do this SELECT lon, lat FROM (SELECT SDO_GEOMETRY(2001, 8307,
SDO_POINT_TYPE(lon, lat, NULL), NULL, NULL) geom FROM large) c,
states s WHERE SDO_ANYINTERACT(s.geom,c.geom) = 'TRUE' AND s.state_abrv='CA'
Use Case – Point In Polygon 12c
0
50
100
150
200
250
300
11g Relate
12c SVA PiP
12c SVA PiP (8)
Elapsed Time (s)
SELECT lon, lat FROM TABLE(mdsys.sdo_PointInPolygon( CURSOR(select lon, lat from large),
(select geom from states where state_abrv='CA‘ ) , .05))
• /*+ PARALLEL(#) */ – Near linear performance boost
• Sdo_PointInPolygon
Use Case – Joins
Which objects are in which region?
And there?
What’s in there? And here?
Use Case – Joins
• SDO_JOIN – For bulk joins – sdo_join('LARGE', 'GEOM', 'STATES',
'GEOM', 'mask=anyinteract'
• Works in 11g
• Faster in 12c 0
100
200
300
400
500
600
700
11g 12c 12c SVA
Elapsed Time (s)
Neustar’s ElementOne on 12c SDO_JOIN – ZIP & Block Group Tables
• Join using mask=anyinteract
0 500
1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 5,000
11g 12c 12c SVA
6.5x faster performance 11m 53s vs 1hr 17m 26s
SDO_JOIN
Neustar’s ElementOne on 12c SDO_CS.TRANSFORM – US block groups
0 200 400 600 800
1,000 1,200 1,400 1,600
11g 12c 12c SVA
10x faster performance 2m 10s vs 22m 26s
SDO CS Coordinate Transform
Neustar’s ElementOne on 12c SDO_GEOM.VALIDATE_GEOM Benchmarks
• Validate all US county geometries
0
5
10
15
20
25
30
35
11g 12c 12c SVA
1.9x faster performance 15s vs 29s
SDO_GEOM Validate Geometry with Context
• Reduction in redo/undo generation using global temporary tables resulting in 1.7x performance boost
• 2x performance gains with inline PL/SQL functions • Up to 10x faster processing time with constructing large
CLOBs / BLOBs • Faster response times with cross session result set
cache
Neustar’s ElementOne on 12c Miscellaneous Features
Q&A