data interoperability an introduction · •work with data in hundreds of formats, both native and...
TRANSCRIPT
What is the Data Interoperability Extension?
• Work with data in hundreds of formats, both native and non-native to ArcGIS• Simple format conversion or complex data transformation
- Industry jargon is “ETL” – Extract Transform Load
• What do we mean by “transformation”?- Attribute manipulation- Geometry manipulation- Feature level granularity
• No coding required- Even when reaching out to web services
Connect Data and Apps and Automate Workflows
Apps are optimized to work with a limited set of formats but you need to move data from app to app
What is the Data Interoperability Extension?
• Three workflows are enabled by the extension1. Direct Read of Formats
- Hundreds delivered out of the box, you can create your own
2. Quick Format Conversion- Typically from any format to File Geodatabase
3. Spatial ETL Geoprocessing- Graphical authoring environment of data transformation tools- Work within the ArcGIS Geoprocessing environment
What workflows are enabled?
AcGIS Data Interoperability is available for Desktop and Server environment
How do people use it?Top 6 Use Cases
No Domain Limitations
ArcGIS Desktop - The DetailsDirect Read
DemoDirect Read in ArcCatalog
Direct Read
• Enable Data Interoperability• Browse to formats on disk in 10.x Catalog tree
- Recognized by extension
• Looks like a Feature Dataset container icon• Use data in maps, geoprocessing• Optionally make an Interoperability Connection
- Off-disk formats like GeoJSON- Format Parameters Accessible
Demo Recap
Move Your DataQuick Translation
Quick Import & Quick Export
• Simple, Fast data movement• Default Behavior for the Format
- No attribute manipulation- No coordinate system manipulation
• Move data to editable format- Suit Target App
To and From Any Format
DemoQuick Import from Mapinfo TABQuick Export to PDF
Quick Tools
• Simple!• Easily used in Modelbuilder
- Select Data chooses element- Automates Geoprocessing
Demo Recap
ArcGIS Desktop - The DetailsSpatial ETL
What is ETL? A quick primer…
• Extract, Transform, Load
The process of moving data from A to B
ArcGIS
Spatial ETL – ArcGIS Desktop 10.x and ArcGIS Pro
• Transform- Any Number of Sources and Formats- Attribute Schema- Interactions – Relational, Topological- Geometry
- Coordinates- Feature Type
- Coordinate System- Styling- Expressions, SQL, Python- Files & Databases- ArcGIS Portal & ArcGIS Online- Web Services
The Full Power of Data Interoperability
DemoTransform Shapefile to Geodatabase with a Spatial ETL Tool, modifying attributes and feature types
Spatial ETL
• Workbench Application• Readers• Transformers• Writers
Demo Recap
Example Use Case #1 – Integrate Multiple Formats
• Unlimited format combinations in any workflow
Why Data Interoperability?
CityMap.gml
CityBuildings.dwg
CityPlan.xslx
KML
Example Use Case #2 – Distribute Data Amongst Systems
• Unlimited system combinations in any workflow
Why Data Interoperability?
CityMap.gml
CityBuildings.dwg
CityPlan.xslx
Data Interoperability
DemoIntegrate multiple sources and distribute to multiple destinations
Example Use Case #3 – Migrate Data Between Systems
• Commonly between DBMS, CAD or GIS technologies• Increasingly between Cloud and On Premise technologies• Fine grained control over insert, update, delete behavior
Why Data Interoperability?
Example Use Case #4 – Data Validation
• Create your own validation tests• Examples:
- Consider the previous feature in some custom sort order- Build expressions using multiple attributes- Use statistical analysis to drive tests- Reach out to web services for real time checks- Cast geometry to another type temporarily
• Create reports of issues found
Why Data Interoperability?
DemoValidate data and output QA report
Example Use Case #5 – Detect Changes
• Distributed Environments Synchronization Challenge- Mobile- Office- Portal- Cloud
• Historical Lineage Not Managed by ArcGIS- Brute force change detection becomes necessary
• Detect geometry or attribute changes• Detect feature existence changes
Why Data Interoperability?
DemoDetect changes in polygon data
Example Use Case #6 – Automation
• Spatial ETL Tools are citizens of the ArcGIS Geoprocessing environment- Use in Modelbuilder- Use Spatial ETL Tools in standalone Python scripts- Use ArcPy inside Workbench- Publish to ArcGIS Server (Windows)
• ArcGIS Pro 1.3 interoperates with FME- Read and Write .fmw files- FME 2016.1 Engine
• Publish maps that use non-native formats copied to a Data Store folder
Why Data Interoperability?
Try it, Learn it
• 60-day Trial:• http://www.esri.com/software/arcgis/extensions/datainteroperability/evaluate• Courses – search ‘ETL’ in the Training Catalog:
- Transforming Data Using Extract, Transform, and Load Processes- http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2571- Controlling Data Translations Using Extract, Transform, and Load Processes- http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2737- Data Validation checklist and webinar – fme.ly/validation- FME and ArcGIS Online resources – fme.ly/agol
Why Data Interoperability?
Please take our Survey – Session 42 Offering 226Your feedback allows us to help maintain high standards and to help presenters
Find the session you want to review
Find your event in the Esri Events App
Scroll down to the bottom of the session
Answer survey questions and submit
