the best come from fresh ingredients: creating cad files from an enterprise sde server
DESCRIPTION
TRANSCRIPT
The Best Dishes Are Made With Fresh Ingredients: Creating CAD Files from an Enterprise SDE Server
Matthew FreidGIS Analyst – Bureau of Environmental ServicesCity of Portland Oregon
GIS to CAD Conversion:Project Goals
Provide most current data to all users Regardless of format & software used to access it
Provide data in native formats and in a structure that’s immediately useable No end user translation or management required
Make the same data and formatting available to internal and external users
City of Portland Bureau of Environmental Services
145,000+ accounts in 150 sq mi service area
2,200+ miles of pipe in combined, sanitary and storm system
14,000 sumps and sedimentation manholes that collect storm water runoff
CSO abatement program final years of 20 year, $ 1.4 billion program
Matthew Freid: GIS Analyst
Provide user support, database mgmt, app design/ integration, business intelligence
Focus on Collection System Asset Database
Our system combines an IMS/CMS with a GIS Data Maintenance Environment (DME)
GIS Data Delivery: High Availability
Data produced by DME is housed in SDE / SQL Server on Enterprise HUB for internal and external users, along with 100’s of other feature classes
Accessed with: Out-of-the box desktop GIS Custom applications for workgroups and
mobile users Web applications
GIS Data Delivery: High Availability
Used by maintenance workers, modelers and planners throughout the bureau and city
Integrated access to as-builts, easements and current data from IMS via www.portlandmaps.com
So … lots of the most current data available
Who’s Missing From This Picture???
Before DME was implemented Assets mapped in AutoCAD Batch exports converted for GIS users
Downsides Data consistency Lack of continuous coverage/integration with IMS Currency of other datasets from other sources
The Big Advantage CAD users had the most up to date files to work
with in their native format
From Best to Worst
Change in data input systems = no direct access to current data in familiar way!
Users could use GIS software to query and analyze the assets data, but… had to use out of date CAD files for designing
projects Result?
Create and use copies of final set of dwg’s
Initial Solutions: Nothing Is Ever Easy
Recreating file structure and extents? Not practical with GIS software
Direct access to SDE within AutoCAD? Cumbersome
Another need: Complete, self-contained set of CAD files for
distribution to outside consultants and contractors
Try and Try Again
Proprietary tool Ran initial translation from SDE to DWG using
AutoCAD - without complete formatting
Added formatting on the fly for a specific user defined area (BMAP)
Issues Preprocessing crashed often, needed human
intervention Can’t make files for users outside the city Sometimes didn’t handle datasets from outside
the bureau
Find A Better Way
Users liked the BMAP approach Get a file on demand within AutoCAD
Try FME? Already used for AutoCAD to SDE conversion so
…
Find A Better Way
New approach – use FME as a stable environment to:
Produce a complete set of fully formatted DWG files for external users
Deliver final files via BMAP tool for internal ACAD users
Allow any data on Enterprise GIS HUB to be incorporated into the DWG’s
Outside Help
Worked with Forte Consulting Our long suffering support and solution
provider for our initial conversion
Used FME Workbench and some Mapping files Goal: City staff could take over and manage
the process over time
Hooks to SDE
Inputs come directly from Enterprise SDE Servers Outputs to temp dir of DWG’s
Workbench File
Attributes and Transformers
Outputs
Beyond Workbench
2+ temporary DWG’s produced and loaded into a single sub-directory
Leave Workbench so we can produce a single DWG that contains features with different sets of OBJECT DATA attribution
Consultant helped us create a mapping file to accommodate this
Mapping File
DEFAULT_MACRO LogFileName mergeAcadLOG_FILENAME $(LogFileName)Count MAX_TO_LOG 50000LOG_APPEND NO
FME_DEBUG UNGROUPED UNCORRELATED MAPPING_FILE
READER_TYPE MULTI_READERMULTI_READER_TYPE{*} AUTOCAD_ODMULTI_READER_KEYWORD{*} AUTOCAD_OD
DEFAULT_MACRO SourceDataset C:\FMEdevelopment\temp_dwg_dirMULTI_READER_DATASET $(SourceDataset)
INCLUDE $(FME_MF_DIR)/../inc/FirstODautocadObject.fmi
FACTORY_DEF * SamplingFactory \ SAMPLE_RATE 1 \ INPUT FEATURE_TYPE * @Log()
MULTI_READER *
AUTOCAD_OD *
The FMI File
Controls Attribution and Field definitions Easy way to add new datasets Self documenting nature of a text file system
has it’s advantages
The FMI file
DEFAULT_MACRO DestDataset C:\FMEdevelopment\results\TEST.dwg
# Writer Information WRITER_TYPE AUTOCAD_OD
AUTOCAD_OD_DATASET $(DestDataset)
DEFAULT_MACRO AUTOCAD_Versions Release2007 AUTOCAD_OD_RELEASE $(AUTOCAD_Versions)
AUTOCAD_OD_AUTO_CREATE_LAYERS YES
AUTOCAD_OD_DEF TAXLOTS \ autocad_color 164 \ autocad_linetype CONTINUOUS \ autocad_layer_frozen no \ autocad_layer_hidden no \ autocad_layer_locked no \ autocad_od_entity_key_attr autocad_od_entity_key \ PROPERTYID char(7) \ RNO char(30) \ STATE_ID char(7) \ OWNER1 char(7) \ OWNER2 char(7) \ OWNER3 char(7) \ OWNERADDR char(7) \ OWNERCITY char(7) \ OWNERSTATE char(7) \ SITEADDR char(37) \ SITECITY char(37) \ SITE_STATE char(37) \ SITEZIP char(37)
Batch File Control
We use batch files to execute the Workbench and Mapping files
Allows for flexibility changing parameters such as server information
Allows for defining output map extent on the fly
Forte developed a simple interface to help us generate batch files
Batch File Control
fme C:\FMEDevelopment\workbench\NewSpodyOdy.fmw --GEODATABASE_SDE_IN_USERID_GEODATABASE_SDE XXXXXXX --GEODATABASE_SDE_IN_PASSWORD_GEODATABASE_SDE XXX --GEODATABASE_SDE_IN_INSTANCE_GEODATABASE_SDE
sde:sqlserver:gissqlprod1\gis92 --GEODATABASE_SDE_IN_SERVER_GEODATABASE_SDE gissqlprod1 --SourceDataset_GEODATABASE_SDE egh_public --LOG_FILE C:\FMEDevelopment\logs\1n2e36_P1.log --DestDataset_ACAD_3 C:\FMEDevelopment\temp_dwg_dir\1n2e36_p1.dwg --DestDataset_ACAD C:\FMEDevelopment\temp_dwg_dir\1n2e36_p2.dwg --XMIN 7685134.24619048 --XMAX 7690721.73190476 --YMIN 681689.792857143 --YMAX 687130.780952381 --REL_ACAD Release2004 --BlockTemplateFile C:\FMEDevelopment\template\portland_blocks.dwg --__WB_ODBC_SOURCE C:\FMEDevelopment\data\BatchDriver.xls
fme C:\FMEDevelopment\etc\mergeDWG.fme --LogFileName C:\FMEDevelopment\logs\merge_1n2e36.log --SourceDataset C:\FMEDevelopment\temp_dwg_dir --DestDataset C:\FMEDevelopment\results\1n2e36.dwg --BlockTemplateFile C:\FMEDevelopment\template\portland_blocks.dwg --ACAD_Versions Release2004
Output Example
Output Attribution
Results for CAD Users
More attribution
More datasets as input
Most current data available
Output standard for all internal and external users
Results For Me
A STABLE system to export from GIS to CAD that meets users expectation for file structure and contents
System is manageable – changes in GIS data or CAD standards can be easily accommodated going forward
Going Forward
Server-based solution for web-based ETL
Other formats such as Microstation
Citywide standard CAD files for all users with complete set of utilities
Utilize future versions of FME to streamline process and add new functionality
Thank You!
Questions?
For more information: Matthew Freid –
[email protected] Bureau of Environmental Services
Portland Oregon www.portlandmaps.com