gis and g&g database synchronization · process. we have successfully developed a tool to...

©2005 ChevronTexaco Corporation - All Rights Reserved

GIS and G&G Database Synchronization

Ananta Dwi Bodhitama*Mursyid Nento*David Stodola+

* Data Management, ChevronTexaco Indonesia Business Unit+ GIS/Remote Sensing, ChevronTexaco Energy Technology Company

2005 ESRI Petroleum User GroupHouston TX, 7-9 March 2005

©2005 ChevronTexaco Corporation - All Rights Reserved2

Abstract

Geological and Geophysical (G&G) master database is a corporate wide master database that stores up-to-date and validated G&G data coming from project databases located in different sites. ChevronTexaco Indonesia Business Unit uses Finder as its G&G master database and ArcSDE as its GIS database.

Finder keeps important subsurface information and we want to have this information available in GIS applications. Having both subsurface and the already existing surface information available in one place is very useful in decision making process.

We have successfully developed a tool to synchronize data from Finder to ArcSDE. The synchronization process takes account important subsurface information: well location, well path, seismic navigation, seismic line and seismic fault. The synchronization process runs very quickly and is set to run periodically on a daily basis to ensure up-to-date consistency with new entry in Finder.

*ArcSDE is a trademark of ESRI, Finder is a trademark of Schlumberger


Introduction

Geographic Information System (GIS) database provides data for various GIS applications.

Geological and Geophysical (G&G) master database is a corporate wide master database that stores up-to-date and validated G&G data.

Indonesia Business Unit (IBU) uses Finder as its G&G master database and Arc Spatial Data Engine (ArcSDE) as its GIS database.


Indonesia Business Unit

Where are we?

IBU,Sumatra

Australia

Pacific Ocean

Atlantic Ocean


IBU Concession Blocks

Where are we? Geographic Regions

Kisaran (New Block)

Balam

Bangko

Bekasap

Duri

Libo

Minas

Petahapan

Petani

Kuantan

25 km


Problem?

ArcSDE: Arc Spatial Data Engine that acts as master spatial database and serves GIS-based applications.

Finder: IBU master database that stores the most recent and up-to-date G&G data.

Need to have all spatial data (surface & subsurface) in ArcSDE better information for decision making.

Gap: No efficient way to store and keep up-to-date G&G data in ArcSDE.


IBU Database Flow Diagram

FINDER

ArcSDE

GISRumbai

OpenWorks

•F93RBIP•MINAS•E_T•RUMBAI

•OW3RBIP•REGI_PRO_RBI•CSBS_PRO_RBI

FINDER

OpenWorks

•F93DRIP•DURI•BEKASAP

•OW3DRIP•DURI_PRO•CSBN_PRO_RBI

OpenWorks

•OW3MNSP•MINA_PRO_MNS

Minas

Duri

SLN HO

SDEsynchInterpretation DataMaster Data


SDEsynch

Custom program built in C. Based on program created by David Stodola of GIS/Remote Sensing Team of ETC. Rewritten to meet specific IBU needs.

Solution for dealing the mentioned gap by providing synchronization of G&G data and store them in the ArcSDE database.

Part of Spatial Data Consolidation Project under UTx Master Data Consolidation.

Synchronized Data: Well Top & Bottom Location, Well Path, Seismic Navigation, Seismic Line, Seismic Fault, Oil & Gas Field Boundary.


8 Types of Data Transferred

FINDER

•Spatial Data•Well Top Location•Well Bottom Location•Well Path•Seismic Navigation•Seismic Line•Seismic Fault•Oil Field Boundary•Gas Field Boundary

•Attribute Data•Unique well identifier (UWI)•Well name & Number•Surface elevation •Well status

ArcSDE

GIS

• Subsurface Data•Well Top Location•Well Bottom Location•Well Path•Seismic Navigation•Seismic Line•Seismic Fault•Oil Field Boundary•Gas Field Boundary

•Surface Data•Land Matter•HCT•PGT•Soil•Facilities

ArcMAP

SynchronizationUsing SDEsynch


Problem: multipoint check?

16001100002

15001000001

………

YaXaWell

Finder

16001100002

15001000001

………

YbXbWell

GIS

?

• Simply compare if Xa=Xb and Ya=Yb

…

1102

1001

Xa2

…

1602

1501

Ya2

…

-

…

…

…

1600

1500

Ya1

…

-

1050

Xan

-1100002

15501000001

………

YanXa1Survey

Finder

• Compare Xa1=Xb1 and Xa2=Xb2 and … Xan=Xbn• Compare Ya1=Yb1 and Ya2=Yb2 and … Yan=Ybn

??


Checksum

Minimize comparison processes during synchronization by concatenating sum of all digits as most significant digits of the sum itself.

Can accurately detect data changes without checking every point.

Helpful in multipoint data like well path where the number of point needed to construct line or polygon is high.


Checksum

Formula:(X + Y)1 + (X + Y)2 … + (X + Y)n = SS1 + S2 … + Sm = CChecksum = C || S

Where X & Y = X/Y pair location of a pointn = number of points needed to construct line or polygonS = sum of all X & Y m = number of digits in SC = sum of all digits of SChecksum = concatenation of C and S


Checksum

Example:Checksum for well path with 5 directional survey points{(0,0), (10,10), (20,20), (30,30), (40,40)}

Where S = (0+0) + (10+10) + (20+20) + (30+30) + (40+40) S = 200C = 2 + 0 + 0 = 2Checksum = C || S

= 2 || 200= 2200

Only 2 values compared instead of many.


Problem: binary storage?

16001100002

15001000001

………

YXWell

Finder

OK002

OK001

……

ShapeWell

GIS

?

• Use ArcSDE API to convert Finder ‘text’ data to shape

bbb.dat002

aaa.dat001

……

Binary FilesOil Field

Finder

• Need to have conversion from binary to text ??

More aaa.dat==================================….brâèbbe'A@0ò@šÑ'A+ø@`Øb@þö@@ë`ús÷@ µ÷@@ë÷@ÀÞ ÷@Ú e÷@qþö@ce÷@æö@`ošÑ'A`0Öõ@ +Ñ'AÀÙõ@`-Ð'AšÝõ@`9Ð'A


Use Nlbrowser

Finder utility to read Nlist files, proprietary of Schlumberger, used to keep most array-type data in Finder like logs and seismic data as well.

There is no Finder API available cannot integrate Nlist related operations into C routine code.

Executable program with parameters where in the routine parameters are taken from Oracle database records.

Invoke by using execve() C function.


SDEsynch Algorithm

Open connection to ArcSDE

Process input file

Loop for every input files record Create database linkCreate copy table from Finder & checksumDelete orphaned recordsDelete updated recordsInsert new records and create shapeDrop database link

Close ArcSDE connection


SDEsynch Process/Data Flow

FINDERInput FileStart SDEsynch

GIS

1 2

3Delete Orphaned Data

Delete Updated Data

Insert Data

Temporary File

Nlbrowser

45

6

6b6a

6c

7

8

End SDEsynch

9


SDEsynch Process/Data Flow

1. Read input file containing project & instance names.2. Create database link to Finder.3. Copy Finder tables to GIS database & checksum.4. Comparing Finder tables and GIS tables by first

deleting orphaned data in GIS database.5. Deleting GIS data that are different from Finder data

(different means need to be updated).6. Insert new data to GIS. Only new data are inserted.

6a. In case data is stored in Nlist (ex: fault)6b. Launch Nlbrowser and keep extracted data in a

temporary file then (6c.) Back to insert routine7. Use SDE API to insert data8. Loop for every input file entry until eof (9.)


ArcSDE Server Setup

Update tnsnames.ora in ArcSDEAdd SDE instance name in /etc/services Install ArcSDE / ArcMap API & librariesInstall C compiler & libraries

Create 8 feature class layers using ArcCatalogWELL_TOP_FINDER (point)WELL_BASE_FINDER (point)WELL_PATH_FINDER (line)SEIS_NAV_FINDER (polygon)SEIS_LINE_FINDER (line)SEIS_FAULT_FINDER (line)FIELD_OIL_FINDER (polygon)FIELD_GAS_FINDER (polygon)


Synchronization Setup

Update input file with projects to synchronize.

Create Oracle user-id that can access listed project in each instance.

Setup script for mail notifications.

Run the SDEsynch programs periodically using crontab.

DONE!


Development Tools

C programming language

ArcSDE C API (Application Programming Interface) version 8.1 for Solaris 8

Gcc C compiler version 2.95 for Solaris 8

Xemacs editing tools (optional)

Makefile (optional)


From Text to Map

SynchronizationUsing SDEsynch

Oracle ‘Textual’ Data ‘Map’ in GIS-based application


From Text to Map

Seismic Navigation

Seismic Line


From Text to Map

Seismic Fault

Well Path

WellLocation


From Text to Map

Oil Field Boundary

Gas Field Boundary


Access From Web Browser

Seismic Fault

Well Path

WellLocation


Data Synchronized (as March 2005)

11364 wells (top & bottom)

985 well deviation surveys

75 seismic navigations

35751 seismic lines

663 seismic faults

8 gas fields

127 oil fields


Results

All IBU Finder projects are synchronized to GIS database on a daily basis.

Able to create shape directly from ArcSDE. Previously, a special application like ArcMap needed to perform this task.

Time saving. The synchronization process runs for less than 5 minutes. Manual update requires much more time, estimated about 2 hours, and any changes can be very difficult to spot.


Acknowledgement

Data Management Team - ChevronTexaco Indonesia Business Unit (CT IBU)

IT - CT IBU

Technology Support - CT IBU

Management - CT IBU

GIS/Remote Sensing Team - Chevron Texaco Energy Technology Company (CT ETC)


References

ArcSDE Developer Help – ArcSDE 8.1 for Client CDROM

Finder 9.1 & 9.3 Data Model – Schlumberger

Oracle 8 : Database Administration – Oracle University


Author Information

Ananta Dwi Bodhitama, Data Management – CT IBURumbai, Sumatra, IndonesiaEmail: [email protected]

Mursyid Nento, Data Management – CT IBURumbai, Sumatra, IndonesiaEmail: [email protected]

David Stodola, GIS/Remote Sensing – CT ETCSan Ramon, CA, USEmail: [email protected]


Thank you


Backup Slides


Well Top Location

Finder Tables:WELL_HDRNODES

SDE Layers:WELL_TOP_FINDER

Source Data Type: Oracle Table

SDE API: SE_shape_generate_point

Routine: Single loop


Well Bottom Location

Finder Tables:WELL_HDRNODES

SDE Layers:WELL_BASE_FINDER


SDE API: SE_shape_generate_point

Routine: Single loop


Well Path

Finder Tables:WELL_DIR_SRVY_PTSWELL_HDRNODES

SDE Layers:WELL_PATH_FINDER


SDE API: SE_shape_generate_line

Routine: Double loop + use checksum


Seismic Navigation

Finder Tables:SEIS_3D_HDR

SDE Layers:SEIS_NAV_FINDER


SDE API: SE_shape_generate_polygon

Routine: Single loop + use checksum


Seismic Line

Finder Tables:SEIS_LINE_HDR

SDE Layers:SEIS_LINE_FINDER



Routine: Single loop + use checksum


Seismic Fault

Finder Tables:SEIS_FAULT_TRACE

SDE Layers:SEIS_FAULT_FINDER

Source Data Type: Nlist File + Oracle Table


Routine: Nlbrowser + process fork


Oil Field Boundary

Finder Tables:LINESGEM_OBJECTS

SDE Layers:FIELD_OIL_FINDER





Gas Field Boundary

Finder Tables:LINESGEM_OBJECTS

SDE Layers:FIELD_GAS_FINDER





ArcSDE Server Setup

Detail columns of WELL_TOP_FINDER point class layer:

Objectid NumberWell_UWI Varchar(50)Project_Na Varchar(50)Instance_N Varchar(50)Longitude FloatLatitude FloatShape Number


ArcSDE Server Setup

Detail columns of WELL_BASE_FINDER point class layer:

Objectid NumberWell_UWI Varchar(50)Project_Na Varchar(50)Instance_N Varchar(50)Longitude FloatLatitude FloatShape Number


ArcSDE Server Setup

Detail columns of WELL_PATH_FINDER line class layer:

Objectid NumberUWI Varchar(50)Survey_Point NumberChecksum NumberProject_Na Varchar(50)Instance_N Varchar(50)Shape Number


ArcSDE Server Setup

Detail columns of SEIS_NAV_FINDER polygon class layer:

Objectid NumberSurvey Varchar(50)X1 NumberY1 NumberX2 NumberY2 NumberX3 NumberY3 NumberX4 NumberY4 NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape Number


ArcSDE Server Setup

Detail columns of SEIS_LINE_FINDER line class layer:

Objectid NumberSurvey Varchar(50)Lower_Left_X NumberLower_Left_Y NumberUpper_Right_X NumberUpper_Right_Y NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberLine_Name Varchar(50)Line_Type Varchar(50)Shape Number


ArcSDE Server Setup

Detail columns of SEIS_FAULT_FINDER line class layer:

Objectid NumberHor_Code Varchar(50)Fault_Code Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date


ArcSDE Server Setup

Detail columns of FIELD_OIL_FINDER line class layer:

Objectid NumberObject_Code Varchar(50)Field_Name Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date


ArcSDE Server Setup

Detail columns of FIELD_GAS_FINDER line class layer:

Objectid NumberObject_Code Varchar(50)Field_Name Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date


Program Execution

Command line:SDEsynchFinder <input_file>

Input file format: <project_name> <instance_name>

Environment variables for running SDEsynch:ARCHOME = /rbappl/users/sde/arcexe81SDEHOME = /rbappl/users/sde/sdeexe81 ORACLE_HOME = /rbappl/users/ora8i/app/oracle/product/8.1.7LD_LIBRARY_PATH = $SDEHOME/lib:$ARCHOME/lib:$ORACLE_HOME/lib


Input Files

Current input file for SDE-Finder Synchronization:

MINAS F93RBIPE_T F93RBIPRUMBAI F93RBIPDURI F93DRIPBEKASAP F93DRIP

Current input file for SDE-Finder Synchronization (for Oil & Gas Field Boundary):

RUMBAI F93RBIP


Process Fork

Create a child process using fork() system call.

Fork is needed before calling execve() because execve() does not create new process and overlays the current process with new one.

Current process or parent process then waits for return code from child process before continuing execution.


Author Brief Biography

Ananta Dwi Bodhitama

Title: IT Engineer. Have been working for 5 years for CT IBU. Keen interest in G&G and GIS software development. Have attended various IT and earth science courses as well.

Formal Education:

Bachelor of Computer Science (1998) University of Indonesia, Indonesia.

Master of Applied Information Technology (2002)Universite de Rennes I, France. (scholarship)

gis and g&g database synchronization · process. we have successfully developed a tool to...

Documents