arcgis server performance and scalability—performance … · 2010-04-01 · performance factors...

68
ArcGIS Server Performance and ScalabilityPerformance Factors and Optimization Eric Miller and Andrew Sakowicz Eric Miller and Andrew Sakowicz

Upload: others

Post on 11-Jun-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGIS Server Performance and Scalability—yPerformance Factors and Optimization

Eric Miller and Andrew SakowiczEric Miller and Andrew Sakowicz

Page 2: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

IntroductionsIntroductionsIntroductionsIntroductions

•• Who are we?Who are we?–– ArcGISArcGIS Server Product DevelopmentServer Product Development

E t i I l t tiE t i I l t ti–– Enterprise Implementation Enterprise Implementation

•• Assumptions about you, our audience:Assumptions about you, our audience:–– Familiar with Familiar with ArcGISArcGIS DesktopDesktop–– Basic knowledge of Basic knowledge of ArcGISArcGIS Server Architecture and conceptsServer Architecture and concepts–– Comfortable with Web architecturesComfortable with Web architectures

22Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 3: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Schedule and OverviewSchedule and OverviewSchedule and OverviewSchedule and Overview

ArcGISArcGIS Server deploymentServer deployment and optimization strategiesand optimization strategiesArcGISArcGIS Server deploymentServer deployment and optimization strategiesand optimization strategies

–– Case Studies: Optimizing web apps that consume Case Studies: Optimizing web apps that consume ArcGISArcGIS Server ServicesServer Services–– Key performance factors in Key performance factors in ArcGISArcGIS Server system componentsServer system components–– Optimizing components prior to deploymentOptimizing components prior to deployment–– Recommended deployment architecturesRecommended deployment architecturesRecommended deployment architecturesRecommended deployment architectures–– Primer: Tuning Primer: Tuning ArcGISArcGIS Server deployments inServer deployments in--situsitu

Question and answer period.Question and answer period.

33

Please complete the session survey!Please complete the session survey!Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 4: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Case Studies: Poorly performing WebCase Studies: Poorly performing WebCase Studies: Poorly performing Web Case Studies: Poorly performing Web ApplicationsApplicationsDemos:Demos:

Demo: Client-side graphics vs. dynamic map serviceg p y p

Demo: Dynamic MapService Map Draw

44Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 5: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

•• Machine ArchitectureMachine Architecture

•• Data SourcesData Sources•• Data SourcesData Sources

•• ArcGISArcGIS Server FrameworkServer Framework

•• ArcGISArcGIS Server servicesServer services

•• Client applications and the ADFClient applications and the ADF

Goal: Show how to optimize these components to deliverGoal: Show how to optimize these components to deliver

55

Goal: Show how to optimize these components to deliver Goal: Show how to optimize these components to deliver highly performing highly performing ArcGISArcGIS Server deployments.Server deployments.

Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 6: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Machine ArchitecturePerformance Factors: Machine ArchitecturePerformance Factors: Machine ArchitecturePerformance Factors: Machine ArchitectureSelect adequate hardware to support desired performance and loadSelect adequate hardware to support desired performance and load

•• PerformancePerformance–– CPU SpeedCPU SpeedCPU SpeedCPU Speed

•• Published benchmarks: http://www.spec.org/cpu2006/results/cint2006.htmlPublished benchmarks: http://www.spec.org/cpu2006/results/cint2006.html•• Mapping: Select CPU with the highest Baseline CINT Rate/CoreMapping: Select CPU with the highest Baseline CINT Rate/Core•• GP: Select CPU with highest Baseline CFP Rate/CoreGP: Select CPU with highest Baseline CFP Rate/Core•• GP: Select CPU with highest Baseline CFP Rate/CoreGP: Select CPU with highest Baseline CFP Rate/Core

–– Disk speedDisk speed•• 15K RPM SAS Disks15K RPM SAS Disks

N t k l tN t k l t–– Network latencyNetwork latency•• Fast switchesFast switches

–– Memory Bus SpeedMemory Bus Speed

•• CapacityCapacity–– # CPU cores# CPU cores–– # CPU cores# CPU cores–– Network bandwidthNetwork bandwidth

•• Gigabit switches and NICs in Full Duplex modeGigabit switches and NICs in Full Duplex mode# Di k d ll# Di k d ll–– # Disks and controllers# Disks and controllers

–– Amount of RAMAmount of RAM66Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 7: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesSelect data source that provide optimal performanceSelect data source that provide optimal performance

•• VectorVectorVectorVector–– Data provider: DBMS, Data provider: DBMS, FileGDBFileGDB, , ShapefileShapefile, SDC, SDC–– Storage type: ESRI Storage type: ESRI ST_GeometryST_Geometry, Oracle SDO, , Oracle SDO,

SQL Server GeometrySQL Server GeometrySQL Server GeometrySQL Server Geometry

•• RasterRaster–– Data provider: Image Server, Data provider: Image Server, FileGDBFileGDB, DBMS, SDC, DBMS, SDC–– Compression: LZ77, JPEG, PNGCompression: LZ77, JPEG, PNG

Conduct a simple benchmark to select optimal data source formatConduct a simple benchmark to select optimal data source format

77

Conduct a simple benchmark to select optimal data source formatConduct a simple benchmark to select optimal data source formatAttend Attend ArcGISArcGIS Server Performance and Scalability Server Performance and Scalability –– Testing sessionTesting session

Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 8: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesRepresentative comparison of throughput variances based on workspace of Representative comparison of throughput variances based on workspace of service source data.service source data.

140000

Low Complexity Map: Throughput vs. data source

100000

120000

Hr)

60000

80000

roug

hput

(Tr/H

20000

40000

Thr

0FGDB_Local_URL SHP_Local_URL Ora11g_AS_URL SQLSvr_AS_URL Postgres_AS_URL

Machine Specs: 8 core: 4 2 5GHz Dual core CPU 8GB RAM

88

Machine Specs: 8-core: 4, 2.5GHz Dual-core CPU, 8GB RAM

Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 9: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Data SourcesPerformance Factors: Data Sources

•• Local vs. UNCLocal vs. UNC–– Locally sourced data always results in better throughput.Locally sourced data always results in better throughput.y y gy y g

120000High Complexity Map Service: Local vs. UNC (1 SOC)

100000

Hr)

60000

80000

put (

Tr/H

40000

Thro

ugh

0

20000

T

99

0FGDB_Local_URL FGDB_UNC_URL SHP_Local_URL SHP_UNC_URL

Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 10: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesPerformance Factors: Data SourcesOptimize DBMS configuration and conduct maintenanceOptimize DBMS configuration and conduct maintenance

•• DBMS configurationDBMS configuration

•• Create and maintain (rebuild) attribute indexesCreate and maintain (rebuild) attribute indexes•• Create and maintain (rebuild) attribute indexesCreate and maintain (rebuild) attribute indexes

•• Updating DBMS statisticsUpdating DBMS statistics

•• Versioning managementVersioning managementR il d tR il d t–– Reconcile and postReconcile and post

–– CompressCompress

1010

NonNon-- optimal DBMS may be a source of significant performance degradationoptimal DBMS may be a source of significant performance degradationDeveloper Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 11: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkPerformance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkSOMSOM

•• 165 map draw requests/sec per core165 map draw requests/sec per core•• 60% CPU utilization on SOM machine60% CPU utilization on SOM machine•• 0 027 seconds average response time0 027 seconds average response time•• 0.027 seconds average response time0.027 seconds average response time

1111Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 12: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkPerformance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkWeb Service ComponentsWeb Service Components

•• Web service handlersWeb service handlers–– Easily overloaded by MIME data (REST/SOAP returning MIME)Easily overloaded by MIME data (REST/SOAP returning MIME)

•• Network saturationNetwork saturation–– 72 cores can approach 40% Gigabit utilization72 cores can approach 40% Gigabit utilization–– Network I/O (fullNetwork I/O (full--duplex)duplex)

•• CPU utilizationCPU utilization

–– Use multiple Web InstancesUse multiple Web Instances

•• Virtual DirectoriesVirtual Directories–– Virtual output directory bottlenecksVirtual output directory bottlenecks

•• Network saturationNetwork saturation•• Disk performanceDisk performancepp

–– Use multiple output directories on different machines.Use multiple output directories on different machines.1212Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 13: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkPerformance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkWeb Service ComponentsWeb Service Components•• MIME MIME vsvs URL return typeURL return type–– For serving maps, MIME can scale better than URLFor serving maps, MIME can scale better than URLFor serving maps, MIME can scale better than URLFor serving maps, MIME can scale better than URL

•• Disk and UNC shares are often bottlenecks before network bandwidthDisk and UNC shares are often bottlenecks before network bandwidth

MIME performs better with smaller imagesMIME performs better with smaller images–– MIME performs better with smaller imagesMIME performs better with smaller images

180000MIME vs. URL

120000140000160000

(Tr/H

r)

6000080000

100000120000

ughp

ut ( High Complexity Map

(large images)Low Complexity Map

0200004000060000

Thro

u p y p(small images)

1313Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

0FGDB_Local_MIME FGDB_Local_URL

Page 14: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkPerformance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkWeb ServicesWeb Services

•• SecuritySecurity–– SSLSSL

–– LSASSLSASS•• http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.ahttp://support.esri.com/index.cfm?fa=knowledgebase.techarticles.a

rticleShow&d=32620rticleShow&d=32620rticleShow&d 32620rticleShow&d 32620

–– Web Tier SecurityWeb Tier SecurityUser/Role StoresUser/Role Stores•• User/Role StoresUser/Role Stores

•• Token ServerToken Server

1414Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 15: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkPerformance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkSOCSOC•• Optimal number of instances does vary significantly by complexity of Optimal number of instances does vary significantly by complexity of

service (complex map or GP model) and choice of source data type.service (complex map or GP model) and choice of source data type.V i i ti f 1 t 4 hi hV i i ti f 1 t 4 hi h i l ti i t CPUi l ti i t CPU•• Varies in practice from 1 to 4 highVaries in practice from 1 to 4 high--isolation instances per CPU coreisolation instances per CPU core

Variance of Service Instances by Source Data Type

3

3.5

hrou

ghpu

t

Variance of Service Instances by Source Data Type

1 5

2

2.5

core

at M

ax T

h

0.5

1

1.5

nces

per

CPU

0FGDB_Local FGDB_UNC SHP_Local SHP_UNC Ora11g_AS Ora11g_DC SQLSvr_AS SQLSvr_DC

Inst

a

1515Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 16: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server ServicesServer ServicesMapServiceMapService•• MxdMxd

–– Optimizing map symbols for performanceOptimizing map symbols for performancep g p y pp g p y p

–– Spatial indexSpatial index

–– Projecting on the flyProjecting on the fly

–– Optimizing map text and labels for performanceOptimizing map text and labels for performance•• Use annotationsUse annotations•• Avoid Avoid maplexmaplex

–– Use fast joins (no cross db joins)Use fast joins (no cross db joins)

–– Avoid wavelet compressionAvoid wavelet compression--based raster types (MrSid,JPEG2000)based raster types (MrSid,JPEG2000)

1616Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 17: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server ServicesServer ServicesMapServiceMapService

•• MxdMxd•• MxdMxd–– Simplify dataSimplify data

–– Set scale dependencySet scale dependency

•• Tuning Tools: Tuning Tools: gg–– MxdPerfStatMxdPerfStat

ArcMapArcMap 9 3 1 Analyzer Tool9 3 1 Analyzer Tool–– ArcMapArcMap 9.3.1 Analyzer Tool9.3.1 Analyzer Tool

1717Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 18: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Performance Factors: ArcGISArcGIS Server Map ServicesServer Map ServicesOptimizing Optimizing mxdmxd with with MXDPerfStatMXDPerfStathttp://arcscripts.esri.com/scripts.asp?eLang=&eProd=&perPage=10&eQuery=mxdperfstathttp://arcscripts.esri.com/scripts.asp?eLang=&eProd=&perPage=10&eQuery=mxdperfstatp p p p g p g y pp p p p g p g y pC:>C:>mxdperfstatmxdperfstat --mxdmxd Portland_Dev09_Bad.mxd Portland_Dev09_Bad.mxd --xyxy 7655029;652614 7655029;652614 --scale 8000 scale 8000

1818Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 19: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Demo: Optimizing Demo: Optimizing mxdmxd with with MXDPerfStatMXDPerfStat

•• Focus on performance impact of:Focus on performance impact of:–– Scale dependency (Scale dependency (SDE.TaxSDE.Tax Lots)Lots)–– Labeling on dense features (Labeling on dense features (TaxlotDenseLabelTaxlotDenseLabel))g (g ( ))–– Query Definition (Tax Lots Query Def)Query Definition (Tax Lots Query Def)

1919Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 20: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Demo: Optimizing Demo: Optimizing mxdmxd with with MXDPerfStatMXDPerfStat

2020Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 21: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services MapServiceMapService

O ti i iO ti i i dd ithith A MA M 9 3 1 A l T l9 3 1 A l T lOptimizing Optimizing mxdmxd with new with new ArcMapArcMap 9.3.1 Analyze Tool9.3.1 Analyze Tool

2121Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 22: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services MapServiceMapService

•• Optimizing MXDs with the new Optimizing MXDs with the new ArcMapArcMap 9.3.1 Analyze Tool: 9.3.1 Analyze Tool: DEMODEMO

2222Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 23: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Performance Factors: ArcGISArcGIS Server Services (Server Services (MapServiceMapService))Uncorrected vs. Corrected Source Uncorrected vs. Corrected Source ArcMapArcMap DocumentDocumentpp

/Hr)

65800

ansa

ctio

ns/

Corrected(Unloaded TT: 0.34 sec)

(2 1 Instances/core)

ghpu

t (Tr

a (2.1 Instances/core)

Uncorrected

2520Thro

u (Unloaded TT: 11.97 sec)(1.6 Instances/core)

13 17

Number of Instances2323Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 24: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server ServicesServer ServicesMapServiceMapService

•• Cache base mapCache base map

•• Display the operational layer as a cached map serviceDisplay the operational layer as a cached map service•• Display the operational layer as a cached map serviceDisplay the operational layer as a cached map service–– if your operational dataset changes infrequentlyif your operational dataset changes infrequently

•• Display the operational layer as clientDisplay the operational layer as client--side graphicsside graphics–– Do not fetch large amounts of featuresDo not fetch large amounts of features

•• Can take a long time to stream features to clientCan take a long time to stream features to clientCan take a long time to stream features to client.Can take a long time to stream features to client.•• Can take a long time to render in client.Can take a long time to render in client.

2424Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 25: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Performance Factors: ArcGISArcGIS Server ServicesServer ServicesMapServiceMapService

D Cli tD Cli t id hi d i iid hi d i iDemo: ClientDemo: Client--side graphics vs. dynamic map serviceside graphics vs. dynamic map service

2525Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 26: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance factors: Performance factors: ArcGISArcGIS Server Services Server Services MapServiceMapService

D M i iD M i iDemo: Measuring map service Demo: Measuring map service

private function onCreationComplete():void

{

logger.text += getTimer() + ' ms: Application creation completed.\n';

myThematicLayer.addEventListener(Event.COMPLETE, onTilesLoaded)

myMap.addEventListener(PanEvent.PAN_START, onPanStart);

myMap addEventListener(ZoomEvent ZOOM START onZoomStart);myMap.addEventListener(ZoomEvent.ZOOM_START, onZoomStart);

}

private function onTilesLoaded(event:Event):void

{

if (panStartFlag){

TimeTileComplete = getTimer() - TimeTileStart ;

TimeTileStart = 0

logger.text += "Tiles took " + TimeTileComplete + " ms to pan.\n";gg p p \ ;

panStartFlag = false;

}

if (zoomStartFlag){

Ti Til C l t tTi () Ti Til St tTimeTileComplete = getTimer() - TimeTileStart ;

TimeTileStart = 0

logger.text += "Tiles took " + TimeTileComplete + " ms to Zoom.\n";

zoomStartFlag= false;

2626

}

}Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 27: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services MapServiceMapServiceDemo: Measuring clientDemo: Measuring client--side graphicsside graphics

private function doThematicQuery():void

{

myGraphicsLayer.clear();

T k l t tQ URL t t t St i ()queryTask.url = textQueryURL.text.toString();

logger.text += getTimer() + " ms: Starting query...\n";

TimeQueryStart = getTimer();

queryTask.execute( query, new AsyncResponder( onResult, onFault ));

myGraphicsLayer.addEventListener(FlexEvent.UPDATE_COMPLETE,graphicsLayerUpdated);

function onResult( featureSet : FeatureSet, token : Object = null ) : void

{

TimeQueryDone = getTimer();TimeQueryDone = getTimer();

var TimeQueryDiff:uint = getTimer() - TimeQueryStart;

logger.text += getTimer() + " ms: Query took " + TimeQueryDiff + " ms";

logger.text += " [" + myGraphicsLayer.numGraphics + " features].\n";

}

2727Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 28: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GPServiceGPService

•• Use Local Jobs Directory Use Local Jobs Directory ––Greatest single performance factor.Greatest single performance factor.

9 3 1 ll i l d l t9 3 1 ll i l d l t––9.3.1 allows simple deployment9.3.1 allows simple deployment

S b t lS b t l tt•• Subset very large Subset very large rastersrasters

•• Use native types like GRIDUse native types like GRIDUse native types like GRIDUse native types like GRID

•• Use Layers instead of raw data from dbUse Layers instead of raw data from db

•• Use inUse in--memory feature data setsmemory feature data sets

2828Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 29: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GPServiceGPService•• PrePre--compute intermediate steps when possiblecompute intermediate steps when possible

•• Use local paths to data and resourcesUse local paths to data and resources

•• Avoid unneeded coordinate transformationsAvoid unneeded coordinate transformations

•• Add attribute indexesAdd attribute indexes•• Add attribute indexesAdd attribute indexes

•• Simplify dataSimplify datap yp y

Detailed instructions on the Detailed instructions on the Resource CenterResource Center at:at:htt // bh l i / i d kt /9 3/i d f ?id 1061& id 1044&thtt // bh l i / i d kt /9 3/i d f ?id 1061& id 1044&thttp://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=1061&pid=1044&tohttp://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=1061&pid=1044&to

picname=Performance_tips_for_geoprocessing_servicespicname=Performance_tips_for_geoprocessing_services

2929Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 30: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Image ServiceImage Service

•• JPEG compressed TIFF is the best performer by far (10JPEG compressed TIFF is the best performer by far (10--100x faster)100x faster)

•• TiledTiled rastersrasters even faster than single TIFFeven faster than single TIFF

3030Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 31: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeocodeGeocode and Globe Servicesand Globe Services

•• GeocodeGeocode ServicesServices–– Use Use ArcSDEArcSDE address locators for single address address locators for single address geocodinggeocoding–– Use local locator files instead of UNC locator files for batch Use local locator files instead of UNC locator files for batch U U CU U C

geocodinggeocoding

•• Globe ServicesGlobe Services•• Globe ServicesGlobe Services–– Use BillUse Bill--boarded label annotations.boarded label annotations.

3131Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 32: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Mobile ServiceMobile Service

•• Document PreparationDocument Preparation–– Minimize operational layersMinimize operational layers–– Cache Cache basemapbasemap layerslayersCC pp yy

•• Service ConfigurationService ConfigurationT t k t t l i h i d 250 MB t id t kT t k t t l i h i d 250 MB t id t k–– Try to keep total service cache size under 250 MB to avoid network Try to keep total service cache size under 250 MB to avoid network swapping. swapping.

–– Upload/Download size limits (200K upload/4MB download) Upload/Download size limits (200K upload/4MB download) http://www banmanpro com/support2/File Upload limits asphttp://www banmanpro com/support2/File Upload limits asphttp://www.banmanpro.com/support2/File_Upload_limits.asphttp://www.banmanpro.com/support2/File_Upload_limits.asp

•• Usage considerationsUsage considerations–– Avoid batch postings in favor of frequent updatesAvoid batch postings in favor of frequent updates

3232Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 33: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeodataGeodata ServiceService

•• Data PreparationData Preparation–– Perform regular version maintenance (keep versioning tree small, Perform regular version maintenance (keep versioning tree small,

compress, schedule synchronizations, etc) compress, schedule synchronizations, etc) http://blogs.esri.com/Dev/blogs/geodatabase/archive/2008/11/25/Geodahttp://blogs.esri.com/Dev/blogs/geodatabase/archive/2008/11/25/Geodatabasetabase--replicationreplication--andand--compress.aspxcompress.aspx

–– Use 1 way replicas over 2 way replicas when possibleUse 1 way replicas over 2 way replicas when possible

–– Consider 2 way replicas instead of checkConsider 2 way replicas instead of check--out replicasout replicas

W llW ll d fi d d t d ld fi d d t d l–– WellWell--defined data modeldefined data model

3333Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 34: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors:Performance Factors: ArcGISArcGIS Server ServicesServer ServicesPerformance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeodataGeodata ServiceService

•• Service ConfigurationService ConfigurationService ConfigurationService Configuration–– Server Object usage timeout (set larger than 10 min default)Server Object usage timeout (set larger than 10 min default)

/ f S (200 /4 )/ f S (200 /4 )–– Upload/Download default IIS size limits (200K upload/4MB download)Upload/Download default IIS size limits (200K upload/4MB download)

•• Usage considerationsUsage considerationsgg–– Client HTTP timeout Client HTTP timeout

http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.artichttp://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=35971leShow&d=35971leShow&d 35971leShow&d 35971

–– http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.artichttp://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=35972leShow&d=35972leShow&d=35972leShow&d=35972

3434Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 35: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Performance Factors: Client Applications and thePerformance Factors: Client Applications and thePerformance Factors: Client Applications and the Performance Factors: Client Applications and the ADFADF

ff•• Meet functional requirementsMeet functional requirements•• Ensure highest performance and scalabilityEnsure highest performance and scalability•• Leverage existing skills and standardsLeverage existing skills and standardsg gg g

•• Different API’s have different performance and functionality Different API’s have different performance and functionality profiles e gprofiles e gprofiles, e.g.profiles, e.g.•• .Net.Net ADFADF•• Java ADFJava ADF•• JavaScriptJavaScript

ADF vs. Flex Application Simple Dynamic Map

Rendering Comparison•• JavaScriptJavaScript•• Flex APIFlex API•• SilverLightSilverLight

11.05

1.11.15

Tim

e (s

ec)

Rendering Comparison

0.80.85

0.90.95

.NET ADF FlexRes

p[on

se T

3535

Conduct a simple benchmark to select optimal solutionConduct a simple benchmark to select optimal solutionAttend Attend ArcGISArcGIS Server Performance and Scalability Server Performance and Scalability –– Testing sessionTesting sessionDeveloper Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 36: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Testing Testing ArcGISArcGIS Server Architectures (Data/Maps)Server Architectures (Data/Maps)

MapServiceMapService DescriptionDescription High Complexity

•• High ComplexityHigh Complexity–– 20 vector layers20 vector layers–– 3 raster layers3 raster layers3 raster layers3 raster layers–– Return image size ~ 100Return image size ~ 100--300 K300 K–– Long transaction times (~0.3 sec)Long transaction times (~0.3 sec)

•• Low ComplexityLow Complexity–– 1 vector layer1 vector layer–– Return image size ~ 5Return image size ~ 5--50 K50 K

Low Complexity–– Return image size ~ 5Return image size ~ 5--50 K50 K–– Short transaction times (0.06 sec)Short transaction times (0.06 sec)

•• BothBoth•• BothBoth–– 9.3.1 Fast “Optimized” 9.3.1 Fast “Optimized” MapServiceMapService–– Data local on SOC machinesData local on SOC machines

FGDB dataFGDB data–– FGDB dataFGDB data

3636Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 37: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Testing Testing ArcGISArcGIS Server Architectures (Methodology)Server Architectures (Methodology)•• SoftwareSoftware

–– MicroSoftMicroSoft Team Edition for Software Testers Team Edition for Software Testers –– MicroSoftMicroSoft Load Test Agent/ControllerLoad Test Agent/Controller

•• Unit TestUnit Test–– Export dynamic maps via SOAP serviceExport dynamic maps via SOAP service–– 800x600 pixels800x600 pixels–– PNG 8PNG 8

Random extents and scaleRandom extents and scale--levels that are relevels that are re--used between test runsused between test runs–– Random extents and scaleRandom extents and scale--levels that are relevels that are re--used between test runsused between test runs

•• Load TestsLoad Tests–– Unit test is executed repeatedly by increasing numbers of test clients. Unit test is executed repeatedly by increasing numbers of test clients.

•• HardwareHardware•• Hardware Hardware –– 4 Dual4 Dual--Core 2.5 GHz CPU, 16 GB RAM, 15K RPM local disksCore 2.5 GHz CPU, 16 GB RAM, 15K RPM local disks

•• More Info? See detailed session.More Info? See detailed session.3737Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 38: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “classic” MIMEServer Architectures: “classic” MIMEArcGISArcGIS Server Architectures: classic MIMEServer Architectures: classic MIMENLB + SOM + N SOC + M Web Servers (WS)NLB + SOM + N SOC + M Web Servers (WS)

NLBNLB

WSWS11 WSWS22 WSWSmm……AdvantagesAdvantages

•• “classic”, users are familiar“classic”, users are familiar•• Best performingBest performing

SOMSOM

•• Best performingBest performing•• Easy maintenanceEasy maintenance

DisadvantagesDisadvantages•• SOM is a single point of failureSOM is a single point of failure

SOCSOC11 SOCSOC22 SOCSOCSOCSOC33SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33 ……

3838Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 39: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” MIMEServer Architectures: “Classic” MIMEPerformance CharacteristicsPerformance Characteristics NLBNLB

1800WSWS11 WSWS22 WSWSmm

……

NLBNLB

ArcGIS Server Scalability (High Complexity)

1400

1600

1800

Use

rs

SOMSOM

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

ParametersParameters•• Document Complexity: Document Complexity: HighHigh•• Return Type:Return Type: MIMEMIME1000

1200

1400

ulat

ed U

Return Type: Return Type: MIMEMIME•• Web Servers: Web Servers: 11--22•• SOM Machines: SOM Machines: 11•• SOC Machines:SOC Machines: 11--9 (72 cores)9 (72 cores)600

800

er o

f Sim

SOC Machines: SOC Machines: 11 9 (72 cores)9 (72 cores)

ObservationsObservations•• Linearity breaks down atLinearity breaks down at 77 SOCSOC200

400

Num

be

Linearity breaks down at Linearity breaks down at 77 SOC SOC machines (machines (425,000 425,000 TrTr/Hr/Hr), (), (49 49 corescores))

•• 22ndnd Web Server restores linearityWeb Server restores linearity

01 2 3 4 5 6 7 8 9

Number of SOC Machines yyNumber of SOC Machines

Actual: 1 WS Actual: 2 WS Ideal 3939Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 40: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” MIMEServer Architectures: “Classic” MIMEPerformance CharacteristicsPerformance Characteristics NLBNLB

WSWS11 WSWS22 WSWSmm……

NLBNLB

ArcGIS Server Scalability (Low Complexity)

3500SOMSOM

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……3000

3500

Use

rs

ParametersParameters•• Document Complexity: Document Complexity: LowLow•• Return Type:Return Type: MIMEMIME2000

2500

ulat

ed U

Return Type: Return Type: MIMEMIME•• Web Servers: Web Servers: 11--22•• SOM Machines: SOM Machines: 11•• SOC Machines:SOC Machines: 11--9 (72 cores)9 (72 cores)

1500

r of S

im

SOC Machines: SOC Machines: 11 9 (72 cores)9 (72 cores)

ObservationsObservations•• Linearity breaks down atLinearity breaks down at 88 SOCSOC

500

1000

Num

be

Linearity breaks down at Linearity breaks down at 88 SOC SOC machines (machines (1,150,000 1,150,000 TrTr/Hr/Hr), (), (56 56 corescores))

•• 22ndnd Web Server restores linearityWeb Server restores linearity

01 2 3 4 5 6 7 8 9

N b f SOC M hiyy

Number of SOC Machines

Actual: 1 WS Actual: 2 WS Ideal 4040Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 41: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Server Architectures: “Classic” MIMEClassic” MIMEPerformance CharacteristicsPerformance Characteristics

NLBNLB

Observations (continued)Observations (continued)

WSWS11 WSWS22 WSWSmm

SOMSOM

……

Observations (continued)Observations (continued)

•• Note increase in SOM CPU Note increase in SOM CPU SOM CPU Utilization

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

utilization between tests utilization between tests run with 1 web server run with 1 web server ((blueblue) and 2 web servers ) and 2 web servers (( ))((greengreen).).

IIS CPU Utilization

4141Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 42: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” MIMEServer Architectures: “Classic” MIMEPerformance CharacteristicsPerformance Characteristics

NLBNLB

ConclusionsConclusions

WSWS11 WSWS22 WSWSmm

SOMSOM

……

ConclusionsConclusions

•• Small decrease in SOM Small decrease in SOM

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

1 Web Server2 Web Serversutilization results in utilization results in significant (%10) significant (%10) underutilization of SOC underutilization of SOC

hihimachines.machines.

•• Add additional web server Add additional web server machines when IIS begins machines when IIS begins to compete with SOM for to compete with SOM for CPU.CPU.

SOC CPU Utilization4242Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 43: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “classic” MIMEServer Architectures: “classic” MIMEArcGISArcGIS Server Architectures: classic MIMEServer Architectures: classic MIMENLB + SOM + N SOC + M Web Servers (WS) + X File Servers NLB + SOM + N SOC + M Web Servers (WS) + X File Servers (FS)(FS)

NLB(w/Affinity)NLB(w/Affinity)

Ad tAd t

WSWS11 WSWS22 WSWSmm……

AdvantagesAdvantages•• “classic”, users are familiar“classic”, users are familiar•• Best performing (of URL Best performing (of URL cfgscfgs))

SOMSOMFSFS FSFSFSFS ……

•• Easy maintenanceEasy maintenance

DisadvantagesDisadvantagesSOCSOC11 SOCSOC22 SOCSOCSOCSOC33 DisadvantagesDisadvantages•• SOM Single Point of FailureSOM Single Point of Failure•• FS Single Point of FailureFS Single Point of Failure

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33 ……

•• FS UNC PenaltyFS UNC Penalty

4343Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 44: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics

NLB(w/Affinity)NLB(w/Affinity)

ArcGIS Server Scalability (High Complexity)

1000

WSWS11 WSWS22 WSWSmm

SOMSOM

……

FSFS FSFSFSFS ……

800

900

1000

sers

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

ParametersParameters•• Document Complexity: Document Complexity: HighHigh•• Return Type:Return Type: URLURL

600

700

mulated

 Us

Return Type: Return Type: URLURL•• Web Servers: Web Servers: 11--22•• SOM Machines: SOM Machines: 11•• SOC Machines:SOC Machines: 11--5 (45 cores)5 (45 cores)300

400

500

ber of Sim

SOC Machines: SOC Machines: 11 5 (45 cores)5 (45 cores)

ObservationsObservations•• Linearity breaks down atLinearity breaks down at 44 SOCSOC

100

200

300

Num

b

Linearity breaks down at Linearity breaks down at 44 SOC SOC machines (machines (275,000 275,000 TrTr/Hr/Hr), (), (36 36 corescores))

•• 22ndnd Web Server Web Server does not does not restore restore

0

1 2 3 4 5

Number of SOC Machines linearitylinearityNumber of SOC Machines

Actual Ideal4444Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 45: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics

NLB(w/Affinity)NLB(w/Affinity)

ArcGIS Server Scalability (Low Complexity)

1800

WSWS11 WSWS22 WSWSmm

SOMSOM

……

FSFS FSFSFSFS ……

1400

1600

800

ers

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

ParametersParameters•• Document Complexity: Document Complexity: LowLow•• Return Type:Return Type: URLURL1000

1200

ulated

 Use

Return Type: Return Type: URLURL•• Web Servers: Web Servers: 11--22•• SOM Machines: SOM Machines: 11•• SOC Machines:SOC Machines: 11--5 (45 cores)5 (45 cores)600

800

ber of Sim

SOC Machines: SOC Machines: 11 5 (45 cores)5 (45 cores)

ObservationsObservations•• Linearity breaks down atLinearity breaks down at 33 SOCSOC200

400Num

Linearity breaks down at Linearity breaks down at 33 SOC SOC machines (machines (350,000 350,000 TrTr/Hr/Hr), (), (27 27 corescores))

•• 22ndnd Web Server Web Server does not does not restore restore

0

1 2 3 4 5

b f hi linearitylinearityNumber of SOC Machines

Actual Ideal 4545Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 46: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics

NLB(w/Affinity)NLB(w/Affinity)

WSWS11 WSWS22 WSWSmm

SOMSOM

……

FSFS FSFSFSFS ……

Observations (continued)Observations (continued)Di k I/O UNC h i t iDi k I/O UNC h i t i

SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……

•• Disk I/O on UNC share is not excessive.Disk I/O on UNC share is not excessive.

•• Network traffic to UNC share does not even approach peak disk Network traffic to UNC share does not even approach peak disk I/O transfer rates. I/O transfer rates.

ConclusionsConclusionsConclusionsConclusions•• Inherent latency in UNC/CIFS share is the bottleneck.Inherent latency in UNC/CIFS share is the bottleneck.

A i l i t i l fil If it bA i l i t i l fil If it b•• A single service must use a single file server. If it becomes A single service must use a single file server. If it becomes popular it doesn’t scale very well.popular it doesn’t scale very well.

•• If many services exist, consider mapping each service to its own If many services exist, consider mapping each service to its own UNC share to improve performance.UNC share to improve performance.

4646Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 47: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures (MIME): Active/ActiveServer Architectures (MIME): Active/ActiveArcGISArcGIS Server Architectures (MIME): Active/Active Server Architectures (MIME): Active/Active SOMsSOMs2 SOM + M SOC + N WS2 SOM + M SOC + N WS

WSWS WSWS

NLBNLB

WSWS WSWS

AdvantagesAdvantages

WSWS11 WSWS22

SOMSOM11 SOMSOM22

WSWS33 WSWSnn

AdvantagesAdvantages•• Highly AvailableHighly Available•• Web Servers well utilizedWeb Servers well utilized

SOCSOC SOCSOC SOCSOCSOCSOCDisadvantagesDisadvantages

•• Duplication of service Duplication of service configurationsconfigurations

SOCSOC11 SOCSOC22 SOCSOCmm……SOCSOC33

configurationsconfigurations•• SOMs don’t know how SOCs SOMs don’t know how SOCs

are utilized by other SOMs.are utilized by other SOMs.

4747Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 48: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: Active/PassiveServer Architectures: Active/PassiveArcGISArcGIS Server Architectures: Active/Passive Server Architectures: Active/Passive SOMsSOMs2 SOM + M SOC + N WS2 SOM + M SOC + N WS

NLBNLBNLBNLB

SS WSWS WSWS WSWSWSWS11

AA PP

WSWS22

AA PP

WSWS33

AA PP

WSWSnn

AA PP

AdvantagesAdvantages•• Highly AvailableHighly Available•• Independent SOMsIndependent SOMs

SOMSOM11 SOMSOM22

Independent SOMsIndependent SOMs•• All web servers utilizedAll web servers utilized•• All SOC machines utilizedAll SOC machines utilized

SOCSOC11 SOCSOC22 SOCSOCmm……SOCSOC33

DisadvantagesDisadvantages•• MaintenanceMaintenance

•• Only available in Java at Only available in Java at 9.3.1 (will be available in 9.3.1 (will be available in

.NET at 9.4).NET at 9.4) 4848Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 49: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “AllServer Architectures: “All--inin--ones” (MIMEones” (MIMEArcGISArcGIS Server Architectures: AllServer Architectures: All--inin--ones (MIME ones (MIME or URL)or URL)N WS/SOM/SOC machinesN WS/SOM/SOC machines

Ad tAd t

NLB (w/affinity)NLB (w/affinity)

AdvantagesAdvantages•• Highly AvailableHighly Available•• Some ease of configuration (no Some ease of configuration (no

di t ib t d i t ll ti )di t ib t d i t ll ti )

WSWS11 WSWS22 WSWSnn

SOMSOM11 ……SOMSOM22 SOMSOMnn distributed installation)distributed installation)•• Easily imaged and VirtualizedEasily imaged and Virtualized•• Supports URL or MIME return Supports URL or MIME return

type with same linear scalabilitytype with same linear scalability

SOCSOC11 SOCSOC22 SOCSOCnn

22 nn

type with same linear scalabilitytype with same linear scalability

DisadvantagesDisadvantagesDisadvantagesDisadvantages•• Duplication of service Duplication of service

configurationsconfigurationsR d d f (SOCR d d f (SOC•• Reduced performance (SOCs Reduced performance (SOCs compete with SOM and WS)compete with SOM and WS)

4949Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 50: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” MIMEServer Architectures: “Classic” MIME

1V_4C_16R 4V_1C_2R

ArcGISArcGIS Server Architectures: Classic MIMEServer Architectures: Classic MIMEVirtualization Penalty with Virtualization Penalty with VMWareVMWare

Physical Machine: 4 CPU/16GB RAM

VM: 4 CPU/16GB RAM

Physical Machine: 4 CPU/16GB RAM

VM: 1 CPU/2GB RAM VM: 1 CPU/2GB RAM

SOM SOC WS SOM SOC WS SOC

VM: 1 CPU/2GB RAM VM: 1 CPU/2GB RAM

2V 2C 8R 4V 1C 4R

SOC SOC

Physical Machine: 4 CPU/16GB RAM

VM: 2 CPU/8GB RAM VM: 2 CPU/8GB RAM

2V_2C_8R

Physical Machine: 4 CPU/16GB RAM

VM: 1 CPU/4GB RAM VM: 1 CPU/4GB RAM

4V_1C_4R

SOM SOC WS SOC SOM SOC WS SOC

VM: 1 CPU/4GB RAM VM: 1 CPU/4GB RAM

5050Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

SOC SOC

Page 51: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

ArcGISArcGIS Server Architectures: “Classic” MIMEServer Architectures: “Classic” MIMEArcGISArcGIS Server Architectures: Classic MIMEServer Architectures: Classic MIMEVirtualization Penalty with Virtualization Penalty with VMWareVMWare

5151Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 52: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning PrimerTuning PrimerTuning Primer

•• Profile and tune single user transaction response timeProfile and tune single user transaction response time•• Optimize Optimize ArcMapArcMap mxdmxd documentdocument•• Create attribute indexes for query and search tasksCreate attribute indexes for query and search tasks•• Create attribute indexes for query and search tasksCreate attribute indexes for query and search tasks•• Optimize services (cache, merge, image type)Optimize services (cache, merge, image type)•• Optimize application requestsOptimize application requests

–– Number of requests per transactionNumber of requests per transaction–– Browser cachingBrowser caching

Server cachingServer caching–– Server cachingServer caching•• DBMS configurationDBMS configuration

5252Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 53: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning Primer

Browser

Tuning PrimerTuning PrimerProfile user transaction response timeProfile user transaction response time

Web Server

Total Response Time (t1-t2)

A test is executed at the web browser.

It measures web browser call’s elapsed time (roundtrip between

t1 t2

SOMWait Time

p ( pbrowser and data source)

SOM

Usage Time SOCUsage Time

Search & Retrieval Time

SDE/DBMS

5353Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 54: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning PrimerTuning PrimerTuning PrimerWeb diagnostic tools: Fiddler2Web diagnostic tools: Fiddler2

5454Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 55: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning PrimerTuning PrimerTuning PrimerWeb diagnostic tools: Fiddler2Web diagnostic tools: Fiddler2

•• Understand each request URLUnderstand each request URL•• Verify cache requests are from virtual directory, not dynamic map Verify cache requests are from virtual directory, not dynamic map

serviceservice•• Validate host origin (reverse proxy)Validate host origin (reverse proxy)Validate host origin (reverse proxy)Validate host origin (reverse proxy)•• Profile each transaction response timeProfile each transaction response time

5555Attend Attend ArcGISArcGIS Server Performance and Scalability Server Performance and Scalability –– Testing sessionTesting sessionDeveloper Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 56: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning PrimerTuning PrimerTuning PrimerWeb diagnostic tools: Web diagnostic tools: TamperdataTamperdata, , YslowYslow, Visual , Visual Studio TTStudio TT

5656Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 57: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning Primer

Browser

Tuning PrimerTuning PrimerAnalyze SOM/SOC statisticsAnalyze SOM/SOC statistics

Web Server

Transmission Time

SOMWait Time SOM

Usage Time

AGS context server statistics using ArcCatalog, Manager or logs provide

aggregate and detailed information to help reveal the cause of the

t1 t2

SOCUsage Time performance problem.

Search & Retrieval Time

SDE/DBMS

5757Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 58: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning PrimerTuning PrimerTuning PrimerTuning PrimerAnalyze SOM/SOC statisticsAnalyze SOM/SOC statistics•• ArcCatalogArcCatalog

•• New 9.3.1 detailed log New 9.3.1 detailed log -- set to verboseset to verbose

<Msg time="2009-03-16T12:23:22" type="INFO3" code="103021" target="Portland.MapServer" methodName="FeatureLayer.Draw"

machine="myWebServer" process="2836" thread="3916" elapsed="0.05221">Executingmachine myWebServer process 2836 thread 3916 elapsed 0.05221 Executing query.</Msg>

<Msg time="2009-03-16T12:23:23" type="INFO3" code="103019" t t "P tl d M S " th dN "Si l R d D "target="Portland.MapServer" methodName="SimpleRenderer.Draw"

machine="myWebServer" process="2836" thread="3916">Feature count: 27590</Msg>

<Msg time="2009-03-16T12:23:23" type="INFO3" code="103001"

5858

Msg time 2009 03 16T12:23:23 type INFO3 code 103001 target="Portland.MapServer" methodName="Map.Draw" machine="myWebServer"

process="2836" thread="3916" elapsed="0.67125">End of layer draw: STREETS</Msg> Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 59: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

TuningTuning MxdMxd map documentmap documentTuning Tuning MxdMxd map documentmap documentNew New ArcMapArcMap 9.3.1 Analyze Tool9.3.1 Analyze Tool

5959Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 60: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

TuningTuning MxdMxd map documentmap documentTuning Tuning MxdMxd map documentmap documentmxdperfstatmxdperfstat

http://arcscripts.esri.com/scripts.asp?eLang=&eProd=&perPage=10&eQuery=mxdperfshttp://arcscripts.esri.com/scripts.asp?eLang=&eProd=&perPage=10&eQuery=mxdperfstattattattat

Item At Scale Layer NameRefresh 

Time (sec)Recommendations Features Vertices Labeling

Geography Phase (sec)

Graphics Phase (sec)

Cursor Phase (sec)

DBMS CPU

DBMS LIO

Simplify labeling, symbology: GraphicsPhase=1.42; simplify geometry and/or set label scale; convert polygon to polyline: vertices fetched=200001; i lif t d/ t l b l

41 1,000TaxlotDenseLabel 1.93simplify geometry and/or set label scale: vertices fetched=200001; 1 200,001 TRUE 0.45 1.42 1.04 0.02 266

42 1,000TaxlotDenseNoLabel 0.53simplify geometry: vertices fetched=200001; 1 200,001 FALSE 0.45 0.02 0.9 0.02 140

6060Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 61: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data sourcesTuning Data sources

Browser

Tuning Data sourcesTuning Data sources

Web Server

Transmission Time

SOMWait Time SOM

Usage Time SOCUsage Time

Search & Retrieval Time

SDE/DBMS

6161Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 62: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data SourcesTuning Data Sources –– Oracle TraceOracle TraceTuning Data Sources Tuning Data Sources Oracle TraceOracle Trace

select username, select username, sidsid, serial#, program, , serial#, program, logon_timelogon_timefrom from v$sessionv$session where username='STUDENT';where username='STUDENT';

USERNAME SID SERIAL# PROGRAM LOGON_TIMUSERNAME SID SERIAL# PROGRAM LOGON_TIM------------------------------------------------------------ -------------------- -------------------- -------------------------------------------------------------------------------------------------------------------------------- ----STUDENT 132 31835 gsrvr.exe 23STUDENT 132 31835 gsrvr.exe 23--OCTOCT--0606

SQL> connect sys@gis1_andrews as SQL> connect sys@gis1_andrews as sysdbasysdbaEnter password:Enter password:Connected.Connected.SQL> execute SQL> execute sys.dbms_system.set_evsys.dbms_system.set_ev(132,31835,10046,12,'');(132,31835,10046,12,'');

PL/SQL procedure successfully completed.PL/SQL procedure successfully completed.

6262

DBMS trace is a very powerful diagnostic toolDBMS trace is a very powerful diagnostic toolDeveloper Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 63: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data SourcesTuning Data Sources –– Oracle Trace (continued)Oracle Trace (continued)Tuning Data Sources Tuning Data Sources Oracle Trace (continued)Oracle Trace (continued)Starting Oracle trace using a custom Starting Oracle trace using a custom ArcMapArcMap UIControlUIControl

Private Sub OracleTrace_Click()

. . .

Set pFeatCls = pFeatLyr.FeatureClass

Set pDS = pFeatCls

Set pWS = pDS.Workspace

sTraceName = InputBox("Enter <test_name><email>")

pWS.ExecuteSQL ("alter session set tracefile_identifier = '" & sTraceName & "'")

pWS.ExecuteSQL ("ALTER SESSION SET events '10046 trace name context f l l 12'")forever, level 12'")

. . .

End Sub

6363Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 64: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data SourcesTuning Data Sources –– Oracle Trace (continued)Oracle Trace (continued)Tuning Data Sources Tuning Data Sources Oracle Trace (continued)Oracle Trace (continued)

SQL ID : 71py6481sj3xu

SELECT 1 SHAPE, TAXLOTS.OBJECTID, TAXLOTS.SHAPE.points,TAXLOTS.SHAPE.numpts,, , p , p ,

TAXLOTS.SHAPE.entity,TAXLOTS.SHAPE.minx,TAXLOTS.SHAPE.miny,

TAXLOTS.SHAPE.maxx,TAXLOTS.SHAPE.maxy,TAXLOTS.rowid

FROM SDE.TAXLOTS TAXLOTS WHERE SDE.ST_EnvIntersects(TAXLOTS.SHAPE,:1,:2,:3,:4) = 1

ll t l d di k tcall count cpu elapsed disk query current rows

------- ------ -------- ---------- ---------- ---------- ---------- ----------

Parse 0 0.00 0.00 0 0 0 0

Execute 1 0.07 0.59 115 1734 0 0

Fetch 242 0.78 12.42 2291 26820 0 24175

------- ------ -------- ---------- ---------- ---------- ---------- ----------

total 243 0.85 13.02 2406 28554 0 24175

Elapsed times include waiting on following events:Elapsed times include waiting on following events:

Event waited on Times Max. Wait Total Waited

---------------------------------------- Waited ---------- ------------

SQL*Net message to client 242 0.00 0.00

db file sequential read 2291 0.39 11.69

SQL*Net more data to client 355 0.00 0.02

SQL*Net message from client 242 0.03 0.54

********************************************************************************

6464Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 65: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data SourcesTuning Data Sources –– Oracle Trace (continued)Oracle Trace (continued)Tuning Data Sources Tuning Data Sources Oracle Trace (continued)Oracle Trace (continued)

•• Analyze queryAnalyze query–– Elapsed time [sec] (CPU + wait event)Elapsed time [sec] (CPU + wait event)–– CPU [sec]CPU [sec]C U [ ]C U [ ]–– Query (Oracle blocks e.g. 8K read from memory)Query (Oracle blocks e.g. 8K read from memory)–– Disk (Oracle blocks read from disk)Disk (Oracle blocks read from disk)

W it t [ ] db fil ti l dW it t [ ] db fil ti l d–– Wait event [sec], e.g. db file sequential read Wait event [sec], e.g. db file sequential read –– Rows fetchedRows fetched

•• Example (cost of physical reads):Example (cost of physical reads):–– Elapsed time = 13.02 secElapsed time = 13.02 sec

CPU = 0 85 secCPU = 0 85 sec–– CPU = 0.85 sec CPU = 0.85 sec –– Disk= 2291 blocksDisk= 2291 blocks–– Wait event (db file sequential read )=11.69 secWait event (db file sequential read )=11.69 sec–– Rows fetched = 24175Rows fetched = 24175

6565Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 66: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Tuning Data SourcesTuning Data Sources –– SQL ProfilerSQL ProfilerTuning Data Sources Tuning Data Sources SQL ProfilerSQL Profiler

6666Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010

Page 67: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

SummarySummarySummarySummary•• Today we covered:Today we covered:

–– Case Study: Optimizing a Flex API web applicationCase Study: Optimizing a Flex API web application

R i d k f f t iR i d k f f t i A GISA GIS S t tS t t–– Reviewed key performance factors in Reviewed key performance factors in ArcGISArcGIS Server system componentsServer system components

–– Described strategies for component optimizationDescribed strategies for component optimization

–– Recommended deployment architecturesRecommended deployment architectures

–– Described methodology for tuning Described methodology for tuning ArcGISArcGIS Server deployments inServer deployments in--situsitu

6767

Still have questions?Still have questions?Developer Summit Technical Workshop Developer Summit Technical Workshop

20102010

Page 68: ArcGIS Server Performance and Scalability—Performance … · 2010-04-01 · Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components

Additional ResourcesAdditional ResourcesAdditional ResourcesAdditional ResourcesQuestions, answers and information…Questions, answers and information…

•• Other sessionsOther sessions–– ArcGISArcGIS Server Performance Server Performance

and Scalability and Scalability -- TestingTesting

•• ESRI Resource CentersESRI Resource Centers–– PPTs, code and videoPPTs, code and video

yy gg

resources.esri.comresources.esri.com

6868Developer Summit Technical Workshop Developer Summit Technical Workshop 20102010