No Slide Title

The Integration of 2 Science Gateways:CyberGIS + OpenTopographyChoonhan Youn, Nancy Wilkins-Diehr, SDSCChristopher Crosby, UNAVCO (formerly SDSC)Anand Padmanabhan, Myunghwa Hwang, Yan Liu, Shaowen WangUniversity of Illinois at Urbana-ChampaignSAN DIEGO SUPERCOMPUTER CENTERWhat are Science Gateways?Community-designed applications, often Web-based, used to conduct scienceCommonly known as web portalsGateways term coined in 2003 in the TeraGrid programMany examples in many fieldsCyberGISProtein Data BanknanoHUBSAN DIEGO SUPERCOMPUTER CENTERA natural result of the impact of the Internet on worldwide communication and information retrieval

Implications on the conduct of science are still evolving1980s, Early gateways, National Center for Biotechnology Information BLAST server, search results sent by email, still a working portal today1989 World Wide Web developed at CERN1992 Mosaic web browser developed1995 International Protein Data Bank Enhanced by Computer Browser2004 TeraGrid project director Rick Stevens recognized growth in scientific portal development and proposed the Science Gateway ProgramToday, Web 3.0 and programmatic exchange of data between web pages Simultaneous explosion of digital informationGrowing analysis needs in many, many scientific areasSensors, telescopes, satellites, digital images, video, genome sequencers#1 machine on Top500 today over 10,000x more powerful than all combined entries on the first list in 1993Only 20 years since the release of Mosaic!SAN DIEGO SUPERCOMPUTER CENTER3

vt100 in the 1980s and alogin window on Ranger todaySAN DIEGO SUPERCOMPUTER CENTER4Why gateways?Increasing utility of the Web for scienceAnd increased need to deal with big dataFrom sensors, instruments (telescopes, genome sequencers), supercomputersCommunity-designed interfaces directly address community needsComplex tasks best not re-addressed by every scientistCoupling multi-scale codesKeeping large numbers of bioinformatics programs up to dateManaging thousands of ensemble jobsDemocratized access to supercomputersAnyone regardless of location can have access to top quality resourcesScalable support - questions on gateway use go to gateway developers

5SAN DIEGO SUPERCOMPUTER CENTERGateways on NSFs front page

67/16/12SAN DIEGO SUPERCOMPUTER CENTERToday, there are approximately 35 gateways using XSEDE

SAN DIEGO SUPERCOMPUTER CENTER7The ProblemCoupling of two independent geospatial software environments OpenTopography (OT)CyberGIS GatewayDemonstrate this coupling in action driven by an applicationViewshed application on CyberGIS gateway is a good candidateConsumes high-resolution Digital Elevation Model (DEM) dataDisconnect between data-driven and analytics-driven gatewaysSeamless fusion of large spatial data and upscale analytics tools without losing usabilityAbstract away complex technicality of software integration8SAN DIEGO SUPERCOMPUTER CENTERGoalsImprove usabilityData need to be easily available to users when CyberGIS analytics is being plannedSeamless access to OpenTopography (OT) data through the CyberGIS gatewayAccess OT data through common user interfaceService integration and chaining Allow the gateway users to directly apply viewshed analysis to OT datasetsReuse existing user interfaces when possibleBenefits both communities

9SAN DIEGO SUPERCOMPUTER CENTEROpenTopographyNSF Facility funded by Earth Sciences Instrumentation and FacilitiesOffice of CyberinfrastructureAim to increase the amount of science-oriented LiDAR data available onlineEnhanced Web-based processing capabilitiesWith a focus on computationally intensive tasksCommunity supportSoftware tools, tutorials, short courses, and workshops www.opentopography.orgSAN DIEGO SUPERCOMPUTER CENTEROpenTopographyService-Oriented Architecture

OGC Catalogue InterfaceCSW ServerMetadata Management ServerSAN DIEGO SUPERCOMPUTER CENTERCyberGIS GatewayOnline collaborative geospatial problem solving environment Enables easy access to CyberGIS analytics and data sourcesProvides transparent access to a rich set of cyberinfrastructure environments Represents a broad approach to CyberGIS Widely accessible by general public 12

SAN DIEGO SUPERCOMPUTER CENTERApplication Driver - Viewshed Analysis Given terrain data, viewshed computes visible regionsWell known spatial analysis methodHigh resolution data for improved quality of the analysisOT as a data source

13Viewshed analysis on CyberGIS GatewayComputation done on the Forge GPU cluster at NCSA and the cloud infrastructure of the CyberInfrastructure and Geospatial Laboratory

SAN DIEGO SUPERCOMPUTER CENTERIntegration ChallengesUser interfacesSeparately developed interfaces need to be bridgedData discoveryCapabilities for interactive data discovery neededService chainingServices are to be integrated to provide users with an illusion of a single service SecurityConnecting multiple security domains14SAN DIEGO SUPERCOMPUTER CENTERAnand starts hereIntegration ApproachGISolve Open Service APIs Token-based single sign-onWorkflow for composing and interacting with composite servicesMetadata ServicesShared user interface components via librariesSecurityServiceChainingDataDiscoveryUser InterfaceGatewayServiceIntegrationlevelSAN DIEGO SUPERCOMPUTER CENTER

SecurityOpal used by OT to wrap applications as Web servicesOpal itself comes from a third gateway!Opal modified to work with GISolve Open Service Security APIREST-based APICyberGIS deploys token-based identity serverAuthentication and authorizationSAN DIEGO SUPERCOMPUTER CENTEROT Services used in CyberGISCount CloudEstimate the number of points in a selected bounding boxData SelectionGiven a bounding box, retrieve LIDAR point cloud dataPoints2GridGenerate DEMs from point cloud data using a variety of gridding functions (min, max, mean, idw)FormatTranslationConversion between data formats ARC Grid files to GeoTIFF

17SAN DIEGO SUPERCOMPUTER CENTERService ChainingOT Services used to generate Digital Elevation Models (DEMs) needed by the viewshed analysis applicationServices chained and invoked as part of pre-processing step by GISolve middlewareSubmit, check status, and get results stepsWorkflow to streamline service invocations Services use GISolve Open Service APIs to authenticate user requests18SAN DIEGO SUPERCOMPUTER CENTERData DiscoveryEnabled through metadata servicesFacilitate the discovery of and access to OT data sources Two distinct metadata sources usedGoogle Fusion TablesVector (polygon) boundaries of OT datasetsCSW (Catalogue Service for the Web) metadataCSW service APIs enable users to publish, browse, and search for specific metadata using CSW protocolSupports HTTP binding OGC Standard Catalogue ServiceMetadata schema : ISO 19139

19SAN DIEGO SUPERCOMPUTER CENTERWorkflow20GISolve MiddlewareOpen Service APIsMetadata ServiceCount ServiceData Access ServiceData Processing ServiceViewshedInterfaceCyberGIS GatewayService ChainingCyberGISIdentity ServiceTokenOpenTopographyOpal Web ServicesCyberInfrastructureService InfrastructureSAN DIEGO SUPERCOMPUTER CENTERExisting User Interfaces- CyberGIS & OT

Data Selection & Viewshed Analysis LiDAR Data Search & DEM Generation

fSAN DIEGO SUPERCOMPUTER CENTERMyunghwa beginsReusing OT user interfaceUser interface components sharedVia OT libraries

SAN DIEGO SUPERCOMPUTER CENTERLink user interfaces

SAN DIEGO SUPERCOMPUTER CENTERValidate input and collect metadataRestrictions on number of cloud points that can be retrieved and number of cells in a DEMViewpoints must be within the spatial extent of datasetsMetadata necessary for data transformationID, coordinate system, bounding box

SAN DIEGO SUPERCOMPUTER CENTER24

Interaction with OT Web Services

Google Fusion Table & CSW Metadata ServicesCount Cloud ServiceSAN DIEGO SUPERCOMPUTER CENTERWhat did we achieve

SAN DIEGO SUPERCOMPUTER CENTERConcluding RemarksGateways as a means to democratize scienceImportance of interoperation of gatewaysEspecially in GIS where layering of data is so usefulApplication-drivenHigh res LiDAR and high-end computingStandard-basedEnables interoperabilityPrinciplesReusability ExtensibilityReliabilityScalabilityGroundbreaking knowledge gained for integrating service-oriented geospatial software environments

27SAN DIEGO SUPERCOMPUTER CENTERNancy beginsAcknowledgementsNational Science FoundationBCS-0846655OCI-1047916OCI-0503697 TeraGrid SES070004NColleagueshttp://www.cigi.illinois.edu/doku.php/people/index 28Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science FoundationSAN DIEGO SUPERCOMPUTER CENTERThank youQuestions?Comments?Discussion?SAN DIEGO SUPERCOMPUTER CENTER