teragrid information services john-paul “jp” navarro teragrid grid infrastructure group...
TRANSCRIPT
TeraGrid Information Services
John-Paul “JP” Navarro
TeraGrid Grid Infrastructure Group “GIG”Area Co-Director for Software Integration and
Information ServicesUniversity of Chicago, Argonne National Laboratory
GCE07November 12, 2007
November 12, 2007 GCE 07 2
Grids include interconnected hardware components, coordinated software and(grid) services, and institutions and groups that operate them.
To effectively use grids, users need access to information about the hardware, software, grid services,and the institutions and groups operating them.
The TeraGrid's Information Services vision is to:
1) define a coordinated way for TeraGrid participants to publish about the services they offer,
2) define a way for the TeraGrid to aggregate and index the information from all TeraGrid participants, and
3) to publish this information to the public in a form that can easily be used by other software, users, and TeraGrid service providers themselves.
This talk will introduce the TeraGrid's Information Service strategy, the high-level architecture, current and future content, and the methods available to users, applications, and gateways to access TeraGrid Information Services content.
Abstract
November 12, 2007 GCE 07 3
• Collection of Information Grid Services• Service providers publish local information• TeraGrid wide aggregating/indexing for
discovery• Primarily focused on public information• Primarily accessible thru software
interfaces• Using standards based interfaces• Reliable, scalable, and fast• Initially focused on TeraGrid information• Able to include partner/community
information
TeraGrid Information Services Elements
November 12, 2007 GCE 07 4
TG Information Services IS [NOT]
IS NOT ISA central database (Data Warehouse)
A central index/aggregation (Google)
A new user interface A way software (user interfaces) can access information
A single implementation/tool Evolving set of software tools
A single software interface Several useful interfaces (small set)
A specific set of information Phased growing collection of information
Changed data ownership Ownership maintained as appropriate
Way to manage scientific information
Way to manage Grid meta-data
A data management system (database)
An information publishing system
A coordinated way to index and publish public [Tera]Grid information thru software interfaces.
November 12, 2007 GCE 07 5
Clients
Clients
Clients
High-Level Components
CacheCache
WS/RESTHTTP GET
WS/SOAP
WS MDS4
TomcatWebMDS
Apache 2.0
TeraGrid WideInformation Services
WS/SOAPWS MDS4
Service ProviderInformation Services
TeraGridWide
Respositories
TeraGridWide
Respositories
November 12, 2007 GCE 07 6
Services Provider Information ServicesContent:
Locally owned and maintained informationOriginates anywhere the service provider wishes
Services:1 general purpose MDS service2 scheduling MDS services: authenticated and public (merging)
TeraGrid Wide Information ServicesContent:
Aggregate/index service provider informationAdditional central information (TGCDB, GIG operated services, …)Cached (service providers services can be down)Authenticated registrations
Services:Several redundant servers (99.5% plus availability)Information caching (persistence)Several MDS4 services (WS/SOAP)WebMDS/Tomcat, Apache 2.0, … services (WS/REST)Content published in: HTML, XHTML/XML, XML, Atom, RSS, …
Service Provider vs TG Wide Services
November 12, 2007 GCE 07 7
WS/* (Tomcat 5.0, Apache 2.0)Benefits
Very common web services platformSupports several web service interfaces (including simple)Supports multiple styles like REST, Web 2.0Can be highly scalable
ContentMany formats: HTML, XHTML/XML, XML, RSS/Atom, …
WebMDS (Globus 4.0.5/VDT 1.7.1)Benefits
Live MDS4 content accessXPath supportXSLT transforms
ContentMany formats: HTML, XHTML/XML, XML, RSS/Atom
WS/SOAP (Globus 4.0.5/VDT 1.7.1 MDS4)Benefits
Indexing, TriggerRegistration, Publish, SubscribeSecurity/AuthorizationRobust WSRF interface
ContentXML
Tools
November 12, 2007 GCE 07 8
High-Availability Design
…
info.dyn.teragrid.org
info.teragrid.org
TeraGrid Dynamic DNS
Information Services administrators select servers
Changes propagate globally with a 15 minute TTL
Clients
Dynamically Changes
Doesn’t Change
Service Provider Information Services
TG wide information services
November 12, 2007 GCE 07 9
Information Services Users
User Documentationhttp://www.teragrid.org/
User Portalhttp://portal.teragrid.org/
Gateways
Peer Grids
User Applications
info.teragrid.org
IncaTesting Harness
November 12, 2007 GCE 07 10
What’s in Production?
Services– TeraGrid Resource Provider Information Services– TeraGrid Wide Aggregating/Indexing Information Services
Content (since when)– Scheduling information for User Portal (Spring)
• Scheduler load, Queue contents (restricted)– CTSS 4 capabilities kits (August)
• Which capability kits are available on each resource• What software is available in each kit on each resource• What services are available from each kit on each resource
– TeraGrid Central Database (tgcdb) keys and descriptions (October)
Queue Contents in User Portal
November 12, 2007 GCE 07 12
CTSS 4 Capability Kits
For each capability kit on each resource–Current support level, and target support level
•Development, Testing, Production–Support organization and contact–Inca status URL–Multiple version of a kit with different support levels
November 12, 2007 GCE 07 13
CTSS 4 Capability Kit Software
For each kit software component on each resource–Name, version, how to access it–Multiple versions of a single component
November 12, 2007 GCE 07 14
CTSS 4 Capability Kit Services
For each kit service on each resource–Name, type, version, and Endpoint (contact location)–GSI OpenSSH, GridFTP, SRB servers, PreWS & WS GRAM, MDS4–Multiple services of the same type
CTSS Capability Kit Availability
Where are the GridFTP services?
November 12, 2007 GCE 07 17
What’s in Development?
Expanded content– Local HPC Software– Extended GridFTP service information– (Meta)Scheduling support information
Core Extension– Information Services Metadata (registration w/o
aggregation)Information Access
– tginfo, universal command line query tool– WS/REST, Web 2.0 style information access– Multiple formats: CSV TEXT, RSS/Atom, XML, …– GLUE 2.0
Community publishing– Community supported capabilities– Community information services registration
November 12, 2007 GCE 07 18
Accessing TeraGrid IS from software
Learn what information is available– http://info.teragrid.org/
Choose your access method and client software– WS/SOAP: GT4 Java core, or client toolkit– WS/REST: Any tool that can issue HTTP GET
Code TG Information Services queries– Using GT4 access
• https://info.teragrid.org:8446/wsrf/services/DefaultIndexServices (XPATH)
• https://mds.teragrid.org:8443/wsrf/services/DefaultIndexServices
– Using HTTP GET• http://info.teragrid.org:8080/webmds/webmds?
info=tgislocal&xsl=<VIEW>• http://info.teragrid.org/restdemo/html/tg/services• http://info.teragrid.org/restdemo/html/tg/sites
(Optional) Resource Selection– List of TG ResourceIDs
November 12, 2007 GCE 07 19
GatewaysPublish or just register to TeraGrid Wide Information
Services
Data collectionsData collections register to TeraGrid Wide Information
ServicesData collections access method, service Endpoint, paths
Community software areasWhich resources have each CSAWhat software is available in each CSA, how to access it
Service ProviderPlanned and unplanned outage informationPolicies
Peer grids/interoperabilityResources, services available on peer grids (OSG, EGEE,
…)
…….
[Not so] Farfetched Possibilities
November 12, 2007 GCE 07 20
Find out more:http://info.teragrid.org/ (links to content and
documentation)
Request content:mailto: [email protected] or [email protected]
Discuss Information Services content, requirements, and design:E-mail list [email protected]
View current Information Services contentUser Portal (scheduler load & queue contents):
https://portal.teragrid.org:443/gridsphere/gridsphere?cid=resources
User Documentation (CTSS 4 kits, software, services):http://www.teragrid.org/userinfo/software/ctss.php
Information Service Main Page:http://info.teragrid.org/
More Information