sparro group, university of regina 1 portal software: browser-based monte carlo zisis papandreou...
TRANSCRIPT
SPARRO Group, University of Regina 1
Portal Software: Browser-based Monte Carlo
Zisis PapandreouUniversity of Regina
GlueX Collaboration MeetingJLab, December 11-13, 2003
SPARRO Group, University of Regina2
Outline
Grid2003 Conference Grid Services
Regina Portal Application framework, Globus Meta queuing
Collaboration tools: GlueX Forum, Database Administrator
SPARRO Group, University of Regina3
Grid2003 Conference (Ed, Zisis)
Phoenix, AZ, November 16-17, 2003 Concurrent with SC2003 (8,000 hackers) Interactive and coached tutorial:
How to Build a Grid Service using the Globus Toolkit® 3 (Argonne people)
Written in Java Built (JDK) and run on Ed’s RH9.0 linux laptop
SPARRO Group, University of Regina4
GT3 Architecture Core: Grid Services Security: SSL, X.509 Base: Job Services,
Index Services, RFT Data: Replica
Management (data subsets)
Other: foreign grid services
OGSA
SPARRO Group, University of Regina5
javac Math.javajava org.apache.axis.wsdl.Java2WSDL-P MathPortType -S MathService -l http://localhost/ogsa/services/core/first/MathService -n http://first.core.gt3tutorial/Math gt3tutorial.core.first.impl.Math
package gt3tutorial.core.first.impl;public interface Math { public int add(int a, int b); public int subtract(int a, int b); public int multiply(int a, int b); public float divide(int a, int b); }
Creating a Grid Service
Write WSDL directly Generate WSDL from
interface language (Java) Decorate it Generate stubs Implement it
Web Services Description Language
SPARRO Group, University of Regina6
Exercises accomplished Grid Services / GT3 FileShare Service
deployment File sharing with
identifiers Adding service data Register with
community index
Lifetime management File searching Security Transience: create
and destroy FileShares
Grid Service Publishing
SPARRO Group, University of Regina7
Demonstration:FileShare Grid Service
(Ed Brash)
SPARRO Group, University of Regina8
Regina Portal: Beefed up
Glasgow: job submission, review (PHP)
New features: ColdFusion Application Framework (CFML) User registration & authentication Integration with GridPort TK, Globus Upload XML Geometry and compile Experiment or commit executables to db
https://www.phys.uregina.ca/GlueX/
SPARRO Group, University of Regina9
Demonstration:Regina Portal(Zisis Papandreou)
SPARRO Group, University of Regina10
Grid Computing “The Grid”: distributed computing infrastructure (both
software and hardware) that provides access to high-end computational capabilities
Grid computing enables the shared use of computational resources such as data archives and simulations.
Resource owners offer resources subject to constraints Resource consumers also place constraints on properties
SPARRO Group, University of Regina11
Queuing, Scheduling &Meta Scheduling
(Aadila Soobrattee)
Regina openSpace cluster (50 CPUs) is governed by OpenPBS local queuing software
Need capabilities of scheduling workload among a computer grid where local scheduling software and local policies may differ: meta-scheduler
Schemes: Centralized: surface info, locals dispatch jobs Hierarchical: meta and local share info; no flexible re-routing Distributed: scheduler/dispatcher model; flexible, scalable
SPARRO Group, University of Regina12
Distributed Scheduling
Meta-scheduler
Local Scheduler/ Dispatcher
Local Scheduler/ Dispatcher
Local Scheduler/ Dispatcher
Jobs
Job flowInformation flow
Meta schedulers at every site Jobs submitted to meta scheduler where job originates Meta schedulers query each other to collect instantaneous load
info; job is transferred to the site with the lowest load Dagman, Condor/G, Nimrod/G, GT3/Platform, Silver/Maui
SPARRO Group, University of Regina13
Silver/Maui
Silver provides grid scheduling services for clusters using OpenPBS, can operate on all major UNIX operating systems and is capable of operating with or without Globus
Jobs Jobs
Jobs
Meta-Scheduler
Local Scheduler
Meta-Scheduler
Local Scheduler
Meta-Scheduler
Local Scheduler
Silver:•reduces average job queue time by enabling access to a larger set of computer resources•intelligently decomposes jobs into multi-system sub-jobs to improve job turnaround time•manages site-dependent job environment and resource requirements•supports jobs which require co-allocated resources which expand multiple systems•supports jobs too large to run at any single site by spanning jobs across multiple sites
Silver:•reduces average job queue time by enabling access to a larger set of computer resources•intelligently decomposes jobs into multi-system sub-jobs to improve job turnaround time•manages site-dependent job environment and resource requirements•supports jobs which require co-allocated resources which expand multiple systems•supports jobs too large to run at any single site by spanning jobs across multiple sites
http://www.supercluster.org/
SPARRO Group, University of Regina14
1. User logs into Globus and gets Globus credential2. User submits job from site A via sqsub client3. sqsub translates job to global job and stores job and credential in SILVER's global queue4. SILVER determines list of feasible resources based on resources requested5. SILVER queries Globus and determines machines that support the specific credential6. SILVER queries each feasible resource and obtains matrix of resource availability times7. SILVER determines resource associated with earliest job completion time (site B)8. SILVER reserves resources on site B9. SILVER stages input data and executable to site B using Globus file transfer (gssftp)10. SILVER translates global job to RSL and submits job to Globus with 'run on site B' request11. Globus translates RSL job to resource manager command language and submits the job to resource manager on site B.12. MAUI detects job, executes it on reserved resources13. job completes14. MAUI creates accounting record, removes reservation, and updates SILVER15. Output data is staged to requested destination using Globus file transfer
SPARRO Group, University of Regina15
Collaboration Tools GlueX Forum (at UConn)
http://zeus.phys.uconn.edu/forums/gluex.org/index.php Web-based email forum (written in PHP) Category-based email, threads, moderators
PostgreSQL Database Administration pgAdminIII (win & unix) Remote connection to db servers Create/delete tables, enter data, view data
SPARRO Group, University of Regina16
What’s next? Expand portal further:
Install Meta Queuing System at Regina Web/Grid Services (document uploading, indexing and
searching) XML Data Transfer (Web Distributed Data Exchange –
WDDX) Communications:
Teleconferencing: AT&T Videoconferencing: VRVS or Polycom Email threaded discussions: GlueX Forum