Overview of Grid Computing Environments Proposed GGF Information Document G.Fox, D. Gannon, M. Pierce, M. Thomas PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN Source of Report Last Half of 2001: Call from GCE RG for Papers for Journal special issues 28 papers submitted and reviewed Published in Concurrency and Computation: Practice and Experience Vol. 14, Grid Computing Environments Special Issue 13-15, Fall Augmented by chapters (about 14) in Fran Berman, Geoffrey Fox and Tony Hey, Grid Computing: Making the Global Infrastructure a Reality, ISBN , John Wiley & Sons Ltd, Chichester, SeeOther key papers which we knew about Raw (HPC) Resources Database Aggregation Portal System Services Application Service Portal Services Portal Services Grid Computing Environments User Services Core Grid Application Service OGSA (OGSI) Interfaces The fuzzy definition of Grid Computing Environments Two Major Areas Programming the User Side of the Grid which involves discussing computing model for Grid Controlling user interaction rendering any output and allowing user input in some (web) page. This includes aggregation of multiple data sources in a single portal page (Jetspeed). Has natural overlaps with information (commercial) portals Classification of GCE Papers in Programming the User Side of the Grid Technology for building GCE systems -Interface with backend Infrastructure e.g. Community Grid Kits, GPDK Problem Solving Environments Domain specific collection of tools and user interface. E.g. XCAT, Polder, SCIRun, Astrophysics Collaboratory GCE Tools Support parameter sweep, visualization, job status, files, security, workflow.. GCE Shell Portals providing a general interface to many Grid capabilities Analogous (not usually command line) to role of UNIX shell providing access to UNIX tools and user programs, files Note UNIX has core system and higher level tools accessed by Shell E.g. Unicore, Hotpage, Mississippi Portal PSEs often built on top of GCEShell portals Aspects of Programming Model I See review of programming model by GGF APM RG Chapter 21 of Wiley Book and web page Handling of the basic components of a distributed computing system files, computing and data resources, programs, and accounts. The GCE will typically interface with an environment like Globus or a batch scheduler like PBS to actually handle the back-end resources. However the GCE will present the user interfaces to handle these resources. We can follow the lead of UNIX (and Legion) and define a basic GCEShell providing access to the core distributed computing functions. JXTA also builds some Grid-like capabilities with a modification of UNIX shell model. GCEShell can have a command line or more visually appealing graphical user interface. Implications of 3-Tier Model The 3-tier model implies that any given capability (say run a matrix inversion program) can appear at multiple levels. Maybe there is a backend parallel computer running an MPI job; this is front-ended perhaps as a service by some middle-tier component running on a totally different computer, which could even be in a different security domain. One can interact with this service at either level; a high performance I/O transfer at the parallel computing level and/or by a slower middle-tier protocol like SOAP at the service level. These two (or more) calls (component interactions) can represent different functions or the middle tier call can be coupled with a high performance mirror. Typically the middle tier provides control and the back end raw data transfer. Raw (HPC) Resources Database Aggregation Portal System Services Application Service Portal Services Portal Services Grid Computing Environments User Services Core Grid Application Service Application Metadata Actual Application OGSA (OGSI) Interfaces The fuzzy definition of Grid Computing Environments Technology for building GCE Systems CoG Kits for Java Perl Python CORBA . GPDK extending CoG kits with JavaBean and JSP middleware Grid Portal Toolkit Other such interfaces with also C and XML tools Event Service Simulations GCEShell Portals and PSEs One can divide GCE capabilities into generic (copy file) or domain specific (generate the multigrid solver suitable to simulate Middle Earth) Correspondingly one finds portal frameworks or GCEShell Portals presenting general capabilities Grid or web-based Problem Solving Environments optimized for a particular domain PSEs are often (and perhaps should be) built on top of GCFShell Portals Typical Layered Architecture GCEShell Portals Globus GT2 Services Manage backend resources Interface Middleware with backend resources Core Middleware Services Application Services Problem Solving Environments User Interface-Client Aggregate component Interfaces Jetspeed GCE Tools Data Management File manipulation in all tiers including client, middleware and back end Security In all tiers and providing interfaces to core Grid Security Workflow or Programming the Grid Grid versions of MPI Higher-level tools include visualization or support of parameter sweep applications UNIX Shell has mix of lower level and higher level tools More on Programming Models Basic 2-level programming model is assumed by most projects First you use classic (parallel) programming to produce simulation nuggets (wrapped as application web services) JDBC / OGSA-DAI to package data resources as objects or services Then you need to compose (orchestrate) the control and dataflow between nuggets Many different models for how this is to be done and can call this workflow Next thrust of GCE RG? Some work on optimization between levels as in GrADS project Research in 2-level Programming Models Basic user interface to middle tier proxies controlling backend (software) resources Component Models like ICENI (Imperial College) or DoE Common Component Architecture Commercial workflow engines as in BPEL4WS Scripting front-ends as in Matlab or Python Network server model as in NetSolve or Ninf Computational Economies Semantic Grid technologies (ontology based integration of resources) as in MyGrid or DiscoveryNet Agents? Programming Infrastructure (Hosting Environment) Different approachs assume different run-time (implementation) and user programming ether BPEL4WS thinks about specifying interactions between components; Matlab interface invokes components from a script Java Interface to OGSI WG emphasizes that we dont have an established implementation even if you pick a language Jini JXTA Servlet Enterprise JavaBeans EJB Or perhaps some future SJB (Scientific JavaBean) supporting high volume dataflow better than EJB (short transactions) Web Services as a Portlet Each Web Service naturally has a user interface specified as just another port Customizable for universal access This gives each Web Service a Portlet view specified (in XML as always) by WSRP (Web services for Remote Portals) So component model for resources automatically gives a component model for user interfaces When you build your application, you define portlet at same time Application or Content source WSDL Web Service S R W P Application as a WS General Application Ports Interface with other Web Services User Face of Web Service WSRP Ports define WS as a Portlet Web Services have other ports (Grid Service) to be OGSI compliant Online Knowledge Center built from Portlets Web Services provide a component model for the middleware (see large common component architecture effort in Dept. of Energy) Should match each WSDL component with a corresponding user interface component Thus one must use a component model for the portal with again an XML specification (portalML) of portal component A set of UI Components Content Provider WSDL Web Service F I U O F I R O Portal Aggregate WS-User Facing Fragments Render Other WS User Facing Ports Other WS Resource or Service-facing Ports User-facing Ports User Facing Ports for Web Service User Profile Application or Content source WSDL Web Service F I U O F I R O Render Portal (Aggregator) Selector Filter Control Channel Customized View Selection View Control Channel Customized View Customized User-Facing Ports As used in Universal Access Portlet XML RSS, OCS, or other Local or remote HTML Local files JSP or VM Local templates WebPage Remote HTML Portlet Portlets User implemented using Portal API Portlets Data PortletController Screen Manager HTML PSML PortletControl ECS JSP template ECS ECS Root to HTML ECS Turbine Servlet Jetspeed Architecture Portlets and Portal Stacks User interfaces to Portal services (Code Submission, Job Monitoring, File Management for Host X) are all managed as portlets. Users, administrators can customize their portal interfaces to just precisely the services they want. Core Grid Services User facing Web Service Ports Application Grid Web Services Aggregation Portals (Jetspeed) Message Security, Information Services Jetspeed Computing Portal: Choose Portlets 4 available portlets linking to Web Services I choose two Choose Portlet Layout Choose 1-column Layout Original 2-column Layout Lists user files on selected host, noahsark. File operations include Upload, download, Copy, rename, crossload Tabs indicate available portlet interfaces. File management Sample page with several portlets: proxy credential manager, submission, monitoring Provide information about application and host parameters Select application to edit Administer Grid Portal