chinagrid experience with gt4 hai jin huazhong university of science and technology [email protected]
TRANSCRIPT
ChinaGrid Experience ChinaGrid Experience with GT4with GT4
Hai JinHai JinHuazhong University of Science and TechnHuazhong University of Science and Techn
ologyology
[email protected]@hust.edu.cn
2
2
Outline
• Overview of ChinaGrid
• Experiences with GT4 Highly Available Dynamic Infrastructure
Execution Management
Data Management and Transferring
Security
• Future Works
3
3
ChinaGrid
• First phase from 2003 to 2005
• 12 key universities as initiative, 20 key universities now
• 15 Tflop/s computing power
• 150TB storage
• Run 5 domain-specific application grids
• Supported by CGSP
4
4
Layered Infrastructure of ChinaGrid
High performance computing environment(campus grid)
ChinaGrid Supporting Platform (CGSP)
NUDT
THU
HUST
ZSUPKU SJTU XJTU
NEUSCUT
BUAA
SEU
SDU
Remoteeducation
grid
Imageprocessing
grid
Fluiddynamics
grid Massiveinformationprocessing gridBioinformatics
grid
6
6
CGSP Overview
Domain Manager Grid Monitor
Execution Management
ModulesInformation Center Data Manager
Security Manager
Service Contianer
CGSP System modules (services)
Portal Engine Grid Developing Toolkits
User Developer
Interoperating Broker
Alien Grid
Client
7
7
GSI
WS-Security
CAS
SimpleCA
Data Managemen
tSecurity
WSCore
Resource Managemen
t
Information Services
Authz Framework
RFT
RLS
OGSI-DAI
New GridFTP
XIO
JAVAWS Core(WSRF)
C WS Core(WSRF)
MDS2
WS-Index
Pre-WSGRAM
WS-GRAM
CSF(contribution)
pyGlobus(contributed)
Python Bindings(contributed)
Data Center(CGSP)
Data Center(CGSP)
Portal SDK(CGSP)
Portal SDK(CGSP)
Programming Toolkits(CGSP)
Programming Toolkits(CGSP)
Resource-Index(CGSP)
Resource-Index(CGSP)
HyperService(CGSP)
HyperService(CGSP)
Proxy(CGSP)
Proxy(CGSP)
MetaData-Man(CGSP)
MetaData-Man(CGSP)
InterDomain(CGSP)
InterDomain(CGSP)
Topologic(CGSP)
Topologic(CGSP)
RemoteDeploy(CGSP)
RemoteDeploy(CGSP)
Job Manager(CGSP)
Job Manager(CGSP)
WorkFlow(CGSP)
WorkFlow(CGSP)
HotDeploy(CGSP)
HotDeploy(CGSP)
GT3.9 and CGSP 1.0
9
9
Goals of Cooperation
• ChinaGrid contributions to global grid community
• Keep the advanced feature of ChinaGrid
• Improve the interoperability widely for whole grid community
11
11
Experience with GT4: HAND
• HAND: Highly Available dyNamic Deployment InfrastructureChinaGrid and Globus started sharing ideas and
codes from June 2005
Hot deploy branch now almost ready for release
• CGSP 2.0 release candidates also import certain features from Globus
13
13
HAND (2)• Container level
deployment advantages Clear workflow
• Service level deployment advantages Flexible management
15
15
HAND (4)• Achievement
A release candidate for GT 4.2
A co-authored paper submitted to HPDC’06
A excellent cooperation and communication channel
Win-win experience on international cooperation
17
17
Experience with GT4:Execution Management
• Early experience to enhance the BPEL (ActiveBPEL) to WSRF-compatible
• Support for JSDL specification
• Support multiple schedule in a common frameworkWSRF compatible Web serviceHyper Services (Virtual Service)Composite ServicesLegacy application encapsulated in GRS
18
18
SubmissionSubmission JobSubmitter
JobSubmitterMonitorMonitor ControlControl Workflow
Deployment
WorkflowDeployment
JobManager
JobManager
JobExecutors
JobExecutors
JMIJMIWorkflow Agent
Workflow Agent
JobQueue
JobQueue
Monitor/ControlAgent
Monitor/ControlAgent
Inter-domainAgent
Inter-domainAgent
Workflow ManagerWorkflow Manager Execution EngineExecution Engine
Load BalancerLoad BalancerDatabaseDatabase
StateDatabase
StateDatabase
BPELEngine
BPELEngine
BPELEngine
BPELEngine
BPELEngine
BPELEngine
GeneralServiceInvoker
GeneralServiceInvoker
WorkflowInvoker
WorkflowInvoker
JSDLParser
andGRS
Invoker
JSDLParser
andGRS
Invoker
CGSPService
Container
CGSPService
Container
GT4.0.xJava
WS Core
GT4.0.xJava
WS Core
CGSPGRS
Container
CGSPGRS
Container
ApacheAxis1.2.xContainer
ApacheAxis1.2.xContainer
Execu
tion
Man
agem
ent (2)
19
19
Experience with GT4:Data Management &
Transferring
CGSP Data Management
heterogeneous
data intensive
limited storage
mass
widely distributed
distributed
education
mass storage
research
low access latency
collaboration
other applications
Storage Resource Applications
Users
20
20
Data Management & Transferring (2)
• Application-level Striped Transfer Based on GridFTP protocol
Striping algorithm and slice & replica selection algorithm
Parallel transfer
• Graphical User Interfaces A common graphical transfer
client which can serve as:
A GridFTP Client A CGSP Transfer Client A user-level file system viewabl
e through Windows Explorer Remote I/O Dragging and Dropping ….
UserSpaceService(USS)
StorageResourceService(SRS)
FileTransferService(FTS)
Storage Status Notification
Transfer Status Notification
Access HistoryBandwidth
Storage Resource Storage Resource Storage Resource
Transfer Client
Replication Operations
Third Party Transfer
Striping Algorithm ConfigurationReplication Algorithm Configuration
Algorithm Queries
1. Transfer Request 2. Slice Assigment
3. GridFTP Transfer of Slices
3. GridFTP Transfer of Slices
Third Party TransferRequest
21
21
Experience with GT4: Security
GT• GSI is composed of a set of
command-line tools to manage certificates, and a set of Java classes to easily integrate security into our web services. GSI offers programmers the following features Transport-level and message-level
security Authentication through X.509
digital certificates Several authorization schemes Credential delegation and single
sign-on Different levels of security:
container, service, and resource
CGSP• User certificate at Portal level
Username-passwd & Credential (pkcs12) Sys-service & Gen-service
• Job is executed by cgsp-module with user info including username,group and domain info
)
• Job security is guaranteed by message-level security
• Support the composite job security cgsp-ac-config.xml
• Support dynamic security resource properties SecurityContextService SecurityTokenCenter FeedbackService
22
22
Future Works• High dynamicity and autonomy containers
CDDLM, WSDM, WS-ReliableMessage
• Complicated deployment policies and infrastructureWS-Agreement, BPEL4Deployment
• ActiveBPEL engine running on GT4
• Interoperable middleware or agentsGRS and GRAMInformation Center and MDS4Security
23
23
Thanks!Hai Jin
Huazhong University of Science and TechnologyWuhan, 430074, China
[email protected]://grid.hust.edu.cn/hjin/