biruni grid glite middleware deployment : from biruni grid
Post on 03-Feb-2022
17 Views
Preview:
TRANSCRIPT
BIRUNI GridBIRUNI Grid gLitegLite Middleware Deployment : FromMiddleware Deployment : From
Presentation TitleBIRUNI Grid BIRUNI Grid gLitegLite Middleware Deployment : From Middleware Deployment : From
Zero to Hero, towards a certified EGEE siteZero to Hero, towards a certified EGEE sitePresentation TitleS k
M. M. FarhanFarhan SjaugiSjaugi, Mohamed Othman, , Mohamed Othman, MohdMohd. . ZulZulYusoffYusoff MohdMohd RafizanRafizan RamlyRamly andand SuhaimiSuhaimi NapisNapisSpeaker
InstitutionE t N
YusoffYusoff, , MohdMohd. . RafizanRafizan RamlyRamly and and SuhaimiSuhaimi NapisNapisGrid Computing Project Officer / Research Assistance
InfoComm Development Center (iDEC) & Department of Communication T h l d N t k F lt f C S i d I f tiEvent NameTechnology and Network, Faculty of Comp. Science and Information
Tech.Universiti Putra Malaysia
farhansj@biruni.upm.my
1
• Introduction
OutlineOutline• Introduction• BIRUNI GRID
– Infrastructure Line-UpInfrastructure Line Up– Network Support– Frameworka e o
• gLite Middleware Deployment– Planningg– Deployment
• Issues and Challenges• Future Plans• Concluding Remarks
2• References
I d iIntroduction
Motivation• Motivation– Academic staffs, research officers/assistants, and post-
graduate students made up of over 80% of the entiregraduate students made up of over 80% of the entire research community in Malaysiai! (Not ncluding undergraduates) Working on Grid computing and its application in various disciplines
– Requires minimal facilities in their labs of respective i iti t t th i R&Duniversities to carry out their R&D
– Need something that they can “play around” andshould anything happens to the facilities (eg Nodesshould anything happens to the facilities (eg. Nodes crashing, hanging, etc) they can simply poweroff and restart!
3
I d iIntroduction
Basically two types of users:• Basically two types of users:– Researchers/students doing research on certain aspects
of GRIDof GRID– Researchers/students using GRID for computation;
(Comp. Science, Bioinformatics, Cheminformatics, ( p , , ,Medical Informatics)
• Need basic Grid facilities and have total control of the entire Grid Ecosystem (all aspects of Grid)
• After acquiring certain level of expertise or if they want more compute power, then they can tap on other production facilitiesH th d f l i d di G id• Hence the need for a learning and discovery Grid
4
BIRUNI G idBIRUNI Grid
UPM has just commisioned BIRUNI GRID with the• UPM has just commisioned BIRUNI GRID with the hope of making it the Nucleus HPC Clusters for A-GridGrid
• Hardware Racking and Power up were done by supppliers while the rest of the configuration and ppp gdeployment was done by UPM Grid Team; Excellent learning opportunities.
5
I f Li UInfrastructure Line-Up
BIRUNI GRID has:• BIRUNI GRID has:– 50 IBM Blade HS21 Servers (2x Intel Xeon Quad Core 2
Ghz with 8 GB rams)Ghz, with 8 GB rams).– 3 IBM x3650 servers, one as Head Node and two as
Storage Nodes.g– 2 IBM DS3000 series SAN with 24 Terabytes of storage
capacity.• BIRUNI GRID consists of three clusters:
– Khaldun Sandbox Cluster (7 worker nodes)– Razi Cluster (28 worker nodes)– Haitham Cluster (10 worker nodes)
6
I f Li UInfrastructure Line-Up
Khaldun• Khaldun– Serve as a platform for research, application
development and grid tutorialdevelopment and grid tutorial.– The environmental provided almost identical with the
production grid.p g– The inter-node interconnection is done through gigabit
ethernet technology.• Razi
– Serve as the main platform for production grid.– More intended for compute-intensive application.– The inter-node interconnection is done through gigabit
ethernet technology.7
I f Li UInfrastructure Line-Up• KhaitamKhaitam
– Serve as the main platform for production grid.– More intended for data-intensive applicationMore intended for data intensive application.– The inter-node interconnection is done through high-
speed infiniband technology.p gy
8
N k SNetwork Support
9
N k SNetwork Support
10
D l F kDevelopment Framework
GRID APPLICATION(Parallel Programming Libraries Java-RMI RPC(Parallel Programming Libraries, Java RMI, RPC,
etc..)
MIDDLEWARE(Distributed Algorithm, Grid Services, etc..)
NETWORK SUPPORT(Reliable High Speed Data Transfer etc )(Reliable High-Speed Data Transfer, etc..)
11
Li Middl D lgLite Middleware Deployment
The gLite distribution is an integrated set of components• The gLite distribution is an integrated set of components designed to enable resource sharing. In other words, this is middleware for building a grid.is middleware for building a grid.
• The gLite middleware is produced by the EGEE project. • The distribution model is to construct different servicesThe distribution model is to construct different services
('node-types') from these components and then ensure easy installation and configuration on the chosen platforms (currently Scientific Linux versions 3 and 4).
• gLite middleware is currently deployed on hundreds of sites as part of the EGEE project and enables global science in a number of disciplines, notably serving the LCG projectLCG project.
12
Li Middl F kgLite Middleware Framework
13
Pl iPlanning
Biruni Grid team members start with minimum• Biruni Grid team members start with minimum experience in Grid Middleware deployment (i.e. gLite Middleware), though most of the team members areMiddleware), though most of the team members are familiar with HPC and system administration.
• Trial-and-Error methodology to familiarize with the gymiddleware.
• Virtualization was chosen as the test-bed.• Work side-by-side with Academia Sinica Grid
Computing (ASGC), Taiwan to accelerate the adoption of the middleware as well as building expertise.
14
D lDeployment
With guidance from ASGC BIRUNI Grid team start• With guidance from ASGC, BIRUNI Grid team start deploying gLite Middleware.
• Start with Computing Element and 14 worker nodes• Start with Computing Element and 14 worker nodes, followed by Storage Element with 1 Tera-Byte storage capacity.g p y
• It takes about two months with intensive communication with ASGC for BIRUNI Grid to deploy and test. Finally BIRUNI Grid to got EGEE-site certified status.
• Incrementally other gLite middleware service and policy are added (i.e. WMS, UI, Grid Portal).
15
I d Ch llIssues and Challenges
Still find difficulties to maintain the grid site from• Still find difficulties to maintain the grid site from error, but….. BIRUNI Grid team learn a lot on how to troubleshoot the problems.troubleshoot the problems.
• Now Razi receives in average 74 jobs per-hour (through EUASIA VO) with its 80 jobs per-hour of ( g ) j pcurrent maximum job capacity.
• The capacity of production grid will be increased from time-to-time.
• Though grid utilization is high, most of the submitted job is come from partners. Still very low utilization from internal organization.
16
I d Ch llIssues and Challenges
17
F PlFuture Plans
Providing backup grid service for EUASIA grid VO• Providing backup grid service for EUASIA grid VO (WMS, LFC).
• Grid Application deployment for• Grid Application deployment for Bioinformatics/Cheminformatics and Medical Informatics and Graphic/Image Renderingp g g
• Promote greater awareness on Grid Computing and shared resources among researchers and stakeholders of Institutions of Higher Learning in Malaysia through the Ministry of Higher Education
hich ill soon be the c stodian of MYREN Phase 2which will soon be the custodian of MYREN Phase 2.• Becoming Certification Authority to serve regional
usersusers. 18
C l di R kConcluding Remarks
Universities not only should have enough resources• Universities not only should have enough resources to build massive compute facilities but also should have access to sandbox, preproduction and alsohave access to sandbox, preproduction and also production Grid
• A small but modest compute facilities where pstudents have the freedom to try out new things without fear of wrecking a big Grid system should be
il blavailable• BIRUNI Grid provide a platform and compute
facilities here st dents and academics learn abo tfacilities where students and academics learn about Grid technology and its applications,
19
C l di R kConcluding Remarks
Happy to have been able to build BIRUNI GRID from• Happy to have been able to build BIRUNI GRID from scratch (nearly ☺ ); very satisfying and at the same time acquire the skill to deploy HPC Clusterstime acquire the skill to deploy HPC Clusters
• UPM wishes to acknowledge Academia Sinica for their undying support right from the planning of y g pp g p gwhat to purchase right up to certification
• All these has been made possible through EUAsiaGrid Project; bridging not just between EU and Asia but also equally important among Asian co ntriescountries
20
R fReferences
BIRUNI Grid Center http://biruni upm my• BIRUNI Grid Center – http://biruni.upm.my• BIRUNI Grid Statistic -
http://goc grid sinica edu tw/gstat/MY-UPM-BIRUNI-http://goc.grid.sinica.edu.tw/gstat/MY-UPM-BIRUNI-01/
• gLite Middleware - http://glite web cern ch/gLite Middleware http://glite.web.cern.ch/• EUAsiaGrid – http://www.euasiagrid.eu/
21
top related