"introduction to the grid" satoshi sekiguchi
TRANSCRIPT
“Introduction to the GRID”
Satoshi SekiguchiDirector, Grid Technology Research Center,
Advanced Industrial Science and Technology, Japan
What is the Grid (1)?
The World Wide Web provides seamless access to information that is stored in many millions of different geographical locations
In contrast, the Grid is a new computing infrastructure which provides seamless access to computing power and data distributed over the globe
The name Grid is chosen by analogy with the electric power grid: plug-in to computing power without worrying where and how it comes from, like a toaster
Slide by courtesy of Fabrizio Gagliardi
What is the Grid? (2)
New applications enabled by the coordinated use of geographically distributed resources
e.g., distributed collaboration, data access and analysis, distributed computing
The Grid relies on advanced software, called middleware, which ensures seamless communication between different computers and different parts of the world
e.g., certificate authorities and policies, protocols for resource discovery/access, programming environment
Original motivation, and support, from high-end science and engineering; but has wide-ranging applicability
e.g., from Grand Challenge to Everyday Business
Slide courtesy by Ian and Carl
Who will use the Grid?
Computational scientists & engineers: large scale modeling of complex structures
Experimental scientists: storing and analyzing large data sets
Collaborations: large scale multi-institutional projects
Training & education: virtual learning rooms and laboratories
Environmentalists: climate monitoring and modeling
Corporations: global enterprises and industrial partnership
Slide by courtesy of Fabrizio Gagliardi
Grid – a quick look
No clear definition, changes day by day
GridGrid
Computing Grid( high speed computing service)
Computing Grid( high speed computing service)
Data Grid( File service, DB integration)Data Grid( File service, DB integration)
PC Grid(Mega computing/ Cycle scavenging)
PC Grid(Mega computing/ Cycle scavenging)
Business Grid( Data Center Grid)Business Grid( Data Center Grid)
Grid
Grid
Meta computing
Virtual Computing Center
Grid Data Center
eGov, Business federation
SETI@HOME-like
Bio, High energy, AstroLarge scale data analysis,
Intra Grid
Technical Business future plan
Access Grid, Sensor Grid
Technology Life Cycle (Grid/Electricity)
Research Lab
Technology
Globus, Unicore
Ninf, SETI@homeGrid Testbed
Pilot user
Soft PKGGrid XX
Grid software Product
Business
Utility Data Center
B. Franklin
Volta's Battery Lamp
Power Plant
Today
GridWorld2004
2004 Apr. 27-28 TFT Hall, Tokyo Japan
The first national scale event for grids in the world
3000 registration, 2000 attended, 20 groups exhibited
GridToday 04
2004 May 26-28 Philadelphia CC
Exhibit for grid technologies in US
400 attended, 26 groups exhibited
Business with Grids
H/W(Server, Storage, Network)
OS (Linux, Solaris, AIX,…, Windows)
System management tool(Work load, DB, Performance monitoring, Resource management)
Applicatiooon(Stock mgmt, Financial mgmt, CRM, )
Business Flow( eCommerce, Business Management,…)
Web service system(XML, SOAP, WSDL )
OGSA(Authentication Authorization, Policy, Resource, )
Grid SI/Consulting
constructionService
GlobusToolkit
CommercialSupport
(Consulting)
ASPApplication
Service Provider
Utility Data Center
(Grid Service) (Mediation)
ResourceBroker
PortalDirectoryservice
grid readyserver computer
(Product Sales)
Construction Tools
Grid middleware
grid readyApplication server
grid readymanagement tools
[Y]2004 2005 2006 2007 2008 2009 2010
807
① Grid System ② Grid Service
49 88 12817065 117
171227
[100 Million JPY = 0.9 M USD]
0
300400500600
594
347
529
391
258
700
①
②
Grid
Bu
siness
Grid Market Growth
200100
~~1156
1383
1114
12851191
1100
10741035
0
1200
1500
1600
[Y]2004 2005 2006 2007 2008 2009 2010
2000
1800
1000
1292
1547
1774
2099
12001245
1886
IDCIDC
System – non GridSystem – non Grid
System - GridSystem - Grid
+ Grid service+ Grid service
[100 Million JPY]
987(2002年水準 )
[by NAREGI report]
Economic Impact in IT Market in Japan
Adopt Grid Technology to Business Scene
IT resource Virtualization/ provisioning/Integration of Web servicesIT resource Virtualization/ provisioning/Integration of Web servicesHigher resource utilization usage for saving TCOHigher resource utilization usage for saving TCOAutomate IT resource management in Data centers Automate IT resource management in Data centers Business Continuity, Business opportunityBusiness Continuity, Business opportunity
Science grid Business Scene
HPC Application・ Bio informatics・Weather forecasting, climate simulation・ Structure Analysis
Adopt Grid technology to businessScene・Higher utilization of IT resources•Dynamic load balancing
• Business Application• Web Service• Engineering/Technical…
Global load distributionGlobal IT resource sharing
More complex, higher TCO
Larger in system scale
3-tier model
Buy more servers for sustain performance
More complex heterogeneity
Processor architecture, performance
OS (Windows, Linux, Unix, …)
Non standard APIsDB, AP server, system management
TCO increases significantly
Save TCO by Web service, Grid
Autonomous IT Infrastructure
Autonomous
Dynamic provisioning according to the demand for the resources in case for access spike, system fail.
Virtualizes computing heterogeneity
Fail
Over load
Stand-byServer on duty
M A
E P
M: MonitorA: AnalysisP: PlanE: Execution
Service on demand Pay Per Use (PPU)
1 2 3 4 5 6 7 8 9 10 11 12
Month
Savings
Option – Pay per Use
Basic Charge
Access Spike Overload
Intra Grid for higher resource utilization
Present in-house system Solution by the business grid
Infra mgmt
Personnel Accounting Production CRM CRM
Business Grid M/W
・ TCO saving・more flexibility in business units
EffectSolution
Grid virtualizes IT resource to share
Problem
・ Vertically fixed resource allocation・ less flexibility in dynamic allocation
shared environment
Infra mgmt Infra mgmt
PersonnelAccounting
Production CRMGeneralaffairs
Off loading access spike
Hot spot
Additional resourcemaintain business continuity
network
Online shop A
Settlement
Delivery
Maker P
Maker Q
Online shop B
Parts Maker
storageserver
Additional resource deals with overflow traffic
Guarantee business continuity
EffectSolution
Faster and easier deployment, and provisioning – much less SE work
Problem
unexpected spike-may lose business opportunity
-may lose reliability
Outsourcing IT resources by the grid service provider
Data Center X
Bussiness A
Bussiness B
Outsourcing
Outsourcing
The in-house system of a company<Grid service provider>
Resource offer
Systems can cooperate between the in-house and the data center by the business grid.
EffectSolution
Constructing system using grid middleware.
Subject
Demand of curtailing system possession cost.
Resources are required for load / peak management.
IDC as a backup center in case of calamityDisaster recovery
Business A
Business B
Design SystemDesign System
Accounting systemAccounting system
Personnel systemPersonnel system
AA Business SystemAA Business System
BB Analysis SystemBB Analysis System
Data protection bases
A CorpDB
B CorpDB
C CorpDB
ResourcePool
A CorpA Corp
B CorpB Corp
C CorpC Corp
C CorpC Corp
C CorpC Corp
Pooled resources→A CorpPooled resources→A Corp
Pooled resources→B CorpPooled resources→B Corp
Pooled resourcesPooled resources
Pooled resources (Additional)Pooled resources (Additional)
AA ComputingParts group
CC Business Parts group
BB BusinessParts group
OtherParts group
AP servers (Parts functional bases)
DB serversWeb/AP servers (Business system bases)
Integrated DB bases
Ne
two
rk ba
ses
Load increases
Automatic extension of pooled resources
Restores automatically to pool resources
・ Adding active resources・ Automatic inclusion
(No need for stopping other systems)
Integrated Operation Management Bases
Monitor Monitor
Mo
nito
r
trouble
Automatic extension / self-restoration
Network
Automatic assignment of a zone
TCO curtailmentCommon pool improve utilization ratio
EffectSolutionGrid enables common IT resource pool to shared back up
Subject
Duplicated applications and IT resources
Many unused resources for every system
Realization of optimizing shared IT resources in a group company
Karaoke Grid @ SC2003
National and InternationalGrid Activities
Significant Grid Activities in Asia-PacNAREGI 1,450M JPY ’04 ’03-’07 MW, AP, HW
Business Grid 2,500M JPY ’04 ’03-’05 MW
Asia Grid Initiative
(ApGrid/APAN)
120M JPY ’02-’04 OP
K*Grid 32M USD ’02-’06 HW, MW, AP, OP
NGO approx.
25M SGD
’04-’05 HW, MW< AP, OP
China Grid
Chinese Taipei
EcoGrid AP
Thai Grid ’04- HW, MW, OP
PRAGMA ’02- AP, OP
1USD=110JPY=1,100KrW=33.8TWD=41.3THB=1.69SGD
ApGrid: Asia Pacific Partnership for Grid Computing
Open community for Grid researchers in Asia Pacific
ApGrid:A meeting point for all Grid researchers in Asia-PacificA communication channel to the GGF, and other grid communities (e.g. TeraGrid, UK-eScience, EUGrid, etc.)A computing testbed for making real grid-collaboration
India
ApGrid Testbed – unique features –
Truly (naturally) multi national/political/institutional VO beyond boundaries
Not an application-dedicated testbed – general platformDiversity of languages, culture, policy, interests, …
Grid BYO – Grass roots approachEach institution contributes his resources for sharingNot a single source funded for the development
Physical resourcesMost contributed resources are small-scale clustersNetworking is there, however the bandwidth is not enough
Achievementshave experiences on running international VOverify the feasibility of this approach for the testbed development
Non-profit international Non-profit international consortium established on consortium established on 3 June 1997. 3 June 1997.
Being a high-performance Being a high-performance network for R&D on network for R&D on advanced next generation advanced next generation applications and services. applications and services.
Providing an advanced Providing an advanced networking environment for networking environment for the research and education the research and education community in Asia-Pacific.community in Asia-Pacific.
Promoting global Promoting global collaboration.collaboration.
Tightly collaborating with TransTightly collaborating with TransPAC.PAC.Having meetings twice a year.Having meetings twice a year.
NSF-funded project lead by NSF-funded project lead by UCSD/SDSC. UCSD/SDSC.
11stst workshop was held in March workshop was held in March 2002. 2002.
7th one – Sep.15-17 ‘04
Establish sustained collaborations Establish sustained collaborations and advance the use of the Grid and advance the use of the Grid technologies for applications.technologies for applications.
Expected outcomes:Expected outcomes:Advance scientific applicationsIncrease productive and effective use of the grid by researchers and scientists in the Pacific RimIncrease interoperability of grid middleware in Pacific Rim and throughout the world
Tightly collaborating with ApGrid.Tightly collaborating with ApGrid.Having workshops 2 times a year.Having workshops 2 times a year.
Pacific Rim Application and Grid Middleware
Assembly
March 2004 ([email protected]) www.ggf.org
Ed
ucatio
n
Par
tner
ship
s
Global Grid Forum
Specifications and Best Practices
Identify where / what specs are critical
Frameworks / Architectures(how specs fit together)
Research and Technology Directions
Application Requirements
WS/I
W3C
OASIS
DMTF
IETF
GGF must collaborate with (leverage) a
variety of standards organizations
working on underlying
and related
technologies.
Korea: Sangsan Lee (GFAC)Japan: Sekiguchi (GFAC), Matsuoka (GFSG), Itoh (GMAC), and co-chairs, etc
March 2004 ([email protected]) www.ggf.org
GGF Meeting Participation
0
100
200
300
400
500
600
GF1 GF3 GF5 GGF2 GGF4 GGF6 GGF8 GGF10
Countries Organizations Group Participants Plenary Participants*GGF3 shows impact of 9-11
TokyoMarch,’03
13th in SeoulMarch,’05
12th in BrusselsSep,19-22 ’04
Roles of the APEC/TEL Grid Workshop
Expected role of APEC/TEL
Government role includes:Encourage basic studiesRemove potential obstacles in advanceEstablish Int’l cooperation framework to speak to other communities, - US, EU, etcNO commitment for each business or project
Business and Science Grid are inseparable from each other:
An ultimate challenge in science grid is always a trailblazer for expanding the business field.
eScience, Large science, Affordable high speed network
Having consensus of shared policy is a MUST for promoting future international grid business opportunity cooperatively,.
security, CA ops, SLA, balance, etc
Grid Challenges
Cope with different management policies of different centersEnsure data security: more is at stake than just money!Guarantee higher reliability for business continuity: prepare for disaster recoveryAgility and resiliency for every business opportunity
Grow rapidly, yet remain reliable for more than a decadeShare data between thousands of scientists with multiple interestsLink major computer centres, not just PCsEnsure all data accessible anywhere, anytime
Business and ENTERPRISEBusiness and ENTERPRISE
SCIENCE and TECHNOLOGYSCIENCE and TECHNOLOGY
Grid TechnologyGrid Technology
Upper MiddleUpper Middle
HPCHPC
Portal/ASPPortal/ASP
Grid Operation CtrGrid Operation Ctr
High speed network (over 1G – 10G)
Ubiquitous GridUbiquitous Grid
IPv6IPv6 SecuritySecurity
Wireless networkTremendous Sensor
Huge datastream
GPSGPS
CamCam
sensorsensor
MobileMobile
Bloodpressure
Bloodpressure
Earth QEarth Q
Database/ExpressionDatabase/Expression
FTTH/xDSL/Wireless AP/100M-FTTH/xDSL/Wireless AP/100M-
DistributedStorage10PB
DistributedStorage10PB
Community InfraCommunity InfraScience and Engineeringplatform
Science and Engineeringplatform
Computing1PFLOPSPC x 100MSC x 10000
Computing1PFLOPSPC x 100MSC x 10000
Peta Scale GridPeta Scale Grid
Nano-techInformatic
s
Nano-techInformatic
s
Access Grid
Access Grid
ChemInfoma
ChemInfoma
Personal Wether s
ervice
Personal Wether s
ervice
Bio-Informatics
Bio-Informatics
MedicalInformatic
s
MedicalInformatic
s
Lower MIddleLower MIddle
P2P gridP2P grid
WebserviceWebservice
eBiz, service
eBiz, service SolutionSolution
UtilityUtility
CommunityCommunity
Server tech
Grid enabled EngineeringGrid enabled Engineering Grid enabled BusinessGrid enabled Business
Grid Architectures and AIST contributions
Grid Fabric
AP partners have Computers: PC, WS, Clusters, Supercomputers; SCE manager;WAN: Internet, APAN, GrangeNet, APII, Tsukuba WAN; Instruments: KEK Accelerators, …
Low Level Grid Middleware
Security, Process, Storage, QoS, etc: Globus, Jxta, CPM, GrACE, OGSA
High Level Grid Middleware
Programming Tools: Ninf-G, Omuni-RPC, Nimrod DPL, MetaCompilerSimulators: Bricks and GridSim, Data farm
Resource Brokers: Nimrod-G
Applications and Portals
ActiveSheet, BioGrid, Drug Design, HEPGrid, ThaiGrid Portal, CFD, Chemical Eng., Environmental and Ecology Preservation,…
Social Framework
Clusters, MPPs,VP, SMPs
Education/training/workshop
Grid RPC/Ninf-GGrid Data Farm
QC Portal
DrugDesign
BioInfoma
Agriculture HEPDG
1 USD = 129 JPY
tools
Summery of my talk
Grid technology has been significantly emerging in science scene.
More demand in business scene.
Grid expands the IT market size and other areas.
Encourage Grid activities in the region is a MUST !
Let’s talk about it for today
Back ups