grid computing
TRANSCRIPT
Agenda► Introduction► Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network Computing (BOINC)
► How to implement the concept of grid computing Grid computing middleware ,Installation configuration and
run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges.
► Some Grid Initiatives
Supercomputer
► A time dependent term which refers to the class of most powerful computer systems world-wide at the time of reference.
► An extremely fast computer that can perform hundreds of millions of instructions per second..
► Examples :PARAM Padma (C-DAC), Columbia Supercomputer (NASA)
PARAM Padma
►PARAM Padma is C-DAC’s next generation high performance scalable computing cluster,currently with a peak computing power of One Teraflop.
►It has 248 processors (IBM POWER4 @ 1GHz ) , 62 nos. of IBM p630 Nodes and 496 GB Main Memory.
configuration
specifications1: No. of Processors 248 (Power 4@1GHz)
2: Internal Storage 4.5 TeraBytes
3: Operating System AIX / LINUX
4: Networks Primary : PARAMNet-II @ 2.5 Gbps Full DuplexBackup : Gigabit thernet @ 1 Gbps Full Duplex
5: Total Weight 9 Tons
6: Cabling (Networks) 12Kms (CAT6 ) & 2.5Kms(Fibre Optic)
7: Total Machine Floor Area 168 Sq.metre (1800sq.ft.)
Agenda► Introduction Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network Computing (BOINC)
► How to implement the concept of grid computing Grid computing middleware ,Installation configuration and run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges.
► Some Grid Initiatives
Grid Computing
►Using distributed computers and resources collectively.
►Usually associated with geographically distributed computers and resources on a high speed network.
►Often about teams sharing resources.
► The primary advantage of Grid computing is that each node can be purchased as commodity hardware, which when combined can produce similar computing resources to a many-CPU supercomputer, but at lower cost.
► The primary performance disadvantage is that the various CPUs and local storage areas do not have high-speed connections. This arrangement is thus well-suited to applications where multiple parallel computations can take place independently, without the need to communicate intermediate results between CPUs.
Evolution, not revolution
► grid can be seen as the latest and most complete evolution of more familiar developments such as distributed computing, the Web, peer-to-peer computing and virtualization technologies.
► Like the Web, grid computing keeps complexity hidden: multiple users enjoy a single, unified experience.
► Unlike the Web, which mainly enables communication, grid computing enables full collaboration toward common business goals.
► Like clusters and distributed computing, grids bring computing resources together.
► Unlike clusters and distributed computing, which need physical proximity and operating homogeneity, grids can be geographically distributed and heterogeneous.
► Like virtualization technologies, grid computing enables the virtualization of IT resources.
► Unlike virtualization technologies, which virtualize a single system, grid computing enables the virtualization of vast and disparate IT resources
► Supercomputer: It can be costly and difficult to write programs so that they can be run in the environment of a supercomputer, which may have a custom operating system.
► Grid computing: If a problem can be adequately parallelized, a "thin" layer of "grid" infrastructure can cause conventional, standalone programs to run on multiple machines (but each given a different part of the same problem)
Why Grid Computing Now?
LocalCluster
Inter PlanetGrid
2100
2100 2100 2100 2100
2100 2100 2100 2100
Personal Device SMPs or SuperComputers
GlobalGrid
EnterpriseCluster/Grid
Scalable ComputingPERFORMANCE
+
Q
o
S
Agenda► Introduction Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network
Computing (BOINC)► How to implement the concept of grid computing
Grid computing middleware ,Installation configuration and run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges.
► Some Grid Initiatives
Design considerations and variations
► One feature of distributed grids is that they can be formed from computing resources belonging to multiple individuals or organizations, so it is easy to assemble volunteer computing networks.
► One disadvantage of this feature is that
the computers which are actually performing the calculations might not be entirely trustworthy. This often involves assigning work randomly to different nodes and checking that at least two different nodes report the same answer for a given work unit.
►Due to the lack of central control over the hardware, there is no way to guarantee that nodes will not drop out of the network at random times. These variations can be accommodated by assigning large work units (thus reducing the need for continuous network connectivity) and reassigning work units when a given node fails to report its results as expected.
►Various middleware projects have created generic infrastructure, to allow various scientific and commercial projects to utilize a particular associated grid, or for the purpose of setting up new grids. BOINC is a common one for academic projects seeking public volunteers.
Berkeley Open Infrastructure for Network
Computing
► The Berkeley Open Infrastructure for Network Computing (BOINC) is a non-commercial middleware system for grid computing.
► The intent of BOINC is to make it possible for researchers to tap into the enormous processing power of personal computers around the world.
► BOINC has over 4,30,000 active computers (hosts) worldwide processing on average 663 TFLOPS as of September 8, 2007.
►Design and structure of BOINC BOINC client-server technology.►BOINC Credit System Validating results before granting
credit. ►Projects using BOINC Framework 1.Cell Computing — biomedical
research. 2.BBC Climate Change Experiment
— (part of Climateprediction.net)
Agenda
► Introduction Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network Computing
(BOINC)► How to implement the concept of grid computing
Grid computing middleware ,Installation configuration and run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges.
► Some Grid Initiatives
►Grid computing middleware S/W which is used to manage the
internal working of a grid computing system. Eg: 1:Alchemi: A .NET-based Enterprise Grid
System and Framework.
How Alchemi Works There are four types of distributed
components (nodes) involved in the construction of Alchemi grids and execution of grid applications: Manager, Executor, User & Cross-Platform Manager.
►A grid is created by installing Executors on each machine that is to be part of the grid and linking them to a central Manager component.
Executors
Dedicated Nondedicated
► dedicated :- meaning the Manager initiates thread execution
directly.► non-dedicated:- meaning that thread execution is initiated by the
Executor. works through firewalls and NAT servers since
there is only one-way communication between the Executor and Manager.
► Dedicated Executors are more suited to an intranet environment
► and non-dedicated Executors are more suited to the Internet environment.
► Users can develop, execute and monitor grid applications using the .NET API and tools which are part of the Alchemi SDK. Alchemi offers a powerful grid thread programming model.
►An optional component (not shown in fig:) is the Cross Platform Manager web service which offers interoperability with custom non-.NET grid middleware.
Installation, Configuration and Operation
► Common Requirements
MANAGER The Manager should be installed on a stable and
reasonably capable machine. The Manager requires SQL Server 2000 or MSDE 2000.
Microsoft .NET Framework 1.1
►The Alchemi Manager can be installed in two modes
1: As a normal Windows desktop application
2: As a windows service. (supported only on Windows NT/2000/XP/2003)
Configuration & Operation
► Role-Based Security Every program connecting to the Manager must
supply a valid username and password. Three default accounts are created during installation: executor (password: executor), user (password: user) and admin (password: admin) belonging to the 'Executors', 'Users' and 'Administrators' groups respectively.
Users are administered via the 'Users' tab of the Alchemi Console (located in the Alchemi SDK)Only.
Executor
The Alchemi Executor can be installed in two modes 1: As a normal Windows desktop application 2: As a windows service. (supported only on Windows NT/2000/XP/2003)
Installation
Configuration & Operation
You need to configure 2 aspects of the Executor:• The host and port of the Manager to connect to.
• Dedicated / non-dedicated execution. A non-dedicated Executor executes grid threads on a voluntary basis (it requests threads to execute from the Manager), while a dedicated Executor is always executing grid threads (it is directly provided grid threads to execute by the Manager).
• A non-dedicated Executor works behind firewalls and NAT servers.
•Click the "Connect" button to connect the Executor to the Manager.
If the Executor is configured for non-dedicated execution, we can start executing by clicking the "Start Executing" button in the "Manage Execution" tab.
► Alchemi Console The Console (Alchemi.Console.exe) is a
grid administration and monitoring tool.
Agenda
► Introduction Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network Computing
(BOINC)► How to implement the concept of grid computing
Grid computing middleware ,Installation configuration and run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges
► Some Grid Initiatives
Essential Utilities and Delivery Networks
(1) WaterWater
Distribution Network
(4) TelephoneTelecom Networks
(3) Gas
(2) Electricity
Power Grid
Computing Grid: Delivering IT services as the 5th utility (Power Grid
inspiration)
eScienceeBusiness
eGovernmenteHealth
MultilingualeEducation
…
Classes of Grid Services / Types of Grids
► Computational Services – CPU cycles Pooling computing power: SETI@Home,
TeraGrid, AusGrid, ChinaGrid, IndiaGrid, UK Grid,…
► Data Services Collaborative data sharing generated by
instruments, sensors, persons: LHC Grid, Napster
► Application Services Access to remote software/libraries and
license management—NetSolve ► Interaction Services
eLearning, Virtual Tables, Group Communication (Access Grid), Gaming
► Knowledge Services The way knowledge is acquired, processed
and managed—data mining.► Utility Computing Services
Towards a market-based Grid computing: Leasing and delivering Grid services as ICT utilities.
Computational Grid
Data Grid
ASP Grid
Interaction Grid
Knowledge Grid
Utility Grid
infra
stru
ctu
re
Users
Grid Challenges
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Construction
What do Grid players want?
► Grid Consumers Execute jobs for solving varying problem size
and complexity Benefit by utilizing distributed resources
wisely Tradeoff timeframe and cost
►Strategy: minimise expenses
► Grid Providers Contribute resources for executing consumer
jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market
opportunity►Strategy: maximise return on investment
Principle 1: Service Oriented Architecture (SOA)
► A SOA is a contractual architecture for offering and consuming software as services.
► There are four entities that make up an SOA service provider, service registry, and service consumer (also known as service requestor).
► The functions or tasks that the service provider offers, along with other functional and technical information required for consumption, are defined in the service definition or contract.
provider
registry
consumer
contract
Principle 2: Market-Oriented (Grid) Computing- (a) Sustained Resourced Sharing and (b)
Effective Management of Shared Resources
Grid Economy
On Demand Assembly of Services: Putting Them All Together
ASP Catalogue
Grid Info Service
Grid Market Directory
GSP(Accounting Service)
GridbusGridBank
GSP(e.g., UofM)
PEGSP
(e.g., VPAC)
PE
GSP(e.g., IBM)
CPUorPE
Grid Service (GS)
(Globus)
Alchemi
GS
GTS
Cluster Scheduler
Job
8
GridResource Broker
2
Visual Application Composer
Application CodeExplore
data1
36
45
Resu
lts9 7
Results+
Cost Info
10
11
Bill
12Data Catalogue
G-Monitor: Setup Experiment
Active Sheet:Spreadsheet Processing on Grid using
Nimrod-G
NimrodProxy
Nimrod-G
World-Wide Grid
What are Grid benefits?►Resource sharing across multiple
administrative boundaries Effective utilisation of the (existing)
resources Dynamic provisioning
►Application Acceleration►Scalability►Virtualisation:
applications, services, resources,…
Agenda
► Introduction Supercomputer Example: PARAM padma► Grid Computing
Definition, Evolution not revolution, Why Grid Computing Now? …
► Design considerations and variations Berkeley Open Infrastructure for Network Computing
(BOINC)► How to implement the concept of grid computing
Grid computing middleware ,Installation configuration and run.
► Service-Oriented Grid Architecture and Gridbus Solutions Utility Networks and Grid Computing, Grid Challenges
► Some Grid Initiatives
Some Grid Initiatives Worldwide► Australia
Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch
► Brazil OurGrid, EasyGrid LNCC-Grid + many others
► China ChinaGrid – Education CNGrid - application
► Europe UK eScience EU Grids.. and many more...
► India Garuda
Japan NAGERI
► Korea...N*Grid
► SingaporeNGP
► USA Globus GridSec AccessGrid TeraGrid Cyberinfrasture and many more...
► Industry Initiatives IBM On Demand Computing HP Adaptive Computing Sun N1 Microsoft - .NET Oracle 10g Infosys – Enterprise Grid Satyam – Business Grid StorageTek –Grid.. and many more
► Public Forums Global Grid Forum Australian Grid Forum Conferences:
► CCGrid► Grid► HPDC► E-Science
http://www.gridcomputing.com
1.3 billion – 3 yrs
1 billion – 5 yrs
450million – 5 yrs
486million – 5 yrs
1.3 billion (Rs)
27 million
2? billion
120million – 5 yrs
► The Department of Information Technology (DIT),
Government of India has funded the Centre for Development of Advanced Computing (C-DAC) to deploy the nation-wide computational grid 'GARUDA' which will connect 17 cities across the country in its Proof of Concept (PoC) phase with an aim to bring "Grid" networked computing to research labs and industry. A total of 45 institutions have been connected. GARUDA will accelerate Indias drive to turn its substantial research investment into tangible economic benefits.
►The GARUDA High-Speed network is a Layer 2/3 MPLS Virtual Private Network (VPN) connecting select 45 institutions across 17 cities at 10/100 Mbps with Stringent Service Level Agreements with the service provider.
Shashwat [email protected]