Cluster Computing

References• HA Linux Project

– http://www.linux-ha.org• Sys Admin

– http://www.samag.com/documents/s=1155/sam0101a/0101a.htm• Load Balancing in Server 2003

– http://channels.lockergnome.com/it/archives/20041022_load_balancing_in_windows_server_2003.phtml

• Microsoft TechNet– http://technet2.microsoft.com/WindowsServer/en/Library/98d46a24-96d8-412c-87d8-

28ace62323d21033.mspx • Force10 Networks

– http://www.force10networks.com/applications/mediarendering.asp • ARS Technica

– http://arstechnica.com/articles/paedia/cpu/klat2.ars • IBM

– http://www-03.ibm.com/servers/eserver/pseries/news/pressreleases/2002/nov/p655.html• Tsunami Technologies

– http://www.tsunamictechnologies.com• University of Buffalo

– http://www.ccr.buffalo.edu/hotpages/content/u2/u2.htm

Who Cares?

• Science Field (SETI)

• Medical Field (Protein Folding/ VR)

• Hollywood (SFX)

• Webmasters

• Anyone who needs massive amounts of computing power – on a “budget”

Outline

• Types of computer clusters

• Who uses them?

• Benefits of each

• Costs

3 Types of Clusters

• High Availability (HA)

• Load Balancing / (LB)

• High Performance/Parallel (HP)

High Availability Clusters

• Two or more servers setup so that if one fails, the other takes over– Share a common data source– Only one accesses data at any time– Monitor each-other by sending “heartbeat” packets

between themselves– When one server fails, the work is migrated over to

the working one– Users experience a brief break in transmission

(typically a few seconds) but not a total loss

HA Clusters ( cont )

Load Balancing Cluster

• A main server distributes data/application requests to any one of many other servers/nodes– A web server could distribute http requests to

appropriate servers based on proximity to client, etc

– Often called a “server farm”– Improve performance by splitting up requests

Load Balancing ( cont )

High Performance Cluster

• Divides computational tasks amongst many different nodes, allowing their shared processing power to be utilized– 296 of the world’s fastest computers are HP

clusters (CNET News)– Nodes can communicate with each-other

while processing – Processor intensive calculations that would

take years on a typical computer, can be completed in a small fraction of the time

HP Clusters ( cont )

Costs

• The price per gigaflop when using a supercomputer is about 5x more expensive than with a cluster

• Clusters have brought the price per GFlop down to around $85

• Supercomputer prices have come down as well though a high-end Sun Fire 15K server with 16 1.0Ghz processors and 64 gigabytes of ram costs $320,000

Costs ( cont )

• Cost of ownership– Important to remember you are not just buying

the hardware inside the computer case– Switches, cabling, and interface hardware

must be purchased as well– Power consumption of a large cluster (for

example 20 nodes) will carry a price tag in excess of $3000 per year

– Cost to free up space for a large cluster– Cost to pay employees to set it up and

monitor it

Case Study

University of Buffalo, New York

800 Dell nodes each housing dual 3.2 GHz Xeon Processors

Case Study

• The University of Buffalo cluster cost $2.3 million to implement– Runs at a peak of around 10 TeraFlops (10 Trillion

Floating Point Operations Per Second)– Put them around #40 on the Top500 most powerful

computer list– Costs $150,000+ per year in electric costs– For now can only be run at 60% power without

causing a power outage– Rendered a video that would have taken a typical

home PC 125 days to render, in 8 hours

Cost Effective Alternative• Many companies offer “cluster on demand”

service– Rent time on clusters from 3rd party companies– Great for companies that do not need that much

computing power, all the time– Typically pay per CPU hour:

• 60 cents per CPU hour• 40 node cluster• 72 hours• $1700 (most companies have minimum hour requirements)

HP Cluster Construction

• MPI (Message Passing Interface)– Most widely used standard– Sought to unify parallel programming– Gives programmers a starting point– Works best when programmed with Fortran,

but C++ works as well

Cluster Construction ( cont )

• Install Fortran ( or C ) Compiler• Configure SSH (need to be able to

execute commands at root without asking for a password)

• Setup a network file system• Install MPI (from anl.gov)• Edit MPI file to add each node’s hostname• Run test programs!

(most steps were taken from http://www-128.ibm.com/developerworks/linux/library/l-cluster2/index.html)

Future

• Create a HP Cluster in the computer lab

• CCSU audio/video server into a high availability setup

Questions?