Distributed Computing Systems

Current Issues in DCS

Dr. Sunny Jeong. spjeong@uic.edu.hk

Mr. Colin Zhang colinzhang@uic.edu.hkWith Thanks to Prof. G. Coulouris, Prof. A.S. Tanenbaum

and Prof. S.C Joo

Cluster, Grid, Cloud …Now…Hadoop?

To Support hadling Big data

Hadoop HDFS + MapReduce– designed so that any node failures are automatically handled by the

framework.

– enables applications to work with thousands of computation-independent computers and petabytes of data.

Facebook Yahoo..of course GOOGLE

Overview

Hadoop Computer Cluster Grid Computing Grid Computing Projects

3

Hadoop Architecture

Wikipedia

MapReduce Architecture

Yuan Luo

HDFS Architecture

Dhruba Borthakur

Clustering Computing

Consists of a set of loosely connected computers that work together as a single system.

The components of a cluster are usually connected to each other through fast LAN, each node with own OS

Availability of low cost microprocessors, high speed networks, and software for high performance distributed computing

7

Grid Computing

Computational Grid: – Focus on CPU scavenging/stealing technology for

complicated operation

Data Grid): Managing and sharing distributed intensive data

Access Grid: For Cooperation between remote computer using audio or video applications

Equipment Grid : Controlling remote machine and analyzing data from the machine.

Grid Project

•SETI@home•BOINC•holding@home•korea@home(stopped in 2011)

Cloud Computing( comparison with Grid)

Service Oriented Transaction oriented No standardization Organization No Standard Technology More less homogeneous gathering (compare to

Grid) Loosely connected

10

Barriers of the Concepts

Shim young cheol