cse-e5430 scalable cloud computing lecture 2 scalable cloud computing (5 cr) 1/36 cse-e5430 scalable...

CSE-E5430 Scalable Cloud Computing (5 cr)

1/36

CSE-E5430 Scalable Cloud ComputingLecture 2

Keijo Heljanko

Department of Computer ScienceSchool of ScienceAalto University

[email protected]

14.9-2015


2/36

Google MapReduce

I A scalable batch processing framework developed atGoogle for computing the Web index

I When dealing with Big Data (a substantial portion of theInternet in the case of Google!), the only viable option is touse hard disks in parallel to store and process it

I Some of the challenges for storage is coming from Webservices to store and distribute pictures and videos

I We need a system that can effectively utilize hard diskparallelism and hide hard disk and other componentfailures from the programmer


3/36

Google MapReduce (cnt.)

I MapReduce is tailored for batch processing with hundredsto thousands of machines running in parallel, typical jobruntimes are from minutes to hours

I As an added bonus we would like to get increasedprogrammer productivity compared to each programmerdeveloping their own tools for utilizing hard disk parallelism


4/36


I The MapReduce framework takes care of all issues relatedto parallelization, synchronization, load balancing, and faulttolerance. All these details are hidden from theprogrammer

I The system needs to be linearly scalable to thousands ofnodes working in parallel. The only way to do this is tohave a very restricted programming model where thecommunication between nodes happens in a carefullycontrolled fashion

I Apache Hadoop is an open source MapReduceimplementation used by Yahoo!, Facebook, and Twitter


5/36

MapReduce and Functional Programming

I Based on the functional programming in the large:I User is only allowed to write side-effect free functions

“Map” and “Reduce”I Re-execution is used for fault tolerance. If a node executing

a Map or a Reduce task fails to produce a result due tohardware failure, the task will be re-executed on anothernode

I Side effects in functions would make this impossible, asone could not re-create the environment in which theoriginal task executed

I One just needs a fault tolerant storage of task inputsI The functions themselves are usually written in a standard

imperative programming language, usually Java


6/36

Why No Side-Effects?

I Side-effect free programs will produce the same outputirregardless of the number of computing nodes used byMapReduce

I Running the code on one machine for debugging purposesproduces the same results as running the same code inparallel

I It is easy to introduce side-effect to MapReduce programsas the framework does not enforce a strict programmingmethodology. However, the behavior of such programs isundefined by the framework, and should therefore beavoided.


7/36

Yahoo! MapReduce Tutorial

I We use a Figures from the excellent MapReduce tutorial ofYahoo! [YDN-MR], available from:http://developer.yahoo.com/hadoop/tutorial/

module4.html

I In functional programming, two list processing conceptsare used

I Mapping a list with a functionI Reducing a list with a function

http://developer.yahoo.com/hadoop/tutorial/module4.html

http://developer.yahoo.com/hadoop/tutorial/module4.html


8/36

Mapping a List

Mapping a list applies the mapping function to each list element(in parallel) and outputs the list of mapped list elements:

Figure: Mapping a List with a Map Function, Figure 4.1 from[YDN-MR]


9/36

Reducing a ListReducing a list iterates over a list sequentially and produces anoutput created by the reduce function:

Figure: Reducing a List with a Reduce Function, Figure 4.2 from[YDN-MR]


10/36

Grouping Map Output by Key to ReducerIn MapReduce the map function outputs (key, value)-pairs.The MapReduce framework groups map outputs by key, andgives each reduce function instance (key, (..., list of

values, ...)) pair as input. Note: Each list of values havingthe same key will be independently processed:

Figure: Keys Divide Map Output to Reducers, Figure 4.3 from[YDN-MR]


11/36

MapReduce Data FlowPractical MapReduce systems split input data into large(64MB+) blocks fed to user defined map functions:

Figure: High Level MapReduce Dataflow, Figure 4.4 from [YDN-MR]


12/36

Recap: Map and Reduce Functions

I The framework only allows a user to write two functions: a“Map” function and a “Reduce” function

I The Map-function is fed blocks of data (block size64-128MB), and it produces (key, value) -pairs

I The framework groups all values with the same key to a(key, (..., list of values, ...)) format, and theseare then fed to the Reduce function

I A special Master node takes care of the scheduling andfault tolerance by re-executing Mappers or Reducers


13/36

MapReduce Diagram

22.1.2010 2

Worker

Worker

Worker

Worker

split 0

split 1

split 2

split 3

split 4

(3)read

(1)fork

outputfile 0(4)

local write

outputfile 1

Userprogram

Master

(1)fork

(2)assignmap

(6)write

Worker

(5)Remote read

(1)fork

(2)assignreduce

Inputfiles

Mapphase

Intermediate files(on local disks)

Reducephase

Outputfiles

Figure: J. Dean and S. Ghemawat: MapReduce: Simplified Data Processing on Large Clusters, OSDI 2004


14/36

Example: Word Count

I Classic word count example from the Hadoop MapReducetutorial:http://hadoop.apache.org/docs/current/

hadoop-mapreduce-client/

hadoop-mapreduce-client-core/MapReduceTutorial.

html

I Consider doing a word count of the following file usingMapReduce:

Hello World Bye World

Hello Hadoop Goodbye Hadoop

http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html





15/36

Example: Word Count (cnt.)

I Consider a Map function that reads in words one a time,and outputs (word, 1) for each parsed input word

I The Map function output is:

(Hello, 1)

(World, 1)

(Bye, 1)

(World, 1)

(Hello, 1)

(Hadoop, 1)

(Goodbye, 1)

(Hadoop, 1)


16/36


I The Shuffle phase between Map and Reduce phasecreates a list of values associated with each key

I The Reduce function input is:

(Bye, (1))

(Goodbye, (1))

(Hadoop, (1, 1))

(Hello, (1, 1))

(World, (1, 1))


17/36


I Consider a reduce function that sums the numbers in thelist for each key and outputs (word, count) pairs. Theoutput of the Reduce function is the output of theMapReduce job:

(Bye, 1)

(Goodbye, 1)

(Hadoop, 2)

(Hello, 2)

(World, 2)


18/36

Phases of MapReduce

1. A Master (In Hadoop terminology: Job Tracker) is startedthat coordinates the execution of a MapReduce job. Note:Master is a single point of failure

2. The master creates a predefined number of M Mapworkers, and assigns each one an input split to work on. Italso later starts a predefined number of R reduce workers

3. Input is assigned to a free Map worker 64-128MB split at atime, and each user defined Map function is fed (key,

value) pairs as input and also produces (key, value)

pairs


19/36

Phases of MapReduce(cnt.)

4. Periodically the Map workers flush their (key, value)

pairs to the local hard disks, partitioning by their key to Rpartitions (default: use hashing), one per reduce worker

5. When all the input splits have been processed, a Shufflephase starts where M × R file transfers are used to sendall of the mapper outputs to the reducer handling each keypartition. After reducer receives the input files, reducersorts (and groups) the (key, value) pairs by the key

6. User defined Reduce functions iterate over the (key,

(..., list of values, ...)) lists, generating output(key, value) pairs files, one per reducer


20/36


I The user just supplies the Map and Reduce functions,nothing more

I The only means of communication between nodes isthrough the shuffle from a mapper to a reducer

I The framework can be used to implement a distributedsorting algorithm by using a custom partitioning function

I The framework does automatic parallelization and faulttolerance by using a centralized Job tracker (Master) and adistributed filesystem that stores all data redundantly oncompute nodes

I Uses functional programming paradigm to guaranteecorrectness of parallelization and to implementfault-tolerance by re-execution


21/36

Detailed Hadoop Data Flow

Figure: Detailed DataFlow in Hadoop, Figure 4.5 from [YDN-MR]


22/36

Adding CombinersCombiners are Reduce functions run on the Map side. Theycan be used if the Reduce function is associative andcommutative.

Figure: Adding a Combiner Function, Figure 4.6 from [YDN-MR]


23/36

Apache Hadoop

I An Open Source implementation of the MapReduceframework, originally developed by Doug Cutting andheavily used by e.g., Yahoo! and Facebook

I “Moving Computation is Cheaper than Moving Data” - Shipcode to data, not data to code.

I Map and Reduce workers are also storage nodes for theunderlying distributed filesystem: Job allocation is first triedto a node having a copy of the data, and if that fails, then toa node in the same rack (to maximize network bandwidth)

I Project Web page: http://hadoop.apache.org/

http://hadoop.apache.org/


24/36

Apache Hadoop (cnt.)

I When deciding whether MapReduce is the correct fit for analgorithm, one has to remember the fixed data-flow patternof MapReduce. The algorithm has to be efficiently mappedto this data-flow pattern in order to efficiently use theunderlying computing hardware

I Builds reliable systems out of unreliable commodityhardware by replicating most components (exceptions:Master/Job Tracker and NameNode in Hadoop DistributedFile System)


25/36

Apache Hadoop (cnt.)

I Tuned for large (gigabytes of data) filesI Designed for very large 1 PB+ data setsI Designed for streaming data accesses in batch processing,

designed for high bandwidth instead of low latencyI For scalability: NOT a POSIX filesystemI Written in Java, runs as a set of user-space daemons


26/36

Hadoop Distributed Filesystem (HDFS)

I A distributed replicated filesystem: All data is replicated bydefault on three different Data Nodes

I Inspired by the Google FilesystemI Each node is usually a Linux compute node with a small

number of hard disks (4-12)I A NameNode that maintains the file locations, many

DataNodes (1000+)


27/36

Hadoop Distributed Filesystem (cnt.)

I Any piece of data is available if at least one datanodereplica is up and running

I Rack optimized: by default one replica written locally,second in the same rack, and a third replica in another rack(to combat against rack failures, e.g., rack switch or rackpower feed)

I Uses large block size, 128 MB is a common default -designed for batch processing

I For scalability: Write once, read many filesystem


28/36

Implications of Write Once

I All applications need to be re-engineered to only dosequential writes. Example systems working on top ofHDFS:

I HBase (Hadoop Database), a database system with onlysequential writes, Google Bigtable clone

I MapReduce batch processing systemI Apache Pig and Hive data mining toolsI Mahout machine learning librariesI Lucene and Solr full text searchI Nutch web crawling


29/36

HDFS ArchitectureI From: HDFS Under The Hood by Sanjay Radia of Yahoo

http://assets.en.oreilly.com/1/event/12/HDFS%20Under%20the%20Hood%20Presentation%201.pdf

http://assets.en.oreilly.com/1/event/12/HDFS%20Under%20the%20Hood%20Presentation%201.pdf


30/36

HDFS Architecture

I NameNode is a single computer that maintains thenamespace (meta-data) of the filesystem. Implementationdetail: Keeps all meta-data in memory, writes logs, anddoes periodic snapshots to the disk

I Recent new feature: Namespace can be split betweenmultiple NameNodes in HDFS Federation:http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/Federation.html

I All data accesses are done directly to the DataNodesI Replica writes are done in a daisy chained (pipelined)

fashion to maximize network utilization

http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/Federation.html


31/36

HDFS Scalability Limits

I 20PB+ deployed HDFS installations (10 000+ hard disks)I 4000+ DataNodesI Single NameNode scalability limits: The HDFS is

NameNode scalability limited for write only workloads toaround HDFS 10 000 clients, K. V. Shvachko: HDFSscalability: the limits to growth:http://www.usenix.org/publications/login/2010-04/

openpdfs/shvachko.pdf

I The HDFS Federation feature was implemented to addressscalability (not fault tolerance) issues

http://www.usenix.org/publications/login/2010-04/openpdfs/shvachko.pdf

http://www.usenix.org/publications/login/2010-04/openpdfs/shvachko.pdf


32/36

HDFS Design Goals

Some of the original HDFS targets have been reached by 2009,some need additional work:

I Capacity: 10 PB (Deployed: 14 PB in 2009, 20+ PB by2010)

I Nodes: 10000 (Deployed: 4000)I Clients: 100000 (Deployed: 15000)I Files: 100000000 (Deployed: 60000000)


33/36

Hadoop Hardware

I Reasonable CPU speed, reasonable RAM amounts foreach node

I 4-12 hard disks per node seem to be the currentsuggestion

I CPU speeds are growing faster than hard disk speeds, sonewer installations are moving to more hard disks / node

I Gigabit Ethernet networking seems to be dominant,moving to 10 Gb Ethernet as prices come down


34/36

Hadoop Network Bandwidth Consideration

I Hadoop is fairly network latency insensitiveI Mapper reads can often be read from the local disk or the

same rack (intra-rack network bandwidth is cheaper thaninter-rack bandwidth)

I For jobs with only small Map output, very little networkbandwidth is used

I For jobs with large Map output, Hadoop likes largeinter-node network bandwidth (Shuffle phase),

I To save network bandwidth, Mappers should produce aminimum amount of output


35/36

Terasort Benchmark

I Hadoop won the Terasort benchmark in 2009 by sorting100 TB in 173 minutes using: 3452 nodes x (2 QuadcoreXeons, 8 GB memory, 4 SATA disks/node)


36/36

Two Large Hadoop Installations

I Yahoo! (2009): 4000 nodes, 16 PB raw disk, 64TB RAM,32K cores

I Facebook (2010): 2000 nodes, 21 PB storage, 64TB RAM,22.4K cores

I 12 TB (compressed) data added per day, 800TB(compressed) data scanned per day

I A. Thusoo, Z. Shao, S. Anthony, D. Borthakur, N. Jain, J.Sen Sarma, R. Murthy, H. Liu: Data warehousing andanalytics infrastructure at Facebook. SIGMOD Conference2010: 1013-1020.http://doi.acm.org/10.1145/1807167.1807278

http://doi.acm.org/10.1145/1807167.1807278

cse-e5430 scalable cloud computing lecture 2 scalable cloud computing (5 cr) 1/36 cse-e5430 scalable...

Documents