7/30/2019 Data-Parallel Architectures And

1/27

Data-Parallel Architectures and

Algorithms

Group Members:

S.Ameed UddinS.Shujah HaiderS.Tauqeer AbbassUsman Khalil

7/30/2019 Data-Parallel Architectures And

2/27

Parallel Computing

Parallel computing is a form of computation in whichmany calculations are carried out simultaneously,

operating on the principle that large problems can oftenbe divided into smaller ones, which are then solvedconcurrently ("in parallel"). There are several differentforms of parallel computing:

BITDATA INSTRUCTION TASK

7/30/2019 Data-Parallel Architectures And

3/27

Data Parallelism

Data parallelism (also known as loop-levelparallelism) is a form of parallelization of computing

across multiple processors in parallel computingenvironments. Data parallelism focuses on distributingthe data across different parallel computing nodes.

Data parallelism is achieved when each processorperforms the same task on different pieces of distributeddata.

7/30/2019 Data-Parallel Architectures And

4/27

An Example

For instance, consider a 2-processor system(CPUs A and B) in a parallel environment, andwe wish to do a BINARY SEARCH on some datad.

Now,A B

7/30/2019 Data-Parallel Architectures And

5/27

Contd.

Here CPU A traverses the Head while CPU Btraverses the Tail.

Data parallelism emphasizes the distributed(parallelized) nature of the data.

Data Parallelism is opposite to task parallelism.

7/30/2019 Data-Parallel Architectures And

6/27

Clarification!

One can confuse data parallelism with themultithreaded programming in C# and JAVA,but the two are different.

In multithreading programming we alter thedurations of threads on processor as needed but

in data parallelism we are distributing the datato our resources.

7/30/2019 Data-Parallel Architectures And

7/27

Types of Data Parallel Arch.

Associative

architecture

and neural

SIMDs SystolicArchitecture

Vector

Architecture

Data Parallel Architecture

7/30/2019 Data-Parallel Architectures And

8/27

SIMD(Single Instruction Multiple Data)

Single instruction, multiple data (SIMD), isa class of parallel computers. It describescomputers with multiple processing elementsthat perform the same operation on multipledata simultaneously. Thus, such machines

exploit data level parallelism.

7/30/2019 Data-Parallel Architectures And

9/27

Contd.

7/30/2019 Data-Parallel Architectures And

10/27

Vector Processor

Avector processor, or array processor, is acentral processing unit (CPU) that implementsan instruction set containing instructions thatoperate on one-dimensional arrays of data calledvectors

. This is in contrast to a scalar processor,whose instructions operate on single data items.The vast majority of CPUs are scalar

7/30/2019 Data-Parallel Architectures And

11/27

Systolic Architecture

In computer architecture, a systolic arrayis apipe network arrangement of processing unitscalled cells. Here cells (i.e. processors), computedata and store it independently of each other.

A good example would be matrix multiplication

7/30/2019 Data-Parallel Architectures And

12/27

Contd.

7/30/2019 Data-Parallel Architectures And

13/27

Advantages and Disadvantages

Pros

Faster

Scalable

Cons

Expensive

Highly specialized for particular applications Difficult to build

7/30/2019 Data-Parallel Architectures And

14/27

Neural Architecture

An artificial neural network (ANN), usuallycalled neural network (NN), is a

mathematical model or computational modelthat is inspired by the structure and/orfunctional aspects of biological neural networks.

7/30/2019 Data-Parallel Architectures And

15/27

Applications

Parallel discrete event simulation

7/30/2019 Data-Parallel Architectures And

16/27

Contd.

Compute results at c,d,e before output from a isknown.

Therefore concurrent processing results in quickercompletion of task.

7/30/2019 Data-Parallel Architectures And

17/27

Matrix Addition

As matrices are stored by the help of multidimensional arrays, different processors add the

smaller chunks of huge matrix.

Sparse matrix( most values are 0) are alsoresolved by data parallel approach.

7/30/2019 Data-Parallel Architectures And

18/27

GPU(Graphics Processing Unit) GPU is a specialized circuit designed to rapidly

manipulate and alter memory in such a way so as to

accelerate the building of images in a frame bufferintended for output to a display. GPUs are used in embedded systems, mobile

phones, personal computers, workstations and gameconsoles.

Modern GPUs are very efficient at manipulatingcomputer graphics, and their highly parallelstructure makes them more effective than general-purpose CPUs.

7/30/2019 Data-Parallel Architectures And

19/27

Contd.

7/30/2019 Data-Parallel Architectures And

20/27

Contd.

All modern Video Accelerating cards, Streamprocessors ,and graphic cards to enhance

gaming experience use parallel architecture toprocess visual data.

7/30/2019 Data-Parallel Architectures And

21/27

Algorithms Implementation Identify concurrency in program Map concurrent pieces to parallel processes

Distribute input, output and intermediate data Manage accesses to shared data by processors Synchronize processors as program executes.

7/30/2019 Data-Parallel Architectures And

22/27

An Example

Model =civic and year=2001 and (color=greenor color=white)

7/30/2019 Data-Parallel Architectures And

23/27

Contd.

7/30/2019 Data-Parallel Architectures And

24/27

Contd.

7/30/2019 Data-Parallel Architectures And

25/27

Granularity

7/30/2019 Data-Parallel Architectures And

26/27

Contd.

7/30/2019 Data-Parallel Architectures And

27/27

THANK YOU!