parallel algorithms & distributed computing
DESCRIPTION
Parallel Algorithms & Distributed Computing. Matt Stimmel Matt White. The Goal: 1 Gigapixel. How?. How?. Imhotep , How are we going to make all these bricks?. The Plan. We’ll use the Hebrew Slaves!. The Plan. +. =. Benford 6100 Brick Mold. Hebrew Slave. Brick. The Process. - PowerPoint PPT PresentationTRANSCRIPT
Parallel Algorithms & Distributed Computing
Matt StimmelMatt White
The Goal: 1 Gigapixel
How?
How?
The Plan
Imhotep, How are we going to make all these bricks?
The Plan
We’ll use the Hebrew Slaves!
The Process
Hebrew Slave
+
Benford 6100 Brick Mold
=
Brick
Re-evaluation
Yeah, this is pretty slow.
Re-evaluation
We’ll use MORE Hebrew Slaves!
The Revised Process
+ =
+ =
+ =
+ =
+ =
+ =
Lots of Hebrew Slaves
Lots of Benford 6100 Brick Molds
Enough bricks for a pyramid!
Thus Parallel Algorithms were born
In computers
Job
Result
Process
Faster!
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
Process
Process
Process
Process
What happens here?
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
Process
Process
Process
Process
Ah Ha!
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
SplitCombine
Process
Process
Process
Process
Sequential!
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
SplitCombine
Process
Process
Process
Process
So, Really, we have…
Job
Result
SplitCombine
Sub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Process
Process
Process
Process
Sequential Parallel Sequential
Amdahl's Law
Speedup = 1
SequentialParallel(1 – Parallel) +
Amdahl's Law
Speedup = 1
SequentialParallel(1 – Parallel) +
Or, as a 5th grader would say, “How many times faster it goes”
Amdahl's Lawhttp://en.wikipedia.org/wiki/File:AmdahlsLaw.svg
Why Distributed Computing?
Process
Process
Process
Process
=
=
=
=
-
The Plan
Imhotep, we can’t afford all these Hebrew Slaves!
The Plan
Well the surrounding
countries have slaves they’re not
using…
The Revised Process: Cheap
+ =
+ =
+ =
+ =
+ =
Slaves from Egypt
Egypt’s Benford 6100 Brick Molds
Enough bricks for a pyramid!
Slaves from Mars Space Brickmakers
Slaves from ChickenLand
Avian Brickmakers
Robot Slaves Brick Factory
Distributed Computing
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
SplitCombine
Process
Process
Process
Process
Internet
Internet
Internet
Internet
Internet
Internet
Internet
Internet
Why Distributed Computing?
Process
Process
Process
Process
=
=
=
=
Internet
Internet
Internet
Internet
-
Not My
Design
The Big Picture
Client
FractalGenImageCombiner
Storage
Control
Data
Server
ServerFunctions:•Manages Connections to Clients•Allows the user to input task parameters. •Divides job into sub-tasks•Displays information about currently running job
Challenges:•Job division• Must divide into a perfect square number of Jobs
•Resolution• Problems occur when the resolution isn’t an integer multiple of the number of
jobs.•Networking• Treated client connections as a Finite State Machine for file transfers. Didn’t work.
• About 1 in 100 transfers failed.
Server
Client
ClientFunctions:•Manages Connection to server•Receives server-generated command line for job•Executes FractalGen on cue•Informs server upon completion
Challenges:•Maintaining a graceful disconnect•Aforementioned file transfer
FractalGen
FractalGenFunctions:•Render the fractal• Specified by the command line
•Save the fractal to the disk
Challenges:•How do you draw a Mandelbrot fractal anyway?• Z = Z2 + C
•Command Line Parsing•Generating large images:• Limit is approx. 8000x8000 due to graphics hardware
•Shader Implementation• Downloaded, then made attempts at optimization.• Iteration count
• Shader Model 3 gives a greater instruction cound allowing form more iterations.• Shader Model 2 has greater compatibility, but cannot render as many iterations.
• Client computer must have a GPU capable of running at least DirectX 9 or else FractalGen will not work.
Image Combiner
ImageCombinerFunctions:•Merge images back together
Challenges:•Finding .bmp files•Memory Allocation• Estimates demanded efficient heap usage• A lot of pointers, a lot of pointer math
•Working with Bitmaps• Multiple color modes – how many bits is a pixel?• “Upside Down” errors
•Running over a network
Client
FractalGen ImageCombiner
•C#•.NET•Multithreaded
•C++• OOP•Memory Allocation
•C#•.NET•XNA•HLSL•Shader Model 3.0
•C#•.NET•Multithreaded
Server
Overview of Technologies Used
How is the task divided into separate jobs?
Breaking up Jobs: Method A
Breaking up Jobs: Method B
Overlays
Method A: Since B didn’t work
We now send a square to each client
Client 1 Client 2
Client 3 Client 4
The clients return separate images of the fractal.
Run the image combiner.
Final Output
The Big Picture
Client
FractalGenImageCombiner
Storage
Control
Data
Server
Science
Science, Step 1
Using a parallel implementation and multiple processors, we will be able to increase performance over the same implementation on a single processor.
Hypothesis:
Science, Step 2
Run the system, splitting job into varying numbers of sub-tasks and varying the number of processors available.
Experiment:
Science, Step 3
Run the system using one client.Control:
Playwrighting, Step 1
Major Dramatic Question:
IsSequential
Parallel≥ 1?
Data
8000x 8000Number of
ClientsNumber of
Jobs Time 1 Time 2 Time 3 Time 4 Avg. Time Speedup "Percent Faster"
Serial 0 0 25 23 24 23 23.75 3.4245614 242.46%
1 Job, 1Client 1 1 82 81 81 81.333333 1 0.00%
4 Jobs, 1 Client 1 4 90 89 94 91 0.8937729 -10.62%
16 Jobs, 1 Client 1 16 105 106 105 103 104.75 0.7764519 -22.35%
100 Jobs, 1 Client 1 100 182 187 187 185.33333 0.4388489 -56.12%
1000 Jobs, 1 Client 1 1000 -100.00%
4 Jobs, 4 Clients 4 4 58 61 53 57.333333 1.4186047 41.86%
16 Jobs, 4 Clients 4 16 60 56 56 54 56.5 1.439528 43.95%
100 Jobs, 4 Clients 4 100 67 68 66 67 1.2139303 21.39%
1000 Jobs, 4 Clients 4 1000 306 306 0.2657952 -73.42%
Data
0 1 4 16 100 10000
50
100
150
200
250
300
350
Average Time by Number of Jobs
014
Number of Jobs
Time
Data
4 16 100 10000
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Number of Jobs
Speedup
Useful
JobSub-task
Sub-task
Sub-task
Sub-task
Sub-result
Sub-result
Sub-result
Sub-result
Result
SplitCombine
Process
Process
Process
Process