shared memory considerations
DESCRIPTION
Shared Memory Considerations. Introduction to Parallel Programming – Part 4. Review & Objectives. Previously: Described method to identify opportunities for parallelism in code segments and applications At the end of this part you should be able to: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/1.jpg)
INTEL CONFIDENTIAL
Shared Memory ConsiderationsIntroduction to Parallel Programming – Part 4
![Page 2: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/2.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Review & Objectives
Previously:Described method to identify opportunities for parallelism in
code segments and applications
At the end of this part you should be able to: Describe the shared-memory model of parallel
programmingDemonstrate how to implement domain and
task decompositions using threadsDecide whether a variable in a multithreaded
program should be shared or private
![Page 3: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/3.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Cooperating Parallel Processes
Parallel computing multiple processes working together to speed the solution of a task
Working together process cooperationKinds of cooperation
Sharing information (communication)Keeping out of each other’s way
(synchronization)
3
![Page 4: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/4.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Shared-Memory Model
4
Shared Memory
Core
PrivateMemory
Core
PrivateMemory
Core
PrivateMemory
![Page 5: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/5.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Evaluating Parallel Models
How do processes share information?How do processes synchronize?In shared-memory model, both accomplished
through shared variablesCommunication: bufferSynchronization: semaphore
5
![Page 6: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/6.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Methodology
Study problem, sequential program, or code segmentLook for opportunities for parallelismUse threads to express parallelism
6
![Page 7: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/7.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
What Is a Process?
A program in some state of executionCodeDataLogical address space
Information about a process includesProcess stateProgram counterValues of core’s registersMemory management information
7
![Page 8: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/8.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
What Is a Thread?
“A unit of control within a process” — Carver and Tai
Main thread executes program’s “main” functionMain thread may create other threads to execute
other functionsThreads have own program counter, copy of core
registers, and stack of activation recordsThreads share process’s data, code, address space,
and other resourcesThreads have lower overhead than processes
8
![Page 9: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/9.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
Utility of Threads
Threads are flexible enough to implementDomain decompositionTask decompositionPipelining
![Page 10: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/10.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition Using Threads
10
SharedMemory
Thread 0 Thread 2
Thread 1
f ( )
f ( )
f ( )
![Page 11: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/11.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition Using Threads
11
SharedMemory
Thread 0 Thread 1
e ( )
g ( )h ( )
f ( )
![Page 12: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/12.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Pipelining Using Threads
12
Thread 0 Thread 2Thread 1
Shared MemoryInput Output
e ( ) f ( ) g ( )Dataset 2
Data sets5, 6, ...
Dataset 4 Data
set 3 Dataset 1
![Page 13: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/13.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Shared versus Private Variables
13
SharedVariables
PrivateVariables
PrivateVariables
Thread
Thread
![Page 14: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/14.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);
14
![Page 15: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/15.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);
15
![Page 16: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/16.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);
16
SharedPrivate
![Page 17: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/17.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k); ...}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
17
![Page 18: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/18.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
18
Thread 0
Thread 1
![Page 19: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/19.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
19
Thread 0
Thread 1
Static variable: Shared
![Page 20: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/20.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
20
Thread 0
Thread 1
Heap variable: Shared
![Page 21: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/21.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
21
Thread 0
Thread 1
Function’s local variables: Private
![Page 22: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/22.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Shared and Private Variables
Shared variablesStatic variablesHeap variablesContents of run-time stack at time of call
Private variablesLoop index variablesRun-time stack of functions invoked by thread
22
![Page 23: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/23.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Shared-Memory Model (Reprise)
23
Shared Memory
Core
PrivateMemory
Core
PrivateMemory
Core
PrivateMemory
![Page 24: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/24.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Threads Model
24
Shared Variables
Thread
PrivateVariables
Thread
PrivateVariables
Thread
PrivateVariables
![Page 25: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/25.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
References
Jim Beveridge and Robert Wiener, Multithreading Applications in Win32®, Addison-Wesley (1997).
David R. Butenhof, Programming with POSIX® Threads, Addison-Wesley, (1997).
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
25
![Page 26: Shared Memory Considerations](https://reader035.vdocuments.net/reader035/viewer/2022062410/56816391550346895dd48713/html5/thumbnails/26.jpg)