02 progpar thread

8/14/2019 02 Progpar Thread

1/76

Outlines

How to Efficiently ProgramHigh Performance Architectures ?

Arnaud LEGRAND, CR CNRS, LIG/INRIA/MescalJean-Louis ROCH, MCF ENSIMAG, LIG/INRIA/Moais

Vincent DANJEAN, MCF UJF, LIG/INRIA/MoaisDerick KONDO, CR INRIA, LIG/INRIA/Mescal

Jean-Franois MHAUT, PR UJF, LIG/INRIA/MescalBruno RAFFIN, CR INRIA, LIG/INRIA/MoaisAlexandre TERMIER, MCF UJF, LIG/Hadas

Some slides come from Samuel THIBAULT, MCF Bordeaux, LaBRI/Runtime
http://find/http://goback/


2/76


3/76

Outlines

Part I: High Performance ArchitecturesPart II: Parallelism and ThreadsPart III: SynchronisationPart IV: Multithreading and Networking

High Performance Architectures

1 Parallel Machines with Shared MemoryILP and multi-cores

Symmetric Multi Processors

2 Parallel Machines with Distributed MemoryClusters

Grids

3 Current Architectures in HPC


4/76

Outlines


Parallelism and Threads

4 Introduction to Threads

5 Kinds of threadsUser threads

Kernel threadsMixed models

6 User Threads and Blocking System CallsScheduler Activations

7 Thread Programming InterfacePOSIX ThreadsLinux POSIX Threads LibrariesBasic POSIX Thread API


5/76

Outlines


Synchronisation

8 Hardware Support

9 Busy-waiting Synchronisation

10 High-level Synchronisation PrimitivesSemaphoresMonitors

11 Some examples with LinuxOld Linux libpthreadNew POSIX Thread Library

P I Hi h P f A hi


6/76

Outlines


Multithreading and Networking

12 A Brief Overview of MPI

13 Mixing Threads and Communication in HPCProblems arises

Discussion about solution


7/76

Parallel Machines with Shared MemoryParallel Machines with Distributed Memory

Current Architectures in HPC

Part I

High Performance Architectures


8/76



Outlines: High Performance Architectures

1 Parallel Machines with Shared MemoryILP and multi-cores

Symmetric Multi Processors

2 Parallel Machines with Distributed MemoryClustersGrids

3 Current Architectures in HPC


9/76



Parallel Architectures

Two main kindsArchitectures with shared memory and architectures withdistributed memory.

Multiprocessors

P

P

P

PMem

Clusters

Fast networkP Mem P Mem P Mem


10/76



ILP and multi-coresSymmetric Multi Processors

Why several processors/cores ?

Limits for monocore processors

superscalar processors: instruction level parallelism

frequency

electrical power

What to do with place available on chips ?

caches (bigger and quicker)

several series of registers (hyperthreaded processors)

several series of cores (multi-core processors)

all of that


11/76

P ll l M hi i h Sh d M


12/76




Towards more and morehierarchical computers

P P M P P M

P P M P P M

P P M P P M

M

M

M

SMT(HyperThreading)

Multi-core

NUMA

P ll l M hi ith Sh d M


13/76




AMD Quad-Core

M

P PL2 $

P PL2 $ L2 $ L2 $

L3 $

P P L2

P P L2 L2 L2

L3

...

Shared L3 cache

NUMA factor ~1.1-1.5

M

P PL2 $

P PL2 $ L2 $ L2 $

L3 $M

P PL2 $

P PL2 $ L2 $ L2 $

L3 $M

Parallel Machines with Shared Memory


14/76




Intel Quad-Core

P PL2 $

P PL2 $

MP P

L2 $

P PL2 $

...

Hierarchical cache levelsP P

L2 $

P PL2 $



15/76




5

dual-quad-core

M

P1 P5 P3 P7

P0 P2P4 P6

Intel

Hierarchical cache levels



16/76



ClustersGrids

Clusters

Composed of a few to hundreds of machines

often homogeneoussame processor, memory, etc.

often linked with a high speed, low latency networkMyrinet, InfinityBand, Quadrix, etc.

Biggest clusters can be split in several parts

computing nodesI/O nodes

front (interactive) node


17/76



18/76

a a e ac es t S a ed e o yParallel Machines with Distributed Memory



Hierarchical ArchitecturesHT technology

multi-core processor

multi processors machine

cluster of machines

grid of clusters and individual machines

Even more complexity

computing on GPUrequire specialized codes but hardware far more powerful

FPGAhardware can be specialized on demand

still lots of work on interface programming here

Introduction to ThreadsKinds of threads


19/76

Kinds of threadsUser Threads and Blocking System Calls

Thread Programming Interface

Part II

Parallelism and Threads


20/76



21/76



Programming on Shared Memory Parallel Machines

Using process

Processors

Operating system Resources management(files, memory, CPU, network, etc)

Process

Mem Mem Mem Mem



22/76



Programming on Shared Memory Parallel Machines

Using threads

Processors

Operating system Resources management(files, memory, CPU, network, etc)

Process

Multithreaded process

Memory



23/76



Introduction to Threads

Why threads ?To take profit from shared memory parallel architectures

SMP, hyperthreaded, multi-core, NUMA, etc. processorsfuture Intel processors: several hundreds cores

To describe the parallelism within the applicationsindependent tasks, I/O overlap, etc.

What will use threads ?User application codes

directly (with thread libraries)POSIX API (IEEE POSIX 1003.1C norm) in C, C++, . . .

with high-level programming languages (Ada, OpenMP, . . . )

Middleware programming environments

demonized tasks (garbage collector, . . . ), . . .


User threadsK l th d


24/76

User Threads and Blocking System CallsThread Programming Interface


User threads

by the kernelEntities managed

Operating system

Processors


threads

Kernel scheduler

User level

User scheduler

Efficiency + Flexibility + SMP - Blocking syscalls -


User threadsKernel threads


25/76



Kernel threads


Operating system

Processors


threads

Kernel scheduler

Kernel level

Efficiency - Flexibility - SMP + Blocking syscalls +


User threadsKernel threads


26/76



Mixed models


Operating system

Processors


threads

Kernel scheduler

User level

User scheduler

Efficiency + Flexibility + SMP + Blocking syscalls limited


27/76


S CScheduler Activations


28/76


Scheduler Activations

User Threads and Blocking System Calls

User level library

Kernel scheduler

User scheduler

Mixed library

Kernel scheduler

User scheduler


U Th d d Bl ki S t C llScheduler Activations


29/76


Sc edu e ct at o s


Idea proposed by Anderson et al. (91)Dialogue (and not monologue) between the user and kernelschedulers

the user scheduler uses system calls

the kernel scheduler uses upcalls

Upcalls

Notify the application of scheduling kernel events

Activationsa new structure to support upcallsa kinf of kernel thread or virtual processor

creating and destruction managed by the kernel


User Threads and Blocking System CallsScheduler Activations


30/76



Instead of:

InterruptionExternal request

spaceUser

Kernel Time




31/76



Instead of:

InterruptionExternal request

spaceUser

Kernel Time

...better use the following schema:

Interruption

Act. A

External request

spaceUser

Kernel Time

upcall(unblocked, preempted, new)

Act. CAct. A (next)

Act. B

upcall(blocked, new)




32/76


Working principle

managed by the kernel

User-level

Operating system

Processors


Activations

Kernel scheduler

threads

User scheduler




33/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

threads

Upcall(New)




34/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

threads


35/76




36/76

g yThread Programming Interface

Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

systemcall

Blocking

threads


37/76

Introduction to ThreadsKinds of threadsUser Threads and Blocking System Calls



38/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

systemcall

Blocking

threads




39/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

systemcall

Blocking

threads


Th d P i I t f



40/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

threads





41/76


Working principle


User-level

Operating system

Processors


Activations

Kernel scheduler

threads

Upcall(Unblocked,Preempted,New)



POSIX ThreadsLinux POSIX Threads LibrariesBasic POSIX Thread API


42/76


Normalisation of the thread interface

Before the normeach Unix had its (slightly) incompatible interface

but same kinds of features was present

POSIX normalisationIEEE POSIX 1003.1C norm (also called POSIX threadsnorm)Only the API is normalised (not the ABI)

POSIX thread libraries can easily be switched at source

level but not at runtimePOSIX threads own

processor registers, stack, etc.signal mask

POSIX threads can be of any kind (user, kernel, etc.)





43/76


Linux POSIX Threads Libraries

LinuxThread (1996) : kernel level, Linux standard threadlibrary for a long time, not fully POSIX compliant

GNU-Pth (1999) : user level, portable, POSIXNGPT (2002) : mixed, based on GNU-Pth, POSIX, notdeveloped anymore

NPTL (2002) : kernel level, POSIX, current Linuxstandard thread library

PM2/Marcel (2001) : mixed, POSIX compliant, lots ofextensions for HPC (scheduling control, etc.)





44/76


Basic POSIX Thread API

Creation/destructionint pthread_create(pthread_t *thread, const

pthread_attr_t *attr, void

*(*start_routine)(void*), void *arg)

void pthread_exit(void*

value_ptr)

int pthread_join(pthread_t thread, void

**value_ptr)

Synchronisation (semaphores)

int sem_init(sem_t *sem, int pshared, unsignedint value)

int sem_wait(sem_t *sem)

int sem_post(sem_t *sem)

int sem_destroy(sem_t*

sem)





45/76


Basic POSIX Thread API (2)

Synchronisation (mutex)int pthread_mutex_init(pthread_mutex_t *mutex,

const pthread_mutexattr_t *attr)

int pthread_mutex_lock(pthread_mutex_t *mutex)

int pthread_mutex_unlock(pthread_mutex_t

*mutex)

int pthread_mutex_destroy(pthread_mutex_t

*mutex)

Synchronisation (conditions)int pthread_cond_init(pthread_cond_t *cond,

const pthread_condattr_t *attr)

int pthread_cond_wait(pthread_cond_t *cond,

pthread_mutex_t *mutex)

int pthread_cond_signal(pthread_cond_t *cond)





46/76

g g


Per thread dataint pthread_key_create(pthread_key_t *key, void

(*destr_function) (void*))

int pthread_key_delete(pthread_key_t key)

int pthread_setspecific(pthread_key_t key,

const void *pointer)

void * pthread_getspecific(pthread_key_t key)





47/76

g g


Per thread dataint pthread_key_create(pthread_key_t *key, void

(*destr_function) (void*))

int pthread_key_delete(pthread_key_t key)

int pthread_setspecific(pthread_key_t key,

const void *pointer)

void * pthread_getspecific(pthread_key_t key)

The new __thread C keyword

used for a global per-thread variableneed support from the compiler and the linker at compiletime and execute time

libraries can have efficient per-thread variables without

disturbing the application Hardware SupportBusy-waiting Synchronisation

High-level Synchronisation PrimitivesSome examples with Linux


48/76

Part III

Synchronisation

Hardware SupportBusy-waiting SynchronisationHigh-level Synchronisation Primitives

Some examples with Linux


49/76

Outlines: Synchronisation

8 Hardware Support

9 Busy-waiting Synchronisation

10 High-level Synchronisation PrimitivesSemaphoresMonitors

11 Some examples with LinuxOld Linux libpthreadNew POSIX Thread Library




50/76

Hardware Support

What happens with incrementations in parallel?

var++; var++;




51/76

Hardware Support

What happens with incrementations in parallel?for (i=0; i


52/76




53/76

Critical section with busy waiting

Example of code

while (! TAS(&var))

;

/* in critical section */

var=0;

Hardware SupportBusy-waiting Synchronisation



54/76


Example of code

while (! TAS(&var))

while (var) ;


var=0;




55/76


Example of code

while (! TAS(&var))

while (var) ;


var=0;

Busy waiting

+ very reactive

+ no OS or lib support required

- use a processor while not doing anything

- does not scale if there are lots of waiters


56/76



SemaphoresMonitors

M i


57/76

Monitors

MutexTwo states: locked or not

Two methods:lock(m) take the mutex

unlock(m) release the mutex (must be done by thethread owning the mutex)

Conditions

waiting thread list (conditions are not related with tests)

Three methods:wait(c, m) sleep on the condition. The mutex is released

atomically during the wait.signal(c) one sleeping thread is wake up

broadcast(c) all sleeping threads are wake up Hardware Support

Busy-waiting SynchronisationHigh-level Synchronisation Primitives


Old Linux libpthreadNew POSIX Thread Library

Old Li lib th d


58/76

Old Linux libpthread

First Linux kernel thread library

limited kernel support available

provides POSIX primitives (mutexes, conditions,

semaphores, etc.)

All internal synchronisation built on signals

lots of play with signal masks

one special (manager) thread used internally to managethread state and synchronisation

race conditions not always handled (not enough kernelsupport)



Old Linux libpthreadNew POSIX Thread Library

NPTL N POSIX Th d Lib


59/76

NPTL: New POSIX Thread Library

New Linux kernel thread library

requires new kernel support (available from Linux 2.6)

specific support in the libc

a lot more efficientfully POSIX compliant

Internal synchronisation based on futex

new kernel objectmutex/condition/semaphore can be fully handled in userspace unless there is contention

A Brief Overview of MPIMixing Threads and Communication in HPC


60/76

Part IV

Multithreading and Networking


Outlines: Multithreading and Networking


61/76

Outlines: Multithreading and Networking

12 A Brief Overview of MPI

13 Mixing Threads and Communication in HPCProblems arisesDiscussion about solution



62/76

Standard in industry

frequently used by engineers, physicians, etc.MPI2 begin to be available

See MPI presentation


Problems arisesDiscussion about solution

Example of problems with MPI


63/76


Token circulation while com-puting on 4 nodes

if (mynode!=0)

MPI_Recv();

req=MPI_Isend(next);

Work(); /* about 1s */

MPI_Wait(req);

if (mynode==0)

MPI_Recv();


64/76





65/76


Recv0 1 2

Isend

Calculus

req

Recv

Calculus

Wait

Isend

data

Recv

Calculus

Wait

Isend

data

req

ack

ack

req

Ignored messageNo reactivity

Token circulation while com-puting on 4 nodes

if (mynode!=0)

MPI_Recv();

req=MPI_Isend(next);

Work(); /* about 1s */

MPI_Wait(req);

if (mynode==0)

MPI_Recv();

expected time: ~ 1 s

observed time: ~ 4 s



Improving MPI reactivity


66/76

Improving MPI reactivity

Possible solutionsadd calls to MPI_test() in the code

using a multithreaded MPI version+ parallelism, communication progression independent from

computations- busy waiting synchronisation less efficient

- scrutation must be managed



Integrate a scrutation server into the scheduler


67/76

Integrate a scrutation server into the scheduler

Scheduler: required for optimal behavioursystem is known by the scheduler

it can choose the best strategy to use

Efficient and reactive scrutationless context switchesguarantee frequency

independent with respect to the number of threads in theapplication

instead of


68/76



Running the scrutation server


69/76


}

{

poll}

{

group

Callback functions:

Frequency- poll- group

init_server()

A

MPI call

threads schedulerUser level

Application

library

MPI





70/76


}

{

poll}

{

group

Req2

ev_wait()

ev_wait()

BC

Req1

A

MPI call


Application

library

MPI





71/76


}

{

poll}

{

group

Req2

BC

Req1

21

requestsAggregated

A

MPI call


Application

library

MPI





72/76

u g t e sc utat o se e

}

{

poll}

{

group

Req2

BC

Req1

21

requestsAggregated

A

MPI call


Application

library

MPI





73/76

g

}

{

poll}

{

group

BC

Req1

1

requestsAggregated

A

MPI call


Application

library

MPI


74/76

Part V

Conclusion

Outlines: Conclusion


75/76

Conclusion


76/76

Multi-threading

cannot be avoided in current HPC

directly or through languages/middlewaresdifficulties to get a efficient scheduling

no perfect universal schedulerthreads must be scheduled with respect to memory (NUMA)

threads and communications must be scheduled together

02 progpar thread

Documents