parallel and concurrent programming programming using ... · parallel and concurrent programming...

Parallel andConcurrent

ProgrammingProgrammingUsing POSIX

Threads

Marwan Burelle

Introduction

Using pthreads

LockingBasic locking

Advanced locking

SynchronisationConditions Variables

Semaphores

Putting it together !

Parallel and Concurrent ProgrammingProgramming Using POSIX Threads

Marwan Burelle

[email protected]://wiki-prog.kh405.net

http://wiki-prog.kh405.net



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Outline

1 Introduction

2 Using pthreads

3 LockingBasic lockingAdvanced locking

4 SynchronisationConditions VariablesSemaphores

5 Putting it together !



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Introduction

Introduction



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Using POSIX Threads API

This lecture is intended to be done in front of a keyboard.This is a main introduction to programming with pthreads.Overview:

• Using pthreads: starting, waiting, cancelling ...• Basic locking: mutex and spin lock• Advanced locking: read/write lock and barrier• More synchronisation: conditions and POSIX

semaphore• Let’s code . . .



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Using pthreads

Using pthreads



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Starting a new thread

pthread_create(3)

intpthread_create(pthread_t *thread,

const pthread_attr_t *attr,void *(*start_routine)(void *),void *arg);

• thread: a pointer to a thread id container.• attr: a pointer to thread attributs.• start_routine: the function executed by the thread.• arg: argument for start_routine.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Starting a new thread (example)

Example:

#include <stdlib.h>#include <unistd.h>#include <stdio.h>#include <pthread.h>

void *start_routine(void *arg){int *res, i;res = malloc(sizeof (int));i = *((int*)arg) ;for(; i; --i)*res += i;

pthread_exit(res);}

int main(){int i=42, *res;pthread_t id;pthread_create(&id,NULL,start_routine ,&i);pthread_join(id, &res);printf("res = %d\n",*res);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Waiting for a thread

pthread_join(3)

int pthread_join(pthread_t thread,void **value_ptr);

• thread: the id of the thread we wait for.• value_ptr: a pointer to the container of the result of

the thread. *value_ptrmust a be a valid pointer. Ifvalue_ptr is NULL the result is just discarded.

Don’t forget that the start_routine of a thread returns avoid pointer, so if we want to grab this result, we need topass a pointer on a void pointer.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Join example

Example:

for a complete example, see the example for starting a new thread.

• Wait with result:

pthread_t id;void *res;/* starting thread *//* parall work here *//* waiting for our thread */pthread_join(id, &res);/* now res contains our expected result */

• Just wait:

pthread_t id;/* ... */pthread_join(id, NULL);



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Cancelling a thread

pthread_cancel(3)

int pthread_cancel(pthread_t thread);

• thread: the id of the thread we want to cancel. Thethread must exists (sic!) and be cancelable (seepthread_setcancelstate(3).)

The cancellation of a thread is asynchronous: the functionreturns without waiting for the cancellation to proceed.Other threads joining on the cancelled thread will beinformed with a special value PTHREAD_CANCELED of typevoid* (different from any valid pointer, even NULL.)



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Cancelling a thread (example)

Example:

void *dummy_thread(void *arg){char buf[32];pthread_setcancelstate(PTHREAD_CANCEL_ENABLE ,NULL);arg = arg;read(STDOUT_FILENO , buf, 31); // Blocking I/Opthread_exit(NULL);

}void *waiting_thread(void *arg){pthread_t id;void *res;id = *((pthread_t*)arg);pthread_join(id, &res); // Joining the other threadif (res == PTHREAD_CANCELED)printf("thread %d has been cancelled.\n", (int)id);

elseprintf("thread %d returns res = %p.\n", (int)id, res);

pthread_exit(NULL);}int main(){pthread_t idd, idw;pthread_create(&idd,NULL,dummy_thread ,NULL);pthread_create(&idw,NULL,waiting_thread ,&idd);pthread_cancel(idd); // Cancelling first threadpthread_join(idw,NULL);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Compilation

• You need to link against a suitable pthreads’s lib.• On most systems this is done by adding -pthread

flags to the compiler.• Some systems offer various threading lib. For

example, on FreeBSD you can use libkse ou libthrby adding -lkse or -lthr.

• Be aware that pthreads are POSIX and not ANSI, andthus you must avoid compilation flags that preventPOSIX parts of libc to be defined (mostly on systemusing glibc.) So, do not use -ansi.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Examples

Example:


long long acker(long long m, long long n){return (m?(n?(acker(m-1,acker(m,n-1))):acker(m-1,1)):n+1);

}

void *worker(void __attribute__((unused)) *_){pthread_t id = pthread_self();printf("%#x: %lli\n",(size_t)id,acker(3,11));pthread_exit(NULL);

}

int main(){pthread_t id1, id2;pthread_create(&id1,NULL,worker,NULL);pthread_create(&id2,NULL,worker,NULL);pthread_join(id1,NULL);pthread_join(id2,NULL);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Locking

Locking



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Introduction to locks

• Locks are facilities to enforce critical section.• Basically, locks have two operations: lock and unlock:

• lock: if the lock is free, lock-it, otherwise wait until theactual owner free-it.

• unlock: if the current thread is the owner, it free thelock (undefined behavior otherwise.)

• Locks often come with a try_lock: do not wait if thelock is not free.

• pthreads offers two kind of locks: mutex and spin lock.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Basic locking

Basic locking



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


mutex

• A mutex is a standard lock with passive waiting• mutex are recursive: you can relock a lock you already

have (whithout blocking.)• Normally when a mutex wait for a lock it pushes the

current thread on sleep using the system.• You can set various options on mutex using thepthread_mutexattr* familly of functions (availableoptions vary on each system.)



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


mutex (example)// example04.c : basic counter with mutex#include <stdlib.h>#include <unistd.h>#include <stdio.h>#include <pthread.h>struct s_tharg {pthread_mutex_t m; size_t count;};

int event(){char buf[1];return (read(STDIN_FILENO , buf, 1));

}

void *guardian(void *arg){struct s_tharg *info = arg;while (event()){pthread_mutex_lock(&(info->m));++info->count;pthread_mutex_unlock(&(info->m));

}pthread_exit(NULL);

}

int main(){pthread_t th[2];struct s_tharg *info = malloc(sizeof (struct s_tharg));info->count = 0;pthread_mutex_init(&(info->m),NULL);pthread_create(th,NULL,guardian ,info);pthread_create(th+1,NULL,guardian ,info);pthread_join(th[0],NULL);pthread_join(th[1],NULL);printf("count = %d\n",info->count);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


mutex (example)

Compile the previous example with:> gcc -W -Wall -Werror -std=c99 -pedantic -pthread example04.c -o example04

You should give it a bounded input to avoid interractionwith end-of-line:> echo -n "a string with a known length" | ./example04count = 28



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


spin lock

• A spin lock is a standard lock with active waiting.• spin lock are intended to be basic: not recursive and

minimal safety.• A spin lock won’t push the process to sleep when

blocking but will spin (i.e. loop) until the lock is freed.• Thanks to spin waiting, spin lock does not implies

context switching.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


spin lock (example)

Example:

struct s_spinarg{pthread_spinlock_t lock;size_t count;

};

void *spin_guardian(void *arg){struct s_spinarg *info = arg;while(event()) {pthread_spinlock_lock(info->lock);++info->count;pthread_spinlock_unlock(info->lock);


}

As you can see, spin locks and mutexes work quiet the same...



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


What kind of lock to use

• mutexes are a all-purpose lock.• spin locks are quick and dirty locks.• mutexes are generaly considered to be more safe than

spin locks.• spin locks have less guarantees but eat less system

ressources (no context switching), despite the fact thatthey use ressources for waiting.

• Nor mutexes, nor spin locks are fair.• You should prefer spin locks for small critical section

(few instructions, no blocking operations ... )• Use mutexes when not sure.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Advanced locking

Advanced locking



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Complex locking: readers and writers

• The classic use case of readers and writers is quitedifficult to solve juste using basic locks.

• The problem is simple: We have a shared buffer ofdata between various threads. Each thread have apredefined role:• a reader only read the shared buffer. Several readers

can access the buffer at the same time.• a writer write to the shared buffer (and may read it

also.) Only one writer can access the buffer at thesame time.

• POSIX threads offer a builtin solution:pthread_rwlock*.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


pthread_rwlock* (example 1/4)

Example:

#include <stdlib.h>#include <unistd.h>#include <string.h>#include <stdio.h>#include <pthread.h>

#define MAX 256

struct s_rwbuf{char *buf;size_t size, cont;pthread_rwlock_t lock;pthread_mutex_t m;

};

int cont(struct s_rwbuf *info){int cont=0;pthread_mutex_lock(&(info->m));cont = info->cont;pthread_mutex_unlock(&(info->m));return cont;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores



Example:

void *reader(void *arg){struct s_rwbuf *info = arg;size_t pos, len;char buf[MAX];while(cont(info)){pos = random() % (info->size / 4);len = random() % ((info->size - pos)/2);pthread_rwlock_rdlock(&(info->lock));memcpy(buf,info->buf + pos*2,len*2);pthread_rwlock_unlock(&(info->lock));write(STDOUT_FILENO ,buf,len*2);


}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores



Example:

void *writer(void *arg){struct s_rwbuf *info = arg;size_t pos, len;char buf[MAX], c=’a’;while(cont(info)){pos = random() % (info->size / 2);len = random() % ((info->size - pos))/2;for (size_t i=0; i<len; i++){buf[i*2] = c;buf[i*2 + 1] = (c+3)<’z’ ? c+3 : ’a’+3;

}c = c<’z’ ? c+1 : ’a’;pthread_rwlock_wrlock(&(info->lock));memcpy(info->buf + pos,buf,len*2);pthread_rwlock_unlock(&(info->lock));


}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores



Example:

int main(){struct s_rwbuf *info = malloc(sizeof (struct s_rwbuf));pthread_t th[2];char b[1];info->buf = calloc(MAX,1);info->size = MAX;pthread_rwlock_init(&(info->lock),NULL);pthread_mutex_init(&(info->m),NULL);info->cont = 1;srandom(42);pthread_create(th,NULL,writer,info);pthread_create(th+1,NULL,reader,info);read(STDIN_FILENO , b, 1);pthread_mutex_lock(&(info->m));info->cont = 0;pthread_mutex_unlock(&(info->m));pthread_join(th[0],NULL);pthread_join(th[1],NULL);write(STDOUT_FILENO ,"\n",1);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Waiting every one: barrier

• An other classical issue is the wave synchronisation:every threads must terminate the stage n beforanyone can go to stage n + 1.

• A simple solution is to join every threads at the end ofstage and create new ones at the begining.Continuation information (i.e. thread’s local data) canbe transmitted using thread’s parameter and returnedvalue.

• POSIX threads provide pthread_barrier* as abuiltin solution for that issue:• The barrier is initialized with an integer constant n;• At the end of each stage, every thread try the barrier;• The barrier is locked until n threads are waiting on it.• The n−th thread is gave a special value, indicating it is

the last arrived.

• Using barrier saves system ressources consumed bythe join/create strategy.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Parallel Sums (1/2)

Example:


#define SIZE 64

struct s_info{int *tab;size_t size, pos;pthread_barrier_t *bar;

};

void *worker(void *arg){struct s_info *info = arg;size_t pos = info->pos;for (size_t s = info->size / 2; s; s>>=1){if (pos<s)info->tab[pos] += info->tab[pos+s];

pthread_barrier_wait(info->bar);}

pthread_exit(NULL);}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Parallel Sums (2/2)

Example:

int main(){int tab[SIZE];pthread_t th[SIZE/2];pthread_barrier_t bar;for (size_t i=0; i<SIZE; i++)tab[i] = i;

pthread_barrier_init(&bar,NULL,SIZE/2);for (size_t i=0; i<SIZE/2; i++){struct s_info *info = malloc(sizeof (struct s_info));info->tab = tab;info->bar = &bar;info->size = SIZE;info->pos = i;pthread_create(th+i, NULL, worker, info);

}for (size_t i=0; i<SIZE/2; i++)pthread_join(th[i],NULL);

printf("sum = %d\n",tab[0]);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Synchronisation

Synchronisation



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Cases where locking is not sufficient

• Locks and barrier are not always sufficient, manysynchronisation issues need more grease to work.

• Usual synchronisation issues:• Bounded number of threads in the same critical

sections;• Fair access to ressources;• Waiting on condition changes.

• The usual solutions for those kind of problems are:condition variables, semaphores or monitor.

• POSIX provides the first two, but anyway, the later ismore or less a design pattern rather than a builtinsolution.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Conditions Variables

Conditions Variables



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Waiting for a change

• Synchronisation often involve waiting for a change ona shared ressource.

• Polling for the change means locking the ressource,testing it and if necessary unlock and loop.

• Even with mutex (or any kind of non spinning lock)this implies spin waiting with many lock/unlockoperations.

• Condition variables are nice solution for this issue:• we create a condition variable c for the tested

ressources;• threads polling the change will wait on c;• threads commiting the change will wake (all or one)

waiting threads after the change occurs.

• Normally, condition variables does not garanteefairness.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Conditions in POSIX Threads

• POSIX threads provide pthread_cond* for conditionhandling.

• Each condition variable must be used in conjunctionwith a mutex.

• The usual schema for waiter is:

// first lockpthread_lock(m);// loop until condition is truewhile (!condition)// wait for a changepthread_cond_wait(c,m);

• Waiting on a condition will first unlock the mutex andthen push the thread to sleep mode. When awake, thethread will acquire the mutex and then comebackfrom the wait functions.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Producer/Consumer

Example:


typedef struct s_queue *t_queue;struct s_queue{void *value;t_queue next;

};

#define QSIZE (sizeof (struct s_queue))

t_queue push(void *x, t_queue q){// classic circular linked list based queue ...

}

void *take(t_queue *q){// classic circular linked list based queue ...

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Producer/Consumer

Example:

struct s_info{t_queue q;pthread_mutex_t *m;pthread_cond_t *c;

};

void *produce(){char *s;s = malloc(8);for (size_t i=0; i<7; ++i)s[i] = ’a’+i;

s[7] = 0;return s;

}

void consume(char *s){printf("consumed: %s\n",s);free(s);

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Producer/Consumer

Example:

void *producer(void *arg){struct s_info *info = arg;for (;;) {void *data;data = produce();pthread_mutex_lock(info->m);info->q = push(data,info->q);pthread_mutex_unlock(info->m);pthread_cond_signal(info->c);


}void *consumer(void *arg){struct s_info *info = arg;for (;;) {void *data;pthread_mutex_lock(info->m);while (!(info->q))pthread_cond_wait(info->c,info->m);

data = take(&(info->q));pthread_mutex_unlock(info->m);consume(data);


}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Producer/Consumer

Example:

int main(){pthread_t th[2];pthread_cond_t cond;pthread_mutex_t mutex;struct s_info *info = malloc(sizeof (struct s_info));pthread_mutex_init(&mutex, NULL);pthread_cond_init(&cond, NULL);info->c = &cond;info->m = &mutex;info->q = NULL;pthread_create(th, NULL, consumer , info);pthread_create(th+1, NULL, producer , info);pthread_join(th[0], NULL);pthread_join(th[1], NULL);return 0;

}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Semaphores

Semaphores



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


What the hell is that ?

• A semaphore is a shared counter with a specificsemantics for the decrease/increase operations.

• Normally, a semaphore maintain a FIFO waitingqueue.

• The two classic operations are:• P: if the counter is strictly positive, decrease it (by

one), otherwise the calling thread is push to sleep,waiting for the counter be positive again.

• V: increase the counter, waking the first waitingthread when needed.

• Since semaphores use a queue, synchronisation usingsemaphores can consider fair: each thread will wait afinite time for the protected ressource. The property iseven more precise, since a waiting thread will see (atleast) every other threads accessing the ressourceexactly one time before it.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Semaphore’s classics

• The counter value of the semaphore can be initializewith any positive integer (zero inclusive.)

• A semaphore with an initial value of 1 can act as a fairmutex.

• Semaphore can be used as a condition counter,simplifying classic problems such asProducer/Consumer.

• Operations’ name P and V comes from Dijkstra’s firstSemaphores’ presentation and probably meansomething in dutch. But, implementations often usemore explicit names like wait for P and post for V.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Producer/Consumer with semaphores

Example:mutex = new semaphore(1)size = new semaphore(0)q = new queueproducer():for (;;)data = produce()P(mutex)q <- push(data,q)V(mutex)V(size)

consumer():for (;;)P(size)P(mutex)data <- take(q)V(mutex)consume(data)



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


The POSIX Semaphores API

• POSIX provides a threadsafe semaphores API.• Available oprations are:

#include <semaphore.h>

int sem_init(sem_t *sem, int pshared, unsigned value);int sem_post(sem_t *sem);int sem_wait(sem_t *sem);int sem_trywait(sem_t *sem);

• POSIX semaphore can be shared across processes(using sem_open(3).) But most implementationsdoesn’t provide this facilities (thus pshared argumentshould always be 0.)

• POSIX semaphore are fair.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Using POSIX semaphores

Example:

// Just modify the Producer/Consumer example using mutex and condition// initilization:// sem_init(info->m,0,1) and sem_init(info->size,0,0)void *producer(void *arg){struct s_info *info = arg;for (;;) {void *data;data = produce();sem_wait(info->m);info->q = push(data,info->q);sem_post(info->m);sem_post(info->size);


}void *consumer(void *arg){struct s_info *info = arg;for (;;) {void *data;sem_wait(info->size);sem_wait(info->m);data = take(&(info->q));sem_post(info->m);consume(data);


}



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores






Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Let’s Code !

• Based on previous description, we now want toimplement a little example using lot of POSIX threadsconstructions.

• Our program will be a small toy server. Let’s see theprovided service:• Our server store a bunch of data (a string) and clients

can retrieve or update it.• The protocol recognize only to query:

• GET\r\n\r\n : retrieve stored data, may block if anupdate is performed by another clients.

• SET\r\n<data>\r\n\r\n: update the stored datausing characters in <data>.

• We will use TCP as transport protocol, but we willfollow HTTP fashion: one query per connexion.

• The stored data is shared among all clients. For sakeof simplicity we will limite the data size to 64 bytes.



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Simple String Server

Our server must enforce the following constraints:• The string data is shared upon every connections.• The string data retrieve by client must be coherent:

each update is atomic.• The server must support multiple connections at the

same time.• Connections must be ended by the client (in order to

prevent the so-called TIME_WAIT issue.)• To avoid blocking clients, we should also add a

timeout on the server side.• Server acknowledge successful update with OK\r\n• When the server send the data, it marks the end with

the sequence \r\n



Threads

Marwan Burelle

Introduction

Using pthreads


Advanced locking


Semaphores


Simple String Server

The server will follow this given schema:• The server uses a threads pool to handle connections.• Connections are dispatch upon threads in the pool

using a simple tasks queue.• The data is protected using read/write lock technics.• We must enforce a synchronisation point upon

initialization of threads in the pool (threads begin towait on the queue all together.)

• Upon update, the server will only use the first 64 bytesof the provided string (the rest is just silently ignored.)

• Some code is provided on the page:http://wiki-prog.kh405.net/index.php/Programmation:Parallel

http://wiki-prog.kh405.net/index.php/Programmation:Parallel

http://wiki-prog.kh405.net/index.php/Programmation:Parallel

parallel and concurrent programming programming using ... · parallel and concurrent programming...

Documents