elec-c7310 sovellusohjelmointi lecture 11: advanced interfaces

ELEC-C7310 Sovellusohjelmointi

Lecture 11: Advanced interfaces

Risto Järvinen

November 29, 2021

Lecture contents

OpenMP, more threads

Additional system specific APIs.

Scaling up I/O multiplexing

Filesystem monitoring

Zero-copy tricks

Other stuff

ELEC-C7310 Sovellusohjelmointi Lecture 11: Advanced interfaces 2/31Risto Järvinen 29.11.2021

OpenMP

Instead of explicit function calls, let compiler do the work.

Barrier-style operation on multicores.

Fallback to single-core operation.

De-facto standard by OpenMP ARB, ref www.openmp.org.

For C/C++ and Fortran.

Supported by GCC 4.2+ (v2.5), compile with "-fopenmp", header omp.h


OpenMP API: Pragmas

Pragmas are instructions to compiler, not actually parts of the language.

Pragmas are instructions to compiler, not actually parts of the language.

#pragma omp parallel Creates a block that is run parallel

#pragma omp for Creates a loop that is run in parallel

#pragma omp task [clause [[, clause] ...] structured-block] Creates a child task that can

run in a separate thread

#pragma omp taskwait Current task suspends execution until all children tasks have

completed execution.

... Many more...


OpenMP API: Clauses

private(list) defines which variables are local; values are undefined on entry and exit.

shared(list) defines which variables are shared between threads.

firstprivate(list) defines which variables are local; value initialized to value at the

beginning of construct.

lastprivate(list) defines which variables are local, but will be updated at the end of the

construct.

... Many more.. depending on pragma.


OpenMP API: Functions

void omp_set_num_threads ( i n t num_threads ) ;

i n t omp_get_num_threads ( void ) ;

i n t omp_get_max_threads ( void ) ;

i n t omp_get_thread_num( void ) ;

i n t omp_get_num_procs ( void ) ;

i n t omp_in_para l le l ( void ) ;

void omp_set_dynamic ( i n t dynamic_threads ) ;

i n t omp_get_dynamic ( void ) ;

void omp_set_nested ( i n t nested ) ;

i n t omp_get_nested ( void ) ;


OpenMP Lock API

void omp_in i t_ lock ( omp_lock_t * lock ) ;

void omp_in i t_nest_ lock ( omp_nest_lock_t * lock ) ;

void omp_destroy_lock ( omp_lock_t * lock ) ;

void omp_destroy_nest_lock ( omp_nest_lock_t * lock ) ;

void omp_set_lock ( omp_lock_t * lock ) ;

void omp_set_nest_lock ( omp_nest_lock_t * lock ) ;

void omp_unset_lock ( omp_lock_t * lock ) ;

void omp_unset_nest_lock ( omp_nest_lock_t * lock ) ;

i n t omp_test_lock ( omp_lock_t * lock ) ;

i n t omp_test_nest_lock ( omp_nest_lock_t * lock ) ;


OpenMP Lock Semantics

omp_init*_lock routine initializes a lock.

omp_destroy*_lock routine uninitializes a lock.

omp_set*_lock routine waits until the lock is available, and then sets it.

omp_unset*_lock routine unsets the lock.

omp_test*_lock routine tests a simple lock, and sets it if it is available.

Nestable locks can be set multiple times by the same thread before being unset.


OpenMP Time API

double omp_get_wtime(void); Returns elapsed wall clock time in seconds.

double omp_get_wtick(void); Returns the precision of wtime in seconds.


OpenMP Data Sharing

Shared memory, most variables are shared by default.

Global variables are shared, including file handles and static variables.

Stack variables in functions called from parallel regions are not shared.

Automatic variables within statement blocks are not shared.

Changing sharing using clauses: shared, private, firstprivate, lastprivate.


OpenMP Synchronization

critical, section is run one thread at a time.

atomic, storage location is updated atomically.

barrier, only one thread runs after barrier.

ordered, section is executed in loop iteration order.

flush, makes threads temporary view of memory consistent.

locks, explicit locking


OpenMP vs POSIX Threads

OpenMP is simple for loop-format programs (data processing)

Pragmas have fallbacks

OpenMP is tied to loop model, not all loops parallelize.

OpenMP needs compiler support

Be careful with race conditions with sharing data!


Scaling I/O Multiplexing

Select()/poll() have inherent performance limits when using large number of file

descriptors.

Reminder:

i n t se lec t ( i n t nfds , fd_se t * readfds ,

fd_se t * w r i t e f ds , fd_se t * except fds ,

struct t imeva l * t imeout ) ;

i n t p o l l ( struct p o l l f d * fds , n fds_ t nfds ,

i n t t imeout ) ;


Linux-specific: Epoll

Variant of poll() interface.

Create a new epoll file descriptor with epoll_create().

Add/remove/modify monitored file descriptors with epoll_ctl().

Wait for events using epoll_wait().


Epoll API

#include <sys / e p o l l . h>

i n t epo l l _c r ea te ( i n t s ize ) ;

i n t epo l l_c rea te1 ( i n t f l a g s ) ;

i n t e p o l l _ c t l ( i n t epfd , i n t op , i n t fd , struct epo l l_even t * event ) ;

i n t epo l l _wa i t ( i n t epfd , struct epo l l_even t * events ,

i n t maxevents , i n t t imeout ) ;

i n t epo l l _pwa i t ( i n t epfd , struct epo l l_even t * events , i n t maxevents ,

i n t t imeout , const s i g s e t _ t * sigmask ) ;


FreeBSD-specific: Kqueue

Variant of poll() interface.

Create new kqueue file descriptor with kqueue().

Add and receive event sources with kevent().

In addition to monitoring file descriptors, can monitor file modification, signals,

asynchronous I/O events, child process reaping and timers.


Kqueue API

i n t kqueue ( void ) ;

i n t kevent ( i n t kq , const struct kevent * change l i s t , i n t nchanges ,

struct kevent * e v e n t l i s t , i n t nevents , const struct t imespec * t imeout ) :

struct kevent {

u i n t p t _ t i d e n t ; / / i d e n t i f i e r f o r event

short f i l t e r ; / / f i l t e r f o r event

u_shor t f l a g s ; / / a c t i o n f l a g s f o r kq

u _ i n t f f l a g s ; / / f i l t e r f l a g value

i n t p t r _ t data ; / / f i l t e r data value

void * udata ; / / opaque i d e n t i f i e r

}


Solaris-specific: /dev/poll

Open /dev/poll file to create a file descriptor.

Add event sources with write(), receive with ioctl().

Like kqueue.

Drawbacks: can’t handle signals, doesn’t handle descriptor removal nicely.


Linux-specific: Signalfd

Create a file descriptor of receiving signals.

The file descriptor can then be multiplexed with select/poll/etc.

Pending signals are consumed using read() on the FD.

Read buffer must be multiple of sizeof(struct signalfd_signalinfo)).


Signalfd API

#include <sys / s i g n a l f d . h>

i n t s i g n a l f d ( i n t fd , const s i g s e t _ t *mask , i n t f l a g s ) ;


POSIX-defined: Interval Timers

Creates a timer that will deliver a timeout using a sigev (signal or thread).


Interval Timer API

#include < s i g n a l . h>

#include <t ime . h>

i n t t imer_crea te ( c l o c k i d _ t c lock id , struct s igevent * evp ,

t i m e r_ t * t i m e r i d ) ;

i n t t imer_se t t ime ( t i m e r_ t t i m e r i d , i n t f l ags ,

const struct i t imerspec * new_value ,

struct i t imerspec * o ld_va lue ) ;

i n t t imer_ge t t ime ( t i m e r_ t t i m e r i d ,

struct i t imerspec * cu r r_va lue ) ;

i n t t imer_de le te ( t i m e r_ t t i m e r i d ) ;


Linux-specific: Timerfd

Create a file descriptor to deliver timeouts.

Works like interval timers, but deliver just event on FD.


Timerfd API

#include <sys / t im e r f d . h>

i n t t im e r f d_c r ea te ( i n t c lock id , i n t f l a g s ) ;

i n t t im e r f d_se t t im e ( i n t fd , i n t f l ags ,

const struct i t imerspec * new_value ,

struct i t imerspec * o ld_va lue ) ;

i n t t imer fd_ge t t ime ( i n t fd ,

struct i t imerspec * cur r_va lue ) ;


Linux-specific: Eventfd

Creates a file descriptor used for generic event wait/notify mechanism.

Has lower overhead than with pipe(). Only a 64-bit counter.

Can be used with kernel/userspace communications.

Read() returns value and sets the counter to zero, blocks if counter was zero. Write()

adds value to counter.

Recently semaphore-like operation also added: read() returns 1 and decrements

counter by 1. If counter was 0, blocks. (Flag EFD_SEMAPHORE)


Eventfd API

#include <sys / event fd . h>

i n t event fd ( unsigned in t i n i t v a l , i n t f l a g s ) ;


Linux-specific: Inotify

Provides a way to monitor filesystem using events.

Creates a file descriptor for monitoring with inotify_init().

Watch list can be edited with inotify_add_watch()/inotify_rm_watch().

Read() returns struct inotify_events.

(Supercedes dnotify, that required FD per monitored directory.)


Inotify API

# include <sys / i n o t i f y . h>

i n t i n o t i f y _ i n i t ( void ) ;

i n t i n o t i f y _ i n i t 1 ( i n t f l a g s ) ;

i n t i no t i f y_add_watch ( i n t fd , const char * pathname , u i n t 3 2 _ t mask ) ;

i n t i no t i f y_ rm_watch ( i n t fd , u i n t 3 2 _ t wd ) ;

struct i n o t i f y _ e v e n t {

i n t wd ; / * Watch d e s c r i p t o r * /

u i n t 3 2 _ t mask ; / * Mask o f events * /

u i n t 3 2 _ t cookie ; / * Unique cookie assoc ia t ing r e l a t e d

events ( f o r rename ( 2 ) ) * /

u i n t 3 2 _ t len ; / * Size o f name f i e l d * /

char name [ ] ; / * Op t iona l n u l l − terminated name * /

} ;


System-specific: Sendfile

#include <sys / s e n d f i l e . h>

ss i ze_ t s e n d f i l e ( i n t out_fd , i n t in_ fd ,

o f f _ t * o f f se t , s i ze_ t count ) ;

Other UNIX systems implement sendfile() with different semantics.

Idea is to avoid reading stuff to buffer just to write it out.


Linux-specific: Pipe tricks

ss i ze_ t s p l i c e ( i n t fd_ in , l o f f _ t * o f f _ i n , i n t fd_out ,

l o f f _ t * o f f_ou t , s i ze_ t len , unsigned in t f l a g s ) ;

ss i ze_ t tee ( i n t fd_ in , i n t fd_out , s i ze_ t len ,

unsigned in t f l a g s ) ;

ss i ze_ t vmspl ice ( i n t fd , const struct iovec * iov ,

unsigned long nr_segs , unsigned in t f l a g s ) ;

Splice() moves data between two file descriptors, one must be a pipe.

Tee() duplicates data from a pipe to pipe.

Vmsplice() maps memory into a pipe.


So what is the best thing to do?

Exam on 8.12.: Course exam == automatic enrollment. Remote exam.

There are no definitive answers, unfortunately.

C10k, does it apply?

Why Threads Are a Bad Idea.

Why Events Are A Bad Idea.

The Problem with Threads.


elec-c7310 sovellusohjelmointi lecture 11: advanced interfaces

Documents