Kernel timing issues

An introduction to the use of kernel timers and work queues

Kernel timers

• Linux offers a facility that lets drivers put a process to sleep until a fixed amount of time has elapsed (as measured in jiffies)

• When the timer expires, a driver-defined action will be performed, which can ‘wake up’ the process that was put to sleep, or could perform some alternative action (for example, the kernel timer could re-start)

jiffies

• unsigned long volatile jiffies; • global kernel variable (used by scheduler)• initialized to zero when system reboots• gets incremented during a timer interrupt• so it counts ‘clock-ticks’ since cpu restart• ‘tick-frequency’ is a ‘configuration’ option• On our machines: HZ=250 (in ‘.config’)

jiffies overflow

• Won’t overflow for at least 16 months• Linux kernel got modified to ‘fix’ overflow• Now the declaration is in ‘linux/jiffies.h’:

unsigned long long jiffies_64; and a new instruction in ‘do_timer()’

(*(u64*)&jiffies_64)++;which compiles to assembly language as

add $1, jiffies+0adc $0, jiffies+4

Kernel timer syntax• Declare a timer: struct timer_list mytimer;• Initialize this timer: init_timer( &mytimer );

mytimer.func = mytimeraction;mytimer.data = (unsigned long)mydata;mytimer.expires = <number-of-jiffies>

• Install this timer: add_timer( &mytimer );• Modify this timer: mod_timer( &mytimer, <jifs> );• Delete this timer: del_timer( &mytimer );• Delete it safely: del_timer_sync( &mytimer);

A kernel-timer caution

• A kernel timer’s timeout-action cannot do anything that could cause the current task to ‘sleep’ (such as copying data between user-space and kernel-space, or trying to allocate more kernel memory)

• However, to aid debugging, a timer CAN use ‘printk()’ within its timeout-routine

‘trytimer.c’

• We have posted an example that shows how a Linux kernel timer can be used to perform a periodic action (such as using ‘printk()’ to issue a message every time the time expires, then restart the timer

• Notice that our demo is not able to issue messages directly to the console – its timer-function executes without a ‘tty’

Delaying work

• If a device-driver needs to perform actions that require using process resources (like a tty), or that may possibly ‘sleep’, then it can defer that work – using a ‘workqueue’

Programming syntax• Declare: struct workqueue_struct *myqueue;

struct work_struct thework;

• Define: void dowork( void *data ) { /* actions */ };

• Initialize: myqueue = create_singlethread_workqueue( “mywork” );INIT_WORK( &thework, dowork, <data-pointer> );

• Schedule: queue_dalayed_work( myqueue, &thework, <delay> );

• Cleanup: if ( !cancel_delayed_work( &thework ) )flush_workqueue( myqueue );

destroy_workqueue( myqueue );

‘tryworkq.c’ and ‘defermsg.c’

• We have posted demo-modules that show the use of workqueues to perform actions later, either as soon as a ‘process context’ is available, or after a prescribed time

• Further details on the options for using an existing kernel workqueue or a workqueue of your own creation may be found in our textbook (Chapter 7 of LDD3)

Applying these ideas

• To demonstrate a programming situation in which using kernel timers is valuable, we created the ‘foo.c’ device-driver, plus an application that uses it (‘watchfoo.cpp’)

• You can compile and install the module, then execute the application: $ watchfoo

• But you will notice there are two ‘problems’ (excess cpu usage and loop-termination)

Reducing CPU’s usage

• The ‘watchfoo’ program rereads ‘/dev/foo’ constantly (numerous times per second), much faster than the human eye can see

• If you run the ‘top’ utility, you will see that a high percentage of the available CPU time is being consumed by ‘watchfoo’

• You can add a kernel timer to the ‘foo.c’ driver to curtail this excessive reading

In-class exercise

• Modify ‘foo.c’ (call it ‘timedfoo.c’) as follows• Create an integer flag-variable (‘ready’) as a

global object in your module• When your ‘read()’ function gets called, it should

sleep until ‘ready’ equals TRUE; it should set ‘ready’ equal to FALSE when it awakens, but should set a timer to expire after 1/10 seconds

• Your timer’s action-function should set ‘ready’ back to TRUE, then wake up any sleeping tasks

Implementation hints

• You need a wait-queue (so your driver’s ‘reader’ tasks can sleep on it)

• You need a timer action-function• You need to organize your timer-function’s

data-items into a single structure (because the timer-function has only one argument)

• Your timer-function must do two things:– Change ‘ready’ to TRUE– Wake up any ‘sleepers’

Deferring work

• Linux supports a ‘workqueue’ mechanism which allows the kernel to defer specified work until some later point in time

• This mechanism has been ‘reworked’ in a major way since our texts were published

• So any device-driver has to be modified if it made any use of a kernel ‘workqueue’

• Changes require grasp of some ‘macros’

‘sizeof’ and ‘offsetof’

• Our GNU compilers permit use of these C/C++ operators on object types

• The ‘sizeof’ operator returns the number of bytes of memory the compiler allocated for storing the specified object

• The ‘offsetof’ operator returns the number of bytes in a structure which precede the specified structure-member

A ‘struct’ examplestruct mystruct {

char w;short x;long y;long long z;} my_instance;

You can use the ‘sizeof’ operator to find out how much memory gets allocated to any ‘struct mystruct’ object, like this:

int nbytes = sizeof( my_instance );

You can use the ‘offsetof’’ operator to find out where within a given structure a particular field occurs, like this:

int offset_z = offsetof( struct mystruct, z );

The ‘container_of()’ macro

• Recent versions of the Linux kernel have introduced a further operator on ‘structs’container_of( ptr, type, member );

• When given a pointer to a field within a structure-object, and the type-name for that that structure-object, and the field-name for that structure’s field-member, then it returns the structure’s address

Using ‘container_of()’struct mystruct {

char w;short x;long y;long long z;} my_instance = { 1, 2, 3, 4 };

If you have a pointer to one of the fields in some instance of a this kind of ‘struct’ object, then you could use the ‘container_of()’ macro to get a pointer to that ‘struct’ object itself, like this: long *ptr = &my_instance.y; struct mystruct *p = container_of( ptr, struct mystruct, y );

This would be useful if you now wanted to access other members:

printk( “w=%d x=%d y=%d z=%d \n”, p->w, p->x, p->y, p->z );

• #include <linux/workqueue.h>• void dowork( struct work_struct *data );• DECLARE_DELAYED_WORK( mywork,

dowork );• struct workqueue_struct *myqueue;• myqueue =

create_singlethread_workqueue( “mywork” );

‘workqueue’ syntax#include <linux/workqueue.h>

struct workqueue_struct *myqueue; // pointer to your workqueuevoid dowork( struct work_struct *data ); // your function’s prototypeDECLARE_DELAYED_WORK( mywork, dowork );

int init_module( void ){

myqueue = create_singlethread_workqueue( “mywork” );if ( !queue_delayed_work( myqueue, &mywork, HZ*5 ) )

return –EBUSY;return 0; // SUCCESS

}void cleanup_module( void ){

destroy_workqueue( myqueue ); }

‘tryworkq.c’

void dowork( struct work_struct *data ){

printk( “\n\n I am doing the delayed work right now \n” );}

In this example the delayed work consists of simply printing a message to the kernel’s logfile -- you can view by typing the ‘dmesg’ command

Notice that the ‘action’ function in this example ignores its ‘data’ argument

An improved example

• Our ‘announce.c’ module shows how an LKM could display its messages within a window on the Linux graphical desktop

• It uses the ‘tty_struct’ object which exists in the process-descriptor for the ‘insmod’ task which you launch to install the LKM

• We shall see how this same idea can be used in a waitqueue’s ‘action’ functions

‘timer’ verses ‘workqueue’

• Any kernel-timer’s action-function will be executed in ‘atomic’ context – just like an interrupt service routine: it cannot ‘sleep’, and it cannot access any user-space data

• But any workqueue’s action-function will be executed by a kernel-thread – and thus it possesses a ‘process’ context, so it can be ‘scheduled’ and ‘sleep’ if necessary – though it, too, cannot access user-space

If ‘dowork()’ needs data…// data items needed by your ‘dowork’ function are packaged in a ‘struct’ struct mydata {

char *msg;struct tty_struct *tty;} my_data = { “\nHello\n”, NULL };

// your module-initialization function sets up your ‘struct delayed_work’ object// and it can also finish initializing your ‘my_data’ object’s member-fields

myqueue = create_singlethread_workqueue( “mywork” );INIT_DELAYED_WORK( &mywork, dowork );my_data.tty = current->signal->tty;

// then your action-function can access members of your ‘my_data’ object like thisvoid dowork( struct work_struct *data ){

struct mydata *dp = container_of( &my_data.msg, struct mydata, msg );struct tty_struct *tty = dp->tty;tty->driver->write( tty, dp->msg, strlen( dp->msg ) );

}

‘defermsg.c’

• This LKM will display a message within a desktop window after a 10-second delay

• It illustrates a use of the ‘container_of()’ macro (as is needed by the reworked API for the Linux kernel’s workqueues)

• Our course-website has a link to an online article by author Jonathan Corbet giving details of ongoing kernel changes in 2.6

Summary of tonight’s demos• ‘foo.c’ and ‘watchfoo.cpp’• ‘announce.c’• ‘trytimer.c’• ‘trymacro.c’• ‘tryworkq.c’• ‘defermsg.c’

• EXERCISE: Modify the ‘foo.c’ device-driver to use a kernel timer in it’s ‘read()’ method