Kqueues for Fun and Profit

This page gives examples of kqueue usage, I am going to assume you are otherwise competent at C programming on UNIX in the examples and commentary.

Disclaimer

This resource was created by Peter Werner not myself. In all likelihood I am as much a novice with kqueue as you are. I am hosting this as I found it to be one of the most useful resources on kqueue I ever found and was dismayed when it disappeared from the web. Thanks to the WayBack Machine I was able to recover it and give it a new home.

I hope you find this as useful as I have and still do.

On to the Fun and Profit

Christian Biere pointed out an error in the description of EVFILT_READ and the listen backlog. The backlog is set per the normal listen(2), and the current backlog size will be returned in the kevent. You don’t need to put the backlog int he structure when you add the kevent. The NetBSD manpage is a little bit clearer:

Sockets which have previously been passed to listen() return when there is an incoming connection pending. data contains the size of the listen backlog (i.e., the number of connections ready to be accepted with accept(2).)

Thanks Christian!

What are they?

Kqueues were developed by Jonathan Lemon and first appeared in FreeBSD 4.1. They are also found in OpenBSD and NetBSD (these examples were all written on OpenBSD). From the manpage:

kqueue() provides a generic method of notifying the user when an event happens or a condition holds, based on the results of small pieces of kernel code termed “filters”. A kevent is identified by the (ident, filter) pair; there may only be one unique kevent per kqueue.

or in plainer english, they tell you when stuff happens.

Why are they spiffy?

More scalable than poll() or select(), easy enough to use, reasonably flexible and powerful. There is a paper by Jonathan Lemon on them which goes into more detail, there also slides. Also Neils Provos has a library for event notification called libevent which has support for kqueues.

How do they work?

As mentioned in the manpage snippet above, there are a number of pre-defined filters in the kernel. These filters apply to such things as a file descriptor having data to read or process being sent a signal and so on. The main interaction with kqueues is done with the kevent(2) system call. Using this, you tell the kernel something like “I’d like to know when this file descriptor has data to read”, you can then go off and do other stuff for do a blocking system call until the condition is true (or both). When there is data to be read on the file descriptor, the kernel files out a kevent struct to be retrieved by a call to kevent().

The basic gist

At the time of writing there are 6 defined filters:

Currently it is not possible to load your own filters and I wouldn’t hold your breath for such a feature to be added.

In your code, there are two system calls and one macro that you will use. Firstly, we’ll look at the structure used to set and retrieve events, struct kevent, its fields are such:

struct kevent {
    uintptr_t   ident;      /*< identifier for this event   */
    short       filter;     /*< filter for event            */
    u_short     flags;      /*< action flags for kqueue     */
    u_int       fflags;     /*< filter flag value           */
    intptr_t    data;       /*< filter data value           */
    void        *udata;     /*< opaque user data identifier */
};

The ident is either a file descriptor, signal number or process id (keeping in mind OpenBSD doesn’t support AIO). The filter is the filter type (EVFILT_XXX), the flags say what to do with the kevent (add, change, etc …). The fflags are filter specific flags (see proc.c) and data is filter specific data, in general used to return data more often than receive it (though EVFILT_READ on a listening socket is an exception). Finally udata is a pointer set by the user that remains untouched by the kernel, and will be present when the specified event occurs and is retrieved via kevent(2). See kqfile.c for an example of this.

kqueue(2)

The returns a descriptor used for subsequent kevent(2) calls.

int kq;

kq = kqueue();
if (kq == -1)
    err(1, "kqueue!");

kevent(2)

The kevent(2) system call takes 6 arguments. It’s prototype is:

int
kevent(int kq, const struct kevent *changelist, int nchanges,
        struct kevent *eventlist, int nevents,
        const struct timespec *timeout);

The first is the kqueue descriptor you obtained with a call to kqueue(). The second is an array of kevent structures you want to add or change, the third is the number of structures in the array. The fourth is an array of kevent structures used to receive events from the kernel, and the file is the size of this array. The sixth and final argument is an optional timeout. It is perhaps useful to think of it a bit like select(2) on steroids. It is not necessary to have all the arguments, so typical usage might look like:

/* set event (referred to as the changelist in the documentation) */
i = kevent(kq, &ke, 1, NULL, 0, NULL);

/* retreive event (the eventlist in the documentation) */
i = kevent(kq, NULL, 0, &ke, 1, NULL);

Errors returned from kevent can be a little involved if you have both items in the changelist and the eventlist. See kqerror.c example for details. Otherwise, if you only have items in the changelist, it returns 0 on success or -1 on error. If you only have items in the eventlist it returns the number of events placed in the eventlist, 0 on timeout or -1 on error.

EV_SET()

This is a macro defined in sys/event.h, it is for setting the items in the kevent struct without having to type them out. From the manpage:

EV_SET(&kev, ident, filter, flags, fflags, data, udata);

where kev is the struct kevent you want to fill in, the rest are the values to use.

Examples

Each of the examples takes you to a page describing the example and pointing out anything noteworthy. The (source) links give you the source which is also available on the html description pages. Reading The basic gist section above first is probably a good idea.