Table of

8.1.2010The year I started blogging (blogware)
9.1.2010Linux initramfs with iSCSI and bonding support for PXE booting
9.1.2010Using manually tweaked PTX assembly in your CUDA 2 program
9.1.2010OpenCL autoconf m4 macro
9.1.2010Mandelbrot with MPI
10.1.2010Using dynamic libraries for modular client threads
11.1.2010Creating an OpenGL 3 context with GLX
11.1.2010Creating a double buffered X window with the DBE X extension
12.1.2010A simple random file read benchmark
14.12.2011Change local passwords via RoundCube safer
5.1.2012Multi-GPU CUDA stress test
6.1.2012CUDA (Driver API) + nvcc autoconf macro
29.5.2012CUDA (or OpenGL) video capture in Linux
31.7.2012GPGPU abstraction framework (CUDA/OpenCL + OpenGL)
7.8.2012OpenGL (4.3) compute shader example
10.12.2012GPGPU face-off: K20 vs 7970 vs GTX680 vs M2050 vs GTX580
4.8.2013DAViCal with Windows Phone 8 GDR2
5.5.2015Sample pattern generator


Using dynamic libraries for modular client threads

If you are writing a modular Linux software you would like to extend using dynamic libraries that are loaded at runtime and launched as separate threads, I can show you one way to do it.

I also include some simple communication: the client can make resource requests to the parent. The parent creates the requested resources when it has the time, and signals the client. The client interface is compatible with C and C++.
In my example the parent is written in C++, and the client in C. Both can execute each other's code. The example proceeds as follows:

Let's start off with the parent main:

#include "resource.h"
#include "client.h"
#include "unistd.h"

int main() {
    std::list<Client*> d_clients;
    try {
        d_clients.push_back(new Client("test", "./"));
    } catch (std::string e) {
        printf("Couldn't open the client library (%s)\n", e.c_str());
        return 1;

    while (true) {
        // Satisfying the requests
        ResourceRequest *req;
        while (g_resMgr.popRequest(&req)) {
            printf("main: Satifsying request \"%s\"\n", req->description().c_str());

            try {
                // Now that we were able to satisfy it, we're finding the requester
                // and telling the request to inform it.
                for (std::list<Client*>::iterator i = d_clients.begin(); i != d_clients.end(); ++i)
                    if ((*i)->isID(req->requester()))
            } catch (std::string e) {
                printf("\tNo can do.  Removing the request and continuing.  (Reason: %s)\n", e.c_str());
            delete req;

        // Let's not busyloop, even if it is an example

This is the parent-side client interface:

#ifndef _CLIENT_H
#define _CLIENT_H

#include <string>
#include <list>
#include <pthread.h>

class Client {
    Client(std::string, std::string);
    bool isID(pthread_t);
    std::string name();
    void addSomeInt(int);

    std::string d_name;
    void *d_dlhandle;
    pthread_t d_threadHandle;
    void (*clientCall)(int);
    int d_someInt;

#endif // CLIENT_H

#include <dlfcn.h>
#include "client.h"
#include "client_interface.h"
#include "resource.h"

// This is the implementation of the getSomeInt callback that clients call
void getSomeInt(int *handle, struct lock *lock) {
    ResourceRequest *req = new ResourceRequestSomeInt(lock, handle);
    printf("Added req %ld\n", req);

Client::Client(std::string name, std::string libFile) : d_name(name) {
    d_dlhandle = dlopen(libFile.c_str(), RTLD_NOW);

    char *error;
    if (error = dlerror())
        throw std::string("Trying to open library file for client " + name + ": " + std::string(error));

    // Finding the entry point
    void* (*start)(void*) = (void* (*)(void*)) dlsym(d_dlhandle, "launch");
    if (error = dlerror())
        throw std::string("Trying to find entry point for client " + name + ": " + std::string(error));

    // Fetching pointers to whatever calls we need
    clientCall = (void (*)(int)) dlsym(d_dlhandle, "clientCall");
    if (error = dlerror()) {
        clientCall = NULL;
            std::string("Could not find clientCall method for client " + name + ": " + std::string(error)).c_str());

    // Test call to client
    if (clientCall)

    // Creating a function struct
    struct callbackFunctions *cf = new struct callbackFunctions;
    cf->getSomeInt = getSomeInt;
    cf->genLock = genLock;
    cf->tryLock = tryLock;
    cf->lock = lock;
    cf->unlock = unlock;

    // Spawning a thread
    if (pthread_create(&d_threadHandle, NULL, *start, (void*)cf))
        throw std::string("Failed trying to spawn a thread for client " + name);

bool Client::isID(pthread_t id) {
    return pthread_equal(d_threadHandle, id);

std::string Client::name() {
    return d_name;

void Client::addSomeInt(int i) {
    d_someInt = i;

This is the actual client, in C:

#include "stdio.h"
#include "client_interface.h"
#include "stdlib.h"

/* This is supposed to get called from the main program */
void clientCall(int i) {
    printf("Client got called with %d\n", i);

void* launch(void* d) {
    struct callbackFunctions *cf = (struct callbackFunctions*)d;
    struct lock d_lock = (*cf->genLock)();
    int someInt;
    /* Let's use a callback to the main program code.
     * The call gets executed of course by the client thread.
     * The call is actually a request, to be satisfied by the main
     * program thread when it has the time.
    (*cf->getSomeInt)(&someInt, &d_lock);

    /* Waiting for the request to be satisfied by the main program */
    if (d_lock.rvalue == SUCCESS)
        printf("Client got its request satisfied.  Value %d\n", someInt);
        printf("Client got refused\n");

    return NULL;

Now the following is for support -- the callback framework and the locking mechanism:


#include "lock.h"

#ifdef __cplusplus
extern "C" {

// Since we cannot have the lock function code in two places at once,
// we're only having it in the main program, and passing pointers for
// them to the client.
struct callbackFunctions {
    // Enter your own callbacks here
    void (*getSomeInt)(int*,struct lock*);

    // Lock functions
    struct lock (*genLock)();
    int (*tryLock)(struct lock*);
    void (*lock)(struct lock*);
    void (*unlock)(struct lock*);

#ifdef __cplusplus
#endif // extern "C"


#ifndef _LOCK_H
#define _LOCK_H

#include <pthread.h>

#define SUCCESS 0
#define FAILURE -1

#ifdef __cplusplus
extern "C" {

// As we will be using the lock for communication, we also
// pass through a return value (either SUCCESS or FAILURE).
struct lock {
    pthread_mutex_t mutex;
    int rvalue;

struct lock genLock();

// Tries to lock without blocking.
// Returns true or false if locking was or was not possible, respectively.
int tryLock(struct lock*);

// Locks.  Blocks until locking is possible.
void lock(struct lock*);

// Unlocks, no matter who locked it in the first place.
// Works for locked and unlocked locks similarly.
void unlock(struct lock*);

#ifdef __cplusplus
#endif // extern "C"

#endif // LOCK_H

#include "lock.h"

struct lock genLock() {
    struct lock newLock;
    pthread_mutex_init(&newLock.mutex, NULL);
    return newLock;

int tryLock(struct lock* lock) {
    return pthread_mutex_trylock(&lock->mutex);

void lock(struct lock* lock) {

void unlock(struct lock* lock) {

Next comes the framework for resource requesting. Now this is almost beside the point of the blog entry, and entirely optional. Everyone most likely need some kind of communication, but whether an example of resource requesting is relevant to you is another question. I'm personally using a structure like this for requesting OpenGL resources from the parent.

#ifndef _RESOURCE_H
#define _RESOURCE_H

#include <string>
#include <stack>
#include "lock.h"
#include "client.h"

class ResourceRequest {
    ResourceRequest(struct lock*);
    void setDescription(std::string);
    std::string description();
    virtual void satisfy() {}
    virtual void inform(Client*) {}
    std::string error(std::string);
    std::string error(const char*);
    pthread_t requester();

    struct lock *d_lock;
    std::string d_name;
    pthread_t d_requester; // This gets set as the CURRENT thread

// This is an example request which merely provides an integer to the client
class ResourceRequestSomeInt : public ResourceRequest {
    ResourceRequestSomeInt(struct lock*, int *data);
    void satisfy();
    void inform(Client*);

    int *d_handle;

class ResourceManager {
    void addRequest(ResourceRequest*);
    // False if none exists.
    bool popRequest(ResourceRequest**);

    struct lock d_reqLock;
    std::stack<ResourceRequest*> d_reqs;

extern ResourceManager g_resMgr;

#endif // _RESOURCE_H

#include "resource.h"
#include <cstdio>

ResourceManager g_resMgr;

ResourceManager::ResourceManager() {

void ResourceManager::addRequest(ResourceRequest *req) {

// We fill in a new request as the parameter, unless there
// isn't any, in which case we return false.
bool ResourceManager::popRequest(ResourceRequest **req) {
    if (d_reqs.empty())
        return false;

    *req =;
    return true;

ResourceRequest::ResourceRequest(struct lock* l) : d_lock(l) {
    d_requester = pthread_self();
// When the request is satisfied, we let the request die, and the
// parent class will signal the client.
ResourceRequest::~ResourceRequest() {
    printf("Destroying the request and releasing the lock\n");
void ResourceRequest::setDescription(std::string name) {
    d_name = name;
std::string ResourceRequest::description() {
    return d_name;
std::string ResourceRequest::error(std::string msg) {
    return std::string("A problem with request \"" + description() + "\": " + msg);
std::string ResourceRequest::error(const char* msg) {
    return error(std::string(msg));
pthread_t ResourceRequest::requester() {
    return d_requester;

// We delegate locking and signalling to the parent, but set the
// handle for the data here.
ResourceRequestSomeInt::ResourceRequestSomeInt(struct lock *l, int *data) :
    ResourceRequest(l), d_handle(data) {

void ResourceRequestSomeInt::satisfy() {
    printf("Satisfying a SomeInt request\n");
    *d_handle = 123;

    // Now we can mark the operation as success
    d_lock->rvalue = SUCCESS;

// This is definitely optional.  In case you would like to have
// the client request also supplied to the Client class in parent, use this.
void ResourceRequestSomeInt::inform(Client *c) {
    printf("Informing client %s for req %s\n", c->name().c_str(), description().c_str());

Finally, you can compile the above code like this:

    g++ -ldl -lpthread -o main main.cpp client.cpp resource.cpp lock.cpp

    gcc -shared -fPIC -o client-module.c



Remember to export symbols in the parent by compiling with -rdynamic, if you wish to use them in a client. 
- wili

Nick     E-mail   (optional)

Is this spam? (answer "no")