CTypes - a Python module to access c-data types and libraries

An introduction to using Python's CTYPES module to interface to compiled libraries written in C.

The first example is inspired by this useful reference:

[1] Alex Holkner, 2006, http://ojs.pythonpapers.org/index.php/tpp/article/download/23/23

We assume that Python > 2.5 is used, and the examples below are run on Linux (Ubuntu 13.04), or Mac OS X (10.9). The IPython version is 4.0.6.

Hans Fangohr, 2012-2016, part of FEEG6002 Advanced Computational Modelling I.

In [1]:
import ctypes as ct

Native C data types are available as c_int, c_char, c_float, and with the prefix u for the unsigned integer variety (c_uint). There are some exceptions, such as the unsigned char name which is c_ubyte (and not c_uchar). A full table is available in [1].

Example 1: Unsigned char c-type in Python

Here we use the Unsigned BYTE to demonstrate the overflow of the data type (as would happen in C) when we exceed 255:

In [2]:
i = ct.c_ubyte(255)
In [3]:
i
Out[3]:
c_ubyte(255)

To access the value of the c-type, we use the value attribute:

In [4]:
i.value
Out[4]:
255

Now increase this by one (to exceed 255):

In [5]:
i.value = i.value + 1
In [6]:
i
Out[6]:
c_ubyte(0)
In [7]:
i.value
Out[7]:
0

Example 2: Accessing a function in C-library from Python

We need a C-library file

The %% file mylib.c command writes the content below in a file mylib.c (part of IPython).

In [8]:
%%file mylib.c  
#include <stdio.h>
int mysum(int a, int b) {
    int c = a + b;
    return c;
}
Overwriting mylib.c

Now we compile this file into a library file mylib.so (we assume that Linux is used here):

In [9]:
!gcc -shared -fPIC -o mylib.so mylib.c

Let's see that the file has been created (and let's not worry about the funny file permissions - it's a long story).

In [10]:
ls -l mylib*
-rw-r--r--+ 1 sesg6025  staff    79 21 Nov 17:43 mylib.c
-rwxr-xr-x+ 1 sesg6025  staff  4184 21 Nov 17:43 mylib.so*

Step 1: Accessing the C-library

There are three steps to use a C library function.

First, get a handle to the library

In [11]:
lib = ct.CDLL('./mylib.so')
In [12]:
lib
Out[12]:
<CDLL './mylib.so', handle 103c5a840 at 0x102d1b860>

Step 2: Setting input and output arguments of the function

Second, for the function we want to use, we need to tell ctypes what the input and output types are:

In [13]:
lib.mysum.argtypes = (ct.c_int, ct.c_int) # input are two ints
In [14]:
lib.mysum.restype = ct.c_int              # return value is also int

Step 3: Calling the function

Third, now we can call the C code from Python:

In [15]:
a = lib.mysum(3, 4)
print(a)
7

Summary: accessing function from Python

In [16]:
%reset
import ctypes as ct
lib = ct.CDLL('./mylib.so')
lib.mysum.argtypes = (ct.c_int, ct.c_int) # input are two ints
lib.mysum.restype = ct.c_int              # return value is also int
a = lib.mysum(3, 4)
print(a)
Once deleted, variables cannot be recovered. Proceed (y/[n])? y
7

Example 3: Passing arrays

We create the shared library file source code:

In [17]:
%%file arrayexample.c

void multiply_array(int *a, int n) {
    /* Given an array of int (a) of length n,
    multiply every element by 2 */
    int i;
    for (i = 0; i < n; i++) {
        a[i] = a[i] * 2;
    }
}
Overwriting arrayexample.c

Compile the shared library,

In [18]:
!gcc -shared -oarrayexample.so arrayexample.c

and get a handle to it

In [19]:
lib = ct.CDLL('./arrayexample.so')

We need to create the C array data structure type, as needed to communicate with multiply_array:

In [20]:
# this create a new type (which is an array of 5 C int)
my_array_type = ct.c_int * 5

And create a variable instance of this type:

In [21]:
my_array = my_array_type() 
In [22]:
my_array
Out[22]:
<__main__.c_int_Array_5 at 0x102d34d08>

The resulting array my_array is initialised to zero (that differs from C's behaviour). Just to confirm the zero data is in the array (and to show that we can index the C array like a normal Python list), we print each element:

In [23]:
for i in range(5): 
    print(my_array[i])
0
0
0
0
0

To print the array, we could also have used:

In [24]:
my_array[:]
Out[24]:
[0, 0, 0, 0, 0]

We can use indexing to populate the array with data, i.e.

In [25]:
my_array[0] = 0
my_array[1:5] = [1, -2, 3, 10]

and print the array again:

In [26]:
for i in range(5):
    print(my_array[i]),
0
1
-2
3
10

Now we have our data object my_array populated with data.

Before we can call the multiply_array function to work on this data, we need to tell ctypes about the interface (i.e. input and output) of the C function multiply_array. Input values are (i) a pointer to int, and (ii) an int.

The ctypes function POINTER creates pointer types for a given type:

In [27]:
lib.multiply_array.argtypes = (ct.POINTER(ct.c_int), ct.c_int)

Finally, we need to set the return type of multiply_array.

Here is a somewhat unusual design decision in ctypes: the return type of our function multiply_array is void, so one would expect that there is a ct.c_void type, but instead the agreement is that we should use None, i.e.

In [28]:
lib.multiply_array.restype = None 

We are nearly ready to call our C function with our data my_array.

We start by converting our array my_array into a pointer (at the C-level an array variable int a[] is the same as a pointer int *a but not in this Python representation). We do this by casting the array into a pointer of type c_int:

In [29]:
my_array_ptr = ct.cast(my_array, ct.POINTER(ct.c_int))
lib.multiply_array(my_array_ptr, 5)

Check that we have actually modified the data:

In [30]:
for i in range(5):
    print(my_array[i]),
0
2
-4
6
20

Because we use the nice Python representation of a C-array my_array, we can use len(my_array) to get the length of the array (wouldn't work in C of course), so we could also write:

In [31]:
my_array[0:5] = [0, 1, 2, 3, 4]
In [32]:
lib.multiply_array(my_array_ptr, len(my_array))
for i in range(len(my_array)):
    print(my_array[i]),
    
0
2
4
6
8

Summary: Passing arrays to functions

In [33]:
%reset

import ctypes as ct

lib = ct.CDLL('./arrayexample.so')

# the int array we need
my_array_type = ct.c_int * 5
my_array = my_array_type()
my_array[0:5] = [0, 1, -2, 3, 10]     # with our real data

# setting input and output arguments
lib.multiply_array.argtypes = (ct.POINTER(ct.c_int), ct.c_int)
lib.multiply_array.restype = None 

# calling the function
my_array_ptr = ct.cast(my_array, ct.POINTER(ct.c_int))
lib.multiply_array(my_array_ptr, 5)

# looking at the modified data 
for i in range(5):
    print(my_array[i]),
Once deleted, variables cannot be recovered. Proceed (y/[n])? y
0
2
-4
6
20

Example 4: Performance study

Here we use a naive and slow method to apprixmate $\pi$ in Python

In [34]:
import math
def f(x):
    return math.sqrt(1. - x * x)

def pi(n):
    s = 0.5 * (f(-1) + f(1))
    h = 2. / n
    for i in range(1, n):
        x = -1 + i * h
        s += f(x)
    return s * h * 2

Define the number of subdivisions:

In [35]:
N = 100000
In [36]:
t_python = %timeit -o pi(N)
10 loops, best of 3: 50.1 ms per loop

Now let's write the same function in C and use it via ctypes:

In [37]:
%%file libpi.c
#include <math.h>
double pi(long n) {
 double s=0;        
 double h=2./n;      
 double x;
 long i;
 for (i=1; i<n; i++) {
	 x = -1+i*h;
	 s += sqrt(1.-x*x);
 }
 return s*h*2; 
}
Overwriting libpi.c
In [38]:
!gcc -O3 -fPIC -shared -o libpi.so libpi.c -lm
libpi = ct.CDLL('./libpi.so')
cpi = libpi.pi
cpi.argtypes = (ct.c_long, )   # needs to be tuple, even if only one argument
cpi.restype = ct.c_double
           
In [39]:
t_ctypes = %timeit -o cpi(N)
1000 loops, best of 3: 532 ┬Ás per loop
In [40]:
print("Execution of compiled C code vs naive Python code is {:.2f} times faster".
      format(t_python.best / t_ctypes.best))
Execution of compiled C code vs naive Python code is 94.16 times faster

When to use ctypes?

Ctypes are useful to interface existing (or new) C libraries from pure Python.

See https://docs.python.org/2/library/ctypes.html for a tutorial.