Need help with C extension module

[chris]

This is my first attempt at undertaking a C extension module. I want
to wrap an existing C library so I can call the functions from Python.
There are only two functions I'm interested in calling. I did mess
with Pyrex a bit and Swig, to no avail, so I turned to doing it by
hand. Using the example in Programming Python, I did get the easier of
the two functions working--only takes a string parameter. I'm stuck
now on the other function and not sure how to wrap it, because it
involves some structs. Here's a simplified version of the C:

struct In
{
int x;
char* s;
... (only primitive data types)
};

struct Out
{
int y;
char* s;
... (only primitive data types)
};

Out* func(In* x, Out* y);

So the function takes pointers to the two structs, and fills out the
output struct and also returns the pointer to it. I would envision the
Python looking like

in = In()
in.y = 1
in.s = "abc"
....

out = func(in)

maybe? Just no idea how to deal with the structs in the C extension
module code.

Any tips appreciated.

Thanks,
Chris

[Robert Kern]

Re: Need help with C extension module

chris wrote:[color=blue]
> This is my first attempt at undertaking a C extension module. I want
> to wrap an existing C library so I can call the functions from Python.
> There are only two functions I'm interested in calling. I did mess
> with Pyrex a bit and Swig, to no avail, so I turned to doing it by
> hand. Using the example in Programming Python, I did get the easier of
> the two functions working--only takes a string parameter. I'm stuck
> now on the other function and not sure how to wrap it, because it
> involves some structs. Here's a simplified version of the C:
>
> struct In
> {
> int x;
> char* s;
> ... (only primitive data types)
> };
>
> struct Out
> {
> int y;
> char* s;
> ... (only primitive data types)
> };
>
> Out* func(In* x, Out* y);
>
> So the function takes pointers to the two structs, and fills out the
> output struct and also returns the pointer to it. I would envision the
> Python looking like
>
> in = In()
> in.y = 1
> in.s = "abc"
> ...
>
> out = func(in)
>
> maybe? Just no idea how to deal with the structs in the C extension
> module code.[/color]

It's *not* a straightforward task. You have to define an extension type. See

http://docs.python.org/ext/defining-new-types.html

However, Pyrex really does make defining extension types much, much
easier. I highly suggest trying to debug your Pyrex code before learning
all of the boilerplate necessary to create extension types in pure C.

--
Robert Kern
rkern@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[chris]

Re: Need help with C extension module

Any tips on what the pyrex should look like for my example?

[Robert Kern]

Re: Need help with C extension module

chris wrote:[color=blue]
> Any tips on what the pyrex should look like for my example?[/color]

# Untested and off the top of my head; please read the Pyrex
# documentation and correct my mistakes before using any of this!
# Notably, I'm pretty sure the __init__ and __dealloc__ bits are
# slightly wrong.

cdef extern from "Python.h":
void *PyMem_Malloc(u nsigned int size)
void PyMem_Free(void *buf)

ctypedef struct In:
int x
char *s
# ...

ctypedef struct Out:
int y
char *s
# ...

cdef Out *func(Int *x, Out *y)

cdef class Output:
cdef Out *output

def __init__(self):
self.output = <Out*>PyMem_Mal loc(sizeof(Out) )

def __dealloc__(sel f):
PyMem_Free(self .output)

property y:
def __get__(self):
return self.output.y

# ...

cdef class Input:
cdef In *input

def __init__(self):
self.input = <In*>PyMem_Mall oc(sizeof(In))

def __dealloc__(sel f):
PyMem_Free(self .input)

property x:
def __get__(self):
return self.input.x

def __set__(self, value):
self.input.x = value

# ...

def func(self):
output = Output()
func(self.input , output.output)
return output

--
Robert Kern
rkern@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[chris]

Re: Need help with C extension module

Many thanks Robert. That will be a good starting point.

-Chris

http://auslunch.com/

http://fetidcascade.com/

http://strombergers.com/python/

[chris]

Re: Need help with C extension module

Ok, I found further examples on the Internet and got something working
(it seems), but I have a question about the memory management. The
example I found did not include any of the PyMem_... functions.

Here's roughly what I have working:

cdef extern from "my.h":
cdef struct inputs:
char *x

cdef struct outputs:
int y

outputs *func(inputs *x, outputs *y)
int init(char* fname)

class Analyzer:
def __init__(self, fname):
init(fname)

# inp is my python "Inputs" object.
def myfunc(self, inp):
cdef inputs* i
i.x= inp.x

cdef outputs* o
o = func(i, o)
return o.y

class Inputs:
def __init__(self):
self.x = ""

So there is no explicit memory management going on there as in Robert's
example. Is this ok?

Thanks,
Chris

[Ken Seehart]

Re: Need help with C extension module

chris wrote:[color=blue]
> This is my first attempt at undertaking a C extension module. I want
> to wrap an existing C library so I can call the functions from Python.
> There are only two functions I'm interested in calling. I did mess
> with Pyrex a bit and Swig, to no avail, so I turned to doing it by
> hand. Using the example in Programming Python, I did get the easier of
> the two functions working--only takes a string parameter. I'm stuck
> now on the other function and not sure how to wrap it, because it
> involves some structs. Here's a simplified version of the C:
>
> struct In
> {
> int x;
> char* s;
> ... (only primitive data types)
> };
>
> struct Out
> {
> int y;
> char* s;
> ... (only primitive data types)
> };
>
> Out* func(In* x, Out* y);
>
> So the function takes pointers to the two structs, and fills out the
> output struct and also returns the pointer to it. I would envision the
> Python looking like
>
> in = In()
> in.y = 1
> in.s = "abc"
> ...
>
> out = func(in)
>
> maybe? Just no idea how to deal with the structs in the C extension
> module code.
>
> Any tips appreciated.
>
> Thanks,
> Chris
>[/color]

Since others have responded about Pyrex, I'll just add my two cents,
which may or not apply to you.

The idea of filling in a struct, passing it to a function, then
returning a struct, is a typical c-ism. If the only purpose of the
structs is to pass data to the function and receive the result, read on.
On the other hand, if the structures represent "things" in the
object-oriented sense, just ignore what I am saying.

I would wrap the function such that the python code simply takes a set
of parameters and returns a tuple. This is much more pythonic, and much
easier to code.

y,s,... = func(x,s,...)

My untested pseudocode would look something like this:

static PyObject *map_sweep(map_ _object *self, PyObject *args)
{
In input;
Out output;

if (PyArg_ParseTup le(args, "is...", &(In.x), &(In.s), ...))
{
return NULL;
}

func(&In, &Out);

return Py_BuildValue(" is...", Out.y, Out.s, ...);
}