Three ways to run Python programs from C

[kudos]

One question, that often comes up is: How to call Python from my C program? There are numerous ways, so let us review three of them.

System Approach

We could call Python from C program by using the system function, which basically means, we do need have an already written python program, or you would need to make a C program write a python program to disk. Here is a quick example:

#include <stdio.h>

int main(){

char *prg = (char*)"print \"hello from python\"\n";
FILE *f= (FILE*)fopen("test2.py","w");
fprintf(f, prg);
fclose(f);

system("python test2.py");

}

Let's assume that the program was saved as test2.c, then you would compile it the following way:

gcc test1.c

Then you would run it like this:

./a.out

The bad thing about this approach is; what if we were to do anything in python, and we wanted Python to output something, which we then could process in C? We could make our Python program write to file, then we could have the C program read the outputted file from the Python program, but then you have to make some modifications to your python program and you would have a lot of files with result floating around on your harddrive (like there isn't enough there already!)


Process Approach

Let's make a toy program, we want to add two number in Python, which are given to us from a C program. So the C program would supply us with the two numbers, the two numbers would then be sent to a Python program and the result would be written from the C program through the printf command.

#include <stdio.h>
#include <stdlib.h>

int main(){
int number1 = 10;
int number2 = 32;
int number3 = -1;

/* the python program as a C string */
char *prg = (char*)"import sys\nprint int(sys.argv[1])+int(sys.argv[2])\n";
/* make a C string which call Python */
char *input = (char*)malloc(sizeof(100)*sizeof(char));
/* a C string which will store the results */
char *input2 = (char*)malloc(sizeof(100)*sizeof(char));
FILE *f= (FILE*)fopen("test2.py","w");
fprintf(f, prg);
fclose(f);
sprintf(input,"python test2.py %d %d",number1,number2);
f = popen(input,"r");
fgets(input2,100,f);
fclose(f);
number3 = atoi(input2);
printf("%d\n",number3+10);

Here we use popen to open a process, and in this case, the process happens to be the python interpreter running the program which we defined as a C string. We can read the result of our the process, which in our case is going to be 42. Then we convert this string into an int with atoi.

Let's assume that the program was saved as test2.c, then you would compile it the following way:

gcc test2.c

Then you would run it like this:

./a.out

Embedded Approach

What if you device didn't have an operating system? Or maybe you don't want your program to interfere with your operating system? One possibility is to embed python into your C program. Now, what does "embed" actually mean? Well, it means, make the interpreter part of your C program

First, what kind of system are you on? (you probably would stop somewhere in the last section in Windows, because I don't think popen is called popen in the Windows api).

Let's assume that you are using Ubuntu or something similar. First, we need to Figure out the flags for compiling and embedding Python into your C program. Where are the include files, and where are the library file?

I start by figuring out what kind of Python I run on my system

python --version

Which returns :

Python 2.6.5

Now I want to figure out where the .h file is, so I type:

locate Python.h

which returns :

/usr/include/python2.6/Python.h

Then I want to find the library file(which basically is the python interpreter):

locate libpython2.6.a

/usr/lib/python2.6/config/libpython2.6.a

Then I got all the things I need, now, let's make sure that we get this works before I do anything else.

We start by making an empty C program

#include <Python.h>
#include <stdio.h>

int main(){

}

Do we get this to compile? Let's save it as test3.c

gcc -I/usr/include/python2.6/ -L/usr/lib/python2.6/config/ -lpython2.6 test3.c

Did it work? If yes, then let's continue to make an interesting C with Python program in test3.c. The program should do what it did earlier; we have two numbers (number1 and number2), we want python to add these two numbers, then we want to get the result of the addition in Python and place it in a variable called number3 in C

#include <stdio.h>
#include <Python.h>

int main(){
int number1 = 10;
int number2 = 32;
Py_Initialize();
char *input = (char*)malloc(sizeof(100)*sizeof(char));
sprintf(input,"number3 = %d + %d",number1,number2);
PyRun_SimpleString(input);
PyObject * module = PyImport_AddModule("__main__");
PyObject * dictionary = PyModule_GetDict(module);
PyObject * result = PyDict_GetItemString(dictionary, "number3");
long number3 = PyInt_AS_LONG(result);
printf("%d\n",(int)number3);
Py_Finalize();
}

You would typically run it as:

./a.out

In this case, we do not need to call the Python as a separate process, because Python is already part of your C program. Now which one should you use? The cons of the embedded version you have to bare careful when you compile your C program. The pros is that you don't interfere with the processes on your computer, write files to your harddrive etc.

[Oralloy]

It seems to me that you would use each of the solutions to solve a particular category of problem.

BTW, I do not like the "system" approach, as you are constrained to creating a string that is exactly compatible with Bash (or sh) command line input.

The fork() and exec() approach is probably better. If for no other reason than it does not require encoding of arguments.

The "process approach" has the same issues as the "system" approach.

Running python as an embedded interpreter is fine, but what is the point?

• If all you want to do is run Python, why bother with C at all?
• This might be valuable when you are dynamically creating non-trivial Python, but few applications will do such.
• This is not valuable if all you are doing is creating python to match a few parameters. Included/excluded code blocks can be guarded in a fixed program much easier.

The bottom line is that executing a sub-process only makes sense when:

• The target language lets you solve a problem easily, which would otherwise be difficult in C.
• You do not have access to the source of the sub-process.
• You do not want to re-implement an existing, debugged program in C. (which is quite reasonable).
• Parallel execution is valuable - the sub-process does something while the C process does something else.

The model you will select depends strongly on your goals. Do you need just the output from Python based on invocation parameters, or do you want to transfer data back and forth between the two?

Your question is really good, although there is no explicit answer.

Cheers!
Orally