Multi Threading embedded python

[amit]

Hello,

I am embedding a python script in a C++ application. The script can be
called simultaneously from multiple threads.

What is the correct way to implement this situation:

1) Have unique python interpreter instantiations ( Py_Initialize() ) for
each thread.

2) Have one python interpreter, and implement a lock on it so it can't
be called simultaneously by multiple threads?

Thanks
Amit

[Pierre Barbier de Reuille]

Re: Multi Threading embedded python

Well, depends on what you want to achieve :)

First, I don't think you can call Py_Initialize on many threads. You
have special function to initialise different interpreters on per-thread
basis.

However, the main problem is: do you want to share data across your
threads ? If the answer is 'no' then use one interpreter by thread, this
is, by far, the simplest solution (and the most efficient ?).

But if you *do* need to share data, then you are in big trouble man :)
As far as I tried it, multi-threading and Python don't go along very
well ! At least, not with system-threads. Basically, to run some Python
code, you have to hold the GIL (Global Interpreter Lock) and you cannot
*test* it, you have to try holding it, so be prepare to block your
threads that would want to access Python !

What I would recommend (and that's what I do in my own software) is to
use a single thread in which your Python interpreter is running. Then,
use a message system in C++ to send commands and get them evaluated and
sent back (if needed). By doing so, you'll avoid a lot of problems, and
you won't loose significantly performances (or if you care about that,
then Python is definitly not the language you need) nor parrallelism
(you wouldn't be able to run many Python threads at the same time anyways).

Well, I hope that help,

Pierre

amit a écrit :[color=blue]
> Hello,
>
> I am embedding a python script in a C++ application. The script can be
> called simultaneously from multiple threads.
>
> What is the correct way to implement this situation:
>
> 1) Have unique python interpreter instantiations ( Py_Initialize() ) for
> each thread.
>
> 2) Have one python interpreter, and implement a lock on it so it can't
> be called simultaneously by multiple threads?
>
> Thanks
> Amit[/color]