Python 2.3 problem with extention-embedding in multi-threaded andmulti sub-interpreter environment

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Atul Kshirsagar

    Python 2.3 problem with extention-embedding in multi-threaded andmulti sub-interpreter environment

    I am embedding python in my C++ application. I am using Python *2.3.2* with
    a C++ extention DLL in multi-threaded environment. I am using SWIG-1.3.19 to
    generate C++ to Python interface.

    Now to explain it in details,
    1. Python initialization [Py_Initialize()] and finalization [Py_Finalize()]
    is done in the *main* thread.
    2. For each new thread I create a separate sub-interpreter
    [Py_NewInterpret er()].
    3. Using PyRun_String("i mport myModule"...) before execution of python
    script, extention module
    is imported.
    4. Each thread executes *multiple* python script using PyEval_EvalCode ()
    using the class objects in
    my extention DLL.
    5. Each sub-interpreter is destroyed [Py_EndInterpret er()] at the end of
    that particular thread.

    I am observing that;
    As explained above when multiple threads are running. And as one of these
    threads finishes, in other running threads I start getting
    "TypeError: 'NoneType' object is not callable" error on the methods called
    on class objects in extention module.

    The same code *works fine* with Python 2.2.2.

    I have found these links more or less talking about the same problem
    migrating from 2.2 to 2.3.




    I *guess* what is happening is global variables are zapped to "NoneType"
    when one thread finishes and other thread trying to access them through the
    Python script (step 4.) this error is generated. But it *works* sometimes
    when(*guess*) the running thread is at step 3. and by importing the module
    the global variables are re-initialized for "Type" information.

    I tried using reload(myModule ) to solve the problem but that is generating
    big memory leak every time it is called.

    Is this a know issue with 2.3 (interpreter?) ? Or is there a change for 2.3
    in the way embedding should be done in a multi-threaded and
    multi-sub-interpreter environment ?

    Anybody for help ?
    Thanks,
    Atul


Working...