How to release the GIL from C?

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I am trying to release the GIL in a multi-threaded program (efforts
detailed below) without any success.

In the main thread during startup, I do the following:

Py_InitializeEx (0); /* Python shouldn't handle signals */
PyEval_InitThre ads();

/* release the GIL */
... various pieces of code tried here ...

Note that no other threads exist yet when the code above is run.

After startup, there are around 30 threads and my handler routines do
work in them.

void handler(...)
{
PyGILState_STAT E gilstate=PyGILS tate_Ensure();

... do the interesting stuff ...

PyGILState_Rele ase(gilstate);
}

No matter what I try, the PyGILState_Ensu re() hangs trying to acquire
the GIL:

#0 0x00002afd50ce2 43d in sem_wait () from /lib/libpthread.so.0
#1 0x00002afd54715 57d in PyThread_acquir e_lock () from
/usr/lib/libpython2.5.so .1.0
#2 0x00002afd546e4 4d4 in PyEval_RestoreT hread () from
/usr/lib/libpython2.5.so .1.0
#3 0x00002afd5470b b8a in PyGILState_Ensu re () from
/usr/lib/libpython2.5.so .1.0

Here are the various things I have tried to release the GIL (the
documentation says they all do, but that doesn't appear to be the case
:-( Unfortunately the doc is all geared around saving the interpreter
state and then later restoring it. I have nowhere to save it in the
main thread nor do I know when the main thread is having handlers
invoked in it again anyway in order to restore it.

1. (Based on Py_BEGIN_ALLOW_ THREADS)
PyThreadState_S wap(NULL);
PyEval_ReleaseL ock();

2. (Based on high level Py_BEGIN_ALLOW_ THREADS)
PyEval_SaveThre ad();

3. Random code I found on the web
PyThreadState *tstate = PyThreadState_G et();
PyThreadState_C lear(tstate);
PyEval_ReleaseT hread(tstate);

4. Something simple
PyEval_ReleaseL ock();

5. Other code I found on the web
PyThreadState_S wap(NULL);

Does anyone know exactly which sequence of calls I should be making?

Thanks,

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGVKKSmOO fHg372QQRAi4VAJ 9KO8z/WnSgPOlCjEOOJgD Cf20nCwCeJHUE
kkhvMsy6p3qflvG KolttWTo=
=4Z8h
-----END PGP SIGNATURE-----

[Roger Binns]

Re: How to release the GIL from C?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Roger Binns wrote:

I am trying to release the GIL in a multi-threaded program (efforts
detailed below) without any success.

The ultimate cause was that the program forked to go into daemon mode.
I had called PyOS_AfterFork( ) as the documents directed. What I hadn't
realised (and isn't documented) is that AfterFork() reinitialises the
Python threading state by making a new GIL and acquires it. The
acquiring bit meant none of my other threads could ever get it.

It is still unclear from the docs exactly which combination of functions
dealing with threadstate, threads and lock you need to call to just
unilaterally give up the GIL (ie when you don't have following block
that wants to reacquire the GIL)

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGVNW0mOO fHg372QQRAuyOAJ 4pCCIK779XgvaUd KBtSa+nHElrHQCg iueP
n/0uMFCSH3SrQhMXd m2Jb/o=
=uh4D
-----END PGP SIGNATURE-----