Bus error in PyGILState_Release (callbacks from other threads)

[geoffschmidt@gmail.com]

[Note: I don't check the mailbox in the header. Please send any
correspondence to the address listed below.]

I'm trying to write an extension in C that delivers callbacks to
Python. The C code starts several threads, and I'd like one of the new
threads that is started to be able to deliver callbacks to Python. I
thought I could do this by wrapping the callback function in
PyGILState_Ensu re / PyGILState_Rele ase. When I do this, the Python code
in the callback in between those two calls certainly works, but when
PyGILState_Rele ase is called, the process dies with a bus error!

Is this supposed to work, or am I doing something terribly wrong? (Code
attached below.) This is the official Python 2.4.3 build for OS X; I'm
running OS X 10.4.6.

My examination of the source suggests that head_mutex in pystate.c is
becoming 0x20, which is not so good, since it is supposed to be a
pointer. (in pystate.c: PyGILState_Rele ase calls
PyThreadState_D eleteCurrent, which calls tstate_delete_c ommon, which
calls HEAD_UNLOCK, which calls PyThread_releas e_lock in
thread_pthread. h; this function's first action is to call
pthread_mutex_l ock, which dies on a bus error because the value of
head_mutex that HEAD_UNLOCK got and passed to PyThread_releas e_lock
pointed off into outer space. This is educated guesswork -- gdb's not
reporting a whole lot in the call stack.) Interestingly, the prior call
to HEAD_LOCK in tstate_delete_c ommon succeeds, so it sort of looks like
memory is being corrupted in between the HEAD_LOCK at around
pystate.c:245 and the HEAD_UNLOCK around pystate.c:254.

HEAD_LOCK(); // <-- guess: this is succeeding
for (p = &interp->tstate_head; ; p = &(*p)->next) {
if (*p == NULL)
Py_FatalError(
"PyThreadState_ Delete: invalid tstate");
if (*p == tstate)
break;
}
*p = tstate->next;
HEAD_UNLOCK(); // <-- guess: this is resulting in a bus error

Here's some Pyrex code that crashes 100% of the time for me:

--- snip (foo.pyx) ---
cdef extern from "stdio.h":
int printf(char *str, ...)

cdef extern from "Python.h":
ctypedef int PyGILState_STAT E
PyGILState_STAT E PyGILState_Ensu re()
void PyGILState_Rele ase(PyGILState_ STATE gstate)

cdef extern from "pthread.h" :
ctypedef void *pthread_t # it'll do
int pthread_create( pthread_t *thread, void *attr,
void *(*start_routin e)(void *), void *arg)

cdef extern void *func(void *x):
printf("Enterin g func(%p)\n", x)

cdef PyGILState_STAT E st
printf("PyGILSt ate_Ensure\n")
st = PyGILState_Ensu re()
printf("PyGILSt ate_Release\n")
PyGILState_Rele ase(st)
printf("Leaving func\n")

def callFuncDirectl y():
func(NULL)

def callFuncInThrea d():
cdef pthread_t thr
pthread_create( &thr, NULL, func, NULL);

--- snip (setup.py) ---

from distutils.core import setup
from distutils.exten sion import Extension
from Pyrex.Distutils import build_ext

setup(
name = 'foo',
ext_modules = [Extension("foo" , ["foo.pyx"])],
cmdclass = {'build_ext': build_ext},
)

--- end ---

I ran 'python setup.py build_ext --inplace', then started python in
that directory, did 'import foo', and then 'foo.callFuncDi rectly()'. No
crash. Then I called 'foo.callFuncIn Thread()'. Prints
'PyGILState_Rel ease' and then falls over.

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION _FAILURE at address: 0x00000020
[Switching to process 25649 thread 0x313]
0x900017dc in pthread_mutex_l ock ()
(gdb) bt
#0 0x900017dc in pthread_mutex_l ock ()
#1 0x002be654 in PyThread_releas e_lock (lock=0x20) at
/Volumes/Data/Users/ronald/Universal/python24-fat/Python/thread_pthread. h:439
#2 0x00045600 in func (__pyx_v_x=0x0) at foo.c:72
#3 0x9002ba68 in _pthread_body ()

Any help (or even just confirmation along the lines of "this is
supposed to work, file a bug") would be greatly appreciated.

thanks,
Geoff Schmidt
gschmidt [[a t]] gschmidt [[d o t]] org (send correspondence here, not
the address in the header)

[geoffschmidt@gmail.com]

Re: Bus error in PyGILState_Rele ase (callbacks from other threads)

geoffschmidt@gm ail.com wrote:[color=blue]
> I'm trying to write an extension in C that delivers callbacks to
> Python. The C code starts several threads, and I'd like one of the new
> threads that is started to be able to deliver callbacks to Python. I
> thought I could do this by wrapping the callback function in
> PyGILState_Ensu re / PyGILState_Rele ase. When I do this, the Python code
> in the callback in between those two calls certainly works, but when
> PyGILState_Rele ase is called, the process dies with a bus error![/color]

The fine folks on the Pyrex list figured out the problem:
PyEval_InitThre ads() needs to be called somewhere before any of the
threading calls are made. For a fixed version of my test program,
please see the Pyrex list archives.

thanks,
Geoff Schmidt
gschmidt [[a t]] gschmidt [[d o t]] org (send correspondence here, not
the address in the header)