python extension, -pthreads and speed

**Pedro Rodriguez** · Jul 18 '05, 10:27 AM

Re: python extension, -pthreads and speed

On Fri, 23 Apr 2004 09:36:08 +0200, Joseph Heled wrote:
[color=blue]
> Hi,
>
> My python module is built using the recommended distutils.core, which
> uses the -pthread flag. To my amazement this slows down the (-O3) code
> by a factor of two (!2)
>
> My gcc documentation says pthread is a PowerPC flag, but I guess this is
> wrong.
>
> Would my code fail if I drop this flag (Assuming I don't use threads at
> all)?
>
> Why would there be such a speed penalty?
>
> Anyone can shed some light on that?
>
> Thanks, Joseph
>[color=green]
>>python -V[/color]
> Python 2.3
>[color=green]
>>gcc -v[/color]
> Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/3.3/specs
> Configured with: ../gcc-3.3/configure : (reconfigured)
> ../gcc-3.3/configure --enable-languages=c,c++ Thread model: posix gcc
> version 3.3[/color]

My understanding of your question is : my C extension module works slower
when used through Python. If my guess is right, this isa possible hint
on the subject.

If your extension use IOs (fgets, sprintf...) or others standard library calls,
they may be affected by the fact that Python is build with multithreading
enabled, on so all those API start using mutexes to be thread safe, thus the
extract cost. This simple C example exhibit the problem :

#include <stdio.h>

int main(int argc, char *argv[])
{
int i;
FILE *in;

in = fopen("/dev/null", "r");

for (i=0; i< 10000000; i++) {
fgetc(in);
}

close(in);
}

$ gcc z.c -O3 -o z
$ time ./z

real 0m12.355s
user 0m5.810s
sys 0m6.550s

$ gcc z.c -O3 -o z -pthread
$ time ./z

real 0m16.055s
user 0m10.610s
sys 0m5.440s

Cheers,
Pedro

**Rolf Magnus** · Jul 18 '05, 10:29 AM

Re: python extension, -pthreads and speed

Pedro Rodriguez wrote:
[color=blue]
> If your extension use IOs (fgets, sprintf...) or others standard
> library calls, they may be affected by the fact that Python is build
> with multithreading enabled, on so all those API start using mutexes
> to be thread safe, thus the extract cost.[/color]

Another candidate for such slowdowns is malloc.

python extension, -pthreads and speed

python extension, -pthreads and speed

Comment

Comment