extending python with a C-written dll

[Steve Holden]

Re: extending python with a C-written dll

Jean-Baptiste PERIN wrote:
[color=blue]
>[color=green]
>>[color=darkred]
>>> >> error: Python was built with version 6 of Visual Studio, and
>>> extensions need to
>>> be built with the same version of the compiler, but it isn't installed.[/color]
>>
>>
>>
>> How do you expect to be able to write your own extension if you don't
>> have
>> the environment to do so in the first place? that error has nothing to do
>> with python itself, but with you not having installed the right compiler.
>> THe same problem will arise building your own extension.[/color]
>
>
> I know it has nothing to do with python ..
> It just makes me sad to see that python cannot get rid of Visual Studoi
> to be extended . It's just amazing !!
>
> Python is free
> The compiler I use is free
> The code I plan to produce will be free
>
> Where can I find a free version of Visual Studio ?
>
> the only thing I use that is not free is the operating system and it's
> because I had no choice .. my company decided for it
>[color=green]
>> Others suggested trying the ctypes binary installer. Do so.[/color]
>
>
> Yes .. that's what I'm going to do .. but I find it frustrating ..[/color]

Perhaps you aren't aware that Microsoft make a free tool chain available
that can be used to create binary extensions? There has been some
discussion of this recently, and I managed to compile MySQLdb and a
couple of the mx extensions, so it's eminently doable [the PIL is
proving a little less tractable due to dependencies on other libraries
that I have yet to resolve].

The downloads are quite large, but if you want to try it, take a look at

http://www.vrplumber.com/programming/mstoolkit/

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119

[Jean Brouwers]

Re: extending python with a C-written dll


First, you might check the calldll module available here

<http://www.nightmare.c om/software.html>

Take a look at Pyrex, it may fit your requirements to access C from
Python.

<http://nz.cosc.canterb ury.ac.nz/~greg/python/Pyrex/>

If you do not have these already, you may need the Win32 API extension

<http://www.python.org/windows/win32/>


/Jean Brouwers



In article <cq6qi8$hk9$1@s 5.feed.news.ole ane.net>, Jean-Baptiste PERIN
<jb_perin@yahoo .fr> wrote:
[color=blue]
> Grant Edwards a écrit :[color=green]
> > On 2004-12-20, Jean-Baptiste PERIN <jb_perin@yahoo .fr> wrote:
> >
> >[color=darkred]
> >>I'm trying to make a windows dll reachable from a python script..[/color]
> >
> >
> > FWIW, you can call dll's using the ctypes modules without
> > mucking around in C. There may be performance reasons to build
> > a "real" python module, but I haven't run across them yet.
> >[/color]
>
> Not really a performance reason .. but mainly because I have a huge code
> produced in C and no time to port it in python ..
> Moreover .. I need the python program to interact with the C code
> ..(perfom calls to dll-defined functions)
>
> I don't know ctypes module .. do you really think it can help ?
> Do you have a link to quickly learn about it ?
>
> The only thing I found is :
> http://www.python.org/workshops/1994...Classes_5.html
>
>
>
>
>[/color]

[John Machin]

Re: extending python with a C-written dll


Jean-Baptiste PERIN wrote:[color=blue]
> Hi,
>
> I'm trying to make a windows dll reachable from a python script ..[/color]
and[color=blue]
> I'm encountering troubles during link step ..
>
> I use "lcc" to compile and link
> I use python 2.3 under win XP[/color]
[snip][color=blue]
> Here are the errors :
> ---------------------
> Error c:\...\plugins\ hello.c 14 undefined reference to[/color]
__imp__Py_Build Value[color=blue]
> Error c:\...\plugins\ hello.c 46 undefined reference to[/color]
__imp__Py_InitM odule4
[snip][color=blue]
> return Py_BuildValue(" i", 1);[/color]
[snip][color=blue]
> void __declspec(dlle xport) inithello()
> {
> (void)Py_InitMo dule3("hello", helloMethods, MHello__doc__);
> Can anyone help me?[/color]

Yes. Option 1: *Maybe* you can help yourself. Have you not noticed from
the above that the compiler is magically prepending "__imp__" to the
names of external functions? However the .lib that comes with Python
won't have that "__imp__". You would have to bridge that gap, somehow.
Have you searched the lcc documentation? Are you prepared for the next
several problems that may arise?

By the way, lcc also *appears* to change "...Module3 " to "...Module4 "
:-)

Option 2: You have been helped already. There are two free compilers
which work quite well for developing extensions in C for win32
platforms; start reading here:

http://docs.python.org/inst/tweak-flags.html#SECTION000620000000000000000

[Fredrik Lundh]

Re: extending python with a C-written dll

John Machin wrote:
[color=blue]
> By the way, lcc also *appears* to change "...Module3 " to "...Module4 " :-)[/color]

$ more Include/modsupport.h

....

#define Py_InitModule3( name, methods, doc) \
Py_InitModule4( name, methods, doc, (PyObject *)NULL, \
PYTHON_API_VERS ION)

....

</F>

[Jean-Baptiste PERIN]

Re: extending python with a C-written dll

[color=blue]
>
> That's got nothing to do with Python. You have to compile
> extensions using a compiler that has an ABI that's compatible
> with the ABI of the compiler used to compile Python.
>
> You appear to be using a binary distribution of Python that was
> compiled with MSVS 6.0, therefore, you have to compile
> extensions with a compiler that has an ABI compatible with MSVS
> 6.0.
>
> It's no use complaining to us bout your C compiler not being
> ABI compatible with MSVS 6.0.
>
> If you don't like MSVS, then your options are to either use a
> different Python binary distro that was compiled with an ABI
> that's compatible with your C compiler, or build just build it
> yourself using whatever compiler you want. I suspect the latter
> will also require re-compiling Blender.
>[/color]

According to what I read in this forum, MinGW is an interesting option..
do you know if it is ABI compatible with MSVC?

otherwise.. I'm gonna explore the solution which consist in building the
dll with cygwin's gcc .. I've already recompiled Blender with gcc .. I
know that, in Blender code, there are C++ function that are available
from Blender's python ..
I've started exploring the makefile.. apparently they are using dlltool
in conjonction with gcc ..

thank you very much for your help ..

[Jean-Baptiste PERIN]

Re: extending python with a C-written dll

Steve Holden a écrit :
[color=blue]
> Jean-Baptiste PERIN wrote:
>[color=green]
>>[color=darkred]
>>>
>>>> >> error: Python was built with version 6 of Visual Studio, and
>>>> extensions need to
>>>> be built with the same version of the compiler, but it isn't installed.
>>>
>>>
>>>
>>>
>>> How do you expect to be able to write your own extension if you don't
>>> have
>>> the environment to do so in the first place? that error has nothing
>>> to do
>>> with python itself, but with you not having installed the right
>>> compiler.
>>> THe same problem will arise building your own extension.[/color]
>>
>>
>>
>> I know it has nothing to do with python ..
>> It just makes me sad to see that python cannot get rid of Visual
>> Studoi to be extended . It's just amazing !!
>>
>> Python is free
>> The compiler I use is free
>> The code I plan to produce will be free
>>
>> Where can I find a free version of Visual Studio ?
>>
>> the only thing I use that is not free is the operating system and it's
>> because I had no choice .. my company decided for it
>>[color=darkred]
>>> Others suggested trying the ctypes binary installer. Do so.[/color]
>>
>>
>>
>> Yes .. that's what I'm going to do .. but I find it frustrating ..[/color]
>
>
> Perhaps you aren't aware that Microsoft make a free tool chain available
> that can be used to create binary extensions? There has been some
> discussion of this recently, and I managed to compile MySQLdb and a
> couple of the mx extensions, so it's eminently doable [the PIL is
> proving a little less tractable due to dependencies on other libraries
> that I have yet to resolve].
>
> The downloads are quite large, but if you want to try it, take a look at
>
> http://www.vrplumber.com/programming/mstoolkit/
>
> regards
> Steve[/color]


If nothing else works .. I'll try that way ..
So far, I've tried to keep as far as possible from MS things ..

thank you for your help ...

[Jean-Baptiste PERIN]

Re: extending python with a C-written dll

Jean Brouwers a écrit :
[color=blue]
> First, you might check the calldll module available here
>
> <http://www.nightmare.c om/software.html>[/color]

I have one or two things to try before ..but it looks interesting ..
thank you very much ..
[color=blue]
> Take a look at Pyrex, it may fit your requirements to access C from
> Python.
>
> <http://nz.cosc.canterb ury.ac.nz/~greg/python/Pyrex/>[/color]

Yes, pirex looks very powerful ..I heard about it already .. I know
someone who used it .. and the result was just amazing ..
there's also SWIG which looks very sexy
http://www.swig.org/tutorial.html

My current problem is a compilation one .. I need to first solve it
before exploring these tools ..


[color=blue]
>
> If you do not have these already, you may need the Win32 API extension
>
> <http://www.python.org/windows/win32/>
>
>
> /Jean Brouwers[/color]


Thank you very much Jean

[Jean-Baptiste PERIN]

Re: extending python with a C-written dll

[color=blue]
> Note that MinGW32 is a tested path for building (free) C extensions. I
> don't know if anyone has a tested way of using lcc.[/color]

Yes, very interesting
I can already conclude that either I don't know how to use lcc
or it doesn't suit my needs ..
in both case I have to quickly get rid of it .. before I throw the
computer through the window ..

I plan to first test the cygwin's gcc compiler before goign to MinGW32
[color=blue]
> With MinGW32, your build step will be the delightfully easy:
>
> python setup.py build --compiler=mingw3 2[/color]


Thank you very much for your help

[Jean-Baptiste PERIN]

Re: extending python with a C-written dll

John Machin a écrit :
[color=blue]
> Jean-Baptiste PERIN wrote:
>[color=green]
>>Hi,
>>
>>I'm trying to make a windows dll reachable from a python script ..[/color]
>
> and
>[color=green]
>>I'm encountering troubles during link step ..
>>
>>I use "lcc" to compile and link
>>I use python 2.3 under win XP[/color]
>
> [snip]
>[color=green]
>>Here are the errors :
>>---------------------
>>Error c:\...\plugins\ hello.c 14 undefined reference to[/color]
>
> __imp__Py_Build Value
>[color=green]
>>Error c:\...\plugins\ hello.c 46 undefined reference to[/color]
>
> __imp__Py_InitM odule4
> [snip]
>[color=green]
>> return Py_BuildValue(" i", 1);[/color]
>
> [snip]
>[color=green]
>>void __declspec(dlle xport) inithello()
>>{
>> (void)Py_InitMo dule3("hello", helloMethods, MHello__doc__);
>>Can anyone help me?[/color]
>
>
> Yes. Option 1: *Maybe* you can help yourself. Have you not noticed from
> the above that the compiler is magically prepending "__imp__" to the
> names of external functions? However the .lib that comes with Python
> won't have that "__imp__". You would have to bridge that gap, somehow.
> Have you searched the lcc documentation? Are you prepared for the next
> several problems that may arise?[/color]

Indeed, it's something I can't understand (the __imp__ stuff)..
Nothing appear about that in the
I have posted a question on lcc forum .. the author of lcc answered

"This happens because somewhere you are declaring the functions
Py_BuildValue
and
Py_InitModule

as __declspec(dlli mport) "

He's right !!
He adviced me to change that ..
I don't want to make changes to python code .. so .. bye bye lcc

long life to gcc
[color=blue]
> By the way, lcc also *appears* to change "...Module3 " to "...Module4 "
> :-)[/color]

this one is explainable to me .. it is due to the fact that both
function are equivalent

in mod support.h:

#define Py_InitModule3( name, methods, doc) \
Py_InitModule4( name, methods, doc, (PyObject *)NULL, \
PYTHON_API_VERS ION)

[color=blue]
> Option 2: You have been helped already. There are two free compilers
> which work quite well for developing extensions in C for win32
> platforms; start reading here:
>
> http://docs.python.org/inst/tweak-fl...00000000000000
>[/color]

I'm currently attracted by the cygwin solution .. that I find very sexy !!
If I fail .. I will explore the MingW ..

Thank you very much for helping me ..

[Scott David Daniels]

Re: extending python with a C-written dll

Jean-Baptiste PERIN wrote:[color=blue]
> I can already conclude that either I don't know how to use lcc
> or it doesn't suit my needs ..
> I plan to first test the cygwin's gcc compiler before goign to MinGW32[/color]

MinGW32 is "Minimalist Gnu for Windows -- 32-bit," and _works_ at
being ABI compatible with Microsoft's ABI and its C runtime. Cygwin
is a way of building a *nix-compatible toolset _on_ Windows, not _in_
windows. What that means (as far as I understand the issues) is that
you'll need to build a Cygwin python in order to use Cygwin modules.
I put in a bit of effort to make sure Python could build for that
compiler (not by MinGW-specific code, but by cleaning up an issue in
the CPython header file for non-MS compilers). There are toolchain
issues to consider (not just the compiler itself, , but function call
conventions, global access, linker/loader, and debugger symbols and
access are involved).

Python 2.4's jump to a new C compiler on windows is a well-justified
decision (Windows C/C++ 6.0's stuff is getting long in the tooth), but
short-term disruptive to small-time Python C module development. The
change does not portend a continuous series of compiler-jumps, but
rather a hope to not become hopelessly mired in ancient history. It
is getting hard (or is impossible, I don't know which) to purchase
Windows C/C++ 6.0 for money these days, and this will only get harder.

In short, I urge you to go with MinGW if you are hoping to write for
windows with a GNU compiler; you'll be swimming upstream a bit less.

--Scott David Daniels
Scott.Daniels@A cm.Org