Win32 Libs for 2.4

[Robin Becker]

Does anyone know if it is feasible to have static libraries for both 2.3 and 2.4
compatible extensions. I'm worrying about libjpeg etc in a win32 environment.

--
Robin Becker

[Daniel Dittmar]

Re: Win32 Libs for 2.4

Robin Becker wrote:[color=blue]
> Does anyone know if it is feasible to have static libraries for both 2.3
> and 2.4 compatible extensions. I'm worrying about libjpeg etc in a win32
> environment.[/color]

Could you be a bit more specific:

do you want to create a binary python extension that is compatible with
both Python 2.3 and Python 2.4.

Or do you want to create a static lib containing both the original C lib
and the code for your extension? This Lib should then be linked with the
proper python2*.lib to create a version specific binary extension.

Or something else?

The first one is not possible as far as I know because python2*.lib
tries to load a specific python2*.dll. You could try to replace static
DLL import with dynamic DLL import (loading the DLL explizit and getting
the function pointers by name). That would be a lot of work and I don't
know how well it works for global variables (like all the type objects).
On the other hand, many people find it annoying that you have to build
new exte8ensions for every Python version, so this could be something a
lot of people have an interest in.

I solved this problem by putting .pyd for several versions into the
download. The right one is picked at runtime by checking sys.version.

Creating one lib which is then linked to python2*.lib works most of the
time, but you'll probably get a warning during loading of the module
that the extension has been compiled for a different version of the API.

Daniel

[Robin Becker]

Re: Win32 Libs for 2.4

Daniel Dittmar wrote:[color=blue]
> Robin Becker wrote:
>[color=green]
>> Does anyone know if it is feasible to have static libraries for both
>> 2.3 and 2.4 compatible extensions. I'm worrying about libjpeg etc in a
>> win32 environment.[/color]
>
>
> Could you be a bit more specific:
>
> do you want to create a binary python extension that is compatible with
> both Python 2.3 and Python 2.4.
>[/color]

not this one
[color=blue]
> Or do you want to create a static lib containing both the original C lib
> and the code for your extension? This Lib should then be linked with the
> proper python2*.lib to create a version specific binary extension.
>[/color]

not this one either
[color=blue]
> Or something else?
>[/color]

actually I want to build the PIL extension for 2.4 as pyd and include various
libraries eg zlib and jpeg. To avoid the missing dlls issue we have done this in
the past by incorporating the zlib/jpeg code using static libraries for both
zlib and jpeg.

It seems I can use the static lib built with MSC 6 with the extension code
compiled with MSC 7.1. At least the extnsion build doesn't complain. Whether
this is really allowed is another matter.
.....[color=blue]
> Daniel[/color]


--
Robin Becker

[Daniel Dittmar]

Re: Win32 Libs for 2.4

Robin Becker wrote:[color=blue]
> actually I want to build the PIL extension for 2.4 as pyd and include
> various libraries eg zlib and jpeg. To avoid the missing dlls issue we
> have done this in the past by incorporating the zlib/jpeg code using
> static libraries for both zlib and jpeg.
>
> It seems I can use the static lib built with MSC 6 with the extension
> code compiled with MSC 7.1. At least the extnsion build doesn't
> complain. Whether this is really allowed is another matter.[/color]

I guess that it won't work if something malloc'ed from the MSC 6 runtime
is free'd by the MSC 7.1 runtime.

zlib is also part of the Python sources on Windows, so it shouldn't be a
problem to build it for PIL.

I'm not sure what effort would be needed to build jpeg lib using MSC
7.1. My guess would be that there is hardly any effort if an nmake file
exists and a bit more work if you only have MS VC project file.

Daniel

[Robin Becker]

Re: Win32 Libs for 2.4

Daniel Dittmar wrote:[color=blue]
> Robin Becker wrote:
>[color=green]
>> actually I want to build the PIL extension for 2.4 as pyd and include
>> various libraries eg zlib and jpeg. To avoid the missing dlls issue we
>> have done this in the past by incorporating the zlib/jpeg code using
>> static libraries for both zlib and jpeg.
>>
>> It seems I can use the static lib built with MSC 6 with the extension
>> code compiled with MSC 7.1. At least the extnsion build doesn't
>> complain. Whether this is really allowed is another matter.[/color]
>
>
> I guess that it won't work if something malloc'ed from the MSC 6 runtime
> is free'd by the MSC 7.1 runtime.[/color]
.......

I thought that static .libs didn't make reference to the dll's they need; isn't
that done at load time?

As for rebuilding that's easy; the hard part is having all the versions
available at once. I prefer to keep common code in one place so would prefer to
have only one static lib for these non version dependant things.

[color=blue]
>
> Daniel[/color]


--
Robin Becker

[Daniel Dittmar]

Re: Win32 Libs for 2.4

Robin Becker wrote:[color=blue][color=green]
>> I guess that it won't work if something malloc'ed from the MSC 6
>> runtime is free'd by the MSC 7.1 runtime.[/color]
>
> I thought that static .libs didn't make reference to the dll's they
> need; isn't that done at load time?[/color]

You're right, DLL-phobia made my thinking less than precise.

The other things I could think of might not apply in this case:
- Routines accessing the elements of FILE* directly. I though library
designers had learned not to export this, but stdio.h still shows it and
some macros from that header file might use it (a problem only if the
definitions in MSC 6 and MSC 7.1 differ)
- calling conventions: in the past, that was a problem mostly with
functions returning structs. (Meaning caller and callee would implement
it differently if they were compiled by different compilers)

Daniel

[Martin v. Löwis]

Re: Win32 Libs for 2.4

Robin Becker wrote:[color=blue]
> I thought that static .libs didn't make reference to the dll's they
> need; isn't that done at load time?[/color]

Unfortunately, thanks to Microsoft's infinite wisdom, static libs
*do* reference DLLs. The C lib headers contain things like

#pragma lib("msvcrt.lib ") // or some such

The compiler translates this into a reference to the DLL in the
object file, which the linker will then translate into a reference
to the DLL when linking, even if the DLL was not giving on the linker
command line.

However, I am uncertain how this works with multiple VC versions -
whether the DLL reference is CRT version independent or not.
You should view the resulting extension module in depends.exe,
and check whether it refers to multiplce CRT dlls (which would be
bad).

Regards,
Martin

[Robin Becker]

Re: Win32 Libs for 2.4

Martin v. Löwis wrote:[color=blue]
> Robin Becker wrote:
>[color=green]
>> I thought that static .libs didn't make reference to the dll's they
>> need; isn't that done at load time?[/color]
>
>
> Unfortunately, thanks to Microsoft's infinite wisdom, static libs
> *do* reference DLLs. The C lib headers contain things like
>
> #pragma lib("msvcrt.lib ") // or some such
>[/color]

according to my docs

there's something called a comment pragma used thusly

#pragma comment( lib, "xxx.lib" )

which places a search record into the obj file. I don't think this forces the
linker to load stuff from this module although I can see that it might be
dangerous depending on which obj files are seen first.

[color=blue]
> The compiler translates this into a reference to the DLL in the
> object file, which the linker will then translate into a reference
> to the DLL when linking, even if the DLL was not giving on the linker
> command line.[/color]

I believe this is a weak reference from the documentation.
[color=blue]
>
> However, I am uncertain how this works with multiple VC versions -
> whether the DLL reference is CRT version independent or not.
> You should view the resulting extension module in depends.exe,
> and check whether it refers to multiplce CRT dlls (which would be
> bad).
>[/color]

well in my 2.4 _imaging.pyd I see direct references to only msvcr71.dll,
but there is a direct reference to USER32.dll which references ADVAPI32.dll
which references WINTRUST.dll which then references msvcrt.dll. Fortunately I
don't think this is a problem as Python24.dll also references USER32.dll directly.

Perhaps a genius level cretifiable (whoops Freudian mispelling of certified)
knows whether this is OK.

[color=blue]
> Regards,
> Martin[/color]


--
Robin Becker

[Martin v. Löwis]

Re: Win32 Libs for 2.4

Robin Becker wrote:[color=blue]
> I don't think this
> forces the linker to load stuff from this module although I can see that
> it might be dangerous depending on which obj files are seen first.[/color]

I think you are wrong. In the object, there will be simply a linker
command line option encoded; do "dumpbin /all foo.obj" to see the
linker options.
[color=blue]
> I believe this is a weak reference from the documentation.[/color]

What makes you think so?: the linker option in the object file will
be added to the linker line, unconditionally .
[color=blue]
> Fortunately I don't think this is a problem as Python24.dll also
> references USER32.dll directly.[/color]

Yes, that user32.dll indirectly uses msvcrt.dll should be no problem.

Regards,
Martin

[Robin Becker]

Re: Win32 Libs for 2.4

Martin v. Löwis wrote:[color=blue]
> Robin Becker wrote:
>[color=green]
>> I don't think this forces the linker to load stuff from this module
>> although I can see that it might be dangerous depending on which obj
>> files are seen first.[/color]
>
>
> I think you are wrong. In the object, there will be simply a linker
> command line option encoded; do "dumpbin /all foo.obj" to see the
> linker options.[/color]

yes I understood that, but I thought

Linker Directives
-----------------
-defaultlib:MSVC RT
-defaultlib:OLDN AMES

doesn't force the code to use entrypoints in those libs so I consider it
weak. Anyhow despite these records being in the static library object
files it doesn't seem to have forced msvcrt to be used directly. I
assume that this is because earlier objects have brought in a library
which already satisfies the externals. Or perhaps I'm just lucky in that
the libjpeg routines don't actually use anything from msvcrt.

.......[color=blue]
> Yes, that user32.dll indirectly uses msvcrt.dll should be no problem.
>[/color]
[color=blue]
> Regards,
> Martin[/color]


--
Robin Becker