MinGW and Python

[sturlamolden]

Re: MinGW and Python


Edward Elliott wrote:
[color=blue]
> Couldn't you compile the msvc-python code under gcc/mingw?[/color]

Yes I could, but I cannot compile the code under msvc for comparison. I
only have MinGW. If build the mingw binary then someone else has to
build the msvc binary for comparison. Then we could do pybenches on the
same computer.

Uh .. I actually think it could be an EULA violation to publish mingw
vs. msvc benchmarks without permission from Microsoft. I don't want to
part of anything illegal or have M$ lawyers breathing down my back. If
we are going to do this, then we must do it properly and get the
permissions we need.

[Ross Ridge]

Re: MinGW and Python


Martin v. Löwis wrote:[color=blue]
> Well, there is no native C library on Microsoft Windows: the system
> simply doesn't include an official C library (I know there is crtdll.dll
> and msvcrt.dll, but these aren't "endorsed" system C libraries).[/color]

MSVCRT.DLL has been a standard system compent of Windows since at least
Windows 98. Many other system components depend on it. Essentially,
MSVCRT.DLL is an "undocument ed" part of the Windows API. It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.

Ross Ridge

[sjdevnull@yahoo.com]

Re: MinGW and Python

sturlamolden wrote:[color=blue]
> But as long as the cygwin dll is there, it creates an overhead for any
> system call. The worst overhead is associated with the Unix fork()
> system call, which Cygwin must emulate as there are no Win32
> equivalent. In particular, a fork() on Unix will be optimized with
> copy-on-write semantics in the kernel, but Cygwin cannot emulate that
> as it has no access to the Windows kernel.[/color]

Cygwin doesn't do COW fork() for historical reasons. Windows 95/98
didn't expose support for such a thing, but NT, XP, etc do (pass NULL
as the SectionHandle parameter to ZwCreateProcess or the older
NtCreateProcess to do a COW duplicate of the calling process).

Cygwin at one point had a fork() implementation using this in the
development tree (or being discussed on the mailing lists) but decided
not to use it since it wasn't much faster than full-copy fork for some
reason, and it would've split the code for NT/95.

[Edward Elliott]

Re: MinGW and Python

sturlamolden wrote:[color=blue]
> Edward Elliott wrote:[color=green]
>> Couldn't you compile the msvc-python code under gcc/mingw?[/color]
>
> Yes I could, but I cannot compile the code under msvc for comparison. I
> only have MinGW. If build the mingw binary then someone else has to
> build the msvc binary for comparison. Then we could do pybenches on the
> same computer.[/color]

Sorry, I didn't mean you personally, I meant "you" in the general sense.

[color=blue]
> Uh .. I actually think it could be an EULA violation to publish mingw
> vs. msvc benchmarks without permission from Microsoft. I don't want to
> part of anything illegal or have M$ lawyers breathing down my back. If
> we are going to do this, then we must do it properly and get the
> permissions we need.[/color]

Why, because of a "no benchmarks" clause in the EULA? First, I'm fairly
certain that clause is uneforceable. Second, it's not worth MS's time to
come after you. Third, even if they did, they'd have an incredibly tough
time showing that any harm occured from your actions, leaving them with no
recovery. Fourth, if it somehow came to that, you'd have organizations
like the EFF lining up at your door to take the case gratis. I'm assuming
US law here.

Not saying you should do it, just that the risk in my opinion is extremely
low. I am a law student, but this is not legal advice.

[Edward Elliott]

Re: MinGW and Python

"Martin v. Löwis" wrote:[color=blue]
> Well, you are not compiling with neither mingw, nor cygwin; you are
> compiling with gcc in either case.[/color]

touche, mr. pedant. :)
[color=blue]
> Well, there is no native C library on Microsoft Windows: the system
> simply doesn't include an official C library (I know there is crtdll.dll
> and msvcrt.dll, but these aren't "endorsed" system C libraries).[/color]

don't know what you mean by "endorsed". does it lack features of the C89
ANSI standard?
[color=blue]
> For Windows, that would require not to use any of the standard C
> functionality, since the system doesn't provide that functionality out
> of the box.[/color]

That would be a problem then. So what happens when you compile python with
msvc, and why can't mingw just replicate that?

[Edward Elliott]

Re: MinGW and Python

Ross Ridge wrote:[color=blue]
> MSVCRT.DLL ... It's not
> exactly "endorsed", Microsoft would rather you use it's current
> compiler and runtime, but it is the standard "official" Windows system
> C library.[/color]

Does it comply with the ANSI C89 standard? I'm still not seeing why mingw
can't just link python to it.

[sturlamolden]

Re: MinGW and Python


Edward Elliott wrote:
[color=blue]
> Sorry, I didn't mean you personally, I meant "you" in the general sense.[/color]

OK :-)

I've just tried to build Python 2.4.3 with MinGW (MSYS 1.0.10, GCC
3.4.2):

$ ./configure --prefix=/c/Python243-mingw
$ make

The build then failed on posixmodule.c.


Sturla Molden


../Modules/posixmodule.c:3 08: warning: function declaration isn't a
prototype
../Modules/posixmodule.c: In function `posix_ttyname' :
../Modules/posixmodule.c:1 162: warning: implicit declaration of function
`ttyname'
../Modules/posixmodule.c:1 162: warning: assignment makes pointer from
integer without a cast
../Modules/posixmodule.c: In function `posix_mkdir':
../Modules/posixmodule.c:1 791: error: too many arguments to function
`mkdir'
../Modules/posixmodule.c: In function `posix_execv':
../Modules/posixmodule.c:2 199: warning: passing arg 2 of `execv' from
incompatible pointer type
../Modules/posixmodule.c: In function `posix_execve':
../Modules/posixmodule.c:2 332: warning: passing arg 2 of `execve' from
incompatible pointer type
../Modules/posixmodule.c:2 332: warning: passing arg 3 of `execve' from
incompatible pointer type
../Modules/posixmodule.c: In function `posix_fork':
../Modules/posixmodule.c:2 847: warning: implicit declaration of function
`fork'
../Modules/posixmodule.c: In function `posix_openpty' :
../Modules/posixmodule.c:2 909: error: `O_NOCTTY' undeclared (first use
in this function)
../Modules/posixmodule.c:2 909: error: (Each undeclared identifier is
reported only once
../Modules/posixmodule.c:2 909: error: for each function it appears in.)
../Modules/posixmodule.c:2 912: error: `SIGCHLD' undeclared (first use in
this function)
../Modules/posixmodule.c:2 914: warning: implicit declaration of function
`grantpt'
../Modules/posixmodule.c:2 919: warning: implicit declaration of function
`unlockpt'
../Modules/posixmodule.c:2 924: warning: implicit declaration of function
`ptsname'
../Modules/posixmodule.c:2 924: warning: assignment makes pointer from
integer without a cast
../Modules/posixmodule.c:2 931: warning: implicit declaration of function
`ioctl'
../Modules/posixmodule.c:2 931: error: `I_PUSH' undeclared (first use in
this function)
../Modules/posixmodule.c: In function `posix_getegid' :
../Modules/posixmodule.c:2 973: warning: implicit declaration of function
`getegid'
../Modules/posixmodule.c: In function `posix_geteuid' :
../Modules/posixmodule.c:2 986: warning: implicit declaration of function
`geteuid'
../Modules/posixmodule.c: In function `posix_getgid':
../Modules/posixmodule.c:2 999: warning: implicit declaration of function
`getgid'
../Modules/posixmodule.c: In function `posix_getppid' :
../Modules/posixmodule.c:3 121: warning: implicit declaration of function
`getppid'
../Modules/posixmodule.c: In function `posix_getuid':
../Modules/posixmodule.c:3 163: warning: implicit declaration of function
`getuid'
../Modules/posixmodule.c: In function `posix_kill':
../Modules/posixmodule.c:3 193: warning: implicit declaration of function
`kill'
../Modules/posixmodule.c: In function `posix_wait':
../Modules/posixmodule.c:4 970: warning: implicit declaration of function
`wait'
../Modules/posixmodule.c: In function `posix_pipe':
../Modules/posixmodule.c:5 511: warning: implicit declaration of function
`pipe'
../Modules/posixmodule.c: At top level:
../Modules/posixmodule.c:3 82: warning: 'posix_error_wi th_filename'
defined but not used
../Modules/posixmodule.c:5 34: warning: 'posix_fildes' defined but not
used
../Modules/posixmodule.c:6 223: warning: 'conv_confname' defined but not
used
../Modules/posixmodule.c:7 126: warning: 'setup_confname _table' defined
but not used
make: *** [Modules/posixmodule.o] Error 1

[Neil Hodgson]

Re: MinGW and Python

sturlamolden:
[color=blue]
> Uh .. I actually think it could be an EULA violation to publish mingw
> vs. msvc benchmarks without permission from Microsoft. I don't want to
> part of anything illegal or have M$ lawyers breathing down my back. If
> we are going to do this, then we must do it properly and get the
> permissions we need.[/color]

The EULA attempted to stop disclosing benchmarks of .NET code, not
C++ code. The most recent version of the EULA has replaced that with
permission to disclose benchmarks of .NET code provided some conditions
are met: full disclosure of benchmark code and environment; latest
release of .NET; compliance with 'best practices'. Its still stupid as
it would be in Microsoft's interest to encourage benchmarking .NET but
not quite as stupid as before.

Neil

[sturlamolden]

Re: MinGW and Python


Martin v. Löwis wrote:
[color=blue]
> Please believe me: there is currently no build process that
> gives the same results as the build process used. It might
> be fairly easy to create one, but none exists as of today.[/color]

I tried to build with MinGW this eveing and it failed. I believe you We
need to make a build process for MinGW.

[Robert Kern]

Re: MinGW and Python

Edward Elliott wrote:[color=blue]
> Ross Ridge wrote:
>[color=green]
>>MSVCRT.DLL ... It's not
>>exactly "endorsed", Microsoft would rather you use it's current
>>compiler and runtime, but it is the standard "official" Windows system
>>C library.[/color]
>
> Does it comply with the ANSI C89 standard? I'm still not seeing why mingw
> can't just link python to it.[/color]

It can.

--
Robert Kern
robert.kern@gma il.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[sturlamolden]

Re: MinGW and Python

Look what I just found:

http://jove.prohosting.com/iwave/ipython/pyMinGW.html

A build process for python 2.4.2 (i.e. not the latest) for MinGW.

[Alex Martelli]

Re: MinGW and Python

Edward Elliott <nobody@127.0.0 .1> wrote:
[color=blue]
> Alex Martelli wrote:[color=green]
> > about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3[/color]
>
> this is the second time I've seen that 14% figure. OOC, where does it come
> from? the data sets you posted show an average 12.6% speedup. 14 is an
> odd way to round. :)[/color]

I believe 12.6% is the result if you pick the faster speed as a
denominator (and so it's the right figure for a _slowdown_,
hypothetically moving from fast to slow case), 14% if you pick the
slower speed as a denominator (i.e., for a _speedup_).

Of course if you work with times rather than speeds it's the other way
around.

[color=blue]
> I don't think it's very useful to talk about average speedups from a
> benchmark of equally-weighted feature tests. the data shows wildly varying
> differences in performance for each test. a real-world application could
> be much slower or much faster on either platform depending on its feature
> mix. not the type of thing that's amenable to expression as a single
> value.[/color]

I believe the single figure is a useful summary. Even the most
sophisticated benchmarks are eventually boiled down to single figures,
as in "so many SPECmarks" etc, because in everyday discourse a scalar is
what you can reasonably discuss. Sure, philosophically speaking it
makes no sense to say that a compiler is better or worse than another at
optimization, without adding a lawyer's brief worth of qualifications
each and every time. In real life, it's a useful and practical
temporary simplification, and engineers (as opposed to mathematicians
and philosophers) have a bias towards practical usefulness.


Alex

[Alex Martelli]

Re: MinGW and Python

Edward Elliott <nobody@127.0.0 .1> wrote:
[color=blue]
> Alex Martelli wrote:[color=green]
> > At the same time, if the 14% slowdown is representative, then it's not
> > true that the compiler responsible for it "optimizes as well" as the
> > other; indeed, "does not optimize particularly well", under such a
> > hypothesis, would be far from a "beyond BS" assertion.[/color]
>
> Maybe someone with access to Windows and some free time could compare
> pybench compiled under msvc and mingw. The best way to test the compiler
> is to eliminate all other variables.[/color]

I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.
Unfortunately, I suspect the ex-officio gcc defenders will only react by
finding yet another quibble (anything to avoid admitting that gcc may
not be as good at optimizing as a _Microsoft_ product -- perish the
thought!), therefore I'm disinclined to invest my scarce free time in
that particular endeavour.


Alex

[Edward Elliott]

Re: MinGW and Python

Alex Martelli wrote:[color=blue]
> I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.[/color]

Yep
[color=blue]
> Unfortunately, I suspect the ex-officio gcc defenders will only react by
> finding yet another quibble (anything to avoid admitting that gcc may
> not be as good at optimizing as a _Microsoft_ product -- perish the
> thought!), therefore I'm disinclined to invest my scarce free time in
> that particular endeavour.[/color]

No problem. I don't expect anyone to follow through, it was more a
suggestion to forestall rampant speculation. :)

[Edward Elliott]

Re: MinGW and Python

Alex Martelli wrote:[color=blue]
> I believe the single figure is a useful summary. Even the most
> sophisticated benchmarks are eventually boiled down to single figures,
> as in "so many SPECmarks" etc, because in everyday discourse a scalar is
> what you can reasonably discuss. Sure, philosophically speaking it
> makes no sense to say that a compiler is better or worse than another at
> optimization, without adding a lawyer's brief worth of qualifications[/color]
^^^^^^^^^^^^^^
hey is that a shot at me? ;)
[color=blue]
> each and every time. In real life, it's a useful and practical
> temporary simplification, and engineers (as opposed to mathematicians
> and philosophers) have a bias towards practical usefulness.[/color]

I agree for benchmarks in general. It's more this particular benchmark I
object to as not being representative. It's like the difference between
SPECmark testing various CPU functions and Winstone measuring real-world
application performance. If you're a CPU designer counting coup, SPECmark
matters. For just about everyone else, Winstone tells you more. And when
you're talking about small (less than an order of magnitude) differences in
SPECmark, platform/system/application issues becomes the dominant factor in
performance. A CPU that's twice as fast won't help my i/o-bound server.

From an engineering standpoint, the pybench number isn't that useful to me.
It tells me little about the practical speed of my application on two
different python interpreters. That's all I'm saying. No need to sic the
philosophers on me (a fate I would not wish on my worst enemy). :)