A solution to the MSVCRT vs MSVCR71 problem?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • sturlamolden

    #1

    A solution to the MSVCRT vs MSVCR71 problem?


    This question has been asked many times, and last time I was accused of
    spreading FUD. So now I will rather propose a solution.

    The reason for the problem is as follows:

    The binary installer for Python built by te python.org team is compiled
    with Microsoft Visual Studio 2003. It is linked with the C runtime
    msvcrt71.dll. The copyright to msvcrt71.dll is owned by Microsoft, and
    it is not an integral part of the Windows operating system. This leads
    to two distinct problems:

    Problem 1: You want to use a particular compiler to build a Python
    extension library, but it links with the wrong CRT. Typical examples
    are MinGW, Microsoft Visual C++ 6.0 and Microsoft Visual C++ Express.
    There is a remedy for MinGW, but not for the other two.

    Problem 2: You want to distribute a program created Py2Exe, but has no
    license for Visual Studio 2003. You are therefore not allowed to
    redistribute msvcrt71.dll. But without this DLL your program will not
    work. As a side note: Visual Studio 2003 is out of sale, so if you
    don't have it already you may be out of luck.

    The solution: modify the IAT of to use the correct binary.

    The process is really simple:

    Solution to problem 1:

    Compile with your compiler or choice, never mind which CRT are used.
    Use a 'dumpbin' program, find all references to msvcrt in the binary
    DLL file. Create a dll with name "py_crt" that exports these functions
    but redirects them to msvcrt71. That is, in the file py_crt.def we put
    something like

    EXPORTS
    malloc=msvcr71. malloc
    free=msvcr71.fr ee
    etc.

    Compile the DLL py_crt.dll and then open your pyd file in binary mode.
    Exchange all occurances of the string "msvcrt" (or any other CRT name)
    with "py_crt". Now your binary should work just fine. What you need to
    make sure is just that the name of the proxy has the same number of
    letters as the CRT your compiler linked. So if it is msvcrt81.dll, e.g.
    use something like py_crt81.dll instead of py_crt.dll.

    Solution to problem 2:

    This would be the opposite, involving modifying Python25.dll and all
    pyd and dll files gathered by Py2Exe to use msvcrt.dll instead of
    msvcr71.dll. One could e.g. create a DLL called py_cr71.dll that
    redirects CRT calls to msvcrt.dll or any other CRT of choice. In
    py_cr71.def we would then have:

    EXPORTS
    malloc=msvcrt.m alloc
    free=msvcrt.fre e
    etc.

    And then it is simply a matter of modifying the binaries to load
    py_cr71.dll instead of msvcrt71.dll.

    What we need to make this work:

    1. A dumpbin facility similar to the one that ships with Visual Studio
    that can be freely distributed.

    2. A python script that parses the output from dumpbin, creates the
    proxy and redirects the CRT. (I already have such a script working.)

    Finally, a setup script (setup.py) could automate this task for
    "problem 1" above.

    Another variant of the fake-CRT procedure, which would also work, is to
    create a fake msvcrt71.dll that redirects to your CRT of choice, and
    replace the msvcrt71.dll that came with Python with that. That is
    conceptually simpler, as it involves not binary modifications, but
    someone might mistake your msvcrt71.dll for the msvcrt71.dll from
    Microsoft. Also antivirus-software might not like attempts to load such
    a DLL as it would look like your program is hijacked by a trojan
    (although it is not).

    What do you think?

  • =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

    #2
    Re: A solution to the MSVCRT vs MSVCR71 problem?

    sturlamolden schrieb:
    Problem 2: You want to distribute a program created Py2Exe, but has no
    license for Visual Studio 2003. You are therefore not allowed to
    redistribute msvcrt71.dll. But without this DLL your program will not
    work. As a side note: Visual Studio 2003 is out of sale, so if you
    don't have it already you may be out of luck.
    I believe this problem doesn't exist. Licensees of Python are permitted
    to redistribute mscvr71.dll, as long as they redistribute it in order
    to support pythonxy.dll. The EULA says

    # You also agree not to permit further distribution of the
    # Redistributable s by your end users except you may permit further
    # redistribution of the Redistributable s by your distributors to your
    # end-user customers if your distributors only distribute the
    # Redistributable s in conjunction with, and as part of, the Licensee
    # Software, you comply with all other terms of this EULA, and your
    # distributors comply with all restrictions of this EULA that are
    # applicable to you.

    In this text, "you" is the licensee of VS 2003 (i.e. me, redistributing
    msvcr71.dll as part of Python 2.5), and the "Redistributabl e" is
    msvcr71.dll. The "Licensee Software" is "a software application product
    developed by you that adds significant and primary functionality to the
    Redistributable s", i.e. python25.dll.

    IANAL; this is not legal advise.

    Regards,
    Martin

    Comment

    • Gabriel Genellina

      #3
      Re: A solution to the MSVCRT vs MSVCR71 problem?

      At Sunday 21/1/2007 00:07, sturlamolden wrote:
      >Solution to problem 1:
      >
      >Compile with your compiler or choice, never mind which CRT are used.
      >Use a 'dumpbin' program, find all references to msvcrt in the binary
      >DLL file. Create a dll with name "py_crt" that exports these functions
      >but redirects them to msvcrt71. That is, in the file py_crt.def we put
      >something like
      >
      >EXPORTS
      >malloc=msvcr71 .malloc
      >free=msvcr71.f ree
      >etc.
      >
      >Compile the DLL py_crt.dll and then open your pyd file in binary mode.
      >Exchange all occurances of the string "msvcrt" (or any other CRT name)
      >with "py_crt". Now your binary should work just fine. What you need to
      >make sure is just that the name of the proxy has the same number of
      >letters as the CRT your compiler linked. So if it is msvcrt81.dll, e.g.
      >use something like py_crt81.dll instead of py_crt.dll.
      >
      >Solution to problem 2:
      >[modify external references inside python25.dll to use a different runtime]
      This would only work, if runtime dll's were compatibles between them,
      and they are not. You can't blindly redirect a call to msvcr71.__xyz
      to msvcr80.__xyz and expect that to work magically - it may have a
      different number of arguments, or different types, or even may not
      exist anymore.
      (And what about any symbol exported by ordinal?)


      --
      Gabriel Genellina
      Softlab SRL






      _______________ _______________ _______________ _____
      Preguntá. Respondé. Descubrí.
      Todo lo que querías saber, y lo que ni imaginabas,
      está en Yahoo! Respuestas (Beta).
      ¡Probalo ya!


      Comment

      • =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

        #4
        Re: A solution to the MSVCRT vs MSVCR71 problem?

        Gabriel Genellina schrieb:
        This would only work, if runtime dll's were compatibles between them,
        and they are not. You can't blindly redirect a call to msvcr71.__xyz to
        msvcr80.__xyz and expect that to work magically - it may have a
        different number of arguments, or different types, or even may not exist
        anymore.
        Actually, the libraries *are* binary-compatible (on the ABI level). You
        just can't mix two libraries in a single program easily.
        (And what about any symbol exported by ordinal?)
        That doesn't happen for msvcrt, as the import library links by name.

        Regards,
        Martin

        Comment

        • Gabriel Genellina

          #5
          Re: A solution to the MSVCRT vs MSVCR71 problem?

          At Sunday 21/1/2007 05:38, Martin v. Löwis wrote:
          >Gabriel Genellina schrieb:
          This would only work, if runtime dll's were compatibles between them,
          and they are not. You can't blindly redirect a call to msvcr71.__xyz to
          msvcr80.__xyz and expect that to work magically - it may have a
          different number of arguments, or different types, or even may not exist
          anymore.
          >
          >Actually, the libraries *are* binary-compatible (on the ABI level). You
          >just can't mix two libraries in a single program easily.
          That's a good thing - but is this just by accident, or is documentedsomew here?
          I remember that I tried something like that in
          the past, and failed. (Perhaps earlier versions
          where not fully backwards compatible - or I didn't try hard enough thattime).


          --
          Gabriel Genellina
          Softlab SRL






          _______________ _______________ _______________ _____
          Preguntá. Respondé. Descubrí.
          Todo lo que querías saber, y lo que ni imaginabas,
          está en Yahoo! Respuestas (Beta).
          ¡Probalo ya!


          Comment

          • sturlamolden

            #6
            Re: A solution to the MSVCRT vs MSVCR71 problem?


            Gabriel Genellina wrote:
            This would only work, if runtime dll's were compatibles between them,
            and they are not.
            It is standard C, defined by ANSI and ISO.

            Comment

            • =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

              #7
              Re: A solution to the MSVCRT vs MSVCR71 problem?

              Gabriel Genellina schrieb:
              That's a good thing - but is this just by accident, or is documented
              somewhere?
              It's documented somewhere (although I can't find the documentation
              right now - it explains how you can link object files from a static
              library compiled with an older compiler version against a new version
              of the C library).
              I remember that I tried something like that in the past, and failed.
              (Perhaps earlier versions where not fully backwards compatible - or I
              didn't try hard enough that time).
              It's been that way for ages, atleast since they started to support
              32-bit code. So if you had problems, they might have had a different
              source.

              It's a different thing for the C++ libraries, though.

              Regards,
              Martin

              Comment

              • =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

                #8
                Re: A solution to the MSVCRT vs MSVCR71 problem?

                sturlamolden schrieb:
                >This would only work, if runtime dll's were compatibles between them,
                >and they are not.
                >
                It is standard C, defined by ANSI and ISO.
                ANSI and ISO don't define the ABI, though. For example, the definition
                of the FILE type might (and does) vary across compilers, even if all
                these compilers implement C99.

                Regards,
                Martin

                Comment

                Working...