Python 2.3.2 & Netware

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Jeff Davey

    Python 2.3.2 & Netware

    I spent the day porting & building a .a that I could link into an NLM
    (statically) and run on Netware, for the purpose of using Python in an
    application.

    So far, everything (from the compiling process) works great. I built a
    little test.nlm to see if everything works good:

    #include <Python.h>

    int main(int argc, char *argv[])
    {
    Py_Initialize() ;
    PyRun_SimpleStr ing("print 'Hi there'\n");
    Py_Finalize();
    return 0;
    }

    I then loaded up this NLM (of like, 3.8MB or so) into protected memory
    space on a Netware 5.1 server.

    Near instantly, I get an abend of such:
    Removed address space because of memory protection violation
    Reason: Free detected corrupt preceeding redzone for node

    So, I wade through the Python source to discover what's happening:

    Py_Initialize() is called
    From there we get down to _Py_ReadyTypes( );

    In _Py_ReadyTypes we get to the first line:
    if (PyType_Ready(& PyType_Type) <0)


    In PyType_Ready we initialize the base class (ie: recurse once), and
    proceed to the following (about to do the type type):

    if (add_operators( type) < 0)
    goto error;


    Following that we get to add_operators:
    descr = PyDescr_NewWrap per(type, p, *ptr);

    and further down in PyDescr_NewWrap per..
    descr = (PyWrapperDescO bject *)descr_new(&Py WrapperDescr_Ty pe, type,
    base->name);
    Just to note, the name here is '__cmp__'

    Then in descr_new (getting close now, I promise! :):
    descr->d_name = PyString_Intern FromString(name );

    In that particular function, the abend seems to happen in
    PyString_Intern InPlace(&s)

    So Following that, and this is where the abend happens,

    In PyString_Intern InPlace(PyObjec t **p) you have this:

    if ((t = PyDict_GetItem( interned, (PyObject *)s)) != NULL {
    Py_INCREF(t);
    Py_DECREF(*p); // THE ABEND IS HERE
    *p = t;
    return;
    }

    The abend happens on the Py_DECREF(*p).

    Now, I had a hard time trying to figure out exactly what this type of
    abend means, but from what I understand, that peice of memory has been
    corrupted at some point.

    When I ported this, I modified the pyconfig.h by hand, perhaps this
    problem is caused by inserting a wrong value (which I sort of doubt
    right now...) into the 'type' sizes ?

    Let me know if you have any ideas....

    Thanks!

    Jeff Davey

  • Brad Clements

    #2
    Re: Python 2.3.2 &amp; Netware

    Hmm, I ported Python 1.5.2 to CLIB NetWare 3 years ago (Zope too) and I did
    not have this problem. I don't think the code you're looking at has changed
    very much since then.

    I don't think this statement helps.. Could you try linking your .a with
    Python's main to get a command line interface?

    i.e. removing your existing 'main' code from the picture might clarify. 3.8
    meg seems a bit large, that's with debug info?

    --
    Novell DeveloperNet Sysop #5




    Comment

    • Jeff Davey

      #3
      Re: Python 2.3.2 &amp; Netware


      "Brad Clements" <bkc@Murkworks. com> wrote in message
      news:mailman.62 7.1068565685.70 2.python-list@python.org ...[color=blue]
      > I don't think this statement helps.. Could you try linking your .a with
      > Python's main to get a command line interface?[/color]

      By python's main, I'm assuming you mean Modules/ccpython.cc ? I converted
      this to a C file (it's pretty small) and ran that.... It gets to the same
      place before it dies.
      [color=blue]
      > i.e. removing your existing 'main' code from the picture might clarify.[/color]
      3.8[color=blue]
      > meg seems a bit large, that's with debug info?[/color]

      Sorry, I didn't realize I had left the -g in there, it's about 800k without.

      Jeff


      Comment

      • Jeff Davey

        #4
        Re: Python 2.3.2 &amp; Netware

        Well, I've gotten a little bit further,

        Basically, in the pyconfig.h I changed #define WITH_PYMALLOC to #undef
        WITH_PYMALLOC. There was something about PyObject_Malloc that returns
        improper memory with Netware.

        The new abend I'm getting is a General Protection Fault :), tracking that
        one down right now. Will let you know when I figure it out.

        Jeff


        Comment

        • Jeff Davey

          #5
          Re: Python 2.3.2 &amp; Netware


          "Jeff Davey" <j@submersion.c om> wrote in message
          news:SGbsb.2884 0$jy.14513@clgr ps13...[color=blue]
          > The new abend I'm getting is a General Protection Fault :), tracking that
          > one down right now. Will let you know when I figure it out.[/color]

          I have a bunch of printfs going to see what's up, basically, it finished the
          'Type' type from _Py_ReadyTypes, and starts into bool... Interesting enough,
          the printf I have at the end of the PyType_Ready function, gets corrupted
          (as in, the static string is overwritten by something else) as it finished
          the 'Type' type. It then proceeds to try and PyType_Ready the bool type, it
          gets to the bases = Py_BuildValue(" (0)", .base); line, inside there it dies:

          bases = type->tp_bases;
          if (bases == NULL) {
          if (bases == NULL)
          bases = PyTuple_New(0);
          else
          bases = Py_BuildValue(" 0", base); // ABEND is HERE
          }
          I was going to keep following, but I've sort of come to the conclusion
          there's some major memory management problems (overwriting buffers, or
          something along that way) with getting Python to run on Netware.

          Now, I'm not sure if it'some setting I shouldn't have enabled in the
          pyconfig.h, or something else, but I do know that Netware is a LOT pickier
          than Linux when it comes to memory.

          Any ideas maybe why I'm having so many problems with Python & Memory
          Mangament ?

          Jeff



          Comment

          • Jeff Davey

            #6
            Re: Python 2.3.2 &amp; Netware

            Wooot! Got it going :)

            Another braindead mistake in the pyconfig.h.

            Now I'm hoping the 300KB memory leak Netware is reporting is because Python
            can't find any of the libraries it needs.

            but, I have a python.nlm running on my Netware box :)

            Jeff


            Comment

            • Brad Clements

              #7
              Re: Python 2.3.2 &amp; Netware

              Congrats.. LIBC or CLIB?

              Now the real problem is, how will you handle global data in dynamically
              loaded extension modules? Since library NLMs do not have per-process private
              global data space.

              --
              Novell DeveloperNet Sysop #5

              _
              "Jeff Davey" <j@submersion.c om> wrote in message
              news:Wpfsb.3092 0$jy.29147@clgr ps13...[color=blue]
              > Wooot! Got it going :)
              >
              > Another braindead mistake in the pyconfig.h.
              >
              > Now I'm hoping the 300KB memory leak Netware is reporting is because[/color]
              Python[color=blue]
              > can't find any of the libraries it needs.
              >
              > but, I have a python.nlm running on my Netware box :)
              >
              > Jeff
              >
              >
              > --
              > http://mail.python.org/mailman/listinfo/python-list
              >[/color]




              Comment

              • Jeff Davey

                #8
                Re: Python 2.3.2 &amp; Netware

                Brad Clements wrote:[color=blue]
                > Congrats.. LIBC or CLIB?[/color]

                LibC, it would have been a super PITA to go CLIB, and really, I didn't
                really care about supporting NW 4.11 anymore. Novell as well recommends
                to start all software development on LibC.

                The nice thing about porting it to LibC is a lot of the libraries that
                Python needed were already there (ie: pthreads, etc). LibC has a lot
                'unixishness' in it.
                [color=blue]
                > Now the real problem is, how will you handle global data in dynamically
                > loaded extension modules? Since library NLMs do not have per-process private
                > global data space.[/color]

                Not quite sure, haven't had time to continue on it. Extremely new to
                Python itself, so trying to figure out what Python itself needs. (Ie,
                it's Lib, compiled modules, etc).

                One thing that is interesting is LibC has support for dlopen & friends.


                Jeff

                Comment

                Working...