Python Memory Leak using SWIG

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • cody314@gmail.com

    Python Memory Leak using SWIG

    Versions:
    Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
    SWIG Version 1.3.20

    Hello I have some code that wraps a C++ library so I may use it in
    python. The code basically just gets some data and puts it in the
    PyArrayObject* which is returned as a PyObject*.
    I then call it from python like so:
    self.c = __f2.fdct2_wrap per(x,self.nbs, self.nba,self.a c)

    I then loop (which pretty much only calls this function) over and
    over. I put the variable as a self.c hoping the garbage collector
    would know how to delete it after the class goes out of scope. I also
    tried explicitly deleting the variable (del var) in the loop with no
    success. In all cases quiet a large memory leak occurs (and grows
    rather quickly).

    I believe this is comming from the fact that the thing that is
    returned is a pointer to the data. So the returning object is a
    pointer. The python garbage collector then doesn't know how to delete
    this structure and probably (maybe) just deletes the pointer after the
    class goes out of scope. Leave the data there and causing the memory
    leak issue. I however doesn't know how to tell python that this
    variable is a pointer and to delete whats going to it. Or perhaps
    tell SWIG to delete the data, and return the structure some other way?

    Here is the c++ wrapping code, perhaps there is an easy way to fix
    this memory leak (I believe a lot of SWIG people probably do this)
    perhaps some function call from the python? or some special call from
    the SWIG? Thanks a bunch!

    // set up the list output
    PyListObject* out;
    PyArrayObject* output;
    out = (PyListObject*) PyList_New(0);
    npy_intp dims[2];
    int i,j;

    for(i=0;i<g.siz e();i++)
    {
    // append a list for this scale
    PyList_Append(( PyObject*) out,PyList_New( 0));

    for(j=0;j<g[i].size();j++)
    {
    // set the dimensions for this scale & angle
    dims[0] = g[i][j].n();
    dims[1] = g[i][j].m();

    // make an array for this scale & angle's data
    output = (PyArrayObject* ) PyArray_SimpleN ewFromData(2, dims,
    PyArray_CDOUBLE ,g[i][j]._data);
    Py_INCREF((PyOb ject*) output);

    // append this angle's data to the list for this scale
    PyList_Append(P yList_GetItem(( PyObject*) out,i),(PyObjec t*)
    output);

    // zero the CpxNumMat for this scale & angle, hand ownership to
    numpy
    g[i][j]._data = NULL;
    g[i][j]._m = 0;
    g[i][j]._n = 0;
    output->flags = output->flags | NPY_OWNDATA;
    }
    }

    return (PyObject*) out;

  • Pierre Sangouard

    #2
    Re: Python Memory Leak using SWIG

    cody314@gmail.c om wrote:
    Versions:
    Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
    SWIG Version 1.3.20
    >
    Hello I have some code that wraps a C++ library so I may use it in
    python. The code basically just gets some data and puts it in the
    PyArrayObject* which is returned as a PyObject*.
    I then call it from python like so:
    self.c = __f2.fdct2_wrap per(x,self.nbs, self.nba,self.a c)
    >
    I then loop (which pretty much only calls this function) over and
    over. I put the variable as a self.c hoping the garbage collector
    would know how to delete it after the class goes out of scope. I also
    tried explicitly deleting the variable (del var) in the loop with no
    success. In all cases quiet a large memory leak occurs (and grows
    rather quickly).
    >
    I believe this is comming from the fact that the thing that is
    returned is a pointer to the data. So the returning object is a
    pointer. The python garbage collector then doesn't know how to delete
    this structure and probably (maybe) just deletes the pointer after the
    class goes out of scope. Leave the data there and causing the memory
    leak issue. I however doesn't know how to tell python that this
    variable is a pointer and to delete whats going to it. Or perhaps
    tell SWIG to delete the data, and return the structure some other way?
    >
    Here is the c++ wrapping code, perhaps there is an easy way to fix
    this memory leak (I believe a lot of SWIG people probably do this)
    perhaps some function call from the python? or some special call from
    the SWIG? Thanks a bunch!
    >
    // set up the list output
    PyListObject* out;
    PyArrayObject* output;
    out = (PyListObject*) PyList_New(0);
    npy_intp dims[2];
    int i,j;
    >
    for(i=0;i<g.siz e();i++)
    {
    // append a list for this scale
    PyList_Append(( PyObject*) out,PyList_New( 0));
    >
    for(j=0;j<g[i].size();j++)
    {
    // set the dimensions for this scale & angle
    dims[0] = g[i][j].n();
    dims[1] = g[i][j].m();
    >
    // make an array for this scale & angle's data
    output = (PyArrayObject* ) PyArray_SimpleN ewFromData(2, dims,
    PyArray_CDOUBLE ,g[i][j]._data);
    Py_INCREF((PyOb ject*) output);
    >
    // append this angle's data to the list for this scale
    PyList_Append(P yList_GetItem(( PyObject*) out,i),(PyObjec t*)
    output);
    >
    // zero the CpxNumMat for this scale & angle, hand ownership to
    numpy
    g[i][j]._data = NULL;
    g[i][j]._m = 0;
    g[i][j]._n = 0;
    output->flags = output->flags | NPY_OWNDATA;
    }
    }
    >
    return (PyObject*) out;
    I think %newobject swig directive is a solution to your problem.

    Pierre


    Comment

    Working...