Integrating w/ distutils, hooks?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Terry Hancock

    #1

    Integrating w/ distutils, hooks?

    Some time ago, I got the idea that I wanted to build image
    resources from vector graphic originals, instead of
    marshalling hundreds of tiny little icon images by hand.
    I wrote "BuildImage " to do this for me, and so far, it
    works very well, so I'm trying to make it easier to use
    and available to more developers.

    There is a brief (and somewhat dated) tutorial explaining
    what BuildImage itself does at:


    """
    In as few words as I can put it: "BuildImage is a build
    system for image resources." That is to say, it attempts to
    do for image resources what make does for programs.
    """

    So now, I'm looking for the best way to integrate the
    BuildImage package with distutils to make it the least
    painful for developers to use in their own distutils'
    packages.

    And I really feel this is something I should be talking to
    distutils-sig about. This is a little long -- but the idea
    is to spark some discussion about how I ought best to do
    this:



    I originally tried making my interface look more like
    distutils' interface -- providing a "setup" function, etc.
    I was thinking perhaps I might be able to drop in BuildImage
    scripts instead of distutils scripts in image resource
    sub-packages of some kind.

    But the end-user is unlikely to have BuildImage
    installed, and I really want 'build' and 'install'
    targets of the containing distribution to go without
    a hitch with or without BuildImage (assuming the images are
    already built, that is), while the developer targets
    "sdist" and "bdist" should use BuildImage to create image
    resources and include them in the distribution. Also,
    I'm probably starting to duplicate distutils functionality,
    which is wasteful.

    Perhaps, emulating distutils is somewhat misguided -- I
    certainly don't want to replace distutils, I just want to
    work with it in a way that will be intuitive for the
    developer. What I really want are hooks to attach to
    within distutils.

    *Ideally*, BuildImage would run at the time you would
    normally run Bison/Flex code -- i.e. it actually is a
    "pre-source" operation: you run it to complete your source
    distribution, then you compile C/C++ code, THEN you install
    Python modules, etc. I don't really see that distutils has
    this concept as yet, but it's probably a reasonable
    compromise to run it at the same time as C/C++ builds. (?)

    If that's the right path, then "sdist" should simply include
    all the necessary files for BuildImage, and the installer
    will simply need to have BuildImage installed in their
    Python site-packages in order to build the package.

    If not, then maybe 'sdist' should build the images, but
    certainly include all the BuildImage sources so they can
    be re-built. (?)

    OTOH, "bdist" should go ahead and build the images, store
    them in the correct location in the distribution tree, and
    tell distutils that it should install them along with the
    sources. (Pretty sure of this)

    Is it conventional for 'bdist' to *also* include the source
    files (i.e. in an open-source built distribution)?

    I'm a little confused about HOW to tell distutils what and
    where to install: it seems that I might want to use the
    "data_files " option to setup, or perhaps I want to update
    the MANIFEST or MANIFEST.in files (can I somehow get my
    code called after processing MANIFEST.in to MANIFEST so I
    can then add my additional instructions?).

    Modifying "MANIFEST.i n" seems a little ugly, since the idea
    is that MANIFEST.in is human-edited source, while "MANIFEST"
    is machine-generated. But I don't know how I can get my
    code called to work on MANIFEST *after* it has been
    generated, but *before* it is used to install files.

    Currently, I visualize a typical setup.py module using
    BuildImage as something like this:

    """
    # setup.py using BuildImage

    import distutils
    try:
    # We don't want to fail if BuildImage is not available,
    # because we are also run by the end user
    import BuildImage
    BuildImage.setu p( ... options for BuildImage ... )
    except:
    print """
    You don't have BuildImage installed, so only
    distutils operations will be supported.
    """

    distutils.setup ( ... options for distutils ... )
    """

    So, when the end-user runs the setup script, they will see
    a message warning them that BuildImage targets won't work,
    but distutils will be able to do its job. OTOH, when the
    developer runs the script to build the images, that will
    work too, and BuildImage will do its thing.

    Obviously, though, BuildImage.setu p() has to be able to tell
    distutils the things it needs to know, such as what to do
    with the images after they are built (typically they will be
    in a resource directory under the source tree? But in a
    distribution that insists on a filesystem standard, like
    Debian, they should be under /usr/share/images or perhaps
    /usr/share/pixmaps, etc).

    Since BuildImage won't be run at 'build' or 'install' time,
    these things have to already be prepared for distutils'
    installer.

    But BuildImage could be made aware of the requirements for
    'bdist', 'bdist_rpm', 'bdist_deb', etc., and alter what
    it tells distutils accordingly.

    Anyway, thanks for your time -- any recommendations would
    greatly be appreciated, including references to previous
    discussion or online resources I should be reading.

    Cheers,
    Terry

    --
    Terry Hancock ( hancock at anansispacework s.com )
    Anansi Spaceworks http://www.anansispaceworks.com


  • David Fraser

    #2
    Re: Integrating w/ distutils, hooks?

    I've also found the distutils architecture to be frustrating.
    It seems to be designed to do one thing only and hard to plug in to.
    Most of the time what I want to do is bundle other python modules
    together with my module in a single installer.
    It is possible to write modules so they're more easily included in other
    installers, I often refactor the other module's setup.py to make this
    possible. However it would be great to be able to add another
    "subinstall " that would do this automatically.
    I've also found that the source distribution method is strange - I
    always remove MANIFEST and MANIFEST.in and rewrite them from inside
    setup.py as this gives better control.

    Terry Hancock wrote:[color=blue]
    > Some time ago, I got the idea that I wanted to build image
    > resources from vector graphic originals, instead of
    > marshalling hundreds of tiny little icon images by hand.
    > I wrote "BuildImage " to do this for me, and so far, it
    > works very well, so I'm trying to make it easier to use
    > and available to more developers.
    >
    > There is a brief (and somewhat dated) tutorial explaining
    > what BuildImage itself does at:
    >
    > http://www.anansispaceworks.com/Docu...ion/BuildImage
    > """
    > In as few words as I can put it: "BuildImage is a build
    > system for image resources." That is to say, it attempts to
    > do for image resources what make does for programs.
    > """
    >
    > So now, I'm looking for the best way to integrate the
    > BuildImage package with distutils to make it the least
    > painful for developers to use in their own distutils'
    > packages.
    >
    > And I really feel this is something I should be talking to
    > distutils-sig about. This is a little long -- but the idea
    > is to spark some discussion about how I ought best to do
    > this:
    >
    >
    >
    > I originally tried making my interface look more like
    > distutils' interface -- providing a "setup" function, etc.
    > I was thinking perhaps I might be able to drop in BuildImage
    > scripts instead of distutils scripts in image resource
    > sub-packages of some kind.
    >
    > But the end-user is unlikely to have BuildImage
    > installed, and I really want 'build' and 'install'
    > targets of the containing distribution to go without
    > a hitch with or without BuildImage (assuming the images are
    > already built, that is), while the developer targets
    > "sdist" and "bdist" should use BuildImage to create image
    > resources and include them in the distribution. Also,
    > I'm probably starting to duplicate distutils functionality,
    > which is wasteful.
    >
    > Perhaps, emulating distutils is somewhat misguided -- I
    > certainly don't want to replace distutils, I just want to
    > work with it in a way that will be intuitive for the
    > developer. What I really want are hooks to attach to
    > within distutils.
    >
    > *Ideally*, BuildImage would run at the time you would
    > normally run Bison/Flex code -- i.e. it actually is a
    > "pre-source" operation: you run it to complete your source
    > distribution, then you compile C/C++ code, THEN you install
    > Python modules, etc. I don't really see that distutils has
    > this concept as yet, but it's probably a reasonable
    > compromise to run it at the same time as C/C++ builds. (?)
    >
    > If that's the right path, then "sdist" should simply include
    > all the necessary files for BuildImage, and the installer
    > will simply need to have BuildImage installed in their
    > Python site-packages in order to build the package.
    >
    > If not, then maybe 'sdist' should build the images, but
    > certainly include all the BuildImage sources so they can
    > be re-built. (?)
    >
    > OTOH, "bdist" should go ahead and build the images, store
    > them in the correct location in the distribution tree, and
    > tell distutils that it should install them along with the
    > sources. (Pretty sure of this)
    >
    > Is it conventional for 'bdist' to *also* include the source
    > files (i.e. in an open-source built distribution)?
    >
    > I'm a little confused about HOW to tell distutils what and
    > where to install: it seems that I might want to use the
    > "data_files " option to setup, or perhaps I want to update
    > the MANIFEST or MANIFEST.in files (can I somehow get my
    > code called after processing MANIFEST.in to MANIFEST so I
    > can then add my additional instructions?).
    >
    > Modifying "MANIFEST.i n" seems a little ugly, since the idea
    > is that MANIFEST.in is human-edited source, while "MANIFEST"
    > is machine-generated. But I don't know how I can get my
    > code called to work on MANIFEST *after* it has been
    > generated, but *before* it is used to install files.
    >
    > Currently, I visualize a typical setup.py module using
    > BuildImage as something like this:
    >
    > """
    > # setup.py using BuildImage
    >
    > import distutils
    > try:
    > # We don't want to fail if BuildImage is not available,
    > # because we are also run by the end user
    > import BuildImage
    > BuildImage.setu p( ... options for BuildImage ... )
    > except:
    > print """
    > You don't have BuildImage installed, so only
    > distutils operations will be supported.
    > """
    >
    > distutils.setup ( ... options for distutils ... )
    > """
    >
    > So, when the end-user runs the setup script, they will see
    > a message warning them that BuildImage targets won't work,
    > but distutils will be able to do its job. OTOH, when the
    > developer runs the script to build the images, that will
    > work too, and BuildImage will do its thing.
    >
    > Obviously, though, BuildImage.setu p() has to be able to tell
    > distutils the things it needs to know, such as what to do
    > with the images after they are built (typically they will be
    > in a resource directory under the source tree? But in a
    > distribution that insists on a filesystem standard, like
    > Debian, they should be under /usr/share/images or perhaps
    > /usr/share/pixmaps, etc).
    >
    > Since BuildImage won't be run at 'build' or 'install' time,
    > these things have to already be prepared for distutils'
    > installer.
    >
    > But BuildImage could be made aware of the requirements for
    > 'bdist', 'bdist_rpm', 'bdist_deb', etc., and alter what
    > it tells distutils accordingly.
    >
    > Anyway, thanks for your time -- any recommendations would
    > greatly be appreciated, including references to previous
    > discussion or online resources I should be reading.
    >[/color]

    Comment

    Working...