rotor replacement

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Skip Montanaro

    #61
    Re: What's so funny? WAS Re: rotor replacement

    >> As long as we are discussing cryptography, what's wrong with m2crypto?[color=blue][color=green]
    >> Or, what about Andrew Kuchling's crypto toolkit?[/color][/color]

    Lucas> Umm, is it just me or did we just discuss the legal issues of
    Lucas> that??

    You may have. Whether or not there are legal issues with them is of little
    relevance to the point I was making. Anything Paul writes would probably
    have the same legal entanglements. I was simply pointing out that maybe,
    just maybe, there are already suitable candidates from a technical
    standpoint and that he doesn't need to write anything.

    Skip

    Comment

    • phr@localhost.localdomain

      #62
      Re: What's so funny? WAS Re: rotor replacement

      Skip Montanaro <skip@pobox.com > writes:[color=blue]
      > phr> It is not a whizbang module. It is a stripped-down, basic
      > phr> implementation of a well-accepted set of standards that are being
      > phr> used in thousands of other applications in other languages.
      >
      > Then there should be a library already out there already. All you
      > should need to do is wrap it (manually, with SWIG, whatever).[/color]

      I'm currently using something wrapped with SWIG, but my understanding
      is that core modules are not supposed to depend on SWIG. So a core
      module will likely use some existing primitives wrapped by hand. That
      is what I've offered to do, along with providing a somewhat more
      Pythonic interface (though still a straightforward one) than directly
      wrapping a C library intended for use from C applications.
      [color=blue]
      > phr> There is demand for it. Look at how this thread started: some
      > phr> crypto user noticed that rotor was gone and wanted to know what to
      > phr> do instead.
      >
      > Yes, and putting rotor back would be the wrong thing to do.[/color]

      Correct. The right thing is to replace rotor with something
      reasonable that follows standards.
      [color=blue]
      > phr> The issue of whether there's enough desire for a crypto module to
      > phr> warrant including one in the stdlib was decided a long time ago.
      > phr> The proof of that somebody decided to accept the rotor module into
      > phr> the distro.
      >
      > No, rotor was added in Python's early days (July 1992). Times have changed.[/color]

      I don't see that. There's surely even more demand for crypto now than
      there was in 1992.
      [color=blue]
      > As long as we are discussing cryptography, what's wrong with m2crypto?
      >
      > http://sandbox.rulemaker.net/ngps/m2/[/color]

      It's a good package but it's pretty heavyweight. It depends on both
      SWIG and OpenSSL. I think it's still under development--there's an
      occasional flurry of messages about it on python-crypto, but I haven't
      been following it closely. I'd have a hard time making a case for
      accepting it into the core given the difficulty I'm having making the
      case for something as simple as a block cipher wrapper.
      m2crypto+OpenSS L is at least 100 times as much code as the module I've
      proposed. I think the Python lib should someday have its own
      pure-Python SSL/TLS implementation sort of like the one Java has. But
      if m2crypto went into the lib, I'd use it.
      [color=blue]
      > Why not incorporate it into the standard distribution?[/color]

      I don't have the authority to incorporate anything into the standard
      distribution. All I can do is offer to contribute stuff that I write,
      and let the distro maintainers decide whether to incorporate it.

      I don't have the authority to offer m2crypto, since I'm not the author.
      Only the authors can do that. They haven't done so, as far as I know.
      [color=blue]
      > Or, what about Andrew Kuchling's crypto toolkit?
      >
      > http://www.amk.ca/python/code/crypto.html[/color]

      This is perhaps more suitable than m2crypto but as far as I know,
      Andrew hasn't offered to contribute it. Whatever his reasons are, I
      have to respect them. I think it has more stuff than a core crypto
      module really needs (e.g. numerous semi-obsolete algorithms that
      aren't in widespread use so aren't needed for interoperabilit y) but
      the extra stuff doesn't really get in the way. If it were in the
      core, it would completely fulfill my desires and I would be
      transported with ecstacy. But I've never seen any indication that
      it's headed there.
      [color=blue]
      > I believe both have been around awhile. If crypto-in-the-core is
      > really what's needed why not see if one of them is ready to go?[/color]

      I don't think m2crypto is the answer, though maybe I'm wrong. And if
      Andrew's package is the answer, he would have submitted it already.
      [color=blue]
      > phr> Have you ever used a crypto library in a serious way?
      >
      > Nope, never directly. Don't make this about me. I'm interested in the
      > Python development process and how you'd like to turn it on its head.[/color]

      Either you're the one turning the actual process on its head, or else
      it's already on its head and needs to be turned rightside up. Either
      way, the existing process has certainly been a total failure so far at
      producing good crypto support in the lib.
      [color=blue]
      > phr> It's already the category king, because there are precisely zero
      > phr> other entrants in the category.
      >
      > See my above references. Note, I don't use crypto at all, yet I was aware
      > of both of these (no Googling required).[/color]

      The authors have not offered to contribute them, so they're not in the
      category. The category consists of crypto modules that have actually
      been offered. As I keep saying, I'd love it if someone else offered
      one. I'm not eager for this headache. I just saw that somebody
      really ought to do it, and nobody was doing it, so I decided I was
      elected.
      [color=blue]
      > My guess would be they are substantially more mature than your
      > proposed module.[/color]

      This may sound a little strange, since my module is not yet written,
      but if we take maturity to mean lower future maintenance needs, then I
      wouldn't say m2crypto is precisely more mature. m2crypto uses SWIG
      and is intimately connected to an immensely complex package (OpenSSL)
      anpd has to track new OpenSSL and SWIG releases. OpenSSL has all
      kinds of OS dependencies, has high performance math subsystems written
      in assembly code for various different cpu's, etc. So my guess is
      that m2crypto has a longer development curve and will always need
      significant amounts of maintenance. My module is more like a basic
      portable math library that computes square roots, cosines, etc. and is
      OS and CPU independent. Once written and tested, it's intended to be
      pretty solid and not need much attention.

      Andrew's package is along the same lines as mine, except Andrew's is
      already done, so Andrew's is definitely more mature. I like to think
      that my module's API incorporates some lessons from Andrew's and
      improves on it, but that's not that important in practice.
      [color=blue]
      > phr> I read those as saying that no crypto module will be
      > phr> considered for inclusion whether or not it's the category
      > phr> king, because any such module might conflict with crypto
      > phr> regulations in some countries.
      >
      > That may be a problem, sure. I'm not sure how the discussion here changes
      > that. That's just life as we know it.[/color]

      It doesn't change it. However, it does change my level of interest in
      writing a module, as is reasonable since the only important
      characteristics of my module that are different from existing ones is
      that mine is designed specifically to be suitable for the core and has
      been offered for distribution in the core. Outside the core, the
      existing modules work ok so I use them.
      [color=blue]
      > Why in the heck is inclusion in the standard library a requirement
      > for you to write this thing?[/color]

      There's no other reason to write it. Outside the stdlib, there are
      other modules that are already useable, like the ones you mentioned,
      so I use those.
      [color=blue]
      > If it's useful to you, write it and get on with your life.[/color]

      It's not worth doing unless it's intended for the stdlib. How many
      times do I have to explain this?
      [color=blue]
      > phr> Tell me again how rotor got into the distribution.
      >
      > Python's user community was probably a few hundred people. Guido
      > likely had no thoughts of world domination with the little language that
      > could. Times have changed.[/color]

      If Python is aiming for world domination, it needs to be able to
      compete with Java in security features. Java ships with the whole
      JSSE system and a much fancier low level crypto architecture than the
      one we've been discussing. So I think the Python folks should figure
      out how to incorporate crypto without getting crushed under the iron
      boot (or at least the wooden shoe) of the Dutch government or whatever
      it is that they're worried about.
      [color=blue]
      > You seem think there was a PEP process and distutils and Wikis. I
      > suspect some of the algorithms one might include in a robust crypto
      > toolkit today weren't even invented in 1992.[/color]

      AES wasn't invented then. DES became a US federal standard around 1977.
      [color=blue]
      > So throw away the rotor crutch and put your money where your mouth is.[/color]

      If you mean write yet another crypto module knowing ahead of time that
      it won't go into the stdlib, that would be more like throwing my money
      down the drain.

      Comment

      • phr@localhost.localdomain

        #63
        Re: What's so funny? WAS Re: rotor replacement

        Skip Montanaro <skip@pobox.com > writes:[color=blue][color=green][color=darkred]
        > >> As long as we are discussing cryptography, what's wrong with
        > >> m2crypto? Or, what about Andrew Kuchling's crypto toolkit?[/color][/color]
        >
        > Lucas> Umm, is it just me or did we just discuss the legal issues of
        > Lucas> that??
        >
        > You may have. Whether or not there are legal issues with them is of little
        > relevance to the point I was making. Anything Paul writes would probably
        > have the same legal entanglements.[/color]

        Andrew's toolkit does incorporate some patented algorithms, but those
        aren't critical and could be removed from a stdlib version.
        [color=blue]
        > I was simply pointing out that maybe, just maybe, there are already
        > suitable candidates from a technical standpoint and that he doesn't
        > need to write anything.[/color]

        There really don't appear to be any that are both technically
        suitable, and that the authors are willing to contribute to PSF. I'd
        be delighted to be wrong. Of course that still leaves the legal
        issue.

        Comment

        • Martin v. Löwis

          #64
          Re: What's so funny? WAS Re: rotor replacement

          phr@localhost.l ocaldomain wrote:[color=blue]
          > I don't see why you can't make up your mind enough to issue simple
          > statements like "the Python lib should have a module that does
          > so-and-so[/color]

          I can say that assuming I know what so-and-so is. For the specific
          case of AES, I would say "I don't think the Python lib necessarily
          needs to have an AES module, but I would not object if it had one"
          (the latter part in consideration of consequences that inclusion
          of crypto code might have).
          [color=blue]
          > and it should meet such-and-such requirements[/color]

          I can only say such things if I know such-and-such in detail
          to specify requirements. For the specific case of AES, I don't
          know enough about it to specify requirements. I will have to
          trust others (and by that, I mean *multiple* others)
          [color=blue]
          > so if
          > someone submits one that meets the requirements and passes code review
          > and testing and doesn't have unexpected issues or otherwise fail to
          > meet reasonable expectations, we'll use it".[/color]

          Because I cannot specify requirements, I cannot make such a promise.

          In addition, for any new module, there is one primary requirement
          for acceptance that cannot be fulfilled in code quality: the
          contributor should promise to support the module in a foreseeable
          future (i.e. a couple of years).
          [color=blue]
          > Again, we're talking about straightforward modules whose basic
          > interface needs are obvious.[/color]

          You are talking about such a thing. I don't know enough about
          the functionality to specify what an obvious interface is, or
          to recognize one if I see it.
          [color=blue]
          > I don't know what OMG is, but there is no IETF requirement that any
          > implementations be available in any particular language.[/color]

          See RFC 2026, section 4.1.2. Two independent implementations
          are required for the document to advance to draft (!) standard.

          [color=blue]
          > However, the result of my not writing an AES module is that Python
          > doesn't have an AES module.[/color]

          That's not true. PyCrypto does have AES support.
          [color=blue][color=green]
          >>No, it's three steps
          >>1. decide that you want to do it
          >>2. do it
          >>3. decide whether you are pleased with the result, and only
          >> use it if you are
          >>
          >>IOW, there should not be a blanket guarantee to use it after step 1.[/color]
          >
          >
          > But, it's completely normal to say before step 1 that "if the result
          > of step 2 does so-and-so, then I'll be pleased in step 3",[/color]

          That's what I'm saying: If you distribute the module to users for
          a year, and users express interest and support for your choice of
          API, I'll support inclusion of the module. "do it" involves more than
          just writing the code.
          [color=blue]
          > You and
          > Frederik seem to think there's something inappropriate or
          > self-inflated about wanting that expectation before committing to do a
          > pile of work that's primarily for other people's benefit.[/color]

          It's very easy. If you are primarily do it for other people's
          benefit, and if you don't find any satisfaction in the process
          of doing it - THEN DON'T. I really mean that; this is how
          free software works. People *volunteer* to do things. If they
          don't volunteer - that's perfectly fine.
          [color=blue]
          > I think
          > your stated attitude is completely bizarre, that you can't really
          > believe anything so silly, so you're really just acting bureaucratic,
          > looking for excuses to say no instead of yes to worthwhile proposals.[/color]

          As I said above - for the specific feature in question, I don't
          care enough for the feature itself. Python will be just as useful
          to me with the feature as it is without.

          What I do care about is the effort that I will need to continue
          maintaining Python. I don't want to have to maintain an ill-designed,
          buggy module with no active maintainer, and I don't want to tell
          people that I had to rip the module out just because it doesn't
          work at all.

          Regards,
          Martin

          Comment

          • Paul Rubin

            #65
            Re: What's so funny? WAS Re: rotor replacement

            "Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue][color=green]
            > > I don't see why you can't make up your mind enough to issue simple
            > > statements like "the Python lib should have a module that does
            > > so-and-so[/color]
            >
            > I can say that assuming I know what so-and-so is. For the specific
            > case of AES, I would say "I don't think the Python lib necessarily
            > needs to have an AES module, but I would not object if it had one"[/color]

            Well, ok, you're changing your tune a little bit now, and getting more
            reasonable. Before, you were making blanket statements of any modules
            written for the Python stdlib. Now you're limiting it to AES and basing
            it on some AES-specific things.
            [color=blue]
            > (the latter part in consideration of consequences that inclusion
            > of crypto code might have).[/color]

            I would say the first thing I'd request would be your sense of how
            plausible it is that the current no-crypto policy might be relaxed.
            And I think this is about totally up to Guido, since (reading between
            the lines of those python-dev messages) he's concerned about the Dutch
            crypto restrictions, which makes some sense because he's Dutch and
            still has to deal with the Dutch government from time to time, while
            you or I might not care what the Dutch government thinks.

            I was about to mention that Rop Gongrijp has been operating a crypto
            company in Holland (www.nah6.com) but I now see that it's moved to
            Germany, and I wonder if those Dutch restrictions might be the reason.
            I don't know any details though.
            [color=blue][color=green]
            > > and it should meet such-and-such requirements[/color]
            >
            > I can only say such things if I know such-and-such in detail
            > to specify requirements. For the specific case of AES, I don't
            > know enough about it to specify requirements. I will have to
            > trust others (and by that, I mean *multiple* others)[/color]

            OK. If it helps, I can tell you that the Python crypto crowd hangs
            out on a mailing list called python-crypto and has discussed this
            module at some length.
            [color=blue][color=green]
            > > so if someone submits one that meets the requirements and passes
            > > code review and testing and doesn't have unexpected issues or
            > > otherwise fail to meet reasonable expectations, we'll use it".[/color]
            >
            > Because I cannot specify requirements, I cannot make such a promise.[/color]

            Again OK. I had thought from earlier discussions that you were a
            crypto developer and familiar with this stuff; no problem if you're
            not. However, in that case, you probably wouldn't be using the module
            if it got included. I think the best policy if you don't feel
            comfortable supporting including some module (whether it's crypto or
            something else) that you're not personally going to use, is don't
            support inclusion, but also don't obstruct it. If there's some other
            stdlib maintainers who are knowledgeable about that kind of module and
            have an opinion about it, then go along with them unless there's a
            clear reason not to. Like, if I were a stdlib maintainer and somebody
            submitted a fancy multigrid PDE solver, that's outside my expertise
            and I'd go along with whatever Tim recommended.
            [color=blue]
            > In addition, for any new module, there is one primary requirement
            > for acceptance that cannot be fulfilled in code quality: the
            > contributor should promise to support the module in a foreseeable
            > future (i.e. a couple of years).[/color]

            That's not too unreasonable, though I don't think I've heard it
            mentioned as a requirement before.
            [color=blue]
            > You are talking about such a thing. I don't know enough about the
            > functionality to specify what an obvious interface is, or to
            > recognize one if I see it.[/color]

            The thing to do then is just defer the whole issue to someone on the
            core team who uses that type of function. Only if no one on the team
            has an opinion do you start having to look for stuff like wide use of
            the module in the outside community.
            [color=blue][color=green]
            > > I don't know what OMG is, but there is no IETF requirement that any
            > > implementations be available in any particular language.[/color]
            >
            > See RFC 2026, section 4.1.2. Two independent implementations
            > are required for the document to advance to draft (!) standard.[/color]

            That says nothing about the implementation languages. Both might be
            in Python, or one might be in VHDL and the other in Ada, or whatever.

            This particular module we're discussing is a straightforward
            implementation of AES, DES, and FIPS 81, for which there are thousands
            of implementations in both hardware and software. For the specific
            Python API, there's a Python implementation and the proposed new
            module is a C implementation that does the exact same thing only
            faster. So that's two implementations , though not independent in the
            IETF sense, because they'd both be written by the same person.

            I've never heard of any requirement before that there be two separate
            implementations of every Python stdlib module, by the way. That would
            be silly.
            [color=blue][color=green]
            > > However, the result of my not writing an AES module is that Python
            > > doesn't have an AES module.[/color]
            >
            > That's not true. PyCrypto does have AES support.[/color]

            That's not in any Python distro that I know of. I'm talking about
            modules in the stdlib, not external modules. "Batteries included",
            you know.
            [color=blue]
            > That's what I'm saying: If you distribute the module to users for
            > a year, and users express interest and support for your choice of
            > API, I'll support inclusion of the module. "do it" involves more than
            > just writing the code.[/color]

            Well, that's nice of you to offer to support inclusion of a module
            that you're not likely to use yourself and whose functionality you
            don't really understand (I'm not being sarcastic). If you're going to
            offer such support it makes sense to impose an unusually high standard
            for offering it. I myself would probably never support including any
            such module no matter how long it was distributed, but rather would
            just defer the whole question to people experienced with such modules
            and trust their sense of what the acceptance criteria should be for a
            specific module. That is, I'd abstain from any decision.
            [color=blue]
            > It's very easy. If you are primarily do it for other people's
            > benefit, and if you don't find any satisfaction in the process
            > of doing it - THEN DON'T. I really mean that; this is how
            > free software works. People *volunteer* to do things. If they
            > don't volunteer - that's perfectly fine.[/color]

            That doesn't map well onto the real world. Say I'm a professional
            cook and I have a good recipe for spam, eggs, sausage, and spam. I
            write the PyCon organizers and ask if they'd like me to cook up a big
            batch to serve for breakfast for PyCon attendees (this is as a
            volunteer, all ingredients provided at my own expense, etc).
            Reasonable answers might be:

            1) No, we don't want to serve breakfast at PyCon, we thought about
            it but decided against it. => OK, fine, I don't cook it.

            2) Hey, that sounds good--cook enough for 500 people and bring it to
            the site before the conference starts, and we'll put it on the
            menu unless it's below expectations. => OK, fine, I start
            cooking. Nobody has made an ironclad promise, but the organizers
            have told me their basic intentions, which is good enough for me.

            3) Hmm, that recipe sounds interesting, why don't you send one or
            two portions to the organizers, we'll try them and let you know.
            => OK, this is basically a prototype implementation, like the
            pure-Python version of the AES module, that's fine for testing
            but not fast enough to serve (e.g.) 500 web connections. I
            send the small batch and wait for an answer before deciding to
            start the much bigger job of making a 500-person batch.

            Your answer is more like "make enough for 500 people and bring it to
            the conference site and only then will we even THINK about whether to
            serve it for breakfast. Serving it at PyCon shouldn't matter to you,
            what are you, some kind of egotist? You like cooking, so if you make
            the 500 portions and then the organizers decline them, you can always
            hand them out in the street. If you don't find any satisfaction in
            that, THEN DON'T DO IT."

            If the cook's goal is not just to feed Python users in the street, but
            to actually improve PyCon itself by providing for PyCon to serve
            breakfast to attendees, I hope you can see why he might not think much
            of your answer.
            [color=blue]
            > As I said above - for the specific feature in question, I don't
            > care enough for the feature itself. Python will be just as useful
            > to me with the feature as it is without.[/color]

            I'm sure that's true of lots of modules. If you're the sole maintainer
            of the Python distro, then you have to evaluate every module that anyone
            submits. But since you're part of a reasonable sized group, I think
            it's best to examine the modules you care about, but don't feel that
            you have to have your hands in everything.

            Comment

            • Martin v. Löwis

              #66
              Re: What's so funny? WAS Re: rotor replacement

              Paul Rubin wrote:[color=blue][color=green]
              >>I can say that assuming I know what so-and-so is. For the specific
              >>case of AES, I would say "I don't think the Python lib necessarily
              >>needs to have an AES module, but I would not object if it had one"[/color]
              >
              >
              > Well, ok, you're changing your tune a little bit now, and getting more
              > reasonable. Before, you were making blanket statements of any modules
              > written for the Python stdlib. Now you're limiting it to AES and basing
              > it on some AES-specific things.[/color]

              And I still stand by those blanket statements. Any new module (i.e.
              specific code) should see real users for some time before it can
              be incorporated into Python.

              The question whether I would like to see a certain *feature*
              in Python (whether as a separate module, a language feature, or
              otherwise) is an entirely different question. That I would not
              object to an AES code in general doesn't imply that I will
              agree to any AES module that somebody contributes.
              [color=blue]
              > I would say the first thing I'd request would be your sense of how
              > plausible it is that the current no-crypto policy might be relaxed.[/color]

              There is no such policy in force. Different people have different
              opinions, and those opinions change over time. I cannot speak for
              others.
              [color=blue]
              > Again OK. I had thought from earlier discussions that you were a
              > crypto developer and familiar with this stuff; no problem if you're
              > not. However, in that case, you probably wouldn't be using the module
              > if it got included. I think the best policy if you don't feel
              > comfortable supporting including some module (whether it's crypto or
              > something else) that you're not personally going to use, is don't
              > support inclusion, but also don't obstruct it.[/color]

              Thanks for the advise, but I'll have my own policies. I have included
              several modules in the past which I'm not personally using.
              [color=blue][color=green]
              >>In addition, for any new module, there is one primary requirement
              >>for acceptance that cannot be fulfilled in code quality: the
              >>contributor should promise to support the module in a foreseeable
              >>future (i.e. a couple of years).[/color]
              >
              >
              > That's not too unreasonable, though I don't think I've heard it
              > mentioned as a requirement before.[/color]

              See PEP 2.
              [color=blue]
              > I've never heard of any requirement before that there be two separate
              > implementations of every Python stdlib module, by the way. That would
              > be silly.[/color]

              And I did not suggest such a thing. You said you never heard about
              a process which requires an implementation before deciding that the
              proposed feature is good, and I said that, for an example, IETF even
              requires *two* implementations before deciding that the feature is
              good.
              [color=blue][color=green][color=darkred]
              >>>However, the result of my not writing an AES module is that Python
              >>>doesn't have an AES module.[/color]
              >>
              >>That's not true. PyCrypto does have AES support.[/color]
              >
              > That's not in any Python distro that I know of.[/color]

              Sure, but "Python" is more than the Python core.
              [color=blue]
              > I myself would probably never support including any
              > such module no matter how long it was distributed, but rather would
              > just defer the whole question to people experienced with such modules
              > and trust their sense of what the acceptance criteria should be for a
              > specific module. That is, I'd abstain from any decision.[/color]

              With that attitude, the patches tracker on SF would grow unbounded,
              because we *continuously* get patches that no core developer has
              any personal interest in. I'll try to look at *all* patches, whenever
              I can, and in the process, I have to learn about technologies which
              I've never used before (primarily operating systems, but also
              APIs and protocols).

              Rejecting all these patches would be unfair to the contributors.
              [color=blue]
              > Your answer is more like "make enough for 500 people and bring it to
              > the conference site and only then will we even THINK about whether to
              > serve it for breakfast.[/color]

              I cannot see these as the same thing.
              [color=blue]
              > I'm sure that's true of lots of modules. If you're the sole maintainer
              > of the Python distro, then you have to evaluate every module that anyone
              > submits. But since you're part of a reasonable sized group, I think
              > it's best to examine the modules you care about, but don't feel that
              > you have to have your hands in everything.[/color]

              Thanks again for the advise, but this is not how Python is being
              maintained.

              Regards,
              Martin

              Comment

              • Paul Rubin

                #67
                Re: What's so funny? WAS Re: rotor replacement

                "Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue]
                > And I still stand by those blanket statements. Any new module (i.e.
                > specific code) should see real users for some time before it can
                > be incorporated into Python.[/color]

                Let's see, the urandom module was recently released in 2.4, I think
                initially at my urging. I believe it was written just for 2.4,
                combining a Windows function used only (as far as I know) by its
                author, and a wrapper to read /dev/urandom on Posix systems. It
                didn't have any wide distribution or use before incorporation, and
                didn't need it. It was simple enough that a reasonable code review
                and some testing on the relevant platforms was enough to make make
                sure it worked properly.
                [color=blue][color=green]
                > > I would say the first thing I'd request would be your sense of how
                > > plausible it is that the current no-crypto policy might be relaxed.[/color]
                >
                > There is no such policy in force. Different people have different
                > opinions, and those opinions change over time. I cannot speak for others.[/color]

                If you can't speak for others, how can you say there's no policy in
                force? If Guido says "no crypto", is that something other than a
                policy? And are you speaking for him when you say that? <wink>
                [color=blue]
                > proposed feature is good, and I said that, for an example, IETF even
                > requires *two* implementations before deciding that the feature is good.[/color]

                No, that's completely wrong. IETF requires two implementations before
                publishing a specification--a precise description--as a draft
                standard, which means agreeing on how the feature should work at the
                finest level of detail. That's much different than merely deciding
                that a feature is good and inviting people to work on it.

                IETF often decides and announces that a feature is good long before
                any such details are decided or agreed on. For example, they decided
                that IP connection encryption was a good feature, and appointed a
                working group a general mandate to go figure out the details (IPSEC)
                and write a spec. See? IETF announced in advance that IPSEC was a
                good feature, and the working group members knew of that decision
                before they committed the resources needed to write the draft. And
                when they started, there were zero actual IPSEC implementations ,
                because nobody knew what precise characteristics IPSEC would
                eventually have. The implementations got written in the process of
                developing the spec and presenting it as a standard.

                The two-implementation policy means there has to be two
                implementations before the draft is finally declared complete, not
                before the IETF can announce that it wants a standard to be written
                and intends to use whatever the working group comes up with once the
                draft and two implementations are done. That is what happens in any
                normal professional process. Nobody commits resources to develop
                something (other than for experimentation ) unless they think the end
                result will actually be used. Are you really having trouble
                understanding this?
                [color=blue][color=green][color=darkred]
                > >>That's not true. PyCrypto does have AES support.[/color]
                > > That's not in any Python distro that I know of.[/color]
                >
                > Sure, but "Python" is more than the Python core.[/color]

                Tell me again what "batteries included" means. I personally interpret
                "included" as meaning "included". That means when I install Python,
                the module is right there in the distro. So if it's not in the
                distro, Python itself doesn't have it. "Batteries included" is an
                expression of Pythonic philosophy that says: putting components that
                end-users need in the distro is superior to making the users get them
                from somewhere else separately. I realize that the difference might
                not matter to you, but it does matter to a heck of a lot of other
                users.
                [color=blue]
                > With that attitude, the patches tracker on SF would grow unbounded,
                > because we *continuously* get patches that no core developer has
                > any personal interest in.[/color]

                Obviously those require a different type of consideration. I'm
                talking about patches where there's a core developer with an interest.
                Let's say the cmath module doesn't exist and someone submits it. It
                computes functions like cos(z) for complex arguments. If Tim, the
                numerics expert, then says "this is worth having and I've examined the
                code and it looks good, let's put it in", I'd say that takes care of
                it, and I think that's what typically happens with real modules. If
                nobody on the core team has an informed opinion, then it has to be
                decided some slower and more formal way.

                Comment

                • Terry Hancock

                  #68
                  Re: What's so funny? WAS Re: rotor replacement

                  On Tuesday 25 January 2005 01:51 am, Fredrik Lundh wrote:[color=blue]
                  > ah, you're the guy who flamed me and called me names because PIL didn't
                  > fit some hypothetical GNU-inspired definition of "open source software." it's
                  > always sad when people have to attack those who don't share their religion,
                  > but it's not that much I can do about that.[/color]

                  That is a complete misrepresentati on of the post in question. It was in
                  no manner of interpretation a "flame". You simply took offense out of thin
                  air. And there's not much *I* can do about *that*.

                  I figured that my intent to contribute to your project was sufficient to
                  indicate my good intentions.
                  [color=blue][color=green]
                  > > So, naturally, I just dropped it.[/color]
                  >
                  > which means that you have no idea how easy or hard it is to install PIL today
                  > (hint: things have changed)[/color]

                  Oh? In which version?

                  Are you saying that your installation procedure now honors the "--with-jpeg=" and
                  "--with-png=" flags to configure? Because although you had these flags, in the last
                  version I installed from source, they did not work -- it was still necessary to edit
                  the Makefile.

                  That was either 1.1.3 or 1.1.4, and I'm pretty sure it was the latter.

                  Now if you've actually fixed this, I look forward to trying it out the next
                  time I have to do a complete from-source installation. I'll say "thank you",
                  just in case.

                  Cheers,
                  Terry

                  --
                  --
                  Terry Hancock ( hancock at anansispacework s.com )
                  Anansi Spaceworks http://www.anansispaceworks.com

                  Comment

                  • Martin v. Löwis

                    #69
                    Re: What's so funny? WAS Re: rotor replacement

                    Paul Rubin wrote:[color=blue]
                    > Let's see, the urandom module was recently released in 2.4, I think
                    > initially at my urging.[/color]

                    There is no urandom module in Python 2.4.
                    [color=blue]
                    > If you can't speak for others, how can you say there's no policy in
                    > force?[/color]

                    I should say I'm not aware of a policy.
                    [color=blue]
                    > If Guido says "no crypto", is that something other than a
                    > policy? And are you speaking for him when you say that? <wink>[/color]

                    If he had said such a thing in general, it would be a policy
                    (although it best would be documented somewhere). I don't think
                    he said that, in general, and with the intent of stating a policy.
                    [color=blue]
                    > That's much different than merely deciding
                    > that a feature is good and inviting people to work on it.[/color]

                    We don't normally invite people to work on anything. People
                    pick the things they work on themselves.
                    [color=blue]
                    > IETF often decides and announces that a feature is good long before
                    > any such details are decided or agreed on. For example, they decided
                    > that IP connection encryption was a good feature, and appointed a
                    > working group a general mandate to go figure out the details (IPSEC)
                    > and write a spec. See?[/color]

                    No. They decided that spam machinery in DNS would be a good thing,
                    people started to work on it, and then they decided that it is not
                    such a good thing, after all, because it causes too many problems.

                    So the decision "this is a good thing" is no guarantee for
                    "if it is done, we will publish it as a standard". They might
                    start a process, people might get nearly through it, and then
                    the process stucks or is given up entirely. Happens all the time.
                    [color=blue]
                    > I realize that the difference might
                    > not matter to you, but it does matter to a heck of a lot of other
                    > users.[/color]

                    It does matter for me, yet Python is still more than the core.

                    You might be ignoring that, but it surely is more to the many
                    developers which create Python libraries and distribute them
                    themselves, see



                    THIS IS ALL PYTHON.
                    [color=blue]
                    > Obviously those require a different type of consideration. I'm
                    > talking about patches where there's a core developer with an interest.[/color]

                    I though you were talking about the AES module...

                    Regards,
                    Martin

                    Comment

                    • Paul Rubin

                      #70
                      Re: What's so funny? WAS Re: rotor replacement

                      "Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue][color=green]
                      > > Let's see, the urandom module was recently released in 2.4, I think
                      > > initially at my urging.[/color]
                      >
                      > There is no urandom module in Python 2.4.[/color]

                      Oops, sorry, it's in the os module:



                      The difference is simply a matter of the packaging. It's important
                      functionality and it was added after an amount of review and testing
                      that was deemed to be reasonable. Unless you're saying that if I
                      wanted to add AES to the string module (so you could say
                      'spam and sausage'.aes_en crypt('swordfis h banana')) instead of writing a
                      separate module, then we wouldn't need this discussion. (Putting it
                      in the string module is not a serious proposal but it wouldn't be
                      completely insane.) What matters is the code complexity, not whether
                      something is in a separate module or not.
                      [color=blue][color=green]
                      > > If Guido says "no crypto", is that something other than a
                      > > policy? And are you speaking for him when you say that? <wink>[/color]
                      >
                      > If he had said such a thing in general, it would be a policy
                      > (although it best would be documented somewhere). I don't think
                      > he said that, in general, and with the intent of stating a policy.[/color]

                      Well, if he indicates that it's not a policy and that the question is
                      still open, then I could see getting interested again in writing an
                      AES module. At the moment I continue to see his python-dev post as
                      quite discouraging.
                      [color=blue]
                      > We don't normally invite people to work on anything. People pick the
                      > things they work on themselves.[/color]

                      Not true. For example, you once invited me to work on an ancillary
                      message feature for the socket module (SF bug 814689), and so it's
                      been on my want-to-do-one-of-these-days list since then. I think it's
                      reasonable for me to have taken your message there as an expression of
                      interest, sufficient to get me to want to work on it. So it's bogus
                      to say the Python developers should avoid expressing interest in
                      something that hasn't already been written.
                      [color=blue][color=green]
                      > > IETF often decides and announces that a feature is good long before
                      > > any such details are decided or agreed on.[/color]
                      >
                      > No. They decided that spam machinery in DNS would be a good thing,
                      > people started to work on it, and then they decided that it is not
                      > such a good thing, after all, because it causes too many problems.[/color]

                      There's no contradiction. IETF decided something based on some
                      reasonable beliefs that later turned out to be wrong, so they had to
                      undo the original decision because the original expectations couldn't
                      be met. That happens sometimes. Everyone has worked on projects that
                      ended up failing and getting cancelled for reasons like that. There
                      are few ironclad guarantees of anything, only reasonable beliefs. But
                      real-world organizations are not afraid to start projects based on
                      reasonable beliefs regardless. Otherwise, few things would ever get
                      done.
                      [color=blue]
                      > It does matter for me, yet Python is still more than the core.
                      >
                      > You might be ignoring that, but it surely is more to the many
                      > developers which create Python libraries and distribute them
                      > themselves, see
                      >
                      > http://www.python.org/pypi
                      > THIS IS ALL PYTHON.[/color]

                      No. Those are programs people have written in Python or as Python
                      extensions. I've written programs like that myself. I don't consider
                      them to be part of Python. They are separate programs. If one of
                      them gets accepted into the distro (as this thread is about), then it
                      becomes part of Python. See <http://www.python.org/doc/faq/general.html>:

                      1.1.6 How do I obtain a copy of the Python source?

                      The latest Python source distribution is always available from
                      python.org, at http://www.python.org/download/. The latest development
                      sources can be obtained via anonymous CVS from SourceForge, at
                      http://www.sourceforge.net/projects/python.

                      Python is what you get from following the download instructions and
                      installing the result. In <http://www.python.org/doc/faq/installed.html> ,
                      where it asks "why is Python installed on my machine?", that is what
                      they are referring to by "Python".
                      [color=blue][color=green]
                      > > Obviously those require a different type of consideration. I'm
                      > > talking about patches where there's a core developer with an interest.[/color]
                      >
                      > I though you were talking about the AES module...[/color]

                      No, when this started you were talking about modules in general.

                      Comment

                      • Martin v. Löwis

                        #71
                        Re: What's so funny? WAS Re: rotor replacement

                        Paul Rubin wrote:[color=blue]
                        > Oops, sorry, it's in the os module:
                        >
                        > http://docs.python.org/lib/os-miscfunc.html
                        >
                        > The difference is simply a matter of the packaging.[/color]

                        No, it's not. It also is a matter of code size, and impact. Small
                        additions can be reviewed and studied more easily, and need to be
                        tested on less users. A new module is on a larger scale than
                        a mere new function.
                        [color=blue]
                        > Unless you're saying that if I
                        > wanted to add AES to the string module (so you could say
                        > 'spam and sausage'.aes_en crypt('swordfis h banana')) instead of writing a
                        > separate module, then we wouldn't need this discussion.[/color]

                        Indeed, if it was a single new function to an existing module, I would
                        not require that this be delivered to users first. It is entire new
                        libraries that I worry about.

                        If you would propose a change to the string module to add an aes_encrypt
                        function, I would immediately reject that patch, of course, because that
                        function does not belong to the string module.
                        [color=blue]
                        > What matters is the code complexity, not whether
                        > something is in a separate module or not.[/color]

                        A module *is* typically more complex than a single function. If
                        your new module has only a single new function, we should discuss
                        whether it really needs to be a separate module.
                        [color=blue]
                        > Well, if he indicates that it's not a policy and that the question is
                        > still open, then I could see getting interested again in writing an
                        > AES module. At the moment I continue to see his python-dev post as
                        > quite discouraging.[/color]

                        And, again, I consider this perfectly fine. This would be a volunteer
                        effort, and volunteers are free to work on whatever they please.

                        Furthermore, people who want an AES module for Python could get one
                        from

                        Download Python Cryptographic Toolkit for free. CryptKit is a small, fast cryptographic toolkit for python. It implements Rijndael(AES), SHA 256, Elliptic Curve PKI, Diffie-Hellman key exchange and Nyberg-Ruppel signature/verification.


                        Maybe Bryan Mongeau will contribute this code to Python some day.
                        [color=blue]
                        > Not true. For example, you once invited me to work on an ancillary
                        > message feature for the socket module (SF bug 814689), and so it's
                        > been on my want-to-do-one-of-these-days list since then. I think it's
                        > reasonable for me to have taken your message there as an expression of
                        > interest, sufficient to get me to want to work on it. So it's bogus
                        > to say the Python developers should avoid expressing interest in
                        > something that hasn't already been written.[/color]

                        I did not say that. I said we don't normally invite people to work
                        on anything - I said not that we *should* not invite them. Now that
                        you mention it, I find that there is an important exception from my
                        factual statement: I do regularly ask people reporting bugs
                        or requesting features to work fixing the bugs or implementing the
                        features. It is perfectly fine if they say "no" then. If they say
                        yes, there is an implied promise that I'll review their code when
                        they are done.

                        As it appears to be clear that you are not going to implement
                        an AES module in the foreseeable future, and as it also seems
                        to be clear that you cannot talk me into changing my views on
                        how Python should be developed, I think further discussing this
                        entire thing is pointless.

                        Regards,
                        Martin

                        Comment

                        • Paul Rubin

                          #72
                          Re: What's so funny? WAS Re: rotor replacement

                          "Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue]
                          > Indeed, if it was a single new function to an existing module, I would
                          > not require that this be delivered to users first. It is entire new
                          > libraries that I worry about.[/color]

                          Why is it different if a single new function is added to an existing
                          module, or if the single new function has the boilerplate of a new
                          module wrapped around it?

                          Look at the sha and md5 modules. They are very similar in both
                          interface and implementation. The only internal function that's
                          really different is the update operation; they actually might have
                          been combined into one module that did the other operations with the
                          same code. But, it's also reasonable to have them as separate
                          modules. If users start needing sha256, it could be done the same
                          way, one new update operation and the rest boilerplate, but in
                          practice it would probably be a separate module.

                          Are you saying if there was user demand for an sha256 module and
                          someone wrote one, you'd still require a year of separate distribution?
                          [color=blue]
                          > A module *is* typically more complex than a single function. If
                          > your new module has only a single new function, we should discuss
                          > whether it really needs to be a separate module.[/color]

                          I previously had the mistaken belief that urandom was a new module
                          rather than a function inserted into an existing module. Note that
                          the urandom's implementation is not ultra-trivial.

                          An AES or DES addition to an existing module that implements just one
                          call:
                          ECB(key, data, direction)
                          would be a huge improvement over what we have now. A more complete
                          crypto module would have some additional operations, but ECB is the
                          only one that's really essential. I already have a pure-Python module
                          that does all the other operations using ECB as a subroutine.
                          It's speed isn't great but it's useable in some real applications.
                          It's only the ECB operation that's intolerably slow in Python.

                          If you think a function like that could be added to some existing
                          module with less hassle than adding a new module, then I can write one
                          and submit it.
                          [color=blue][color=green]
                          > > Well, if he indicates that it's not a policy and that the question is
                          > > still open, then I could see getting interested again in writing an
                          > > AES module. At the moment I continue to see his python-dev post as
                          > > quite discouraging.[/color]
                          >
                          > And, again, I consider this perfectly fine. This would be a volunteer
                          > effort, and volunteers are free to work on whatever they please.[/color]

                          Well, volunteers are more likely to work on modules that are mentioned
                          as being welcome by the developers, than modules affected by explicit
                          prior developers' public decisions that cast a chill over the hope of
                          ever getting such a module accepted.
                          [color=blue]
                          > Furthermore, people who want an AES module for Python could get one from[/color]

                          Come on, you're being deliberately obtuse, we've discussed this over
                          and over. There are plenty of AES modules that people can get from
                          somewhere. The topic is what it takes to have an AES module that
                          people don't NEED to get from anywhere, because they already have it
                          from having Python installed. Do I have to keep repeating "batteries
                          included" until you understand what it means?
                          [color=blue]
                          > http://sourceforge.net/projects/cryptkit/
                          >
                          > Maybe Bryan Mongeau will contribute this code to Python some day.[/color]

                          Well, that code has been around for over a year, people are using it,
                          etc. Are you saying you'll support its inclusion if Bryan offers to
                          contribute it? I've examined that module, I wouldn't consider it
                          ideal for the core (besides AES, it has some complicated additional
                          functions that aren't useful to most people), but it would certainly
                          take care of my AES needs (it's apparently missing DES though).
                          [color=blue]
                          > I did not say that. I said we don't normally invite people to work
                          > on anything - I said not that we *should* not invite them.[/color]

                          I would say that inviting people to work on a module for the stdlib
                          means the developers have thought about whether such a module would be
                          useful and worth including, and are indicating that they're favorable
                          to the idea. However, you wrote:

                          In Message-ID: <41f4324a$0$154 7$9b622d9e@news .freenet.de>
                          So if the module was primarily written to be included in the core, I
                          would initially reject it for that very reason. After one year or so
                          in its life, and a recognizable user base, inclusion can be considered.

                          The context was new modules in general, not specifically an AES
                          module. Since "considered " means "thought about", so you said
                          inclusion shouldn't even be thought about until the module is already
                          done. That's completely in conflict with the idea of inviting anyone
                          to work on a new module, since inviting means that there's been some
                          thought.
                          [color=blue]
                          > Now that you mention it, I find that there is an important exception
                          > from my factual statement: I do regularly ask people reporting bugs
                          > or requesting features to work fixing the bugs or implementing the
                          > features. It is perfectly fine if they say "no" then. If they say
                          > yes, there is an implied promise that I'll review their code when
                          > they are done.[/color]

                          I would say there's an implied promise of something more than a code
                          review. There's an implied statement that you agree that the proposed
                          new functionality is useful, which means the patch has a good chance
                          of being accepted to the stdlib if it's not too messy or cumbersome.
                          That's a heck of a lot different from saying "why don't you write that
                          patch and distribute it independently for a year on a purely
                          speculative basis, and then I'll think about whether it's worthwhile
                          to include it or not".
                          [color=blue]
                          > As it appears to be clear that you are not going to implement an AES
                          > module in the foreseeable future,[/color]

                          The reason for that is as far as I can tell, even if I follow 100% of
                          your prescription of writing the module, releasing it independently
                          and supporting it for a year, and then offering to contribute it
                          complete with favorable user reviews and a promise of two years of
                          further support, the probability of it being accepted and included is
                          still close to zero. In more recent messages you've suggested that my
                          reading of that probability is wrong and that it's actually higher
                          than zero.

                          So let me just ask you one final question: suppose I do all that
                          stuff. The question: in your personal opinion, based on the best
                          information you have, what is the your own subjective estimate of the
                          probability?

                          I won't say I'd immediately replace my estimate with yours, but if you
                          name a reasonably high number and tell me that you really believe that
                          number, then I could get interested again.
                          [color=blue]
                          > and as it also seems to be clear that you cannot talk me into
                          > changing my views on how Python should be developed, I think further
                          > discussing this entire thing is pointless.[/color]

                          Well, you can have whatever views you want, but one I've thing I've
                          realized from this thread is that many of the frustrations I often
                          encounter with Python are a direct result of a development process
                          that's successful in some ways but dysfunctional in others.

                          Comment

                          • Skip Montanaro

                            #73
                            Re: What's so funny? WAS Re: rotor replacement

                            [color=blue][color=green]
                            >> http://www.python.org/pypi
                            >> THIS IS ALL PYTHON.[/color][/color]

                            Paul> No. Those are programs people have written in Python or as Python
                            Paul> extensions.

                            What's your point? That I have to download and perhaps install them to use
                            them? In that case, how are these two scenarios different:

                            * I have to download and build the MySQLdb package to talk to MySQL
                            servers from Python code

                            * I have to ensure that the readline library and include files are
                            installed on my system before the readline module (which is included
                            in the core distribution) can be built

                            I and many other people happily use external packages other people have
                            written as well as make stuff available. My guess is that you do as well.
                            If everyone adopted your position that it wasn't Python unless it had been
                            added to the core, we'd all be reinventing lots of wheels or tackling much
                            less challenging tasks, if we programmed in Python at all. Here's an
                            incomplete list of stuff not in the core I have used happily over the past
                            several years to do my jobs using Python:

                            * MySQLdb, Sqlite, pycopg, sybase-python - all database modules
                            * CSV, Object Craft's csv, DSV - csv modules predating csv in the core
                            * SpamBayes
                            * Quixote
                            * Docutils
                            * MoinMoin
                            * Pyrex
                            * Psyco
                            * PyInline
                            * PyGTK
                            * xmlrpclib before it was in the core
                            * MAL's mx.DateTime before the core datetime module was available
                            * timeout_socket before sockets supported timeouts

                            Many of those things I could never have written myself, either for lack of
                            time, lack of skill or both. I'm grateful they were available when I needed
                            them and feel no qualms about using them even though they are not
                            distributed with Python proper.

                            Notice another interesting feature of several of those items: csv,
                            xmlrpclib, mx.DateTime, timeout_socket. They were all modules I used that
                            eventually wound up in the core in some fashion. They didn't go in the core
                            first, then demonstrate their usefulness. It was the other way around.

                            Not everything that is useful belongs in the core distribution. I think you
                            are confusing "batteries included" with "everything , including the kitchen
                            sink".

                            Skip

                            Comment

                            • Skip Montanaro

                              #74
                              Re: What's so funny? WAS Re: rotor replacement

                              [color=blue][color=green]
                              >> What matters is the code complexity, not whether something is in a
                              >> separate module or not.[/color][/color]

                              Martin> A module *is* typically more complex than a single function.

                              And one that deals with cryptography is likely to be even more complex.

                              Skip

                              Comment

                              • Paul Rubin

                                #75
                                Re: What's so funny? WAS Re: rotor replacement

                                Skip Montanaro <skip@pobox.com > writes:[color=blue]
                                > What's your point? That I have to download and perhaps install them to use
                                > them? In that case, how are these two scenarios different:
                                >
                                > * I have to download and build the MySQLdb package to talk to MySQL
                                > servers from Python code
                                >
                                > * I have to ensure that the readline library and include files are
                                > installed on my system before the readline module (which is included
                                > in the core distribution) can be built[/color]

                                The difference is that once Python is installed on your machine and
                                you can get a ">>>" prompt, you have readline available right away but
                                you have to download something to use MySQLdb. Whoever took care of
                                your Python installation, and it may not have been you, also took care
                                of readline. The past several OS distributions I've installed have
                                included Python and readline out of the box, so I never had to think
                                about readline. The last time I used a Python instance that didn't
                                come with the OS (on Windows XP at work), the IT department had
                                installed Python on my desktop before I started using it, so I still
                                didn't have to worry about readline. But any module that doesn't
                                come in the distro, I have to download myself.
                                [color=blue]
                                > I and many other people happily use external packages other people have
                                > written as well as make stuff available. My guess is that you do as well.[/color]

                                No, I don't. I do use them sometimes but I'm unhappy about them. If
                                I can write something using a core module instead of an external
                                module, I prefer to use the core module. So I'll generally use dbm
                                instead of MySQL unless I really need MySQL, which I haven't yet in
                                Python (I've used MySQL with Perl dbi, but Perl, you know, shudder).

                                Also, external module installation scripts often don't work properly,
                                so I end up having to wrestle the code to get it installed. And if a
                                geek like me has such trouble installing external modules, what hope
                                does a normal end-user have? Maybe if you're using Windows, that
                                stuff has been debugged better, but I've had poor results under
                                GNU/Linux.

                                I've had this discussion here before, maybe not with you. What I
                                really want is zero installations of anything. I just want to go to
                                the store and buy a new computer and have a complete OS install with
                                full sources and a full set of applications including Python already
                                installed when I first power it up. My practical approximation is to
                                buy a new computer, immediately reformat the HD to remove the icky
                                Redmond virus, and then install a GNU/Linux distro that includes
                                Python (and readline). If Python really aims for world domination,
                                that means it has to shoot for being preinstalled on almost every new
                                computer the way Windows is now. And all the interesting modules
                                should be there, maybe in a "contrib" directory that gets little or no
                                maintenance priority from the core team.
                                [color=blue]
                                > If everyone adopted your position that it wasn't Python unless it
                                > had been added to the core, we'd all be reinventing lots of wheels
                                > or tackling much less challenging tasks, if we programmed in Python
                                > at all. Here's an incomplete list of stuff not in the core I have
                                > used happily over the past several years to do my jobs using Python:[/color]

                                That makes no sense at all. That list is a list of programs written
                                in the Python language. They are Python programs, where Python is an
                                adjective. Python, the noun referring to a piece of software,
                                generally means the stuff in the Python distro. That doesn't stop
                                programs outside the distro from being useful. Mediawiki is a PHP
                                program. That doesn't mean Mediawiki is part of PHP.
                                [color=blue]
                                > * MySQLdb, Sqlite, pycopg, sybase-python - all database modules[/color]

                                These should all be in the core if Python wants to be a serious
                                competitor to PHP, which comes with interfaces for those db's and
                                several additional ones besides. That these modules are missing are a
                                significant library deficiency.
                                [color=blue]
                                > * CSV, Object Craft's csv, DSV - csv modules predating csv in the core[/color]

                                That's fixed now, csv is in the core.
                                [color=blue]
                                > * SpamBayes[/color]

                                I have the impression this is an application and not a module, or
                                anyway is written mainly to support one application. Should be
                                separate. Also, it's written in Python(?) rather than C, which means
                                the installation headaches from not being in the core aren't so bad.
                                [color=blue]
                                > * Quixote[/color]

                                Don't know what this is.
                                [color=blue]
                                > * Docutils[/color]

                                Should be in the core if it's what I think it is.
                                [color=blue]
                                > * MoinMoin[/color]

                                Application, should be separate. Also, GPL'd, I think. Can't be
                                distributed under PSF license.
                                [color=blue]
                                > * Pyrex[/color]

                                Sort of problematic, would be interesting to have something like this
                                in the core but maybe Pyrex as it currently stands isn't the answer.

                                I have the impression that PyPy is going to depend on Pyrex in a
                                fundamental way, so it will have to be in the core when we dump CPython.
                                [color=blue]
                                > * Psyco[/color]

                                I think this isn't ready for prime time yet. Should go into the core
                                once it is.
                                [color=blue]
                                > * PyInline[/color]

                                Not sure what this is.
                                [color=blue]
                                > * PyGTK[/color]

                                wxPython might be a better choice. wxpython.org quotes Guido as
                                saying "wxPython is the best and most mature cross-platform GUI
                                toolkit, given a number of constraints. The only reason wxPython isn't
                                the standard Python GUI toolkit is that Tkinter was there first". I
                                don't have direct experience with either wxPython or PyGTK, though.
                                If I can get by with Tkinter, I'd rather do that, since Tkinter is in
                                the core.
                                [color=blue]
                                > * xmlrpclib before it was in the core[/color]

                                1. Did you really need this, instead of some more reasonable rpc format?
                                xdrlib has been in the core forever.
                                2. Isn't xmlrpclib written in Python?
                                [color=blue]
                                > * MAL's mx.DateTime before the core datetime module was available[/color]

                                See, as Python improved, those things went into the core.
                                [color=blue]
                                > * timeout_socket before sockets supported timeouts[/color]

                                Could you use sigalarm instead?
                                [color=blue]
                                > Many of those things I could never have written myself, either for
                                > lack of time, lack of skill or both. I'm grateful they were
                                > available when I needed them and feel no qualms about using them
                                > even though they are not distributed with Python proper.[/color]

                                Sure, it's fine if you have all those modules and you write a Python
                                program that uses, say, five of them. External modules aren't so bad
                                when the developer and the end user are the same person. What happens
                                if you send your Python program to a nonprogrammer friend who has just
                                a vanilla Python installation? Now he has to download and install
                                those five modules too. You send him the url's where you got the
                                modules a year ago. What are the chances that the 5 url's even all
                                still work, much less the chance of him being able to install and run
                                all 5 of the modules without needing help? What if the versions he
                                downloads (from separate developers) have gotten out of sync with each
                                other and can't interoperate any more?

                                If Python's maintainers are going to recommend those modules to people
                                as a substitute for providing those functions in the core, it would be
                                a big help if the modules were mirrored on python.org instead of
                                merely linked, since a lot of links turn into 404's over time.
                                [color=blue]
                                > Notice another interesting feature of several of those items: csv,
                                > xmlrpclib, mx.DateTime, timeout_socket. They were all modules I used that
                                > eventually wound up in the core in some fashion. They didn't go in the core
                                > first, then demonstrate their usefulness. It was the other way around.[/color]

                                I'm not sure about timeout_socket and it sounds like it should have
                                just been a patch to the socket module, not a new module. csv is
                                quite a complex module and took a lot of tweaking and PEP editing
                                before standardization . But the need for it was obvious; the only
                                good reason it wasn't in the core ages ago was that no one had done
                                the work of writing it and shaking it out. xmlrpclib, not sure. How
                                long was it in separate distribution? Also, xmlrpc is a pretty new
                                protocol so it took a while before people wanted it. DES has been
                                around since the 1970's and AES has about the same users as DES, so
                                there's a known level of demand. That should be enough to say yes or
                                no to whether there should be a core module or not.

                                Also, your notion of trying to create a "category king" of AES modules
                                doesn't reflect how these things work. There are at least 10
                                different AES modules that provide the same AES function. If somebody
                                is using one, they have no reason to switch to another. If it takes
                                20 visible users for including a module to be worthwhile, and those 10
                                modules have 5 users each, those populations are going to stay stable
                                until one of them goes in the core and becomes the standard. (And
                                actually: mxCrypto is the most capable of these packages and might be
                                the one with the most users, but it's completely unsuitable for the
                                core because of its size).
                                [color=blue]
                                > Not everything that is useful belongs in the core distribution. I
                                > think you are confusing "batteries included" with "everything ,
                                > including the kitchen sink".[/color]

                                Well, if you compare the Python stdlib with the toolkits that come
                                with competing languages, say PHP or Java, you can see that Python's
                                lib could be enhanced with considerably more stuff without being
                                excessive.

                                Comment

                                Working...