rotor replacement

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

Skip Montanaro <skip@pobox.com > writes:[color=blue]
> And one that deals with cryptography is likely to be even more complex.[/color]

No. The AES module would have about the same complexity as the SHA module.

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin <http://phr.cx@NOSPAM.i nvalid> writes:[color=blue]
> actually: mxCrypto is the most capable of these packages and might be
> the one with the most users, but it's completely unsuitable for the
> core because of its size).[/color]

Oops, I should say, mxCrypto itself isn't that large; the issue is
that it needs OpenSSL which is a big unwieldy program. Having
mxCrypto in the core as an OpenSSL interface is a legitimate notion.
But there should be something that doesn't depend on OpenSSL.

[Martin v. Löwis]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin wrote:[color=blue]
> An AES or DES addition to an existing module that implements just one
> call:
> ECB(key, data, direction)
> would be a huge improvement over what we have now.[/color]

Apparently, people disagree on what precisely the API should be. E.g.
cryptkit has

obj = aes(key)
obj.encrypt(dat a)

I think I would prefer explicit encrypt/decrypt methods over a
direction parameter. Whether or not selection of mode is a separate
parameter, or a separate method, might be debatable - I'ld personally
prefer a separate method. However, we would have to ask users.
[color=blue]
> If you think a function like that could be added to some existing
> module with less hassle than adding a new module, then I can write one
> and submit it.[/color]

I would trust my intuition more for a single function than for an
entire API. In this specific proposal, I think I would trust my
intuition and reject the ECB function because of the direction
argument.
[color=blue]
> Come on, you're being deliberately obtuse, we've discussed this over
> and over. There are plenty of AES modules that people can get from
> somewhere. The topic is what it takes to have an AES module that
> people don't NEED to get from anywhere, because they already have it
> from having Python installed. Do I have to keep repeating "batteries
> included" until you understand what it means?[/color]

I fully understand what you desire - to include the module "as a
battery". What makes this decision difficult is that you fail to
understand that I don't want included batteries so much that I
would accept empty or leaking batteries.
[color=blue][color=green]
>>http://sourceforge.net/projects/cryptkit/[/color]
> Well, that code has been around for over a year, people are using it,
> etc. Are you saying you'll support its inclusion if Bryan offers to
> contribute it?[/color]

*Now* you get it. Precisely that. I would ask the users what they
think about the API (shouldn't be too difficult because the module
does have users) and what they think about other aspects (performance,
stability, and so on).
[color=blue]
> I've examined that module, I wouldn't consider it
> ideal for the core (besides AES, it has some complicated additional
> functions that aren't useful to most people)[/color]

Ok, that would be a problem. If this is a simple removal of functions
that you'ld request (which functions?), I'ld try to collect opinions
on that specific issue, and ask Bryan whether he could accept
removal of these functions.
[color=blue]
> So if the module was primarily written to be included in the core, I
> would initially reject it for that very reason. After one year or so
> in its life, and a recognizable user base, inclusion can be considered.
>
> The context was new modules in general, not specifically an AES
> module. Since "considered " means "thought about", so you said
> inclusion shouldn't even be thought about until the module is already
> done. That's completely in conflict with the idea of inviting anyone
> to work on a new module, since inviting means that there's been some
> thought.[/color]

I rarely invite people to work on new modules. For new modules, I
normally propose that they develop the module, and ship it to users
for some time.

I may have made exceptions to this rule in the past, e.g. when the
proposal is to simply wrap an existing C API in a Python module
(like shadow passwords). In this case, both the interface and
the implementation are straight-forward, and I expect no surprises.
For an AES module (or most other modules), I do expect surprises.
[color=blue]
> I would say there's an implied promise of something more than a code
> review. There's an implied statement that you agree that the proposed
> new functionality is useful, which means the patch has a good chance
> of being accepted to the stdlib if it's not too messy or cumbersome.[/color]

I have said many times that I am in favour of including an AES
implementation in the Python distribution, e.g. in

http://mail.python.org/pipermail/python-dev/2003-April/034963.html

What I cannot promise is to include *your* AES implementation,
not without getting user feedback first. The whole notion of
creating the module from scratch just to include it in the core
strikes me as odd - when there are so many AES implementations
out there already that have been proven useful to users.
[color=blue]
> So let me just ask you one final question: suppose I do all that
> stuff. The question: in your personal opinion, based on the best
> information you have, what is the your own subjective estimate of the
> probability?[/color]

Eventually, with hard work, I estimate the chances at, say, 90%. That
is, eventually, unless the code itself shows flaws, the module *will*
be included. However, initially, when first proposed, the chances are
rather like 10%. I.e. people will initially object. Decision processes
take their time, and valid concerns must be responded to. I personally
think that there is a good response to each concern, but it will take
time to find it. Before that, it will take time to find out what
precisely the concern is.

Regards,
Martin

[Martin v. Löwis]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin wrote:[color=blue]
> (And
> actually: mxCrypto is the most capable of these packages and might be
> the one with the most users, but it's completely unsuitable for the
> core because of its size).[/color]

mxCrypto is primarily unsuitable for the core because Marc-Andre Lemburg
will never ever contribute it. He is very concerned about including
crypto code with the Python distribution, so he certainly won't
contribute his own.

Regards,
Martin

[Nick Craig-Wood]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin <http> wrote:[color=blue]
> An AES or DES addition to an existing module that implements just one
> call:
> ECB(key, data, direction)
> would be a huge improvement over what we have now. A more complete
> crypto module would have some additional operations, but ECB is the
> only one that's really essential.[/color]

I would hate to see a module which only implemented ECB. Sure its the
only operation necessary to build the others out of, but its the least
secure mode of any block cipher.

If you don't offer users a choice, then they'll use ECB and just that
along with all its pitfalls, meanwhile thinking that they are secure
because they are using AES/DES...

For those people following along at home (I'm sure everyone who has
contributed to thread knows this already) I tried to find a simple
link to why ECB is bad, this PDF is the best I could come up with, via
Google's Cache.

http://www.google.com/search?q=cache:U5-RsbkSs0MJ:www.cs.chalmers.se/Cs/Grundutb/Kurser/krypto/lect04_4.pdf

--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

[Skip Montanaro]

Re: What's so funny? WAS Re: rotor replacement

Paul> I've had this discussion here before, maybe not with you. What I
Paul> really want is zero installations of anything.

Fine. Go build a sumo distribution and track the normal CPython. The
problem isn't all that new. (Take a look at scipy.org for one take on that
theme. Of course Linux distros have been doing their take on this forever.)
[color=blue][color=green]
>> If everyone adopted your position that it wasn't Python unless it had
>> been added to the core, we'd all be reinventing lots of wheels or
>> tackling much less challenging tasks, if we programmed in Python at
>> all. Here's an incomplete list of stuff not in the core I have used
>> happily over the past several years to do my jobs using Python:[/color][/color]

Paul> That makes no sense at all. That list is a list of programs
Paul> written in the Python language. They are Python programs, where
Paul> Python is an adjective.

No, many of them are just modules or programming frameworks.
[color=blue][color=green]
>> * SpamBayes[/color][/color]

Paul> I have the impression this is an application and not a module,

Yes, you're correct.
[color=blue][color=green]
>> * Quixote[/color][/color]

Paul> Don't know what this is.

Web app framework.
[color=blue][color=green]
>> * Docutils[/color][/color]

Paul> Should be in the core if it's what I think it is.

Probably will be someday.
[color=blue][color=green]
>> * MoinMoin[/color][/color]

Paul> Application, should be separate. Also, GPL'd, I think. Can't be
Paul> distributed under PSF license.

Sure.
[color=blue][color=green]
>> * Psyco[/color][/color]

Paul> I think this isn't ready for prime time yet. Should go into the
Paul> core once it is.

It's getting close for those of us with Intel chips in our boxes.
[color=blue][color=green]
>> * PyInline[/color][/color]

Paul> Not sure what this is.

A module for inlining C code within a Python module. Also see Weave from
the scipy.org folks. It was inspired by the Perl Inline::C module.
[color=blue][color=green]
>> * PyGTK[/color][/color]

Paul> wxPython might be a better choice.

Doesn't matter. At work they decreed GTK as the GUI platform long before I
came along (they also use gtkmm for C++ apps). It's still an example of a
broadly useful package available outside the core distribution.
[color=blue][color=green]
>> * xmlrpclib before it was in the core[/color][/color]

Paul> 1. Did you really need this, instead of some more reasonable rpc
Paul> format?

Yes, for several years I used a homegrown RPC solution behind the Musi-Cal
website that was Python only. Eventually Mojam (a Perl shop) bought
Musi-Cal (a Python shop). I switched to XML-RPC with little effort. At one
point we also had Java talking XML-RPC.

Paul> xdrlib has been in the core forever.

Sure. But it's somewhat lower level than XML-RPC and isn't really an RPC
protocol. It's just a marshalling protocol and is probably not as flexible
as XML-RPC at that.

Paul> 2. Isn't xmlrpclib written in Python?

Yes. The implementation language is just a detail. I also use Fredrik
Lundh's sgmlop library to accelerate XML-RPC and play some other games when
I know I'm talking Python-to-Python (marshal my args, then XML-RPC the
result passing a single argument between the client and server).
[color=blue][color=green]
>> * MAL's mx.DateTime before the core datetime module was available[/color][/color]

Paul> See, as Python improved, those things went into the core.

Sure, than that's what Martin has been trying to tell you about your AES
proposal. Put it out there, refine it, and get it into the core when it's
mature.
[color=blue][color=green]
>> * timeout_socket before sockets supported timeouts[/color][/color]

Paul> Could you use sigalarm instead?

I suppose. That's not the point though. I'm not married to the concept as
you seem to be that something has to be in the core distribution to be of
use to me. I'm perfectly happy incorporating solutions other people
provide. I believe you will find I am in the majority in this regard.
[color=blue][color=green]
>> Many of those things I could never have written myself, either for
>> lack of time, lack of skill or both. I'm grateful they were
>> available when I needed them and feel no qualms about using them even
>> though they are not distributed with Python proper.[/color][/color]

Paul> Sure, it's fine if you have all those modules and you write a
Paul> Python program that uses, say, five of them. External modules
Paul> aren't so bad when the developer and the end user are the same
Paul> person. What happens if you send your Python program to a
Paul> nonprogrammer friend who has just a vanilla Python installation?

I figure out some other packaging solution. In my world most of the
software I write is for my employer, so this is not a problem I face very
often. People use freeze, py2exe, py2app or other packaging solutions to
solve most/all of these problems.

Paul> Now he has to download and install those five modules too. You
Paul> send him the url's where you got the modules a year ago. What are
Paul> the chances that the 5 url's even all still work, much less the
Paul> chance of him being able to install and run all 5 of the modules
Paul> without needing help? What if the versions he downloads (from
Paul> separate developers) have gotten out of sync with each other and
Paul> can't interoperate any more?

This is the well-known "CPAN in Python" problem. People are working on it.
Perhaps you would like to spend some energy helping solve it. If so, join
the catalog-sig.
[color=blue][color=green]
>> Notice another interesting feature of several of those items: csv,
>> xmlrpclib, mx.DateTime, timeout_socket. They were all modules I used
>> that eventually wound up in the core in some fashion. They didn't go
>> in the core first, then demonstrate their usefulness. It was the
>> other way around.[/color][/color]

Paul> I'm not sure about timeout_socket and it sounds like it should
Paul> have just been a patch to the socket module, not a new module.

Sure, but a shim between the socket module and Python modules that used it
was a good first approximation to the problem. (I am also a firm believer
in successive approximation to problem solving, especially when I don't know
enough about the problem to know precisely what form the final solution will
take.)

Paul> csv is quite a complex module and took a lot of tweaking and PEP
Paul> editing before standardization . But the need for it was obvious;
Paul> the only good reason it wasn't in the core ages ago was that no
Paul> one had done the work of writing it and shaking it out.

Actually, there were at least two fairly mature implementations of CSV
modules out there before the PEP was a twinkle in anyone's eye. The authors
of those modules got together and wrote the current PEP and module from
scratch based upon their collective experience. I think the effort of
having a couple versions out in the field followed by joint effort to
produce something worthy of inclusion in the core is an excellent
demonstration of what Martin has been saying all along.

Paul> xmlrpclib, not sure. How long was it in separate distribution?

Not all that long. XML-RPC itself hadn't been around very long before
Fredrik wrote xmlrpclib. Both the protocol and xmlrpclib (as well as
similar modules for other languages) caught on pretty quickly.

Skip

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

"Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue]
> Apparently, people disagree on what precisely the API should be. E.g.
> cryptkit has
>
> obj = aes(key)
> obj.encrypt(dat a)[/color]

I don't disagree about the API. The cryptkit way is better than ECB
example I gave, but the ECB example shows it's possible to do it in
one call.
[color=blue]
> I think I would prefer explicit encrypt/decrypt methods over a
> direction parameter. Whether or not selection of mode is a separate
> parameter, or a separate method, might be debatable[/color]

I prefer separate methods too, however if it was done with a direction
flag instead, it wouldn't really cause a problem. As long as the
functionality is there, I can use it.
[color=blue]
> I would trust my intuition more for a single function than for an
> entire API. In this specific proposal, I think I would trust my
> intuition and reject the ECB function because of the direction argument.[/color]

As an experienced user of a lot of these packages, I can tell you I've
seen it done both ways and I have a slight preference for separate
calls, but it really doesn't matter one way or the other and it's not
worth getting in a debate about it or having a committee design the
API and worry about such trivial issues.

BTW, the main reason to reject the example ECB function is that
creating a key object ("key schedule") from a string can take
significant computation (sort of like compiling a regexp) so the ECB
function for some ciphers would have to cache the object like the
regexp module does. Yuck.

The direction flag question would normally be between:

key = aes.key(key_dat a)
ciphertext = key(plaintext, "e")

or
key = aes.key(key_dat a)
ciphertext = key.encrypt(pla intext)

FWIW, another way to do it, also sometimes preferable, is:

key = aes.ecb(key_dat a, "e") # e for encryption, or "d" for decryption
ciphertext = key(plaintext)

I think the module I proposed did it this last way, but I haven't
looked at it in a while.

The point is that when faced with yet another crypto package, I don't
get in too concerned about which simple API variant it uses to do such
a basic operation. I just care that the operation is available. I
look in the docs to find that package's particular API for that
operation, and I do what the docs say.

I should make it clear that this module is like Python's low-level
"thread" module in that you have to know what you're doing in order to
use it directly without instantly getting in deep trouble. Most
applications would instead use it indirectly through one or more
intermediate layers.
[color=blue]
> I fully understand what you desire - to include the module "as a
> battery". What makes this decision difficult is that you fail to
> understand that I don't want included batteries so much that I
> would accept empty or leaking batteries.[/color]

I do understand that, and the prospect of empty or leaking batteries
is vitally important to considering whether to include a battery
that's included, but for the purposes of an included-battery
discussion, the characteristics of NON-included batteries is not
relevant, given that we know they exist.
[color=blue][color=green][color=darkred]
> >>http://sourceforge.net/projects/cryptkit/ ...>[/color]
> > I've examined that module, I wouldn't consider it
> > ideal for the core (besides AES, it has some complicated additional
> > functions that aren't useful to most people)[/color]
>
> Ok, that would be a problem. If this is a simple removal of functions
> that you'ld request (which functions?),[/color]

OK. First you have to decide whether you want a general crypto
toolkit, or just an AES module. I've been concentrating on just an
AES module (or rather, a generic block cipher module with AES and DES)
since I figure that creates fewer areas of controversy, etc. I think
it's too early to standardize a fancy toolkit. Once there's block
ciphers, we can think about adding more stuff afterwards.

For that module, I'd say remove everything except AES and maybe
SHA256, and ask that DES be added. SHA256 is possibly useful, but
isn't really part of an encryption package; it can be separated out
like the existing sha and md5 modules. Also, it should be brought
into PEP 247 compliance if it's not already.

Rationale: I'd get rid of the entropy module now that os.urandom is
available. Having the OS provide entropy is much better than trying
to do it in user code. I'd get rid of the elliptic curve stuff unless
there's some widely used standard or protocol that needs that
particular implementation. Otherwise, if I want ECC in a Python
program, I'd just do it on characteristic-p curves in pure Python
using Python longs. (Bryan's package uses characteristic-2 curves
which means the arithmetic is all boolean operations, that are more
efficient on binary CPU's, especially small ones. But that means the
module has to be written in C, because doing all those boolean
operations in Python is quite slow. It would be like trying to do
multi-precision arithmetic in Python with Python ints instead of
longs). Once there's a widely accepted standard for ECC like there is
for AES, then I'd want the stdlib to have an implementation of the
standard, but right now there are just a lot of incompatible,
nonstandard approaches running around.

If SHA256 is accepted then SHA512/SHA384 (these are basically the
same) might as well also be. Not many people are using any of these
hash functions right now. Usage will increase over time (they are US
federal standards like AES), and they'll probably be worth adding
eventually. I'm indifferent to whether they're added now.

I think I'd include RC4 under the "toolkit" approach, if it's not
already there. I'd also include a pair of functions like the ones in
p3.py, i.e. an utterly minimal API like:

ciphertext = encrypt_string( key_string, plaintext)
plaintext = decrypt_string( key_string, ciphertext)

that does both encryption and authentication, for key and data strings
of arbitrary size. This would be what people should use instead of
the rotor module. It would be about a 10 line Python function that
calls the block cipher API. The block cipher API itself is intended
for more advanced users.
[color=blue]
> I may have made exceptions to this rule in the past, e.g. when the
> proposal is to simply wrap an existing C API in a Python module
> (like shadow passwords). In this case, both the interface and
> the implementation are straight-forward, and I expect no surprises.[/color]

I'd be happy with an AES module that simply wrapped a C API and that
should be pretty much surprise-free. It would be about like the SHA
module in terms of complexity. What I proposed tries to be a bit more
Pythonic but I can live without that.
[color=blue]
> For an AES module (or most other modules), I do expect surprises.[/color]

Well, the hmac module was added in 2.2 or 2.3, without any fuss. It's
written in Python and is somewhat slow, though. What kind of
development process should it take to replace it in the stdlib with a
C module with the exact same interface?

I think you're imagining a basic AES module to be more complicated
than it really is, because you're not so familiar with this type of
module. Also, possibly because I'm making it sound like a lot of work
to write. But that work is just for the C version, and assumes that
it's me personally writing it. What little experience I've had with
Python's C API has been painful, so I figure on having to spend
considerable time wrestling with it. Someone more adapt with the C
API could probably implement the module with less effort than I'd need.
[color=blue]
> I have said many times that I am in favour of including an AES
> implementation in the Python distribution, e.g. in[/color]

Oh, ok. Earlier you said you wanted user feedback before you could
conclude that there was reason to want an AES module at all.
[color=blue]
> What I cannot promise is to include *your* AES implementation,
> not without getting user feedback first.[/color]

That's fine. But I think it's reasonable to actually approach some
users and say "this module is being considered for addition to the
core--could you try plugging it into your applications that now use
other external modules, and say whether you think it will fill your
needs".

That's impossible if consideration doesn't even start until testing is
complete. All one could say then is "here's yet another crypto
module, that does less than the toolkit you're already using, could
you please temporarily drop whatever you're doing and update your
programs to switch from your old module that works, to a new module
that MIGHT work?". If the new module is accepted into the core, of
course, it becomes worth retrofitting the existing toolkits to use it.
[color=blue]
> The whole notion of creating the module from scratch just to include
> it in the core strikes me as odd - when there are so many AES
> implementations out there already that have been proven useful to users.[/color]

We discussed this already. Here are three possible contexts for
designing a crypto module:

1) You're designing it to support some specific application you're
working on. The design will reflect the needs of that application
and might not be so great for a wider range of applications.
2) You're writing a general purpose module for distribution outside
the core (example: mxCrypto). You'll include lots of different
functions, pre-built implementations of a variety of protocols, etc.
You might include bindings that try to be compatible with other
packages, etc. Maybe this can get added to the core someday,
like numarray, but for now, that's a rather big step.
3) You're designing to add basic functionality to the core. Here,
you try to pick a general purpose API not slanted towards a
particular app, and provide just some standard building blocks
that other stuff can be built around. This is more like the
math module, which just does basics: sqrt, sin, cos, etc., with no
attempt at the stuff in a package like numarray. But if there's
a standard like FIPS 80 and if there's good reason to implement
a big subset of it (which there is), then you may as well
implement the standard completely unless there's a good reason
not to (which there isn't; the less important operations are a
few dozen lines of code total).

I think context #3 gets you something better suited for the core and
none of the existing crypto modules were written that way. The same
is in fact true for many of the non-crypto modules, that seem to have
been written in context #1 and would have been better under context #3.

Also, there's the plain fact that none of the authors of the existing
crypto modules have offered to contribute them. So somebody had to
step up and do something.
[color=blue][color=green]
> > what is the your own subjective estimate of the probability?[/color]
> Eventually, with hard work, I estimate the chances at, say, 90%.[/color]

Hmm, this is very very interesting. I am highly confident that all
the purely technical questions (i.e. everything about the API and the
code quality, etc.) can converge to a consensus-acceptable solution
without much hassle. I had thought there were insurmountable
obstacles of a nontechnical nature, mainly caused by legal issues, and
that these are beyond any influence that I might have by writing or
releasing anything.

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

"Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue]
> mxCrypto is primarily unsuitable for the core because Marc-Andre Lemburg
> will never ever contribute it. He is very concerned about including
> crypto code with the Python distribution, so he certainly won't
> contribute his own.[/color]

Oh wait, I confused mxcrypto and m2crypto. Sorry. Anyway, the
technical considerations are similar.

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

Nick Craig-Wood <nick@craig-wood.com> writes:[color=blue]
> I would hate to see a module which only implemented ECB. Sure its the
> only operation necessary to build the others out of, but its the least
> secure mode of any block cipher.[/color]

It's intended as a building block for other modes. Most applications
shouldn't use it directly.
[color=blue]
> If you don't offer users a choice, then they'll use ECB and just that
> along with all its pitfalls, meanwhile thinking that they are secure
> because they are using AES/DES...[/color]

The documentation has to be written somewhat forcefully to tell users
what not to do. I can help with that. I've had to do that a lot,
supporting crypto packages in projects where the other programmers
haven't used crypto very much.

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

Skip Montanaro <skip@pobox.com > writes:[color=blue][color=green][color=darkred]
> >> * Quixote[/color][/color]
>
> Paul> Don't know what this is.
>
> Web app framework.[/color]

I think Python should add a web app framework to its core, again since
it otherwise can't seriously begin to compete with PHP. However,
there are lots of approaches so this is an example of where your
suggested process of letting a bunch of different implementations
circulate before choosing something is a good idea.
[color=blue][color=green][color=darkred]
> >> * PyInline[/color][/color]
>
> Paul> Not sure what this is.
>
> A module for inlining C code within a Python module. Also see Weave from
> the scipy.org folks. It was inspired by the Perl Inline::C module.[/color]

Hmm, sounds like it has the same issues as Pyrex. I'm also not sure
why you'd want both PyInline and Pyrex.
[color=blue][color=green][color=darkred]
> >> * PyGTK[/color][/color]
>
> Paul> wxPython might be a better choice.
>
> Doesn't matter. At work they decreed GTK as the GUI platform long before I
> came along (they also use gtkmm for C++ apps).[/color]

Can't wxPython use GTK?
[color=blue]
> It's still an example of a broadly useful package available outside
> the core distribution.[/color]

I'd say if access to GTK is widely important functionality, then the
core should provide it in some way (e.g. through wxPython) and that's
enough. If your company wants some different (i.e. nonstandard, if
wxPython becomes the standard) form of access, then it can deal with
the consequences of not following standards.
[color=blue]
> Paul> 2. Isn't xmlrpclib written in Python?
> Yes. The implementation language is just a detail.[/color]

I think it's more than a detail. If an external module is written in
Python, I can download it from wherever and include it with my own app
that I send to an end user. I do the work so the end user doesn't
have to. If it's written in C, then the end user has to deal with it.
[color=blue]
> Paul> See, as Python improved, those things went into the core.
>
> Sure, than that's what Martin has been trying to tell you about your AES
> proposal. Put it out there, refine it, and get it into the core when it's
> mature.[/color]

What kind of refinements are you envisioning? This isn't a web
application framework we're talking about. It's more like the sha
module.
[color=blue]
> Paul> Could you use sigalarm instead?
>
> I suppose. That's not the point though. I'm not married to the concept as
> you seem to be that something has to be in the core distribution to be of
> use to me. I'm perfectly happy incorporating solutions other people
> provide.[/color]

So aren't you happier when the other person provides you with a
solution that installs with one command, instead of a solution that
requires you to download N different modules from who knows where, and
install them separately, all while hoping that they haven't been
tampered with? If I'm trying to provide someone else with a solution,
I'd rather use sigalarm than make the end-user download an extra
module, because I think they'll be happier that way.
[color=blue]
> Paul> What happens if you send your Python program to a
> Paul> nonprogrammer friend who has just a vanilla Python installation?
>
> I figure out some other packaging solution. In my world most of the
> software I write is for my employer, so this is not a problem I face very
> often. People use freeze, py2exe, py2app or other packaging solutions to
> solve most/all of these problems.[/color]

Only those people who think that a cross-platform application is one
that works on both XP Home and XP Pro. That does simplify some
things. Life in a cult is often indeed simpler than life in the real
world <wink>.
[color=blue]
> Actually, there were at least two fairly mature implementations of
> CSV modules out there before the PEP was a twinkle in anyone's eye.
> The authors of those modules got together and wrote the current PEP
> and module from scratch based upon their collective experience.[/color]

Yes, CSV is complicated and benefits from that process just like
web app frameworks do. Let's pick another example, the hmac module
that appeared in Python 2.2. It implements the RFC 2104 HMAC algorithm.

Where are the two mature implementations that circulated before the
hmac module was added? Where were the authors pooling their
collective wisdom? Where was the year of user feedback? The answer
is, nothing like that was needed. HMAC is simple enough for a module
author to read RFC 2104 and implement what it says, run some tests,
and declare the module good to go.
[color=blue]
> I think the effort of having a couple versions out in the field
> followed by joint effort to produce something worthy of inclusion in
> the core is an excellent demonstration of what Martin has been
> saying all along.[/color]

Martin is saying the opposite: that he doesn't understand the point of
writing a new module that synthesizes from experiences with old
modules, instead of just using one of the old modules.

I don't think there's a one-size-fits-all answer to any of these
questions. You have to have your hands in the details of a specific
problem, to arrive at the best way to deal with that problem.

[Martin v. Löwis]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin wrote:[color=blue]
> Oh, ok. Earlier you said you wanted user feedback before you could
> conclude that there was reason to want an AES module at all.[/color]

I believe I never said that. I said that I wanted user feedback to
determine whether *this* AES module (where this is either your
from-scratch implementation, or any other specific implementation
contributed) is desirable.
[color=blue]
> Hmm, this is very very interesting. I am highly confident that all
> the purely technical questions (i.e. everything about the API and the
> code quality, etc.) can converge to a consensus-acceptable solution
> without much hassle. I had thought there were insurmountable
> obstacles of a nontechnical nature, mainly caused by legal issues, and
> that these are beyond any influence that I might have by writing or
> releasing anything.[/color]

These obstacles are indeed real. But I believe they are not
unsurmountable. For example, there is the valid complaint that,
in order to export the code from SF, we need to follow U.S.
export laws. 10 years ago, these would have been unsurmountable.
Today, it is still somewhat painful to comply with these laws,
but this is what the PSF is for, which can fill out the forms
necessary to allow exporting this code from the U.S.A.

Regards,
Martin

[Nick Craig-Wood]

Re: What's so funny? WAS Re: rotor replacement

Skip Montanaro <skip@pobox.com > wrote:
[color=blue]
> Fine. Go build a sumo distribution and track the normal CPython.
> The problem isn't all that new. (Take a look at scipy.org for one
> take on that theme. Of course Linux distros have been doing their
> take on this forever.)[/color]

If I'm writing code just for fun. I'll be doing on Debian Linux, then
I can do

apt-get install python-crypto

and I'm away.

However if I'm writing code for work, it has to work on windows as
well, which introduces a whole extra barrier to using 3rd party
modules. Something else to compile. Something else to stick in the
installer. Basically a whole heap of extra work.

I think one of the special things about Python is its batteries
included approach, and a crypto library would seem to be an obvious
battery to install since it doesn't (or needn't) depend on any other
library or application.

--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

[Nick Craig-Wood]

Re: What's so funny? WAS Re: rotor replacement

Paul Rubin <http> wrote:[color=blue]
> "Martin v. Löwis" <martin@v.loewi s.de> writes:[color=green]
> > Apparently, people disagree on what precisely the API should be. E.g.
> > cryptkit has
> >
> > obj = aes(key)
> > obj.encrypt(dat a)[/color]
>
> I don't disagree about the API. The cryptkit way is better than ECB
> example I gave, but the ECB example shows it's possible to do it in
> one call.[/color]

There is a PEP about this...

API for Block Encryption Algorithms v1.0

http://www.python.org/peps/pep-0272.html

--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

Nick Craig-Wood <nick@craig-wood.com> writes:[color=blue]
> There is a PEP about this...
>
> API for Block Encryption Algorithms v1.0
> http://www.python.org/peps/pep-0272.html[/color]

Yes, I know about that and have been in contact with its author. He
and I are in agreement (or at least were in agreement some time ago)
that the proposed API of the new module is an improvement, at least
for a generic module. PEP 272 seems to document the interface of
something that had been implemented for some particular application.

[Paul Rubin]

Re: What's so funny? WAS Re: rotor replacement

"Martin v. Löwis" <martin@v.loewi s.de> writes:[color=blue][color=green]
> > Oh, ok. Earlier you said you wanted user feedback before you could
> > conclude that there was reason to want an AES module at all.[/color]
>
> I believe I never said that. I said that I wanted user feedback to
> determine whether *this* AES module (where this is either your
> from-scratch implementation, or any other specific implementation
> contributed) is desirable.[/color]

If that's what you're saying now, I'll accept it and not bother
looking for your other posts that came across much differently.
[color=blue][color=green]
> > I had thought there were insurmountable
> > obstacles of a nontechnical nature, mainly caused by legal issues,..[/color]
>
> These obstacles are indeed real. But I believe they are not
> unsurmountable. For example, there is the valid complaint that,
> in order to export the code from SF, we need to follow U.S.
> export laws. 10 years ago, these would have been unsurmountable.[/color]

Well, yes, 10 years ago, SF didn't exist <wink>. But there was an ftp
site run by Michael Johnson that had some special server side checks
that made sure the client was in the USA. That was considered good
enough to comply with the export regs, and another guy and I
distributed crypto code (a program that let you use your PC as an
encrypted voice phone) through that site in 1995.

Of course, every time my co-author and I released a new version
through the controlled ftp site, within a day or so the code somehow
managed to show up on another ftp site in Italy with worldwide access.
We (the authors) always managed to be shocked when that happened. But
we had nothing to do with it, so it wasn't our problem.
[color=blue]
> Today, it is still somewhat painful to comply with these laws,
> but this is what the PSF is for, which can fill out the forms
> necessary to allow exporting this code from the U.S.A.[/color]

Well, complying is painful in the sense of being morally repugnant
(people shouldn't have to notify the government in order to exercise
their free speech rights), but the actual process is pretty easy in
terms of the work required. Python should qualify for the TSU
exception which means you just need to send an email to the BXA,
without needing to fill out any forms. I thought that's what you had
done for the rotor module.