reduce() anomaly?

[Erik Max Francis]

Re: How sum() should really be done

Douglas Alan wrote:
[color=blue]
> Ah, that reminds me -- both sum() and reduce() can be removed from
> Python by extending operator.add so that it will take any number of
> arguments.[/color]

reduce can't, since reduce doesn't require the function passed to be
operator.add.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \
\__/ The golden rule is that there are no golden rules. -- George
Bernard Shaw

[BW Glitch]

Re: Python's simplicity philosophy

Douglas Alan wrote:
[color=blue]
> "Dave Brueck" <dave@pythonapo crypha.com> writes:
>
>[color=green][color=darkred]
>>>Of course I am not joking. I see no good coming from the mantra, when
>>>the mantra should be instead what I said it should be:[/color][/color]
>
>[color=green][color=darkred]
>>>"small, clean, simple, powerful, general, elegant"[/color][/color]
>
>[color=green]
>>It's really a matter of taste - both "versions" mean about the same to me
>>(and to me both mean "get rid of reduce()" ;-) ).[/color]
>
>
> No, my mantra plainly states to keep general and powerful features
> over specific, tailored features. reduce() is more general and
> powerful than sum(), and would thus clearly be preferred by my
> mantra.
>
> The mantra "there should be only one obvious way to do it" apparently
> implies that one should remove powerful, general features like
> reduce() from the language, and clutter it up instead with lots of
> specific, tailored features like overloaded sum() and max().[/color]

That's not what _I_ understand. There's room for powerful features, but
when these features gives trouble to the people who just want something
done, then another approach should be taken.

And I'm not talking about stupid people. I'm talking about the
microbiolgist/chemist/physics/etc who is programming because of need. CS
people would do a better job, but they are more costly to bring up in
any proyect that requires especific knowledge in one area.
[color=blue]
> If so,
> clearly this mantra is harmful, and will ultimately result in Python
> becoming a bloated language filled up with "one obvious way" to solve
> every particular idiom. This would be very bad, and make it less like
> Python and more like Perl.
>[/color]

You feel ok? Perl's mantra is "More Than One Way To Do It"...
[color=blue]
> I can already see what's going to happen with sum(): Ultimately,
> people will realize that they may want to perform more general types
> of sums, using alternate addition operations. (For intance, there may
> be a number of different ways that you might add together vectors --
> e.g, city block geometry vs. normal geometry. Or you may want to add
> together numbers using modular arithmetic, without worrying about
> overflowing into bignums.) So, a new feature will be added to sum()
> to allow an alternate summing function to be passed into sum(). Then
> reduce() will have effectively been put back into the language, only
> its name will have been changed, and its interface will have been
> changed so that everyone who has taken CS-101 and knows off the top of
> their head what reduce() is and does, won't easily be able to find it.
>[/color]

I don't get you. There's a reason why special functions can be
overloaded (__init__, __cmp__, etc.; I don't use others very often).
That would allow for this kind of special treatments. Besides GvR would
not accept your scenario.

Also, whytf do you mention so much CS101? Maybe you took the course with
LISP, assembly and Scheme, but AFAIK, not everyone has/had access to
this course. Many people learned to program way before taking CS101.
IMHO, you think that the only people that should make software is a CS
major.
[color=blue]
> Yes, there are other parts of The Zen of Python that point to the
> powerful and general, rather than the clutter of specific and
> tailored, but nobody seems to quote them these days, and they surely
> are ignoring them when they want to bloat up the language with
> unneccessary features like overloaded sum() and max() functions,
> rather than to rely on trusty, powerful, and elegant reduce(), which
> can easily and lightweightedly do everything that overloaded sum() and
> max() can do and quite a bit more.[/color]

GvR (or BDFL, as most people know him) has been very careful with his
design decisions. I've been only for about 2 years 'round here, but I've
seen why list compressions came by, why there's no ternary operator and
why Python uses indentation as block separations. These are design
decisions that GvR took.

And there's a good reason why they _are_ design decisions. (Try to guess
why. :P)
[color=blue]
>
>[color=green][color=darkred]
>>>To me, there is never *one* obviously "right way" to do anything[/color][/color]
>
>[color=green]
>>Never? I doubt this very much. When you want to add two numbers in a
>>programming language, what's your first impulse? Most likely it is
>>to write "a + b".[/color]
>
>
> Or b + a. Perhaps we should prevent that, since that makes two
> obviously right ways to do it!
>[/color]

Even without any algebra, any kid can tell you that 1 + 2 is the same as
2 + 1. Replace 1 and 2 by a and b and you get the same result.

[snip][color=blue][color=green]
>>And that is why I wouldn't be sad if reduce() were to disappear - I don't
>>use reduce() and _anytime_ I see reduce() in someone's code I have to slow
>>way down and sort of rehearse in my mind what it's supposed to do and see if
>>I can successfully interpret its meaning (and, when nobody's looking, I
>>might even replace it with a for-loop!).[/color]
>
>
> C'mon -- all reduce() is is a generalized sum or product. What's
> there to think about? It's as intuitive as can be. And taught in
> every CS curiculum. What more does one want out of a function?
>
> |>oug[/color]

It wasn't obvious for me until later. reduce() is more likely to be used
for optimization. IIRC, some said that optimization is the root of all evil.

Just because it's _obvious_ to you, it doesn't mean it's obvious to
people who self taught programming.

--
Andres Rosado

-----BEGIN TF FAN CODE BLOCK-----
G+++ G1 G2+ BW++++ MW++ BM+ Rid+ Arm-- FR+ FW-
#3 D+ ADA N++ W OQP MUSH- BC- CN++ OM P75
-----END TF FAN CODE BLOCK-----

"Well, That's Just Prime"
"Shut up, Rattrap."
-- Rattrap and Optimus Primal, innumerable occasions

[Terry Reedy]

Re: Python's simplicity philosophy


"Patrick Maupin" <pmaupin@speake asy.net> wrote in message
news:653b7547.0 311121826.65b6b 2d4@posting.goo gle.com...[color=blue]
> And then there are the corner cases, e.g. sum([]) vs.
> reduce(operator .add,[]) (which throws an exception).[/color]

The proper comparison is to reduce(operator .add, [], 0), which does
not throw an exception either. sum(seq, start=0) is equivalent to
reduce(operator .add, seq, start=0) except that sum excludes seq of
string. (The doc specifically says equivalent for number (int) seqs,
so there might be something funny with non-number, non-string seqs.)
In other words, sum is more-or-less a special case of reduce with the
within-context constants operator.add and default start=0 built in
(and the special special case of seq of strings excluded).

I think it a big mistake (that should be repaired in 3.0) that the
result start value was made optional, leading to unnecessary empty-seq
exceptions. I also think the order is wrong and should also be fixed.
I believe it was placed last, after the seq of update values, so that
it could be made optional (which it should not be). But it should
instead come before the seq, to match the update function (result,
seq-item) arg order. This reversal has confused people, including
Guido.
[color=blue]
> (Having said that, I never
> personally argued that reduce() should be removed from the language,
> but I do agree that it does not have to be part of "core" Python,
> and could easily be relegated to a module.)[/color]

If the builtins are reduced in 3.0, as I generally would like, I would
be fine with moving apply, map, filter, and a repaired version of
reduce to a 'fun'ctional or hof module. But the argument of some
seems to be that this batteries-included language should specifically
exclude even that.

Terry J. Reedy

[Douglas Alan]

Re: How sum() should really be done

Erik Max Francis <max@alcyone.co m> writes:
[color=blue]
> Douglas Alan wrote:[/color]
[color=blue][color=green]
>> Ah, that reminds me -- both sum() and reduce() can be removed from
>> Python by extending operator.add so that it will take any number of
>> arguments.[/color][/color]
[color=blue]
> reduce can't, since reduce doesn't require the function passed to be
> operator.add.[/color]

Well, as I said, for this to be true, *all* binary operators (that it
makes sense to) would have to be upgraded to take an arbitrary number
of arguments, like they do in Lisp.

|>oug

[Douglas Alan]

Re: Python's simplicity philosophy

BW Glitch <bwglitch@hotpo p.com> writes:
[color=blue][color=green]
>> The mantra "there should be only one obvious way to do it" apparently
>> implies that one should remove powerful, general features like
>> reduce() from the language, and clutter it up instead with lots of
>> specific, tailored features like overloaded sum() and max().[/color][/color]
[color=blue]
> That's not what _I_ understand. There's room for powerful features,
> but when these features gives trouble to the people who just want
> something done, then another approach should be taken.[/color]

If there's a problem with people not understaning how to sum numbers
with reduce(), then the problem is with the documentation, not with
reduce() and the documentation should be fixed. It is quite easy to
make this fix. Here it is:

FAQ
---
Q: How do I sum a sequence of numbers?

A: from operator import add
reduce(add, seq)

Problem solved.
[color=blue]
> And I'm not talking about stupid people. I'm talking about the
> microbiolgist/chemist/physics/etc who is programming because of
> need.[/color]

If a microbiologist cannot understand the above, they have no business
being a microbiologist. I learned reduce() in high school, and it
didn't take me any longer than the 60 seconds I claim it will take
anyone with a modicum of intelligence.
[color=blue][color=green]
>> If so, clearly this mantra is harmful, and will ultimately result
>> in Python becoming a bloated language filled up with "one obvious
>> way" to solve every particular idiom. This would be very bad, and
>> make it less like Python and more like Perl.[/color][/color]
[color=blue]
> You feel ok? Perl's mantra is "More Than One Way To Do It"...[/color]

If both the mantras cause a language to have general features
replaced with a larger number of specialized features that accomplish
less, then both mantras are bad.
[color=blue][color=green]
>> I can already see what's going to happen with sum(): Ultimately,
>> people will realize that they may want to perform more general
>> types of sums, using alternate addition operations. (For intance,
>> there may be a number of different ways that you might add together
>> vectors -- e.g, city block geometry vs. normal geometry. Or you
>> may want to add together numbers using modular arithmetic, without
>> worrying about overflowing into bignums.) So, a new feature will
>> be added to sum() to allow an alternate summing function to be
>> passed into sum(). Then reduce() will have effectively been put
>> back into the language, only its name will have been changed, and
>> its interface will have been changed so that everyone who has taken
>> CS-101 and knows off the top of their head what reduce() is and
>> does, won't easily be able to find it.[/color][/color]
[color=blue]
> I don't get you. There's a reason why special functions can be
> overloaded (__init__, __cmp__, etc.; I don't use others very
> often). That would allow for this kind of special
> treatments. Besides GvR would not accept your scenario.[/color]

There are often multiple different ways to add together the same data
types, and you wouldn't want to have to define a new class for each
way of adding. For instance, you wouldn't want to have to define a
new integer class to support modular arithmetic -- you just want to
use a different addition operation.
[color=blue]
> Also, whytf do you mention so much CS101?[/color]

Because anyone who has studied CS, which should include a significant
percentage of programmers, will know instantly where to look for the
summing function if it is called reduce(), but they won't necessarily
know to look for sum(), since languages don't generally have a
function called sum(). And everyone else will not know to look for
either, so they might as well learn a more powerful concept in the
extra 30 seconds it will take them.
[color=blue]
> Maybe you took the course with LISP, assembly and Scheme, but AFAIK,
> not everyone has/had access to this course. Many people learned to
> program way before taking CS101.[/color]

As did, I, and I had no problem with reduce() when I learned it long
before I took CS-101.
[color=blue]
> IMHO, you think that the only people that should make software
> is a CS major.[/color]

Did I say that?
[color=blue][color=green][color=darkred]
>>>> To me, there is never *one* obviously "right way" to do anything[/color][/color][/color]
[color=blue][color=green][color=darkred]
>>> Never? I doubt this very much. When you want to add two numbers in a
>>> programming language, what's your first impulse? Most likely it is
>>> to write "a + b".[/color][/color][/color]
[color=blue][color=green]
>> Or b + a. Perhaps we should prevent that, since that makes two
>> obviously right ways to do it![/color][/color]
[color=blue]
> Even without any algebra, any kid can tell you that 1 + 2 is the same
> as 2 + 1. Replace 1 and 2 by a and b and you get the same result.[/color]

Yes, but they are still two *different* ways to to get to that result.
Starting with a and adding b to it, is not the same thing as starting
with b and adding a to it. It is only the commutative law of
arithmetic, as any good second grade student can tell you, that
guarantees that the result will be the same. On the other hand, not
all mathematical groups are albelian, and consequently, a + b != b + a
for all mathematical groups.
[color=blue][color=green]
>> C'mon -- all reduce() is is a generalized sum or product. What's
>> there to think about? It's as intuitive as can be. And taught in
>> every CS curiculum. What more does one want out of a function?
>> |>oug[/color][/color]
[color=blue]
> It wasn't obvious for me until later. reduce() is more likely to be
> used for optimization. IIRC, some said that optimization is the root
> of all evil.[/color]

I don't know what you are getting at about "optimizati on". Reduce()
exists for notational convenience--i.e., for certain tasks it is easer
to read, write, and understand code written using reduce() than it
would be for the corresponding loop--and understanding it is no more
difficult than understanding that a summing function might let you
specify the addition operation that you'd like to use, since that's
all that reduce() is!
[color=blue]
> Just because it's _obvious_ to you, it doesn't mean it's obvious to
> people who self taught programming.[/color]

It was obvious to me when I was self-taught and I taught myself APL in
high-school. It also seemed obvious enough to all the physicists who
used APL at the lab where I was allowed to hang out to teach myself
APL.

|>oug

[Alex Martelli]

Re: Python's simplicity philosophy

Terry Reedy wrote:
...[color=blue][color=green]
>> personally argued that reduce() should be removed from the language,
>> but I do agree that it does not have to be part of "core" Python,
>> and could easily be relegated to a module.)[/color]
>
> If the builtins are reduced in 3.0, as I generally would like, I would
> be fine with moving apply, map, filter, and a repaired version of
> reduce to a 'fun'ctional or hof module. But the argument of some
> seems to be that this batteries-included language should specifically
> exclude even that.[/color]

A functional module would be neat. A great way to enhance the chance
that there will be one would be starting one today (e.g. on sourceforge),
ideally with both pure-Python and C-helped (or pyrex, etc) implementations ,
and get it reasonably-widely used, debugged, optimized. There's plenty
of such ideas around, but gathering a group of people particularly keen
and knowledgeable about functional programming and hashing out the "best"
design for such a module would seem to be a productive endeavour.


Also, I advocate that 3.0 should have a module or package (call it
"legacy" for example) such that, if you started your program with
some statement such as:

from legacy import *

compatibility with Python 2.4 or thereabouts would be repaired as
much as feasible, to ease running legacy code, and to the expense
of performance, 'neatness' and all such other great things if needed
(e.g., no "repaired" versions or anything -- just compatibility).

One reasonably popular counterproposal to that was to have it as

from __past__ import *

by analogy with today's "from __future__". I'd also like to make it
easy to get this functionality with a commandline switch, like is
the case today with -Q specifically for _division_ legacy issues.


But mostly, each time I mention that on python-dev, I'm rebuffed with
remarks about such issues being way premature today. Somebody's
proposed starting a new list specifically about 3.0, to make sure
remarks and suggestions for it made today are not lost about more
day-to-day python-dev traffic, but I don't think anything's been
done about that yet.


Alex

[Alex Martelli]

Re: How sum() should really be done

Douglas Alan wrote:
[color=blue]
> Erik Max Francis <max@alcyone.co m> writes:
>[color=green]
>> Douglas Alan wrote:[/color]
>[color=green][color=darkred]
>>> Ah, that reminds me -- both sum() and reduce() can be removed from
>>> Python by extending operator.add so that it will take any number of
>>> arguments.[/color][/color]
>[color=green]
>> reduce can't, since reduce doesn't require the function passed to be
>> operator.add.[/color]
>
> Well, as I said, for this to be true, *all* binary operators (that it
> makes sense to) would have to be upgraded to take an arbitrary number
> of arguments, like they do in Lisp.[/color]

Your definition of "operator" appears to be widely at variance with
the normally used one; I've also noticed that in your comparisons of
reduce with APL's / , which DOES require specifically an OPERATOR (of
the binary persuasion) on its left. reduce has no such helpful
constraints: not only does it allow any (callable-as-binary) FUNCTION
as its first argument, but any other CALLABLE at all. Many of the
craziest, most obscure, and worst-performing examples of use of
reduce are indeed based on passing as the first argument some callable
whose behaviour is anything but "operator-like" except with respect to
the detail that it IS callable with two arguments and returns something
that may be passed back in again as the first argument on the next call.
[see note later].


Anyway, to remove 'reduce' by the trick of "upgrading to take an
arbitrary number of arguments", that "upgrade" should be applied to
EVERY callable that's currently subsceptible to being called with
exactly two arguments, AND the semantics of "calling with N arguments"
for N != 2 would have to be patterned on what 'reduce' would do
them -- this may be totally incompatible with what the callable does
now when called with N != 2 arguments, of course. For example,
pow(2, 10, 100)
now returns 24, equal to (2**10) % 100; would you like it to return
107150860718626 732094842504906 000181056140481 170553360744375 038837035105112 493612249319837 881569585812759 467291755314682 518714528569231 404359845775746 985748039345677 748242309854210 746050623711418 779541821530464 749835819412673 987675591655439 460770629145711 964776865421676 604298316526243 868372056680693 76
instead...?-)

I doubt there would be any objection to upgrading the functions in
module operator in the way you request -- offer a patch, or make a
PEP for it first, I would hope it would be taken well (I can't speak
for Guido, of course). But I don't think it would make much more of
a dent in the tiny set of reduce's current use cases.


[note on often-seen abuses of FP built-ins in Python]

Such a typical abuse, for example, is connected with the common idiom:

for item in seq: acc.umul(item)

which simply calls the same one-argument callable on each item of seq.
Clearly the idiom must rely on some side effect, since it ignores the
return values, and therefore it's totally unsuitable for shoehorning
into "functional-programming" idioms -- functional programming is
based on an ABSENCE of side effects.

Of course, that something is totally inappropriate never stops fanatics
of functional programming, that have totally misunderstood what FP is
all about, from doing their favourite shoehorning exercises. So you see:

map(acc.umul, seq)

based on ignoring the len(seq)-long list of results; or, relying on the
fact that acc.umul in fact returns None (which evaluates as false),

filter(acc.umul , seq)

which in this case just ignores an _empty_ list of results (I guess
that's not as bad as ignoring a long list of them...?); or, of course:

reduce(lambda x, y: x.umul(y) or x, seq, acc)

which does depend strictly on acc.umul returning a false result so
that the 'or' will let x (i.e., always acc) be returned; or just to
cover all bases

reduce(lambda x, y: (x.umul(y) or x) and x, seq, acc)


Out of all of these blood-curling abuses, it seems to me that the
ones abusing 'reduce' are the very worst, because of the more
complicated signature of reduce's first argument, compared to the
first argument taken by filter, or map with just one sequence.

To be sure, one also sees abuses of list comprehensions here:

[acc.umul(item) for item in seq]

which basically takes us right back to the "abuse of map" case.
List comprehension is also a rather FP-ish construct, in fact,
or we wouldn't have found it in Haskell to steal/copy it...;-).


Alex

[Alex Martelli]

Re: Python's simplicity philosophy

BW Glitch wrote:
...[color=blue]
> It wasn't obvious for me until later. reduce() is more likely to be used
> for optimization. IIRC, some said that optimization is the root of all
> evil.[/color]

That's *PREMATURE* optimization (and "of all evil IN PROGRAMMING", but
of the two qualifications this one may be less crucial here) -- just like
misquoting "The LOVE OF money is the root of all evil" as "MONEY is the
root of all evil", so does this particular misquote trigger me;-).

Optimization is just fine IN ITS PROPER PLACE -- after "make it work"
and "make it right", there MAY come a time where "make it fast" applies.

It's extremely unlikely that 'reduce' has a good place in an optimization
phase, of course -- even when some operator.foo can be found:

[alex@lancelot tmp]$ timeit.py -c -s'import operator' -s'xs=range(1,32 1)'
'r=reduce(opera tor.mul, xs)'
1000 loops, best of 3: 450 usec per loop

[alex@lancelot tmp]$ timeit.py -c -s'import operator' -s'xs=range(1,32 1)'
'r=1' 'for x in xs: r*=x'
1000 loops, best of 3: 440 usec per loop

reduce shows no advantage compared with a perfectly plain loop, and
when no operator.bar is around and one must use lambda:

[alex@lancelot tmp]$ timeit.py -c -s'import operator' -s'xs=range(1,32 1)'
'r=reduce(lambd a x, y: pow(x, y, 100), xs)'
1000 loops, best of 3: 650 usec per loop

[alex@lancelot tmp]$ timeit.py -c -s'import operator' -s'xs=range(1,32 1)'
'r=1' 'for x in xs: r = pow(r, x, 100)'
1000 loops, best of 3: 480 usec per loop

reduce gets progressively slower and slower than the pian loop. It's
just no bloody good, except maybe for people short-sighted enough to
believe that it's "more concise" than the loop (check out the lengths
of the timed statements above to dispel THAT myth) and who'd rather
slow things down by (say) 35% than stoop to writing a shorter, plainer
loop that every mere mortal would have no trouble understanding.

[color=blue]
> Just because it's _obvious_ to you, it doesn't mean it's obvious to
> people who self taught programming.[/color]

That may be the real motivation for the last-ditch defenders of reduce:
it's one of the few (uselessly) "clever" spots in Python (language and
built-ins) where they may show off their superiority to mere mortals,
happily putting down as sub-human anybody who doesn't "get" higher-order
functions in 10 seconds flat (or less)...;-)


Alex

[Patrick Maupin]

Re: Python's simplicity philosophy

"Terry Reedy" wrote:[color=blue]
> "Patrick Maupin" wrote:[color=green]
> > And then there are the corner cases, e.g. sum([]) vs.
> > reduce(operator .add,[]) (which throws an exception).[/color]
>
> The proper comparison is to reduce(operator .add, [], 0), which does
> not throw an exception either. sum(seq, start=0) is equivalent to
> reduce(operator .add, seq, start=0) except that sum excludes seq of
> string. (The doc specifically says equivalent for number (int) seqs,
> so there might be something funny with non-number, non-string seqs.)
> In other words, sum is more-or-less a special case of reduce with the
> within-context constants operator.add and default start=0 built in
> (and the special special case of seq of strings excluded).[/color]

I agree. Please remember that my post was arguing that it would
take more than 30 seconds to teach someone reduce() instead of sum().
This is precisely because sum() was deliberately chosen to special-case
the most common uses of reduce, including not only the add operator,
but also the default initial value.
[color=blue]
> I think it a big mistake (that should be repaired in 3.0) that the
> result start value was made optional, leading to unnecessary empty-seq
> exceptions. I also think the order is wrong and should also be fixed.
> I believe it was placed last, after the seq of update values, so that
> it could be made optional (which it should not be). But it should
> instead come before the seq, to match the update function (result,
> seq-item) arg order. This reversal has confused people, including
> Guido.[/color]

That makes some sense, but you'd _certainly_ have to move and/or rename
the function in that case. There's breaking some backward compatibility,
and then there's taking a sledgehammer to things which used to work
perfectly.

As an aside, if sum() grew an optional _third_ parameter, which
was the desired operator, sum() would FULLY recreate the functionality
of reduce(), but with the same default behavior that was deemed
desirable enough to create the sum() function. This is similar to
your preferred form in that the starting value is not optional
when you specify the operator (simply because you have to use
three parameters to specify the operator), differing only in
the order of the first two parameters.

Although this is not quite your preferred form, perhaps those in such
a rush to remove reduce() should consider this slight enhancement to
sum(), to mollify those who don't want to see the functionality disappear,
but who could care less about the name of the function which provides
the functionality (e.g. sum(mylist,1,op erator.mul) is slightly
counterintuitiv e).
[color=blue]
> If the builtins are reduced in 3.0, as I generally would like, I would
> be fine with moving apply, map, filter, and a repaired version of
> reduce to a 'fun'ctional or hof module. But the argument of some
> seems to be that this batteries-included language should specifically
> exclude even that.[/color]

I'm taking a wait-and-see attitude on this. I don't think any of the
implementers have such a vested interest in being "right" that these
functions will be removed at all cost. As soon as the implementers
start porting their _own_ code to 3.0, I believe we'll starting getting
useful feedback, either of the form "itertools et al. has everything
I need", or "boy, this SUCKS! I really need map!"

(I'm curious, though, why you included "apply" in this list. I've
personally never needed it since the * enhancement to the call syntax.)

Regards,
Pat

[BW Glitch]

Re: Python's simplicity philosophy

Alex Martelli wrote:
[color=blue]
> BW Glitch wrote:
> ...
>[color=green]
>>It wasn't obvious for me until later. reduce() is more likely to be used
>>for optimization. IIRC, some said that optimization is the root of all
>>evil.[/color]
>
>
> That's *PREMATURE* optimization (and "of all evil IN PROGRAMMING", but
> of the two qualifications this one may be less crucial here) -- just like
> misquoting "The LOVE OF money is the root of all evil" as "MONEY is the
> root of all evil", so does this particular misquote trigger me;-).
>[/color]

Sorry, I didn't remember the quote completely. :S But that was the point
I wanted to make.

Andres Rosado

[Alex Martelli]

Re: Too much builtins (was Re: Python's simplicity philosophy

Ville Vainio wrote:
...[color=blue]
> I also think that reduce, sum, map and filter (and lots of others,
> __builtins__ has *too much stuff*) should be removed from builtins,
> but that will probably take some time (1997 years?). LC's and genexpes
> will take care of most of that stuff. And people can always do:[/color]

There are no way in which LC's and genexp can "take care of" CONSUMING
iterables and iterators -- they PRODUCE them (and itertools also do
some production, and, mostly, composable transformation) . map and
filter, sure; sum, reduce, min, max, can never be replaced by any
LC nor any genexp. I think we need a separate module for _consumers_
of iterables/iterators, and I have repeatedly posted about it. But,
sure, I'd gladly agree that there's no real call for any of these to
be a built-in.

[color=blue]
> from funtional import *
> # map, filter, reduce, curry, ... (I want lots of these :)[/color]

I'd rather put map/filter/reduce in a 'legacy' module -- warts and
all, e.g. map's horrible habit of None-padding shorter sequences,
EEK -- and have a designed-from-scratch functional module without
the warts. What about starting one as a sourceforge project, as I
mentioned elsewhere?

[color=blue]
> There are also tons of functions that should be in sys, math or
> whatever:
>
> reload, repr, divmod, max, min, hash, id, compile, hex...[/color]

Probably, yes. What functions, as opposed to types, do you
think should absolutely be in the builtins rather than in a separate
module, and (unless it's obvious) why? Of course __import__ had
better stay or we won't be able to get anything imported, for
example:-). The attribute-related functions hasattr, getattr,
setattr, and delattr, seem very fundamental (but I'd want the
same status for the item-functions currently over in operator),
as well as (at least some of them -- delattr isn't -- but I do
think that trying to discriminate too finely would make things
too hard to learn) very frequently used in code. What else?
iter, len, pow [for the crucial 3-arguments case], range (or some
preferable variant that returns an iterator), and zip seem pretty
fundamental; chr and ord might be suitable as str methods (in
the case of chr, a staticmethod, no doubt). Many functions that
are quite handy for interactive use, such as locals, globals,
dir, vars, ..., are not all that frequently used in programs --
so they might live in a module that the interactive mode gets
automatically, rather than being built-ins.

Having beginners learn 'import' before they do raw_input (or
output, which should also be a function, not a statement) may
not agree with somebody's didactics, so, we should consider how
to deal with those.

All of this would be perfect for the mailing list on Python 3.0
if the latter existed. Posting it to c.l.py makes it unlikely
Guido will ever consider the discussion's resuts, anyway. The
problem is that, with 3.0 at least 2 years away, there seems to
be little incentive to actually go and make such a list, so that
its archives may come in handy in the future.

[color=blue]
> What's your pet deprecation candidate? I have always thought
> `backticks` as repr has got to be the most useless feature around.[/color]

Pretty bad, yes. 'apply' at least, while useless, doesn't make
Python's syntax any more complicated, while `...` does.


Alex

[Alex Martelli]

Re: Python's simplicity philosophy

Douglas Alan wrote:
...[color=blue]
> The argument that some programmers might be too lazy to want to learn
> powerful, simple, and elegant features that can be taught in seconds,[/color]

Programmers may be quite ready to learn anything whose claimed "power"
IS actually there. But this complex of thread started with somebody
asking for good use cases for reduce, and NONE has been shown.

Programmers need not have any interest in spending what may well be
more than 'seconds' in learning something that will never given them
any *practical* return. Computer "scientists ", maybe, have to be, by
some definition of "scientist" . Fortunately, *programmers* are still
allowed to be very pragmatical types instead.

[color=blue]
> Besides, if you weren't exposed at all to LISP (or a LISP-like
> language) while getting a CS degree, it wasn't a very good CS
> program! They're going to teach you AI techniques in a different[/color]

Aha, showing your true colors: I see you have now moved from a
"CS 101 course" (which is given to majors in MANY disciplines)
to a "CS degree". Quite a different thing. Around here (for the
same length of course, say 3 years), graduates in "informatic al
engineering", for example, may well not know what a Turing machine
is, nor be able to name any "AI technique" of practical use (they
may have learned, e.g. alpha-beta pruning, which historically did
originate within AI, but may be more practical today to learn in
completely different contexts) -- but they're likely to know more
than graduates in "informatic s" (computer science) about, e.g.,
statistics, and/or how to organize a security-audit. (I have to
phrase it in terms of likelihood because most majors do offer quite
a bit of choice in terms of what exact courses you can take).

I wouldn't be surprised to find an "ingegnere informatico" (3-years
degree, i.e. BS-equivalent) who doesn't understand 'reduce' at
first; and once he or she has grasped it, I _would_ be surprised
not to get challenged with an "OK, now, what IS it GOOD for?".

When Plato was asked the same question about Geometry, he had a
slave give the querant a gold coin then throw him out of the school:
caring about USEFULNESS was OH so icky to Greek philosophers! I
would not be surprised if a similar stance (maybe not quite as
overt) lingers in several academic environments. Fortunately,
though, it seems to me that Engineering faculties have managed
to stay pretty free of it (yep, I'm an engineer, and proud of it,
and proud that my daughter has chosen an engineering major too --
my son chose Economics, a lovely calling to be sure, and is having
a much harder time selecting the courses that concentrate on the
_useful_ stuff ["how do I make a cool million before I'm 30" kind
of stuff:-)] from those which are the Economics equivalent of
"Suppose cows were spheres of uniform density"...:-).


Alex

[Terry Reedy]

Re: Python's simplicity philosophy


"Patrick Maupin" <pmaupin@speake asy.net> wrote in message
news:653b7547.0 311140546.153c1 d88@posting.goo gle.com...[color=blue]
> "Terry Reedy" wrote:[color=green]
> > I think it a big mistake (that should be repaired in 3.0) that[/color][/color]
the[color=blue][color=green]
> > result start value was made optional, leading to unnecessary[/color][/color]
empty-seq[color=blue][color=green]
> > exceptions. I also think the order is wrong and should also be[/color][/color]
fixed.[color=blue][color=green]
> > I believe it was placed last, after the seq of update values, so[/color][/color]
that[color=blue][color=green]
> > it could be made optional (which it should not be). But it should
> > instead come before the seq, to match the update function (result,
> > seq-item) arg order. This reversal has confused people, including
> > Guido.[/color]
>
> That makes some sense, but you'd _certainly_ have to move and/or[/color]
rename[color=blue]
> the function in that case.[/color]

I agree on moving. I might rename reduce as induce simultaneously.
[color=blue]
> Although this is not quite your preferred form, perhaps those in[/color]
such[color=blue]
> a rush to remove reduce() should consider this slight enhancement to
> sum(), to mollify those who don't want to see the functionality[/color]
disappear,[color=blue]
> but who could care less about the name of the function which[/color]
provides[color=blue]
> the functionality (e.g. sum(mylist,1,op erator.mul) is slightly
> counterintuitiv e).[/color]

The problem with sum(seq, [base=0, [op = add]]) is that it *still* has
the result base case and the seq params in what I consider to be the
wrong order (unless the optional update function has the order of its
parameters reverses from reduce) -- and for the same reason of
optionality.

I need to write a coherent summary of my views on the ++s and --s of
reduce.
[color=blue]
> functions will be removed at all cost. As soon as the implementers
> start porting their _own_ code to 3.0, I believe we'll starting[/color]
getting[color=blue]
> useful feedback, either of the form "itertools et al. has everything
> I need", or "boy, this SUCKS! I really need map!"[/color]

I presume that part of why Guido wants a year to do 3.0 is just so he
can test alternatives with ports of his own code.
[color=blue]
> (I'm curious, though, why you included "apply" in this list. I've
> personally never needed it since the * enhancement to the call[/color]
syntax.)

I presume you mean as opposed to delete it altogether. Because apply
can be used with map while * and ** cannot and at least one person has
said he uses that.

Terry J. Reedy

[Alex Martelli]

Re: Python's simplicity philosophy

Douglas Alan wrote:
...[color=blue][color=green]
>> That is non-trivial to most, based on my experience in explaining it
>> to other people (which for the most part have been computational
>> physicists, chemists, and biologists).[/color]
>
> I find this truly hard to believe. APL was a favorite among
> physicists who worked at John's Hopkins Applied Physics Laboratory
> where I lived for a year when I was in high school, and you wouldn't[/color]

Interesting. I worked for many years in an environment where physicists
doing research could freely choose between APL and Fortran (IBM Research),
and while there was a hard-core of maybe 10%-15% of them who'd _never_
leave APL for any reason whatsoever, an overwhelmingly larger number
of physicist was at least as keen on Fortran. I have no hard data on
the subject, but it appears to me that Fortran has always been way more
popular than APL among physicists as a whole.
[color=blue]
> thing. In fact, people seemed to like reduce() and friends -- people
> seemed to think it was a much more fun way to program, rather than
> using boring ol' loops.[/color]

....while most physicists I worked with were adamant that they wanted
to continue coding loops and have the _compiler_ vectorize them or
parallelize them or whatever. Even getting them to switch to Linpack
etc from SOME of those loops was a battle at first, though as I recall
the many advantages did eventually win them over.

Anyway, computational scientists using Python should be using Numeric
(if they aren't, they're sadly misguided). Numeric's ufuncs ARE the
right way to do the equivalent of APL's +/ (which is quite a different
beast than ANYusercodedfun ction/ would be...), and show clear and
obvious advantages in so doing:

[alex@lancelot tmp]$ timeit.py -c -s'import operator; xs=range(999)'
'x=reduce(opera tor.add, xs)'
1000 loops, best of 3: 290 usec per loop

[alex@lancelot tmp]$ timeit.py -c -s'xs=range(999) ' 's=sum(xs)'
10000 loops, best of 3: 114 usec per loop

[alex@lancelot tmp]$ timeit.py -c -s'import Numeric as N;
xs=N.arrayrange (999)' 'x=N.add.reduce (xs)'
100000 loops, best of 3: 9.3 usec per loop

Now *THAT* is more like it: 10+ times FASTER than sum, rather than
2+ times SLOWER! Of course, you do have to use it right: in this
snippet, if you initialize xs wrongly...:

[alex@lancelot tmp]$ timeit.py -c -s'import Numeric as N; xs=range(999)'
'x=N.add.reduce (xs)'
100 loops, best of 3: 2.1e+03 usec per loop

....then you can say goodbye to performance, as you see. But when
used skilfully, Numeric (or its successor numarray, I'm sure -- I
just don't have real experience with the latter yet) is just what
numerically heavy computations in Python require.


Alex

[Alex Martelli]

Re: Python's simplicity philosophy

Dave Brueck wrote:
...[color=blue][color=green]
>> Why, pray-tell, would you want an OO program to do:
>>
>> results = [ func(x) for x in sequence ]
>>
>> ... instead of ...
>>
>> results = sequence.map(fu nc) ??[/color]
>
> Because I find the first much more readable (and IMO the "an OO program to
> do" bit is irrelevent from a practical point of view).[/color]

I entirely agree with both points. They're even clearer when the
contrast is between, e.g.:

results = [ x+23 for x in sequence ]

and:

results = sequence.map(la mbda x: x+23)

where using the HOF approach forces you to understand (and read) lambda too.


Alex