PEP 322: Reverse Iteration (REVISED, please comment)

[Alex Martelli]

Re: PEP 322: Reverse Iteration (REVISED, please comment)

Raymond Hettinger wrote:
[color=blue][color=green]
>> The static method approach would clearly document that the
>> result is an iterator (which none of the other names proposed
>> really does IMHO)[/color]
>
> Better name are welcome!
>
> Moving it somewhere else is not open. It is proposed as a builtin for a
> reason -- it is a core looping tool like zip() or enumerate() and it is[/color]

I don't think it has anywhere like the frequency of use of either.
[color=blue]
> Static methods, class methods, and weird descriptors be darned, this is
> not an exercise in how weirdly it can be implemented just to avoid
> having a builtin.[/color]

I just don't think it's worth making a built-in.

[color=blue]
> Would everything be somehow better if Alex's wonderful sum() had
> been implemented as a int.sum() classmethod or was tucked way in
> another module?[/color]

int.sum would be a disaster either way: if it was forced to return
an int it would reduce use cases substantially; if it wasn't it
would be just weird.

math.sum would be arguably equivalent to sum as a built-in -- a
tad less immediately accessible, but perhaps superior in that it
immediately suggests it's about numbers only, so we'd avoid the
icky performance trap you now get with sum(manylists, []) {which
makes sum only 2/3 wonderful at best -- fine with numbers ONLY}.
[color=blue]
> Of course not! Likewise, would zip() or enumerate()
> be successful solutions to lock-step iteration and the loop-counter
> problems if they were iter.zip() and iter.enumerate( )? No, of course not.[/color]

zip is currently an INFERIOR solution for lock-step iteration: your
itertools.izip is much better for that. Unfortunately zip IS
constrained to return a list BUT that fact is not reflected in it
being a classmethod list.zipped (we didn't have classmethods when
zip was introduced, so that wasn't an option). If it had been made
one, then by now we might have a built-in version of izip, sigh.

Your enumerate is excellent, and just as frequently used as izip.
I prefer it as a builtin -- though I wouldn't be heartbroken if it
had been in itertools, but it's better in the builtins _because_
it's so frequently used.

Unfortunately we have too many and inappropriate builtins and they're
not going to go away until 3.0 (years away). This raises the bar for
other not-obviously-indispensable built-ins more than it would be
sensible to do in a hypothetical "greenfield design".

[color=blue]
> Let's put an end to this silliness right now. The idea is offered as a
> builtin or not at all; otherwise, the existing [::-1] starts to look
> better.[/color]

You're clearly overwrought, so I won't "see" that "silliness" and get
offended by it. If the choice was only to have the reverser (by
whatever name) as a built-in or not at all, I'd be -0 on the reverser --
slightly against, though not enough to bother opposing it. If it
could be sited somewhere appropriate (ok, not as iter.reversed if
that's technically unfeasible -- i did say "appropriat e":-) I'd be +1.

[color=blue]
> The challenge with a PEP this simple is that experts feel this
> overpowering urge to apply all their know-how and transform
> in to something other than a clean, fast, simple solution.[/color]

The crux of our disagreement, namecalling apart, might be highlighted
by your assertion in another short and emotional post:
[color=blue]
> This is supposed to be something you can teach in the first half-hour[/color]

I can't imagine "teaching this in the first half-hour" of any Python
course I teach. There are FAR too many other functions, including
non-builtin ones such as sys.exit, and some in itertools too, that are
more important than this, in my own assessment.

So we disagree about frequence of usage, and consequently "warrantedn ess"
as a built-in -- big deal. I honestly don't understand how this purely
technical disagreement can explain these intense emotions.

_I_ am supposed to be the Latin, Mediterranean, hot-blooded enthusiast,
after all, and on average I think I cover the role adequately.

Oh well, with the incredible amount of good things you've done for Python
development, I guess you're surely entitled to blow your top occasionally,
if that's your preference. Also, presumably, to get your pet built-in
into the language, as you've slipped it past Guido. As for name, I
therefore second the neatest suggestion I've seen: since it IS a reversed
iter, and we can't directly call it that, then:

reti

Hey, it IS short, so if it's gonna be so heavily used, good thing, no?-)


Alex

[Bengt Richter]

Re: PEP 322: Reverse Iteration (REVISED, please comment)

On 28 Oct 2003 10:22:44 -0800, python@rcn.com (Raymond Hettinger) wrote:
[color=blue]
>Based on your extensive feedback, PEP 322 has been completely revised.
>The response was strongly positive, but almost everyone preferred
>having a function instead of multiple object methods. The updated
>proposal is at:
>
> www.python.org/peps/pep-0322.html
>
>In a nutshell, it proposes a builtin function that greatly simplifies reverse
>iteration. The core concept is that clarity comes from specifying a
>sequence in a forward direction and then saying, "inreverse( )":
>
> for elem in inreverse(seqn) :
> . . .
>
>Unlike seqn[::-1], this produces a fast iterator instead of a full reversed
>copy.[/color]

I'm thinking that a concise way to make a fast iterator from any slice expression
could subsume the inreverse functionality. IOW, a generator slice corresponding
in spirit to generator expressions. E.g., for the inreverse use,

for elem in seqn'[::-1]:
. . .

(note the apostrophe in front of the slice notation)

Of course, unlike 'inreverse', you wouldn't be limited to going backwards,

for oddelem in seqn'[1::2]:
. . .

Ok, take a deep breath, here comes some more ;-)

How about generator expressions/slices being able to be evaluated in parallel in a 'for' loop
(with '=' instead of 'in' to signify that we're taking elements from each of the
sequences in a tuple and making an intermediate data tuple to unpack for the assignment,
not unpacking the rhs tuple of sequences)? E.g.,

for oddelem, evenelem = seqn'[1::2], seqn'[0::2]:
. . .

This seems like a nice generalization that could use any sequences, not just generator slices.
I.e., like tuple assignment, but the 'for' makes the rhs get treated like the above was short for

for oddelem, evenelem in itertools.izip( seqn'[1::2], seqn'[0::2]):

Note that like tuple assignment, you could unpack as many sequences in parallel as you liked.
(In fact, the single-target case would work too, just by changing 'in' to '=' ;-)

If you like generator slices, it might be nice to have bare generator slices too, implicitly
producing a sequence of integers that might make sense in relation to the same notation used
on a sequence. E.g.,

'[:] <=> (i for i in xrange(0,sys.ma xint,1))
'[start:stop:step] <=> (i for i in xrange(start,st op,step))
'[::-1] <=> (i for i in xrange(-1, -sys.maxint-1, -1))

etc. Thus plain enumerate could be respelled (taking seq via a full generator slice):

for i, elem = '[:], seq'[:]:
. . .

and the reversed order, enumerating with indices -1, -2, etc. would look like

for i, elem = '[::-1], seq'[::-1]:
. . .

Too much at once? ;-)
[color=blue]
>
>Discussions with Guido made it clear that inreverse() will not be extended
>to cover all iterables. The proposal is about simplicity, expression, and
>performance. As such, it would be counter-productive to take in a general
>iterable, run it to completion, save the data in memory, and then iterate
>over the data in reverse.
>[/color]
The same could apply to seq'[::-1]
I assume seq would be queried for __len__ and __getitem__ when about to create
the generator slice.

Regards,
Bengt Richter

[Shu-Hsien Sheu]

Newbie: Functions and class

Hi,

After 2 months of learning in Python, the most difficult part is the
object part. Though I can get most of my works done using very simple
classes, I don't think I get the essence of it. Right now I am thinking
of wrirting a class like:

class hittable(object ):
def __init__(self):
self.table = [{}, {}, {}, {}]
def build(<some parameter>): #some modification to the dictionary
<some codes>
def search(self, aa):
if aa == 0:
<do modification "build" to self.table[0]>
if aa == 1:
<do modification "build" to self.table[1]>

I have trouble implementing the above in real code. I can only get
something by making a seperate function outside the class:
[color=blue][color=green][color=darkred]
>>> def build(dd, i):[/color][/color][/color]
dd.setdefault(i , {})
return dd[color=blue][color=green][color=darkred]
>>> class hittable(object ):[/color][/color][/color]
def __init__(self):
self.table = [{}, {}, {}, {}] #0:peptides, 1: nucleotides,
2:metals, 3:others
def search(self, i):
self.table[i] = build(self.tabl e[i], i)[color=blue][color=green][color=darkred]
>>> kk = hittable()
>>> kk.search(3)
>>> kk.table[/color][/color][/color]
[{}, {}, {}, {3: {}}]

And couldn't find a way to integrate the "build" function into a class
method:
[color=blue][color=green][color=darkred]
>>> class hittable(object ):[/color][/color][/color]
def __init__(self):
self.table = [{}, {}, {}, {}]
def build(self, i):
self.table[i].setdefault(i, {})
def search(self, i):
self.table[i] = self.build(i)[color=blue][color=green][color=darkred]
>>> kk = hittable()
>>> kk.search(3)
>>> kk.table[/color][/color][/color]
[{}, {}, {}, None]

I think I can imagine the above code won't work, though couldn't find a
solution to it.

What am I missing here?

Thanks!

-shushien

[Alex Martelli]

Re: Newbie: Functions and class

Shu-Hsien Sheu wrote:
...[color=blue]
> def build(self, i):
> self.table[i].setdefault(i, {})[/color]

this method has no explicit 'return', so it returns None
[color=blue]
> def search(self, i):
> self.table[i] = self.build(i)[/color]

So, you're assigning None to self.table[i].
[color=blue][color=green][color=darkred]
> >>> kk = hittable()
> >>> kk.search(3)
> >>> kk.table[/color][/color]
> [{}, {}, {}, None]
>
> I think I can imagine the above code won't work, though couldn't find a
> solution to it.
>
> What am I missing here?[/color]

Just forget that wanton assignment in method search! Why would you
want to assign anything new to self.table[i], when self.build(i) has
just modified self.table[i] appropriately?! I.e., change search to:

def search(self, i):
self.build(i)

this doesn't have anything special to do with classes -- rather,
with the care and feeding of dictionaries, I'd say:-).


Alex

[Michele Simionato]

Re: PEP 322: Reverse Iteration (REVISED, please comment)

Alex Martelli <aleax@aleax.it > wrote in message news:<EBVnb.376 111$R32.1246692 7@news2.tin.it> ...[color=blue]
> math.sum would be arguably equivalent to sum as a built-in -- a
> tad less immediately accessible, but perhaps superior in that it
> immediately suggests it's about numbers only, so we'd avoid the
> icky performance trap you now get with sum(manylists, []) {which
> makes sum only 2/3 wonderful at best -- fine with numbers ONLY}.
>[/color]

Too late :-(

I see now that ``math.sum`` would have been a much better solution that a
new built-in. BTW, I remember this was discussed on the list, but
what was the argument against a polymorphic sum?

I mean, why it was not an option to implement a ``sum`` function
calling:

1. int.__add__,flo at.__add__ etc. if the arguments where numbers;
2. string.join if the arguments where lists;
3. something to avoid the performance trap of list.__add__;
4. __add__ for custom defined objects.

BTW, I think the answer should go in the FAQ, since everybody looking
at ``sum`` for the first time would imagine it is polymorphic. I remember
you said your first idea was to make it polymorphic, but this was
rejected due to performances reasons. But why making ``sum`` to
special case according to the arguments (it would be enough to
check the type of the first argument, then we would get an error
if we try to add incompatible types, just as in "".join(["a", 1]))
was a bad idea?

As you say, since ``sum`` is not polymorphic it makes more sense to put
it in the ``math`` module. Now it is too late, but it is unfortunate.


Michele

[Shu-Hsien Sheu]

Re: Newbie: Functions and class

Hi Alex,

Great thanks! What u've suggested was exactly what I was looking for.
Too bad it's such a trivial question:((

I have another trivial question though. Why won't the following work?

class hittable(object ):
def __init__(self):
self = [[], [], [], []]

-shuhsien
[color=blue]
>Just forget that wanton assignment in method search! Why would you
>want to assign anything new to self.table[i], when self.build(i) has
>just modified self.table[i] appropriately?! I.e., change search to:
>
> def search(self, i):
> self.build(i)
>
>this doesn't have anything special to do with classes -- rather,
>with the care and feeding of dictionaries, I'd say:-).
>
>[/color]

[KefX]

Re: Newbie: Functions and class

>class hittable(object ):[color=blue]
> def __init__(self):
> self = [[], [], [], []][/color]

Assignment to 'self' is never all that useful. You're just rebinding a
temporary object; this function is basically a no-op because the assignment to
'self' is discarded once the function exits. I know it's kind of confusing, but
you'll get used to the name binding rules before too long.

If you don't understand, it's the same reason this doesn't work:
x = 2
y = x
x = 3
if x == y:
print 'Hewwo world'
# Why doesn't this print?

The answer is that rebinding x doesn't rebind y, which is clearly a Good Thing
in genera as you can see by the example.

- Kef

[Shu-Hsien Sheu]

Reverting (key, value) pairs in a dictionary

Hi,

How do you turn a dictionary of <original_key >: <original_value > into
<original_value >: <original_key >, when there is no duplicate values?
For instance, I have a dictionary kk:
kk = {'abc': ['B', 'C', 'D', 'E'], 'def':['G', 'H']}

and I want a new dictionary newkk which looks like:
newkk = {'B':'abc', 'C':'abc', 'D':'abc', 'E':'abc', 'G':'def', 'H':'def'}

My codes are:
[color=blue][color=green][color=darkred]
>>> kk = {'abc': ['B', 'C', 'D', 'E'], 'def':['G', 'H']}
>>> new = []
>>> for i, item in kk.items():[/color][/color][/color]
for j in item:
new.append([j, i])[color=blue][color=green][color=darkred]
>>> newkk = dict(new)
>>> new[/color][/color][/color]
[['B', 'abc'], ['C', 'abc'], ['D', 'abc'], ['E', 'abc'], ['G', 'def'],
['H', 'def']][color=blue][color=green][color=darkred]
>>> newkk[/color][/color][/color]
{'C': 'abc', 'B': 'abc', 'E': 'abc', 'D': 'abc', 'G': 'def', 'H': 'def'}

Is there a better way of doing it?

-shuhsien

[Skip Montanaro]

Re: Reverting (key, value) pairs in a dictionary


Shu-Hsien> How do you turn a dictionary of <original_key >:
Shu-Hsien> <original_value > into <original_value >: <original_key >, when
Shu-Hsien> there is no duplicate values?

Shu-Hsien> For instance, I have a dictionary kk:
Shu-Hsien> kk = {'abc': ['B', 'C', 'D', 'E'], 'def':['G', 'H']}

Shu-Hsien> and I want a new dictionary newkk which looks like:
Shu-Hsien> newkk = {'B':'abc', 'C':'abc', 'D':'abc', 'E':'abc', 'G':'def', 'H':'def'}

How about:
[color=blue][color=green][color=darkred]
>>> newkk = {}
>>> for key in kk:[/color][/color][/color]
... val = kk[key]
... newkk.update(di ct(zip(val, [key]*len(val))))
...[color=blue][color=green][color=darkred]
>>> newkk[/color][/color][/color]
{'C': 'abc', 'B': 'abc', 'E': 'abc', 'D': 'abc', 'G': 'def', 'H': 'def'}

Skip

[Edward C. Jones]

Re: Reverting (key, value) pairs in a dictionary

Shu-Hsien Sheu wrote:[color=blue]
> Hi,
>
> How do you turn a dictionary of <original_key >: <original_value > into
> <original_value >: <original_key >, when there is no duplicate values?[/color]

See http://members.tripod.com/~edcjones/MultiDict.py

[Stephen Horne]

Re: Reverting (key, value) pairs in a dictionary

On Thu, 30 Oct 2003 16:27:13 -0500, Shu-Hsien Sheu <sheu@bu.edu>
wrote:
[color=blue]
>Hi,
>
>How do you turn a dictionary of <original_key >: <original_value > into
><original_valu e>: <original_key >, when there is no duplicate values?
>For instance, I have a dictionary kk:
>kk = {'abc': ['B', 'C', 'D', 'E'], 'def':['G', 'H']}
>
>and I want a new dictionary newkk which looks like:
>newkk = {'B':'abc', 'C':'abc', 'D':'abc', 'E':'abc', 'G':'def', 'H':'def'}[/color]

There will be loads of ways of doing this. I would probably use a list
comprehension such as...

result = dict( [(d, k) for k, l in <dict>.items () for d in l] )

Generator comprehensions may make this a tad neater in Python 2.4 ;-)


List comprehensions aren't to everyones taste, though, so some people
would probably find the following clearer...

result = {}

for k, l in <dict>.items () :
for d in l :
result [d] = k

The only flaw in your own code is that creating the list of lists in
'new' is redundant - it is more efficient to create the dictionary
directly.


--
Steve Horne

steve at ninereeds dot fsnet dot co dot uk

[Alex Martelli]

Re: Reverting (key, value) pairs in a dictionary

Shu-Hsien Sheu wrote:
[color=blue]
> How do you turn a dictionary of <original_key >: <original_value > into
> <original_value >: <original_key >, when there is no duplicate values?[/color]

If, as per this spec, each key was mapped to one hashable value (but your
example below indicates this isn't the case!), then

newkk = dict([ (v,k) for k, v in kk.iteritems() ])

would be the solution. But for your example:
[color=blue]
> For instance, I have a dictionary kk:
> kk = {'abc': ['B', 'C', 'D', 'E'], 'def':['G', 'H']}
>
> and I want a new dictionary newkk which looks like:
> newkk = {'B':'abc', 'C':'abc', 'D':'abc', 'E':'abc', 'G':'def', 'H':'def'}[/color]

you need a nested loop on each so-called value (==list of values), so:

newkk = dict([ (v,k) for k, vs in kk.iteritems() for v in vs ])


In python 2.4 (will be a while coming, so DON'T hold your breath) you can
spell this without those annoying extra square brackets in ([ ... ]) and
get a tiny speedup as well as marginally clearer syntax.


Alex

[Alex Martelli]

Re: Newbie: Functions and class

Shu-Hsien Sheu wrote:
[color=blue]
> Hi Alex,
>
> Great thanks! What u've suggested was exactly what I was looking for.
> Too bad it's such a trivial question:(([/color]

"The only bad question is the one not asked":-).
[color=blue]
>
> I have another trivial question though. Why won't the following work?[/color]

Works fine, actually, just probably doesn't do what you THINK it does:-).
[color=blue]
> class hittable(object ):
> def __init__(self):
> self = [[], [], [], []][/color]

this rebinds local variable self in the __init__ metod (to a list of
four lists), and that's all. Doesn't affect any INSTANCE of class
hittable, for example. What are you trying to do?! If you want
"hittable" to return a list of lists then it must be a function,
not a class, obviously:

def hittable(): return [[], [], [], []]

if you want each instance of class hittable to start out with
an ATTRIBUTE holding a list of four empty lists then SAY so:

class hittable(object ):
def __init__(self):
self.putsomenam ehere = [[], [], [], []]


Alex

[Shu-Hsien Sheu]

Re: Newbie: Functions and class

Dear Kef,

I can totally understand the following. However, I cannot see clearly
how this is similiar to the self assignment?
[color=blue]
>If you don't understand, it's the same reason this doesn't work:
>x = 2
>y = x
>x = 3
>if x == y:
> print 'Hello world'
> # Why doesn't this print?
>
>[/color]

Dear Alex,

I think I can see your point. So, we can gerenally say that, only
instances of a class can have assignments; Class name are only a
"reference" thing and doesn't have any "exact" values bind to it. Sorry
my wording might be wrong, but it's the best that I could think of.

thanks!

-shuhsien

[Alex Martelli]

Re: Newbie: Functions and class

Shu-Hsien Sheu wrote:
...[color=blue]
> I think I can see your point. So, we can gerenally say that, only
> instances of a class can have assignments; Class name are only a
> "reference" thing and doesn't have any "exact" values bind to it. Sorry
> my wording might be wrong, but it's the best that I could think of.[/color]

No, this doesn't sound like anything having to do with Python.

Let's go back to your example. In it, you had a function:

def <function_name_ doesnt_matter_W HICH!>(<argumen t_name_ditto>):
<argument_name_ ditto> = [ [], [], [], [] ]

and that was all the function did.

NO MATTER WHAT the name of this function,
NO MATTER WHAT the name of its arguments,
NO MATTER WHERE the function is (inside a class statement,
outside of it, inside another function, on a partridge tree),
NO MATTER HOW you access and call it,

*this function will NEVER, EVER do ANYTHING observable from the outside*.

It's rebinding its local variable name which is its argument's name.

That's ALL it's doing. NOTHING ELSE, *EVER*, under ANY conditions.

Actions of a function regarding the bindings of the functions' own,
intrinsically INTERNAL, local variable names, are not per se ever
observable from the outside.

Therefore, this function's body might as well be:
pass
under ANY conditions.

This has nothing to do with classes. Class names can perfectly
well "have assignments" (for any reasonable readings of this "have").
Classes are objects just like any others and they're first class just
like class instances and the like. The issue has nothing at all to
do with this. It doesn't matter in the least that your function was
within a classbody and had the specialname __init__ and its only
argument had the conventionally-used name self -- this placement and
naming influences WHEN the function may be called, but your function
is STILL a "no-operation", totally independent of when or how it's
being called there's no way it's ever gonna DO anything.


Alex