PEP 289: Generator Expressions (please comment)

[Mike Rovner]

Re: PEP 289: Generator Expressions (please comment)

rzed wrote:[color=blue]
> I like it! How about in conjunction with the print statement?
> print "%s\n" % (s for s in strlist)[/color]

It's not better than

for s in strlist:
print s

or I didn't understand your intent.

Mike

[Donald 'Paddy' McCarthy]

Re: PEP 289: Generator Expressions (please comment)



Paul Rubin wrote:[color=blue]
> python@rcn.com (Raymond Hettinger) writes:
>[color=green]
>>In brief, the PEP proposes a list comprehension style syntax for
>>creating fast, memory efficient generator expressions on the fly:
>>
>> sum(x*x for x in roots)[/color]
>
>
> This is a little confusing. Consider:
>
> 1) [x*x for x in roots]
> 2) (x*x for x in roots)
>
> #2 looks like it should be a "sequence comprehension" and not a
> generator expression. I'd go back to your earlier proposal of
> using a yield keyword if you want a generator expression:
>
> sum(yield x for x in roots)
>[/color]
I too prefer the above use of yield because then you can always
associate generators with the yield keyword.

Pad.

[Rainer Deyke]

Re: PEP 289: Generator Expressions (please comment)

Donald 'Paddy' McCarthy wrote:[color=blue]
> I too prefer the above use of yield because then you can always
> associate generators with the yield keyword.[/color]

I don't, because using 'yield' here would lead to greater confusion when
using generator expressions in generator functions.


yield [x for x in y] # Yields a list
yield x for x in y # Creates and discards a generator
yield yield x for x in y # Yields an iterator


--
Rainer Deyke - rainerd@eldwood .com - http://eldwood.com

[Holger Krekel]

Re: PEP 289: Generator Expressions (please comment)

Raymond Hettinger wrote:[color=blue]
> Peter Norvig's creative thinking triggered renewed interest in PEP 289.
> That led to a number of contributors helping to re-work the pep details
> into a form that has been well received on the python-dev list:
>
> http://www.python.org/peps/pep-0289.html
>
> In brief, the PEP proposes a list comprehension style syntax for
> creating fast, memory efficient generator expressions on the fly:
>
> sum(x*x for x in roots)
> min(d.temperatu re()*9/5 + 32 for d in days)
> Set(word.lower( ) for word in text.split() if len(word) < 5)
> dict( (k, somefunc(k)) for k in keylist )
> dotproduct = sum(x*y for x,y in itertools.izip( xvec, yvec))
> bestplayer, bestscore = max( (p.score, p.name) for p in players )[/color]

Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.

Sometimes it seems that introducing new syntax happens much easier
than improving or adding stdlib-modules.
[color=blue]
> Each of the above runs without creating a full list in memory,
> which saves allocation time, conserves resources, and exploits
> cache locality.[/color]

I do like these properties, though :-)

At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with
'yield' and generators. Maybe the PEP should have an "implementa tion
plan" chapter mentioning such details?

cheers,

holger

[Ville Vainio]

Re: PEP 289: Generator Expressions (please comment)

Holger Krekel <pyth@devel.tri llke.net> writes:
[color=blue]
> At least I hope that generator expressions will only be introduced
> via a __future__ statement in Python 2.4 much like it happened with[/color]

Why? Does it break backwards compatibility somehow?

As far as readability goes, genexps seem to be pretty much as
readable/unreadable as list comprehensions. To me, BTW, the idea looks
great. However, I'm not completely sure about allowing them to be
creatad w/o parens in function calls, seems a bit too magical for my
taste.

--
Ville Vainio http://www.students.tut.fi/~vainio24

[Holger Krekel]

Re: PEP 289: Generator Expressions (please comment)

Ville Vainio wrote:[color=blue]
> Holger Krekel <pyth@devel.tri llke.net> writes:
>[color=green]
> > At least I hope that generator expressions will only be introduced
> > via a __future__ statement in Python 2.4 much like it happened with[/color]
>
> Why? Does it break backwards compatibility somehow?[/color]

OK ok. Probably only backwards readability :-)
[color=blue]
> As far as readability goes, genexps seem to be pretty much as
> readable/unreadable as list comprehensions.[/color]

Yes, i don't like those much, either. But then i am one of this strange
minority which likes map, filter, lambda and passing functions/callables all
around.

But actually generators i do appreciate a lot so i'll probably not only
get used to generator expressions but will enjoy them (more than list
comprehensions at least, i guess).

cheers,

holger

[John Burton]

Re: PEP 289: Generator Expressions (please comment)


"Raymond Hettinger" <python@rcn.com > wrote in message
news:5d83790c.0 310231158.65595 858@posting.goo gle.com...[color=blue]
> Peter Norvig's creative thinking triggered renewed interest in PEP 289.
> That led to a number of contributors helping to re-work the pep details
> into a form that has been well received on the python-dev list:
>
> http://www.python.org/peps/pep-0289.html
>[/color]


This seems a well thought out and useful enhancement to me.
Personally I like the syntax as proposed and would vote yes if this was the
kind of thing we got votes on!

[Chris Perkins]

Re: PEP 289: Generator Expressions (please comment)

Raymond Hettinger:[color=blue]
>
> http://www.python.org/peps/pep-0289.html
>[/color]

I like it. This made me realize that list comprehension sytax is a
special case that never really needed to be special-cased.

In fact, list comprehensions themselves become redundant syntactic
sugar under the new proposal:

[foo(x) for x in bar] ==> list(foo(x) for x in bar)

Chris Perkins

[Brandon Corfman]

Re: PEP 289: Generator Expressions (please comment)

+1

Raymond Hettinger wrote:[color=blue]
> Peter Norvig's creative thinking triggered renewed interest in PEP 289.
> That led to a number of contributors helping to re-work the pep details
> into a form that has been well received on the python-dev list:[/color]

[Skip Montanaro]

Re: PEP 289: Generator Expressions (please comment)

[color=blue][color=green]
>> I too prefer the above use of yield because then you can always
>> associate generators with the yield keyword.[/color][/color]

Rainer> I don't, because using 'yield' here would lead to greater
Rainer> confusion when using generator expressions in generator
Rainer> functions.

There's also the mental confusion about what the yield is doing. Is it
yielding a value back from the function containing the generator expression
or yielding values from the generator expression to the current function?
One of the earliest proposals on python-dev used the yield keyword. I fell
into this exact hole. It took a message or two from others participating in
the thread to extricate myself.

Skip

[Skip Montanaro]

Re: PEP 289: Generator Expressions (please comment)


Ville> However, I'm not completely sure about allowing them to be
Ville> creatad w/o parens in function calls, seems a bit too magical for
Ville> my taste.

Think of parens and genex's the same way you'd think of parens and tuples.
Creating tuples often doesn't require parens either, but sometimes does in
cases where the syntax would be ambiguous). As arguments to function calls,
parens would be required to separate generator expressions from other
arguments. In the one arg case they aren't required (though you're free to
add them if it warms the cockles of your heart ;-).

Skip

[Skip Montanaro]

Re: PEP 289: Generator Expressions (please comment)

[color=blue][color=green]
>> sum(x*x for x in roots)
>> min(d.temperatu re()*9/5 + 32 for d in days)
>> Set(word.lower( ) for word in text.split() if len(word) < 5)
>> dict( (k, somefunc(k)) for k in keylist )
>> dotproduct = sum(x*y for x,y in itertools.izip( xvec, yvec))
>> bestplayer, bestscore = max( (p.score, p.name) for p in players )[/color][/color]

Holger> Although most people on python-dev and here seems to like this
Holger> PEP I think it generally decreases readability. The above
Holger> constructs seem heavy and difficult to parse and thus i am
Holger> afraid that they interfere with the perceived simplicity of
Holger> Python's syntax.

Of course, you can do all of these things today by just turning the args
into list comprehensions:

sum([x*x for x in roots])
min([d.temperature() *9/5 + 32 for d in days])
Set([word.lower() for word in text.split() if len(word) < 5])
dict([(k, somefunc(k)) for k in keylist])
dotproduct = sum([x*y for x,y in itertools.izip( xvec, yvec)])
bestplayer, bestscore = max([(p.score, p.name) for p in players])

[regarding the proposed generator expressions, not my listcomp examples][color=blue][color=green]
>> Each of the above runs without creating a full list in memory, which
>> saves allocation time, conserves resources, and exploits cache
>> locality.[/color][/color]

Which can be a sticking point for using list comprehensions. With generator
expressions in place, list comprehensions become just syntactic sugar for

list(generator expression)

Alex Martelli demonstrated some performance improvements (about 2-to-1 I
think) of generator expressions over the equivalent list comprehensions.

Holger> At least I hope that generator expressions will only be
Holger> introduced via a __future__ statement in Python 2.4 much like it
Holger> happened with 'yield' and generators. Maybe the PEP should have
Holger> an "implementa tion plan" chapter mentioning such details?

I think the plan is to make them available in 2.4. I don't think there's
any plan for a __future__ import because they won't change the semantics of
any existing constructs.

Skip

[Emile van Sebille]

Re: PEP 289: Generator Expressions (please comment)

Raymond Hettinger revives pep 289:[color=blue]
> Peter Norvig's creative thinking triggered renewed interest in PEP 289.
> That led to a number of contributors helping to re-work the pep details
> into a form that has been well received on the python-dev list:
>
> http://www.python.org/peps/pep-0289.html
>
> In brief, the PEP proposes a list comprehension style syntax for
> creating fast, memory efficient generator expressions on the fly:[/color]

What do you get from:
type(x*x for x in roots)

I assume you'd get something like genexp that behaves like an iterator and
that the consuming function accesses items until StopIteration(S topGenxing?
;) is raised, and that a genexp is otherwise a first class object.

From the PEP:
"All free variable bindings are captured at the time this function is
defined, and passed into it using default argument values...[snip]...In
fact, to date, no examples have been found of code where it would be better
to use the execution-time instead of the definition-time value of a free
variable."

Does this imply then that instead of:
dumprec = "\n".join(' %s : %s' % (fld.name, fld.val) for fld in rec)
for rec in file: print dumprec

one would need to write some variation of:
for rec in file:
print "\n".join(' %s : %s' % (fld.name, fld.val) for fld in rec)

or:
def dumprec(rec):
return "\n".join(' %s : %s' % (fld.name, fld.val) for fld in rec)
for rec in file: print dumprec(rec)

I find the first construct easier on the eyes, and it fulfills on the
promise of 'generator expression' as opposed to 'iterable generated
expression'. Although it does feel much more macro-ish...

Overall, +1.


Emile van Sebille
emile@fenx.com

[sdd]

Re: PEP 289: Generator Expressions (please comment)

John Burton wrote:[color=blue]
> ...{about PEP 289)...
>
> This seems a well thought out and useful enhancement to me.
> Personally I like the syntax as proposed and would vote yes
> if this was the kind of thing we got votes on![/color]

Ahhh, you do get a vote, but more by reason than numbers.
The PSF is watching this discussion with bated breath.
This whole discussion in c.l.p is part of examining the
merits of the PEP.

Don't tell anyone that I mentioned the inner workings of
the PSF cabal, or ............... ............... ....~}~~}

[Michele Simionato]

Re: PEP 289: Generator Expressions (please comment)

chrisperkins37@ hotmail.com (Chris Perkins) wrote in message news:<45228044. 0310240442.4903 f866@posting.go ogle.com>...[color=blue]
> Raymond Hettinger:[color=green]
> >
> > http://www.python.org/peps/pep-0289.html
> >[/color]
>
> I like it. This made me realize that list comprehension sytax is a
> special case that never really needed to be special-cased.
>
> In fact, list comprehensions themselves become redundant syntactic
> sugar under the new proposal:
>
> [foo(x) for x in bar] ==> list(foo(x) for x in bar)
>
> Chris Perkins[/color]

.... and here I proclaim list comprehension deprecated!

;) M.