a sequence question

[Chris Wright]

Hi,

1) I want to iterate over a list "N at a time"
sort of like:

# Two at a time... won't work, obviously
[color=blue][color=green][color=darkred]
>>> for a, b in [1,2,3,4]:[/color][/color][/color]
.... print a,b
....
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: unpack non-sequence[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]


Is there a nifty way to do with with list comprehensions,
or do I just have to loop over the list ?

cheers and thanks

chris wright

[Diez B. Roggisch]

Re: a sequence question

l = [1,2,3,4]

for a, b in zip(l[::2], l[1::2]):
print a,b

--
Regards,

Diez B. Roggisch

[Roy Smith]

Re: a sequence question

In article <lzrKd.138680$K 7.56500@news-server.bigpond. net.au>,
Chris Wright <wrightca@hotma il.com> wrote:
[color=blue]
> Hi,
>
> 1) I want to iterate over a list "N at a time"[/color]

You could do it with slicing and zip:
[color=blue][color=green][color=darkred]
>>> l = [1, 2, 3, 4, 5, 6, 7, 8]
>>> zip (l[::2], l[1::2])[/color][/color][/color]
[(1, 2), (3, 4), (5, 6), (7, 8)]

To my eyes, that's a bit cryptic, but it works and it's certainly
compact. I don't use either zip() or extended slicing a lot; perhaps if
I used them more often, the above would be more obvious to me if I read
it in somebody else's code.

The interesting thing would be generalizing this to the "N at a time"
case. I think this works:

def nzip (list0, n):
args = []
for i in range(n):
slice = list0[i::n]
args.append (slice)
return zip (*args)

l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
print nzip (l, 3)

Roy-Smiths-Computer:play$ ./nzip.py
[(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, 11, 12)]

but I haven't given any thought to what happens if the length of the
list isn't a multiple of n (exercise for the reader). It's also
annoying that the above generates a bunch of temporary lists. It would
be cool if there was a way to have the intermediates be generator
expressions, but I'm not that good with that stuff, so I'll leave that
as an exercise for other readers :-)

[F. Petitjean]

Re: a sequence question

Le Fri, 28 Jan 2005 13:59:45 GMT, Chris Wright a écrit :[color=blue]
> Hi,
>
> 1) I want to iterate over a list "N at a time"
>
>
> Is there a nifty way to do with with list comprehensions,
> or do I just have to loop over the list ?
>
> cheers and thanks[/color]
seq = xrange(1, 9) # an iterable [1, 2, ... 8]
N = 2
it = (iter(seq,)*N # a tuple containing N times the *same* iterator on
seq
print zip(*it) # the list you are after
from itertools import izip
help(izip)
it = (iter(seq),)*2
for tup in izip(*it):
print tup[color=blue]
>
> chris wright[/color]

[Duncan Booth]

Re: a sequence question

Chris Wright wrote:
[color=blue]
> 1) I want to iterate over a list "N at a time"
> sort of like:
>
> # Two at a time... won't work, obviously
>[color=green][color=darkred]
> >>> for a, b in [1,2,3,4]:[/color][/color]
> ... print a,b
> ...[/color]

Try this:

l = [1, 2, 3, 4]
for a, b in zip(*[iter(l)]*2):
print a, b

zip(*[iter(seq)]*N) will group by N (but if there are any odd items at the
end it will ignore them).

map(None, *[iter(seq)]*N) will group by N padding the last item with None
if it needs to.

[Michael Hartl]

Re: a sequence question

For problems like this I use a partition function defined in a utils.py
file that I use (based on Peter Norvig's utils file at
http://aima.cs.berkeley.edu/python/utils.py). Using partition, the
problem you posed can be solved by writing

#for a, b in partition([1, 2, 3, 4], 2):
# print a, b

The implementation of partition I use is simple-minded; the previous
posts in this thread suggest some more sophisticated ways to attack it
using generators.

#def partition(seq, partsize):
# """Partitio n a sequence into subsequences of length partsize."""
# ls = len(seq)
# assert ls % partsize == 0, ('length %s, partition size %s\n'
# % (ls, partsize))
# return [seq[i:(i+partsize)] for i in range(0, ls, partsize)]
Michael

[Nick Coghlan]

Re: a sequence question

Duncan Booth wrote:[color=blue]
> Try this:
>
> l = [1, 2, 3, 4]
> for a, b in zip(*[iter(l)]*2):
> print a, b
>
> zip(*[iter(seq)]*N) will group by N (but if there are any odd items at the
> end it will ignore them).
>
> map(None, *[iter(seq)]*N) will group by N padding the last item with None
> if it needs to.[/color]

For anyone else who was as bemused as I was that Duncan's and F. Petitjean's
suggestions actually *work*, this was what I had to do to figure out *why* they
work:

Py> l = [1, 2, 3, 4]
Py> itr = iter(l)
Py> zip(itr) # Put all items from iterator in position 1
[(1,), (2,), (3,), (4,)]
Py> itr = iter(l)
Py> zip(itr, itr) # Put every second item in position 2
[(1, 2), (3, 4)]

Using zip(*[iter(l)]*N) or zip(*(iter(l),) *N) simply extends the above to the
general case.

I'd definitely recommend hiding this trick inside a function. Perhaps something
like (using Michael's function name):

from itertools import izip, repeat, chain

def partition(seq, part_len):
return izip(*((iter(se q),) * part_len))

def padded_partitio n(seq, part_len, pad_val=None):
itr = iter(seq)
if (len(seq) % part_len != 0):
padding = repeat(pad_val, part_len)
itr = chain(itr, padding)
return izip(*((itr,) * part_len))

Py> list(partition( range(10), 2))
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]
Py> list(partition( range(10), 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
Py> list(padded_par tition(range(10 ), 2))
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]
Py> list(padded_par tition(range(10 ), 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
Py> list(padded_par tition(range(10 ), 3, False))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, False, False)]
Py> zip(*padded_par tition(range(10 ), 3))
[(0, 3, 6, 9), (1, 4, 7, None), (2, 5, 8, None)]

Not sure how useful that last example is, but I thought it was cute :)

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[gene.tani@gmail.com]

Re: a sequence question

cookbook's not an easy grep but:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303060

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303279

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347689

[Steven Bethard]

Re: a sequence question

Nick Coghlan wrote:[color=blue]
> I'd definitely recommend hiding this trick inside a function. Perhaps
> something like (using Michael's function name):
>
> from itertools import izip, repeat, chain
>
> def partition(seq, part_len):
> return izip(*((iter(se q),) * part_len))
>
> def padded_partitio n(seq, part_len, pad_val=None):
> itr = iter(seq)
> if (len(seq) % part_len != 0):
> padding = repeat(pad_val, part_len)
> itr = chain(itr, padding)
> return izip(*((itr,) * part_len))[/color]

I think you can write that second one so that it works for iterables
without a __len__:

py> def padded_partitio n(iterable, part_len, pad_val=None):
.... itr = itertools.chain (
.... iter(iterable), itertools.repea t(pad_val, part_len - 1))
.... return itertools.izip( *[itr]*part_len)
....
py> list(padded_par tition(itertool s.islice(iterto ols.count(), 10), 2))
[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]
py> list(padded_par tition(itertool s.islice(iterto ols.count(), 10), 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

I just unconditionally pad the iterable with 1 less than the partition
size... I think that works right, but I haven't tested it any more than
what's shown.

Steve

[todddeluca@gmail.com]

Re: a sequence question


Chris Wright wrote:[color=blue]
> Hi,
>
> 1) I want to iterate over a list "N at a time"
> sort of like:
>
> # Two at a time... won't work, obviously
>[color=green][color=darkred]
> >>> for a, b in [1,2,3,4]:[/color][/color]
> ... print a,b
> ...
> Traceback (most recent call last):
> File "<interacti ve input>", line 1, in ?
> TypeError: unpack non-sequence[color=green][color=darkred]
> >>>[/color][/color]
>
>
> Is there a nifty way to do with with list comprehensions,
> or do I just have to loop over the list ?
>
> cheers and thanks
>
> chris wright[/color]

I wouldn't call this nifty, but it does use list comprehensions:
(n-(len(l)%n))%n is the amount of padding
(len(l)+(n-(len(l)%n))%n)/n is the number of groups (calculated by
adding the padding to the length of l and then dividing by n)
[color=blue][color=green][color=darkred]
>>> l = range(10)
>>> n = 3
>>> [(l+[None]*((n-(len(l)%n))%n))[i*n:(i+1)*n] for i in[/color][/color][/color]
xrange((len(l)+ (n-(len(l)%n))%n)/n)]
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, None, None]]

Regards,
Todd

[Nick Coghlan]

Re: a sequence question

Steven Bethard wrote:[color=blue]
> I think you can write that second one so that it works for iterables
> without a __len__:
>
> py> def padded_partitio n(iterable, part_len, pad_val=None):
> ... itr = itertools.chain (
> ... iter(iterable), itertools.repea t(pad_val, part_len - 1))
> ... return itertools.izip( *[itr]*part_len)
> ...
> py> list(padded_par tition(itertool s.islice(iterto ols.count(), 10), 2))
> [(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]
> py> list(padded_par tition(itertool s.islice(iterto ols.count(), 10), 3))
> [(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
>
> I just unconditionally pad the iterable with 1 less than the partition
> size... I think that works right, but I haven't tested it any more than
> what's shown.[/color]

I think you're right - I was looking at padding unconditionally , but because I
was padding with the actual partition length, it didn't work correctly when the
padding wasn't needed.

Padding with one less than the partition length fixes that quite neatly.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[David Isaac]

Re: a sequence question

"Nick Coghlan" <ncoghlan@iinet .net.au> wrote in message
news:mailman.15 53.1106960946.2 2381.python-list@python.org ...[color=blue]
> Using zip(*[iter(l)]*N) or zip(*(iter(l),) *N) simply extends the above to[/color]
the[color=blue]
> general case.[/color]

Clearly true.
But can you please go into much more detail for a newbie?
I see that [iter(l)]*N produces an N element list with each element being
the same iterator object, but after that

http://www.python.org/doc/2.3.5/lib/built-in-funcs.html

just didn't get me there.

Thanks,
Alan Isaac

[Nick Coghlan]

Re: a sequence question

David Isaac wrote:[color=blue]
> "Nick Coghlan" <ncoghlan@iinet .net.au> wrote in message
> news:mailman.15 53.1106960946.2 2381.python-list@python.org ...
>[color=green]
>>Using zip(*[iter(l)]*N) or zip(*(iter(l),) *N) simply extends the above to[/color]
>
> the
>[color=green]
>>general case.[/color]
>
>
> Clearly true.
> But can you please go into much more detail for a newbie?
> I see that [iter(l)]*N produces an N element list with each element being
> the same iterator object, but after that
> http://www.python.org/doc/2.3.5/lib/built-in-funcs.html
> just didn't get me there.[/color]

See if the following interactive examples clear things up at all:

# The unclear version
Py> itr = iter(range(10))
Py> zipped = zip(*(itr,)*3) # How does this bit work?
Py> print "\n".join(map(s tr, zipped))
(0, 1, 2)
(3, 4, 5)
(6, 7, 8)

# Manual zip, printing as we go
Py> itr = iter(range(10))
Py> try:
.... while 1: print (itr.next(), itr.next(), itr.next())
.... except StopIteration:
.... pass
....
(0, 1, 2)
(3, 4, 5)
(6, 7, 8)

# Manual zip, actually behaving somewhat like the real thing
Py> itr = iter(range(10))
Py> zipped = []
Py> try:
.... while 1: zipped.append(( itr.next(), itr.next(), itr.next()))
.... except StopIteration:
.... pass
....
Py> print "\n".join(map(s tr, zipped))
(0, 1, 2)
(3, 4, 5)
(6, 7, 8)

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[David Isaac]

Re: a sequence question

> Alan Isaac wrote:[color=blue][color=green]
> > I see that [iter(l)]*N produces an N element list with each element[/color][/color]
being[color=blue][color=green]
> > the same iterator object, but after that
> > http://www.python.org/doc/2.3.5/lib/built-in-funcs.html
> > just didn't get me there.[/color][/color]

"Nick Coghlan" <ncoghlan@iinet .net.au> wrote in message
news:mailman.23 88.1108127294.2 2381.python-list@python.org ...[color=blue]
> Py> itr = iter(range(10))
> Py> zipped = zip(*(itr,)*3) # How does this bit work?
> # Manual zip, actually behaving somewhat like the real thing
> Py> itr = iter(range(10))
> Py> zipped = []
> Py> try:
> ... while 1: zipped.append(( itr.next(), itr.next(), itr.next()))
> ... except StopIteration:
> ... pass[/color]


http://www.python.org/doc/2.3.5/lib/built-in-funcs.html says:

"This function returns a list of tuples,
where the i-th tuple contains the i-th element from each of the argument
sequences."

So an "argument sequence" can in fact be any iterable,
and these in turn are asked *in rotation* for their yield, right?
So we pass the (identical) iterables in a tuple or list,
thereby allowing a variable number of arguments.
We unpack the argument list with '*',
which means we have provided three iterables as arguments.
And then zip works as "expected",
once we have learned to expect zip to "rotate" through the arguments.
Is that about right?

If that is right, I still cannot extract it from the doc cited above.
So where should I have looked?

Thanks,
Alan Isaac

[Nick Coghlan]

Re: a sequence question

David Isaac wrote:[color=blue]
> If that is right, I still cannot extract it from the doc cited above.
> So where should I have looked?[/color]

Ouch. The terminology's evolved, and it looks to me like the docs for the older
builtins haven't been updated to track it.

The terminology has pretty much settled to 'iterable' for anything which returns
a sensible result from iter(obj), 'iterator' for any iterable which returns
itself from iter(obj), 'reiterable' for any iterable which is not an iterator,
and 'sequence' for any reiterable which supports len(obj) and integer indexing.

That's not the terminology the older docs use, though, even in the most recent
versions of that page [1].

For most of them it's OK, since the text clarifies what the term means in
context (e.g. that 'sequence' actually means 'iterable' for some function
signatures). zip() doesn't do that though - it actually accepts iterables, but
only talks about sequences.

A bug report on Sourceforge would help in getting the problem fixed for the 2.5
docs (possibly even the 2.4.1 docs if it happens soon). 2.3's a lost cause
though, since 2.3.5 is already out the door and only another security bug is
likely to trigger a new 2.3 release.

For the 'left-to-right' evaluation thing, that's technically an implementation
artifact of the CPython implementation, since the zip() docs don't make any
promises. So updating the docs to include that information would probably be a
bigger issue, as it involves behaviour which is currently not defined by the
library.

Cheers,
Nick.

[1] http://www.python.org/dev/doc/devel/...-in-funcs.html
--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net