question about slicing with a step length

Re: question about slicing with a step length

John Salerno wrote:[color=blue]
> Given:
>
> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
> can someone explain to me why
>
> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?
>
> I thought the first index, whether going forward or backward, was
> inclusive. And there is no index of 10 in this list, so what is it
> referring to?
>
> Thanks.[/color]

Its referring to right after the end of the list, just as the 0 refers
to just before the start of the list. The word "slice" is supposed to
suggest cutting between (or before or after) elements.

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

Re: question about slicing with a step length

John Salerno wrote:[color=blue]
> Given:
>
> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
> can someone explain to me why
>
> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color]

I always have trouble with these. Given the docs[1]:

"""
The slice of s from i to j with step k is defined as the sequence of
items with index x = i + n*k such that $0 \leq n < \frac{j-i}{k}$. In
other words, the indices are i, i+k, i+2*k, i+3*k and so on, stopping
when j is reached (but never including j). If i or j is greater than
len(s), use len(s). If i or j are omitted then they become ``end''
values (which end depends on the sign of k). Note, k cannot be zero.
"""

I would expect that we're looking at the indices
10 + n*(-2)
such that
0 <= n < (0-10)/-2
0 <= n < 5
which should be indices 10, 8, 6, 4, 2 which would skip 10 so as not to
provoke an IndexError, and then give you the list [9, 7, 3, 5, 1].

Clearly that's not what happens. Looks like there's a rule in there
somewhere that says if the start is greater than the length, use the
length instead. That would mean starting at 9 instead of 10, and
getting the indices 9, 7, 5, 3, 1 which would give you the list [10, 8,
6, 4, 2]

Hmm... Is there some documentation I'm missing that says the start
index will be replaced with the length if necessary? I'll file a doc
patch if it looks like I'm not just missing something somewhere else...

STeVe

[1]http://docs.python.org/lib/typesseq.html

Re: question about slicing with a step length

John Salerno wrote:
[color=blue]
> Given:
>
> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
> can someone explain to me why
>
> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color]

I think the documentation is misleading/incomplete when
it comes to negative strides for extended slices.

The relevent sections are here:



[quote]
Some sequences also support "extended slicing" with a
third "step" parameter: a[i:j:k] selects all items of a
with index x where x = i + n*k, n >= 0 and i <= x < j.
[end quote]

and from http://www.python.org/doc/2.3.5/ref/slicings.html

[quote]
It is not an error if i or j lie outside the range of
valid indexes (such items don't exist so they aren't
selected).
[end quote]


The documentation suggests that, given a slice
[10:0:-2], Python looks up indices:

10 + 0*-2, 10 + 1*-2, 10 + 2*-2, ...

or 10, 8, 6, 4, 2, 0, ...

but since *none* of these indices is within the limits
10 <= x and x < 0, the documentation suggests that
the result should be the empty list.

The indices actually selected are:

9, 7, 5, 3, 1

at which point I give up and throw my hands in the air
and promise never to use negative strides with extended
slices.


--
Steven.

Re: question about slicing with a step length

James Stroud wrote:[color=blue]
> John Salerno wrote:[color=green]
>> Given:
>>
>> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>
>> can someone explain to me why
>>
>> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?
>>
>> I thought the first index, whether going forward or backward, was
>> inclusive. And there is no index of 10 in this list, so what is it
>> referring to?
>>
>> Thanks.[/color]
>
> Its referring to right after the end of the list, just as the 0 refers
> to just before the start of the list. The word "slice" is supposed to
> suggest cutting between (or before or after) elements.
>[/color]

But why isn't the first index inclusive in this case?

Re: question about slicing with a step length

John Salerno wrote:[color=blue]
> James Stroud wrote:
>[color=green]
>> John Salerno wrote:
>>[color=darkred]
>>> Given:
>>>
>>> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>>
>>> can someone explain to me why
>>>
>>> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?
>>>
>>> I thought the first index, whether going forward or backward, was
>>> inclusive. And there is no index of 10 in this list, so what is it
>>> referring to?
>>>
>>> Thanks.[/color]
>>
>>
>> Its referring to right after the end of the list, just as the 0 refers
>> to just before the start of the list. The word "slice" is supposed to
>> suggest cutting between (or before or after) elements.
>>[/color]
>
> But why isn't the first index inclusive in this case?[/color]
You skipped it with the -2 step size.

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

Re: question about slicing with a step length


Steven D'Aprano wrote:[color=blue]
> John Salerno wrote:
>[color=green]
> > Given:
> >
> > numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
> >
> > can someone explain to me why
> >
> > numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color]
>
> I think the documentation is misleading/incomplete when
> it comes to negative strides for extended slices.[/color]

Agreed!

[snip...][color=blue]
>
> The indices actually selected are:
>
> 9, 7, 5, 3, 1
>
> at which point I give up and throw my hands in the air
> and promise never to use negative strides with extended
> slices.
>[/color]
!!! Agreed!

I found this which seems an appropriate description:
"""
An extended slice of list x of length n in the form x[j:k:i] selects
every i-th element starting with and including the element at index j
and ending with but not including the element at index k. When either
index is negative, the value of n is added to it before any further
processing occurs. When either index is missing or lies outside of the
list bounds, the minimum or maximum inclusive index is used
automatically.
"""
source: https://trove.homeip.net/ExtendedSlices
André
[color=blue]
> --
> Steven.[/color]

Re: question about slicing with a step length

André wrote:
[color=blue]
> An extended slice of list x of length n in the form x[j:k:i] selects
> every i-th element starting with and including the element at index j[/color]

This makes it sound like the index of 10 should be inclusive.

[color=blue]
> When either index is missing or lies outside of the
> list bounds, the minimum or maximum inclusive index is used
> automatically.[/color]

So does that mean that the example actually is wrong, and Python is
compensating? I tried numbers[9:0:-2] and it returned the same value as
if I had used 10 as the first index, so perhaps 10 shouldn't be there
after all.

Re: question about slicing with a step length

> John Salerno wrote:[color=blue][color=green]
>> Given:
> > numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
> > can someone explain to me why
> > numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color][/color]

It appears that s[i:j:-1] is s[(j+1):(i+1)] .reverse()'ed. For 'numbers',
this is 10, 9, 8, 7, 6, 5, 4, 3, 2]. Then take every other item. Why the
+1? Don't know and not my intuitive expectation. I just know that
extended slicing was developed for and until recently pretty much
restricted to numeric (now numpy).

Steven D'Aprano[color=blue]
> I think the documentation is misleading/incomplete when
> it comes to negative strides for extended slices.[/color]

and Andre "Agreed!" also, and you three aren't the only ones. Maybe some
day I will read the source, think about it more, and post a suggested
revision for comment ... or maybe someone will beat me to it.

Terry Jan Reedy

Re: question about slicing with a step length


Terry Reedy wrote:[color=blue][color=green]
> > John Salerno wrote:[color=darkred]
> >> Given:
> > > numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
> > > can someone explain to me why
> > > numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color][/color]
>
> It appears that s[i:j:-1] is s[(j+1):(i+1)] .reverse()'ed. For 'numbers',
> this is 10, 9, 8, 7, 6, 5, 4, 3, 2]. Then take every other item. Why the
> +1? Don't know and not my intuitive expectation. I just know that
> extended slicing was developed for and until recently pretty much
> restricted to numeric (now numpy).
>[/color]
It's not simply "+1".[color=blue][color=green][color=darkred]
>>> a = range(10)
>>> a[9:0:-2][/color][/color][/color]
[9, 7, 5, 3, 1]
[color=blue][color=green][color=darkred]
>>> a[10:0:-2][/color][/color][/color]
[9, 7, 5, 3, 1]
[color=blue][color=green][color=darkred]
>>> a[11:0:-2][/color][/color][/color]
[9, 7, 5, 3, 1]
[color=blue][color=green][color=darkred]
>>> a[42:0:-2][/color][/color][/color]
[9, 7, 5, 3, 1]
[color=blue]
> Steven D'Aprano[color=green]
> > I think the documentation is misleading/incomplete when
> > it comes to negative strides for extended slices.[/color]
>
> and Andre "Agreed!" also, and you three aren't the only ones. Maybe some
> day I will read the source, think about it more, and post a suggested
> revision for comment ... or maybe someone will beat me to it.[/color]

I, myself, think that the slicing behaviour is broken in this case.[color=blue]
>From the little I understand (without looking at the source),[/color]
adjustment on the boundaries of the slicing "request" (the first 2
numbers) is done first, to make them positive and, I believe, ensure
that they do not extend beyond the boundaries of the sequence. Then,
the actual stepping through is performed.

In my (most likely not-well informed enough opinion), the behaviour
should be the following:
from [i: j: k],
if k > 0 and j <= i : return []
if k< 0 and j >= i : return []
otherwise, build the following list of numbers:
i, i+k, i+2*k,
until i +n*k >= j (if k>0; use <= if k<0)
where the last number is excluded, i.e.
[0: 2: 1] = list(0, 1)
*Then*, exclude from the list just built any negative numbers, or any
number greater than the length of the sequence to be sliced.

So, [10:0:-2] would yield the following list of numbers
[10, 8, 6, 4, 2] before trimming; applying this to
range(10), would end up with result in trimming "10" from the list
built.
Thus, we are looking at indices [8, 6, 4, 2] of range(10), which happen
to be the numbers [8, 6, 4, 2] in this case. We should end up with
the same list if we ask for [42:0:-2]. This is not the observed
behaviour,
as we get [9, 7, 5, 3, 1].
If we ask Python to do[color=blue][color=green][color=darkred]
>>> a[0:10:2][/color][/color][/color]
we get
[0, 2, 4, 6, 8]
which is correct. It seems to me that the following (pseudo-code, not
valid Python)
a[2:10:2] == a[8:0:-2].reversed()
should be true, just like
a[1:10:1] == a[9:0:-1].reversed()
is true.

I *suspect* that "boundary adjustments" are performed first before
creating a sequence of indices, so that there is less "trimming" (as
described above) for the sake of efficiency ... and that this is not
done properly.

Then again, perhaps I simply don't understand what slices are supposed
to do....

André
[color=blue]
>
> Terry Jan Reedy[/color]

Re: question about slicing with a step length

John Salerno wrote:[color=blue]
> Given:
>
> numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>
> can someone explain to me why
>
> numbers[10:0:-2] results in [10, 8, 6, 4, 2]?[/color]

I've filed a bug report:


I suggest the following rewording for extended slices:

"""
To get the slice of s from i to j with step k, first
determine the bounds. If k is positive, and i or j is
greater than len(s), use len(s). If k is negative, and
i or j is greater than len(s)-1, use len(s)-1. Note, k
cannot be zero. If i or j are omitted then they become
``end'' values (which end depends on the sign of k).

The slice of s from i to j with step k is then defined
as the sequence of items with index x = i + n*k such
that 0 <= n < (j - i)/k. In other words, the indices
are i, i+k, i+2*k, i+3*k and so on, stopping when j is
reached (but never including j).
"""

I believe that properly explains the behavior, but if I've messed it up
(entirely likely), please suggest alternate wording on the bug report.

STeVe

Re: question about slicing with a step length


"André" <andre.roberge@ gmail.com> wrote in message
news:1141906509 .952096.165310@ p10g2000cwp.goo glegroups.com.. .
[color=blue]
>Terry Reedy wrote:[color=green]
>> It appears that s[i:j:-1] is s[(j+1):(i+1)] .reverse()'ed. For
>> 'numbers',
>> this is 10, 9, 8, 7, 6, 5, 4, 3, 2]. Then take every other item. Why
>> the
>> +1? Don't know and not my intuitive expectation. I just know that
>> extended slicing was developed for and until recently pretty much[/color][/color]
<> restricted to numeric (now numpy).
[color=blue]
> It's not simply "+1".[/color]

Yes it is. Boundary adjustment follows, as implied by what I wrote.
[color=blue][color=green][color=darkred]
> >>> a = range(10)
> >>> a[9:0:-2]
> >>> a[10:0:-2]
> >>> a[11:0:-2]
> >>> a[42:0:-2][/color][/color][/color]

are all [9, 7, 5, 3, 1] because a[1:10] == a[1:11] == a[1:12] == a[1:43] ==
[1,3,5,7,9], which is then reversed.

Terry Jan Reedy