list.clear() missing?!?

[Steven Bethard]

Re: list.clear() missing?!?

John Salerno wrote:[color=blue]
> Steven Bethard wrote:
>[color=green]
>> I think these are all good reasons for adding a clear method, but
>> being that it has been so hotly contended in the past, I don't think
>> it will get added without a PEP. Anyone out there willing to take out
>> the best examples from this thread and turn it into a PEP?[/color]
>
> What are the usual arguments against adding it?[/color]

I think most of the other folks have already given you the answers, but
the main ones I remember:

(1) There are already at least two ways of spelling it. We don't need
another.

(2) Clearing a list is a pretty uncommon operation. Usually just
creating a new list is fine (and easier).

Disclaimer: I'm happy with adding list.clear(), so it's quite possible
I'm misrepresenting the argument here. I guess that's just all the more
reason to have a PEP for it: to lay out all the pros and cons.

STeVe

[Martin v. Löwis]

Re: list.clear() missing?!?

Steven D'Aprano wrote:[color=blue][color=green][color=darkred]
>>> $ python2.4 -mtimeit 'x = range(100000); '
>>> 100 loops, best of 3: 6.7 msec per loop
>>> $ python2.4 -mtimeit 'x = range(100000); del x[:]'
>>> 100 loops, best of 3: 6.35 msec per loop
>>> $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
>>> 100 loops, best of 3: 6.36 msec per loop
>>> $ python2.4 -mtimeit 'x = range(100000); del x'
>>> 100 loops, best of 3: 6.46 msec per loop
>>>
>>> Why the first benchmark is the slowest? I don't get it... could someone
>>> test this, too?[/color]
>> In the first benchmark, you need space for two lists: the old one and
>> the new one;[/color]
>
> Er, what new list? I see only one list, x = range(100000), which is merely
> created then nothing done to it. Have I missed something?[/color]

See Duncan's explanation. This code is run many times, allocating many
lists. However, only two different lists exist at any point at time.

A Python list consists of two memory blocks: the list proper (a few
bytes), plus the "guts", i.e. a variable-sized block of pointers to
the objects. del x[:] frees the guts.
[color=blue]
> I understood Felipe to be asking, why does it take longer to just create a
> list, than it takes to create a list AND then do something to it?[/color]

Actually, the same code (deallocation of all integers and the list
blocks) appears in either case. However, in the one case it is triggered
explicitly, and before the new allocation; in the other case, the
allocation of the new objects occurs before the old ones are released.

Regards,
Martin

[Mel Wilson]

Re: list.clear() missing?!?

Ville Vainio wrote:[color=blue]
> Fredrik Lundh wrote:[color=green]
>>because Python already has a perfectly valid way to clear a list,
>>perhaps ?
>>
>> del l[:][/color]
>
>
> Ok. That's pretty non-obvious but now that I've seen it I'll probably
> remember it. I did a stupid "while l: l.pop()" loop myself.[/color]

Actually, it's in the Library Reference (that we keep under
our pillows) section 2.3.6.4 Mutable Sequence Types.

Cheers, Mel.

[Raymond Hettinger]

Re: list.clear() missing?!?

[Steven Bethard][color=blue]
> I think these are all good reasons for adding a clear method, but being
> that it has been so hotly contended in the past, I don't think it will
> get added without a PEP. Anyone out there willing to take out the best
> examples from this thread and turn it into a PEP?[/color]

Something this small doesn't need a PEP. I'll just send a note to
Guido asking for a pronouncement.

Here's a draft list of pros and cons (any changes or suggestions are
welcome):

Pros:
-----

* s.clear() is more obvious in intent

* easier to figure-out, look-up, and remember than either s[:]=[] or
del s[:]

* parallels the api for dicts, sets, and deques (increasing the
expecation that lists will too)

* the existing alternatives are a bit perlish

* the OP is shocked, SHOCKED that python got by for 16 years without
list.clear()


Cons:
-----

* makes the api fatter (there are already two ways to do it)

* expanding the api makes it more difficult to write classes that can
be polymorphically substituted for lists

* learning slices is basic to the language (this lesson shouldn't be
skipped)

* while there are valid use cases for re-using lists, the technique is
already overused for unsuccessful attempts to micro-optimize (creating
new lists is surprisingly fast)

* the request is inane, the underlying problem is trivial, and the
relevant idiom is fundamental (api expansions should be saved for rich
new functionality and not become cluttered with infrequently used
redundant entries)

[Felipe Almeida Lessa]

Re: list.clear() missing?!?

Em Qua, 2006-04-12 às 12:40 -0700, Raymond Hettinger escreveu:[color=blue]
> * the existing alternatives are a bit perlish[/color]

I love this argument =D! "perlish".. . lol...

Cheers,

--
Felipe.

[Raymond Hettinger]

Re: list.clear() missing?!?

[Felipe Almeida Lessa][color=blue][color=green]
> > I love benchmarks, so as I was testing the options, I saw something very
> > strange:
> >
> > $ python2.4 -mtimeit 'x = range(100000); '
> > 100 loops, best of 3: 6.7 msec per loop
> > $ python2.4 -mtimeit 'x = range(100000); del x[:]'
> > 100 loops, best of 3: 6.35 msec per loop
> > $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
> > 100 loops, best of 3: 6.36 msec per loop
> > $ python2.4 -mtimeit 'x = range(100000); del x'
> > 100 loops, best of 3: 6.46 msec per loop
> >
> > Why the first benchmark is the slowest? I don't get it... could someone
> > test this, too?[/color][/color]

[Dan Christensen][color=blue]
> I get similar behaviour. No idea why.[/color]

It is an effect of the memory allocator and fragmentation. The first
builds up a list with increasingly larger sizes. It periodically
cannot grow in-place because something is in the way (some other
object) so it needs to move its current entries to another, larger
block and grow from there. In contrast, the other entries are reusing
a the previously cleared out large block.

Just for grins, replace the first with"
'x=None; x=range(100000) '
The assignment to None frees the reference to the previous list and
allows it to be cleared so that its space is immediately available to
the new list being formed by range().

[Peter Hansen]

Re: list.clear() missing?!?

Raymond Hettinger wrote:[color=blue]
> Cons:
> -----
> * learning slices is basic to the language (this lesson shouldn't be
> skipped)[/color]

And yet it doesn't appear to be in the tutorial. I could have missed
it, but I've looked in a number of the obvious places, without actually
going through it (again) from start to finish. Also, googling for
"slice site:docs.pytho n.org", you have to go to the *sixth* entry before
you can find the first mention of "del x[:]" and what it does. I think
given the current docs it's possible to learn all kinds of things about
slicing and still not make the non-intuitive leap that "del x[slice]" is
actually how you spell "delete contents of list in-place".
[color=blue]
> * while there are valid use cases for re-using lists, the technique is
> already overused for unsuccessful attempts to micro-optimize (creating
> new lists is surprisingly fast)[/color]

Not just valid use-cases, but ones for which "x = []" is entirely buggy,
yet not obviously so, especially to newcomers.
[color=blue]
> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)[/color]

The first phrase is insulting and untrue, the second merely untrue (as
even your own list of pros and cons shows), and the last completely
valid and highly relevant, yet not overriding.

-Peter

[Ville Vainio]

Re: list.clear() missing?!?

Raymond Hettinger wrote:
[color=blue]
> * easier to figure-out, look-up, and remember than either s[:]=[] or
> del s[:][/color]

Easier is an understatement - it's something you figure out
automatically. When I want to do something w/ an object, looking at its
methods (via code completion) is the very first thing.
[color=blue]
> * the OP is shocked, SHOCKED that python got by for 16 years without
> list.clear()[/color]

I'm sure you realize I was being sarcastic...
[color=blue]
> * learning slices is basic to the language (this lesson shouldn't be
> skipped)[/color]

Assigning to slices is much less important, and is something I always
never do (and hence forget).
[color=blue]
> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)[/color]

I understand that these are the main arguments. However, as it stands
there is no one *obvious* way to clear a list in-place. I agree that
it's rare to even need it, but when you do a it's a little bit of a
surprise.

[Ville Vainio]

Re: list.clear() missing?!?

Ville Vainio wrote:
[color=blue]
> Assigning to slices is much less important, and is something I always
> never do (and hence forget).[/color]

ALMOST never, of course.

[Alan Morgan]

Re: list.clear() missing?!?

In article <1144870852.886 477.238570@u72g 2000cwu.googleg roups.com>,
Raymond Hettinger <python@rcn.com > wrote:[color=blue]
>[Steven Bethard][color=green]
>> I think these are all good reasons for adding a clear method, but being
>> that it has been so hotly contended in the past, I don't think it will
>> get added without a PEP. Anyone out there willing to take out the best
>> examples from this thread and turn it into a PEP?[/color]
>
>Something this small doesn't need a PEP. I'll just send a note to
>Guido asking for a pronouncement.
>
>Here's a draft list of pros and cons (any changes or suggestions are
>welcome):
>
>Pros:
>-----
>
>* s.clear() is more obvious in intent[/color]

Serious question: Should it work more like "s=[]" or more like
"s[:]=[]". I'm assuming the latter, but the fact that there is
a difference is an argument for not hiding this operation behind
some syntactic sugar.

Alan
--
Defendit numerus

[Steven D'Aprano]

Re: list.clear() missing?!?

On Wed, 12 Apr 2006 12:40:52 -0700, Raymond Hettinger wrote:
[color=blue]
> Something this small doesn't need a PEP. I'll just send a note to
> Guido asking for a pronouncement.[/color]

Raymond, if you're genuinely trying to help get this sorted in the
fairest, simplest way possible, I hope I speak for everyone when
I say thank you, your efforts are appreciated.

But there is this:
[color=blue]
> * the request is inane, the underlying problem is trivial, and the
> relevant idiom is fundamental (api expansions should be saved for rich
> new functionality and not become cluttered with infrequently used
> redundant entries)[/color]

Is this sort of editorialising fair, or just a way of not-so-subtly
encouraging Guido to reject the whole idea, now and forever?

Convenience and obviousness are important for APIs -- that's why lists
have pop, extend and remove methods. The only difference I can see between
a hypothetical clear and these is that clear can be replaced with a
one-liner, while the others need at least two, e.g. for extend:

for item in seq:
L.append(item)

Here is another Pro for your list:

A list.clear method will make deleting items from a list more OO,
consistent with almost everything else you do to lists, and less
procedural. This is especially true if clear() takes an optional index (or
two), allowing sections of the list to be cleared, not just the entire
list.


--
Steven.

[Steven D'Aprano]

Re: list.clear() missing?!?

On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote:
[color=blue]
> Serious question: Should it work more like "s=[]" or more like
> "s[:]=[]". I'm assuming the latter, but the fact that there is
> a difference is an argument for not hiding this operation behind
> some syntactic sugar.[/color]

Er, I don't see how it can possibly work like s = []. That merely
reassigns a new empty list to the name s, it doesn't touch the existing
list (which may or may not be garbage collected soon/immediately
afterwards).

As far as I know, it is impossible -- or at least very hard -- for an
object to know which namesspaces it is in, so it can reassign one name but
not the others. Even if such a thing was possible, I think it is an
absolutely bad idea.



--
Steven.

[Alan Morgan]

Re: list.clear() missing?!?

In article <pan.2006.04.12 .23.23.07.83001 8@REMOVETHIScyb er.com.au>,
Steven D'Aprano <steve@REMOVETH IScyber.com.au> wrote:[color=blue]
>On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote:
>[color=green]
>> Serious question: Should it work more like "s=[]" or more like
>> "s[:]=[]". I'm assuming the latter, but the fact that there is
>> a difference is an argument for not hiding this operation behind
>> some syntactic sugar.[/color]
>
>Er, I don't see how it can possibly work like s = []. That merely
>reassigns a new empty list to the name s, it doesn't touch the existing
>list (which may or may not be garbage collected soon/immediately
>afterwards).[/color]

Right. I was wondering what would happen in this case:

s=[1,2,3]
t=s
s.clear()
t # [] or [1,2,3]??

If you know your basic python it is "obvious" what would happen
if you do s=[] or s[:]=[] instead of s.clear() and I guess it is
equally "obvious" which one s.clear() must mimic. I'm still not
used to dealing with mutable lists.

Alan
--
Defendit numerus

[Peter Hansen]

Re: list.clear() missing?!?

Alan Morgan wrote:[color=blue]
> Right. I was wondering what would happen in this case:
>
> s=[1,2,3]
> t=s
> s.clear()
> t # [] or [1,2,3]??
>
> If you know your basic python it is "obvious" what would happen
> if you do s=[] or s[:]=[] instead of s.clear() and I guess it is
> equally "obvious" which one s.clear() must mimic. I'm still not
> used to dealing with mutable lists.[/color]

If you know your basic python :-), you know that s[:] = [] is doing the
only thing that s.clear() could possibly do, which is changing the
contents of the list which has the name "s" bound to it (and which might
have other names bound to it, just like any object in Python). It
*cannot* be doing the same as "s=[]" which does not operate on the list
but creates an entirely new one and rebinds the name "s" to it.

The only possible answer for your question above is "t is s" and "t ==
[]" because you haven't rebound the names.

-Peter

[Peter Hansen]

Re: list.clear() missing?!?

Steven D'Aprano wrote:[color=blue]
> Convenience and obviousness are important for APIs -- that's why lists
> have pop, extend and remove methods. The only difference I can see between
> a hypothetical clear and these is that clear can be replaced with a
> one-liner, while the others need at least two, e.g. for extend:
>
> for item in seq:
> L.append(item)[/color]

It's not even clear that extend needs two lines:
[color=blue][color=green][color=darkred]
>>> s = range(5)
>>> more = list('abc')
>>> s[:] = s + more
>>> s[/color][/color][/color]
[0, 1, 2, 3, 4, 'a', 'b', 'c']

Okay, it's not obvious, but I don't think s[:]=[] is really any more
obvious as a way to clear the list.

Clearly .extend() needs to be removed from the language as it is an
unnecessary extension to the API using slicing, which everyone should
already know about...

-Peter