Py2.3: Feedback on Sets

**Troels Therkelsen** · Jul 18 '05, 01:29 AM

Re: Py2.3: Feedback on Sets

In article <Jp%Za.256$jw4. 238@nwrdny03.gn ilink.net>, Raymond Hettinger wrote:[color=blue]
> I've gotten lots of feedback on the itertools module
> but have not heard a peep about the new sets module.[/color]

I would have to say that while I have looked at the sets module and read its
documentation, I have not used it much (more below).
[color=blue]
> * Are you overjoyed/outraged by the choice of | and &
> as set operators (instead of + and *)?[/color]

I actually prefer the | & syntax because it makes more intuitive sense to
me, as the operators work identically to how they do when using them as
binary number operators.

[snip]
[color=blue]
> * Do you care that sets can only contain hashable elements?[/color]

No.

[snip]
[color=blue]
> * Are the docs clear? Can you suggest improvements?[/color]

The docs for the sets module, like most of the Python docs, are very good.
The example helps, too.
[color=blue]
> * Are sets helpful in your daily work or does the need arise
> only rarely?[/color]

I rarely need sets explicitly, but sometimes need the logic that they offer.
For example, when wanting to concantenate two lists so that the resulting list
only has unique elements, that can be done with set logic. However, as it
is another module written in Python, and I only need the logic I usually don't
go through with the overhead of creating the Set classes, etc. If it was
better integrated (ie., a set() builtin type constructor like int(), str(),
etc) then I would feel less of a reservation against using sets. I know this
reservation isn't founded in facts, but more in the feeling of trusting
builtin types more than custom classes provided by a module.
[color=blue]
> User feedback is essential to determining the future direction
> of sets (whether it will be implemented in C, change API,
> and/or be given supporting language syntax).[/color]

Don't know if my feedback provided above helps, but you asked for it ;-)

Regards,

Troels Therkelsen

**Carl Banks** · Jul 18 '05, 01:29 AM

Re: Py2.3: Feedback on Sets

Raymond Hettinger wrote:[color=blue]
> I've gotten lots of feedback on the itertools module
> but have not heard a peep about the new sets module.
>
> * Are you overjoyed/outraged by the choice of | and &
> as set operators (instead of + and *)?[/color]

I slightly favor | and &.

[color=blue]
> * Is the support for sets of sets necessary for your work
> and, if so, then is the implementation sufficiently
> powerful?
>
> * Is there a compelling need for additional set methods like
> Set.powerset() and Set.isdisjoint( s) or are the current
> offerings sufficient?[/color]

I imagine isdisjoint would be useful, although it's easy enough to use
bool(s&t).

[color=blue]
> * Does the performance meet your expectations?
>
> * Do you care that sets can only contain hashable elements?
>
> * How about the design constraint that the argument to most
> set methods must be another Set (as opposed to any iterable)?
>
> * Are the docs clear? Can you suggest improvements?[/color]

Yeah: in the library reference, the table entry for s.union(t) should
say "synonym of s|t" instead of repeating the description. This is
especially true because it's not clear from a simple glance whether
"s.union(t) " goes with "s|t" or "s&t", because it sits right between
the two. Better yet, I would change it to a three-column table
(operation, synonym, result).

[color=blue]
> * Are sets helpful in your daily work or does the need arise
> only rarely?[/color]

I haven't used sets yet (or Python 2.3), but I expect to use them a
lot. However, I imagine my typical use would be efficient testing for
membership. I have maybe half a dozen places where I use a dictionary
for that now.

--
CARL BANKS
"You don't run Microsoft Windows. Microsoft Windows runs you."

**Istvan Albert** · Jul 18 '05, 01:30 AM

Re: Py2.3: Feedback on Sets

Raymond Hettinger wrote:

First of all, thanks for the work on it, I need to use sets
in my work all the time. I had written my own
(simplistic) implementation but that adds another layer
of headaches when distributing programs since then
I have to distribute multiple modules.

Sometimes I ended up with a little set function in every
big module. Pretty silly. For me sets are a greatly useful
addition.
[color=blue]
> * Is the support for sets of sets necessary for your work
> and, if so, then is the implementation sufficiently
> powerful?[/color]

One pattern that I constantly need is to remove duplicates from
a sequence. I don't know if this an often enough used pattern to
warrant an API change, for me it would be most useful if I could
get the contents of a set as a sequence right away, without having to
explicitly code it.
[color=blue]
> * Are you overjoyed/outraged by the choice of | and & as
> set operators (instead of + and *)?[/color]

I think that since you have have - as a difference operator it
would make sense to also have + as a union operator. Takes nothing
away from |. The & operator is the right one, * would not be appropriate
IMO.
[color=blue]
> * Do you care that sets can only contain hashable elements?[/color]

I don't really care, on the other hand, it might be better to call the
class HashSet, so that it conveys right away that it uses hashing
to store the elements.
[color=blue]
> * Are the docs clear? Can you suggest improvements?[/color]

I wondered whether it would be better to specify the immutability
of the class at the constructor level.

Then there is the update method. It feels a little bit redundant
since there is an add() method that seems to be doing the same thing
only that add() adds only one element at a time.
Would it be possible to have add() handle all additions, iterable or
not, then scrap update() altogether.

Then just by looking at the docs, it feels a little bit confusing to
have discard() and remove() do essentially the same thing but only one
of them raising an exception. Which one? I already forgot. I don't know
which one I would prefer though.

Another aspect that I did not understand, what is difference between
update() and union_update().

The long winded method names, such as difference_upda te() also feel
redundant when one can achieve the same thing with the -= operator. I
would drop these and instead show in the docs how to accomplish these
with the operators. Would considerably cut down on the documentation,
and apparent complexity.

I'm a big fan of having the minimal number of methods as long it is
easy to obtain the result.

For example methods like x.issubset(y) is the same as bool(x-y) so may
not be all that necessary, just a thought.
[color=blue]
> * Are sets helpful in your daily work or does the need arise
> only rarely?[/color]

I use them very often and they are extremely useful.

thanks again,

Istvan.

**Radovan Garabik** · Jul 18 '05, 01:32 AM

Re: Py2.3: Feedback on Sets

Raymond Hettinger <vze4rx4y@veriz on.net> wrote:[color=blue]
> I've gotten lots of feedback on the itertools module
> but have not heard a peep about the new sets module.
>
> * Are you overjoyed/outraged by the choice of | and &
> as set operators (instead of + and *)?[/color]

I would prefer to have + in addition to |. I do not care
about *, but IMHO + is so intuitive and natural that
it is a pity not to have it (you can add lists, tuples,
strings with +, but not sets???)
[color=blue]
>
> * Is the support for sets of sets necessary for your work
> and, if so, then is the implementation sufficiently
> powerful?[/color]

So far my code used dictionaries with values set to None,
I expect that I will use sets soon, it will be more logical
and the code more readable.
[color=blue]
>
> * Are sets helpful in your daily work or does the need arise
> only rarely?[/color]

daily work, production code, but so far I used lists or
dictionaries instead.
I am somewhat afraid of performance, until setrs are implemented
in C, though

--
-----------------------------------------------------------
| Radovan Garabík http://melkor.dnp.fmph.uniba.sk/~garabik/ |
| __..--^^^--..__ garabik @ kassiopeia.juls .savba.sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!

**Russell E. Owen** · Jul 18 '05, 01:33 AM

Re: Py2.3: Feedback on Sets

In article <bhcp3q$10918l$ 1@ID-89407.news.uni-berlin.de>,
garabik-news-2002-02@kassiopeia.j uls.savba.sk (Radovan Garabik) wrote:
[color=blue]
>Raymond Hettinger <vze4rx4y@veriz on.net> wrote:[color=green]
>> * Is the support for sets of sets necessary for your work
>> and, if so, then is the implementation sufficiently
>> powerful?[/color]
>
>So far my code used dictionaries with values set to None,
>I expect that I will use sets soon, it will be more logical
>and the code more readable.[/color]

Same here.

I don't rely on sets heavily (I do have a few implemented as
dictionaries with value=None) and am not yet ready to make my users
upgrade to Python 2.3.

I suspect the upgrade issue will significantly slow the incorporation of
sets and the other new modules, but that over time they're likely to
become quite popular. I am certainly looking forward to using sets and
csv.

I think it'd speed the adoption of new modules if they were explicitly
written to be compatible with one previous generation of Python (and
documented as such) so users could manually include them with their code
until the current generation of Python had a bit more time to be adopted.

I'm not saying this should be a rule, only suggesting it as a useful
goal. Presumably it'd be easy with some modules and not worth the work
in some cases.

-- Russell

**Chris Reedy** · Jul 18 '05, 01:33 AM

Re: Py2.3: Feedback on Sets

Raymond -

Well now that you ask ...

Raymond Hettinger wrote:[color=blue]
> I've gotten lots of feedback on the itertools module
> but have not heard a peep about the new sets module.
>
> * Are you overjoyed/outraged by the choice of | and &
> as set operators (instead of + and *)?[/color]

I think that choice appeals to me more than + and * (which are already
more overloaded than I would like). I haven't seen any suggestions that
I liked better.
[color=blue]
> * Is the support for sets of sets necessary for your work
> and, if so, then is the implementation sufficiently
> powerful?[/color]

Yes I need it (desperately). Generally works as I need. However, see
more comments below.
[color=blue]
> * Is there a compelling need for additional set methods like
> Set.powerset() and Set.isdisjoint( s) or are the current
> offerings sufficient?[/color]

I haven't felt the need yet. So far I've been satisfied with:

if x & y:

as opposed to

if x.isdisjoint(y)
[color=blue]
> * Does the performance meet your expectations?[/color]

So far. However, so far, I haven't been trying meet any demanding
performance requirements.
[color=blue]
> * Do you care that sets can only contain hashable elements?[/color]

This is an interesting question. In particular, I have found myself on
more than one occasion doing the following:

for x in interesting_obj ects:
x.foo = Set()
while something_to_do :
somex.foo |= something_I_jus t_computed
for x in interesting_obj ects:
x.foo = ImmutableSet(x. foo)
build_some_more _sets(somex.foo )

I'm not sure whether I like having to go back and change all my sets to
Immutable ones after I've finished the computation. (Or whether I just
ought to make x.foo immutable all the time.) I did appreciate the
ImmutableSet type, since it allows me to flag to myself that I don't
expect a set to change further.
[color=blue]
> * How about the design constraint that the argument to most
> set methods must be another Set (as opposed to any iterable)?[/color]

In some cases I've run into that. Since I can create a set with any
iterable I've been able to do:

set op Set(iterable)

I think I might be interested in using a general iterable if that would
get me some advantage (maybe significantly faster).
[color=blue]
> * Are the docs clear? Can you suggest improvements?[/color]

No problems here. However, my background is math, and I've never had
problems with documentation (I started my career learning IBM mainframe
assembly language programming from the reference manuals) so I don't
think I'm a good test case.
[color=blue]
> * Are sets helpful in your daily work or does the need arise
> only rarely?[/color]

I'm working on a project where they are critical. If it hadn't been
supplied I would have implemented one myself. I was using the backported
version of the set module with 2.2 before 2.3 came out.
[color=blue]
> User feedback is essential to determining the future direction
> of sets (whether it will be implemented in C, change API,
> and/or be given supporting language syntax).
>
>
> Raymond Hettinger[/color]

Chris

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 80,000 Newsgroups - 16 Different Servers! =-----

**Skip Montanaro** · Jul 18 '05, 01:33 AM

Re: Py2.3: Feedback on Sets

Russell> I suspect the upgrade issue will significantly slow the
Russell> incorporation of sets and the other new modules, but that over
Russell> time they're likely to become quite popular. I am certainly
Russell> looking forward to using sets and csv.

The csv module (and the _csv module which underpins it) should work with
2.2.3. If they don't, please file a bug report.

Russell> I think it'd speed the adoption of new modules if they were
Russell> explicitly written to be compatible with one previous
Russell> generation of Python (and documented as such) so users could
Russell> manually include them with their code until the current
Russell> generation of Python had a bit more time to be adopted.

That was the intention with the csv module. I wonder if some limitations to
use of sets with 2.2.x could be gotten around by adding a __future__ import?
Maybe itertools is also needed.

Russell> I'm not saying this should be a rule, only suggesting it as a
Russell> useful goal. Presumably it'd be easy with some modules and not
Russell> worth the work in some cases.

Yes, that's a worthwhile goal.

Skip

**Andrew Dalke** · Jul 18 '05, 01:34 AM

Re: Py2.3: Feedback on Sets

Gary Feldman:[color=blue]
> Also, I'd like to see "iterable must be <some type spec>",
> though this is a general flaw in the Python doc and is perhaps
> biased by my C/C++ background where you'd never dream
> of doing a reference manual without explicitly indicating the
> types of every parameter.[/color]

Python uses what is sometimes called "duck typing" (meaning,
if it quacks like a duck...). Lots of objects are iterable - strings,
lists, sets, dict (keys), and user-defined classes. Since you
prefer C++, think of Python more akin to templates. Templates
expect the objects templated on to have certain properties (can
be "+"ed, can be deferenced, has a method named "xyz") and
not that they have given types.
[color=blue]
> Personally, I have hard time imagining where I'd want
> [remove]. If I really cared, I could check beforehand, so I think
> I'd just always use discard.[/color]

I'm the other way around. I find it hard to imagine where
I would call discard. If I want to remove an element from a
set then I want to know right away if that element isn't there.
It's been handy for tracking down bugs in my code.
[color=blue]
> 5.12.2
> engineering_man agement = engineers & programmers[/color]

Actually, I don't like that example because there is too
much text to read through to see the actual symbols used.
[color=blue]
> PS I suppose I should mention my strongest pet peeve
> with the Python documentation, which is the practice of
> putting the member functions on a different page than
> the class overview. But that's not your issue, either.[/color]

And I confess that I like to see everything on one page
and not split up between several pages. That way I can
use my browser's search facility.

Andrew
dalke@dalkescie ntific.com

**Raymond Hettinger** · Jul 18 '05, 01:34 AM

Re: Py2.3: Feedback on Sets

"Skip Montanaro" <skip@pobox.com > wrote in message
news:mailman.10 60798102.26105. python-list@python.org ...[color=blue]
>
> Russell> I suspect the upgrade issue will significantly slow the
> Russell> incorporation of sets and the other new modules, but that over
> Russell> time they're likely to become quite popular. I am certainly
> Russell> looking forward to using sets and csv.
>
> The csv module (and the _csv module which underpins it) should work with
> 2.2.3. If they don't, please file a bug report.
>
> Russell> I think it'd speed the adoption of new modules if they were
> Russell> explicitly written to be compatible with one previous
> Russell> generation of Python (and documented as such) so users could
> Russell> manually include them with their code until the current
> Russell> generation of Python had a bit more time to be adopted.
>
> That was the intention with the csv module. I wonder if some limitations to
> use of sets with 2.2.x could be gotten around by adding a __future__ import?
> Maybe itertools is also needed.[/color]

In the documentation for the itertools module, I intensionally included
pure python versions of each tool that make backporting easy. You
can cut and paste the documentation into a module with
from __future__ import generators and have a Py2.2 version of
itertools that would enable the sets module to run just fine.

Still, why not upgrade to Py2.3? The bug fixes were all ported to 2.2.3
and into Py2.3 so that the essential differences are the new modules
and some minor language improvements.

Raymond Hettinger

**Michael Hudson** · Jul 18 '05, 01:34 AM

Re: Py2.3: Feedback on Sets

"Raymond Hettinger" <vze4rx4y@veriz on.net> writes:
[color=blue]
> I've gotten lots of feedback on the itertools module
> but have not heard a peep about the new sets module.
>
> * Are you overjoyed/outraged by the choice of | and &
> as set operators (instead of + and *)?[/color]

I'd actually rather sets didn't overload any operators at all, but
appreciate that this may be a minority position.

| and & is the only sane choice, however.
[color=blue]
> * Is the support for sets of sets necessary for your work
> and, if so, then is the implementation sufficiently
> powerful?[/color]

I don't use them as much as I should, I suspect.
[color=blue]
> * Is there a compelling need for additional set methods like
> Set.powerset() and Set.isdisjoint( s) or are the current
> offerings sufficient?[/color]

I've not reached for something and not found it there yet.
[color=blue]
> * Does the performance meet your expectations?[/color]

My uses so far have not had even the faintest of performance demands,
so, yes.
[color=blue]
> * Do you care that sets can only contain hashable elements?[/color]

Not yet.

Cheers,
mwh

--
The website looks like the Hi-Score sheet from a Bullshit Bingo
tournament. -- Dan Holdsworth, asr

**Bob Gailer** · Jul 18 '05, 01:35 AM

Re: Py2.3: Feedback on Sets

After giving blanket approval to the docs I now add:

I have a mission to set some new guidelines for Python documentation.
Perhaps this is a good place to start.
Example - currently we have:

class Set( [iterable])
Constructs a new empty Set object. If the optional iterable parameter is
supplied, updates the set with elements obtained from iteration. All of the
elements in iterable should be immutable or be transformable to an
immutable using the protocol described in section
<http://www.python.org/doc/current/lib/immutable-transforms.html #immutable-transforms>5.12 .3.

Problems:
The result of Set appears to be an empty Set object. The fact that it might
be filled is hidden in the parameter description.
The parameter description itself is hidden in the paragraph, making it
harder to find, especially when the reader is in a hurry.

Some suggested guidelines to improve readability and understandabili ty:
1 - label each paragraph so we know what it is about
2 - have a function paragraph that briefly but completely describes the
function
3 - have labeled sections for things that can be so grouped (e.g. parameters)
4 - start the description of each thing in a new paragraph.

Example:

class Set( [iterable])
function: Constructs a new empty Set object and optionally fills it.
parameters:
iterable [optional] if supplied, updates the set with elements
obtained from
iteration. All of the elements in iterable should be immutable or be
transformable to an immutable using the protocol described in
section
<http://www.python.org/doc/current/lib/immutable-transforms.html #immutable-transforms>5.12 .3.

What do you think? If this layout is appealing, let's use the set docs as a
starting point to model this approach. I for one am willing to apply this
model to the rest ot the set docs, and help update other docs, but not all
of them.

BTW I also have a problem with the term "Common uses". "Common" suggests
that these are better, or more frequent. I suggest "Some examples of
application of sets".

I also agree with the suggestion that operations that are synonymous be so
indicated in the table.

Bob Gailer
bgailer@alum.rp i.edu
303 442 2625

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.506 / Virus Database: 303 - Release Date: 8/1/2003

**Russell E. Owen** · Jul 18 '05, 01:35 AM

Re: Py2.3: Feedback on Sets

In article <mailman.106079 8102.26105.pyth on-list@python.org >,
Skip Montanaro <skip@pobox.com > wrote:
[color=blue]
> Russell> I suspect the upgrade issue will significantly slow the
> Russell> incorporation of sets and the other new modules, but that over
> Russell> time they're likely to become quite popular. I am certainly
> Russell> looking forward to using sets and csv.
>
>The csv module (and the _csv module which underpins it) should work with
>2.2.3. If they don't, please file a bug report.[/color]

That's excellent news. It might be worth adding it to the documentation,
e.g. "new in version 2.3 but compatible with version 2.2.x" (surely x is
1 (with True/False) or 0 (without), or was there really some needed
feature change in 2.2.3?).
[color=blue]
>That was the intention with the csv module. I wonder if some limitations to
>use of sets with 2.2.x could be gotten around by adding a __future__ import?
>Maybe itertools is also needed.[/color]

That is an interesting question. Mind you, I have no idea if sets is
compatible with 2.2.x or not; I didn't try since it wasn't documented
and I didn't want to risk missing some obscure bug.

-- Russell

**John Baxter** · Jul 18 '05, 01:36 AM

Re: Py2.3: Feedback on Sets

In article <bhe3cr$8n0$1@s lb9.atl.mindspr ing.net>,
"Andrew Dalke" <adalke@mindspr ing.com> wrote:
[color=blue]
> I read some mention of using "|" instead of "+", so I knew
> to use it. I would have liked +, but not *. I know the logic
> for thinking * but & doesn't have the other connotations
> * has (like [1] * 2, "a"*9)
>[color=green]
> > * Is the support for sets of sets necessary for your work
> > and, if so, then is the implementation sufficiently
> > powerful?[/color][/color]

After years of using Python without sets, I hand built a specialized
intersection a couple of months ago. Knowing the Sets module was
coming, I did only what I needed at that moment, and didn't bother
optimizing it (it takes a few seconds to do what I need...removing a
second or two isn't useful). (I worked around a "need" for difference
by changing the input generation in the overall problem.)

So..."necessary " is too strong here, but "a good thing" is certainly
apt. If I only get to choose yes or no for "necessary" the answer is
"yes".

--John

--
Email to above address discarded by provider's server. Don't bother sending.

**Raymond Hettinger** · Jul 18 '05, 01:36 AM

Re: Py2.3: Feedback on Sets

"Istvan Albert"[color=blue]
> One pattern that I constantly need is to remove duplicates from
> a sequence. I don't know if this an often enough used pattern to
> warrant an API change, for me it would be most useful if I could
> get the contents of a set as a sequence right away, without having to
> explicitly code it.[/color]

[color=blue][color=green][color=darkred]
>>> list(Set('abrac adbra'))[/color][/color][/color]
['a', 'r', 'b', 'c', 'd']

[color=blue][color=green]
> > * Are the docs clear? Can you suggest improvements?[/color]
>
> I wondered whether it would be better to specify the immutability
> of the class at the constructor level.[/color]

ImmutableSet is available as a constructor.

[color=blue]
> Then there is the update method. It feels a little bit redundant
> since there is an add() method that seems to be doing the same thing
> only that add() adds only one element at a time.
> Would it be possible to have add() handle all additions, iterable or
> not, then scrap update() altogether.[/color]

Not really.
Set.update() is for vectorizing high volume additions.
There is some analogy to list.append() vs. list.extend().

[color=blue][color=green]
> > Then just by looking at the docs, it feels a little bit confusing to[/color]
> have discard() and remove() do essentially the same thing but only one
> of them raising an exception. Which one? I already forgot. I don't know
> which one I would prefer though.[/color]

Will clarify the docs.

[color=blue]
> Another aspect that I did not understand, what is difference between
> update() and union_update().[/color]

update() works with any iterable and union_update() only with another Set.
If the API is liberized to allow any iterable for most operations, then
the distinction will vanish.

[color=blue]
> The long winded method names, such as difference_upda te() also feel
> redundant when one can achieve the same thing with the -= operator. I
> would drop these and instead show in the docs how to accomplish these
> with the operators. Would considerably cut down on the documentation,
> and apparent complexity.[/color]

That is a good thought; however,
some find a.union(b) to be more readable than a|b
and some find that a.symmetric_dif ference is more memorable than a^b.

[color=blue]
> For example methods like x.issubset(y) is the same as bool(x-y) so may
> not be all that necessary, just a thought.[/color]

Granted. However:

* issubset has an early out algorithm and consumes contant memory.
In contrast, bool(x-y) builds a whole new set and then throws it away.
* issubset and issuperset are somewhat basic set operations

[color=blue][color=green]
> > * Are sets helpful in your daily work or does the need arise
> > only rarely?[/color]
>
> I use them very often and they are extremely useful.[/color]

Me too.

Raymond Hettinger

Py2.3: Feedback on Sets

Py2.3: Feedback on Sets

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment