invert dictionary with list &c

[Des Small]

Lately I have found myself using a pattern to make new dictionaries
quite often, by which I mean twice:

def invert(d):
nd = {}
[nd.setdefault(v al, []).append(key) for k, v in d]
return nd

def count(l):
d = {}
[d.setdefault(w, 0) += 1 for w in l]
return d

Is this the pythonic way to do such things? Ideally I'd like to write
them as one liners, but I can't see how.

Des
--
"[T]he structural trend in linguistics which took root with the
International Congresses of the twenties and early thirties [...] had
close and effective connections with phenomenology in its Husserlian
and Hegelian versions." -- Roman Jakobson

[anton muhin]

Re: invert dictionary with list &c

Des Small wrote:
[color=blue]
> Lately I have found myself using a pattern to make new dictionaries
> quite often, by which I mean twice:
>
> def invert(d):
> nd = {}
> [nd.setdefault(v al, []).append(key) for k, v in d]
> return nd
>
> def count(l):
> d = {}
> [d.setdefault(w, 0) += 1 for w in l]
> return d
>
> Is this the pythonic way to do such things? Ideally I'd like to write
> them as one liners, but I can't see how.
>
> Des[/color]

Most pythonic way IMHO would be:

def invert(d):
nd = {}
for k, v in d.iteritems():
nd[v] = nd.get(k, []) + [k]
return nd

def count(l):
d = {}
for e in l:
d[e] = d.get(e, 0) + 1
return d

Or to define dict with default values:

import copy

class defdict(dict):
def __init__(self, default = None):
self._default = default
super(dict, self).__init__( self)

def __getitem__(sel f, k):
return self.get(k, copy.deepcopy(s elf._default)) # or setdefault

when

def invert(d):
nd = defdict([])
for k, v in d.iteritems():
nd[v] += [k]
return nd

def count(l):
d = defdict(0)
for e in l:
d[e] += 1
return d

However, if you insist on one-liners, I can suggest some ;):

def count(l):
return reduce(
lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
l, {}
)

def invert(d):
return reduce(
lambda d, (k, v): (d.update(dict([(v, d.get(v, []) + [k])])), d)[1],
d.iteritems(), {}
)

(given in several lines, but can be written in one)

or even

count = lambda l: reduce(
lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
l, {}
)

:)

regards,
anton.

[Mel Wilson]

Re: invert dictionary with list &c

In article <yyrjbrqz3ql5.f sf@pc156.maths. bris.ac.uk>,
Des Small <des.small@bris tol.ac.uk> wrote:[color=blue]
>Lately I have found myself using a pattern to make new dictionaries
>quite often, by which I mean twice:
>
>def invert(d):
> nd = {}
> [nd.setdefault(v al, []).append(key) for k, v in d]
> return nd
>
>def count(l):
> d = {}
> [d.setdefault(w, 0) += 1 for w in l]
> return d
>
>Is this the pythonic way to do such things? Ideally I'd like to write
>them as one liners, but I can't see how.[/color]

Once you've written the above, they are one-liners:

inverted_dict = invert (some_dict)

Why would you need less? There's little doubt in any
reader's mind that inverting a dictionary is really what
you're doing; there's little chance of a typo sneaking in
and messing up the process without anybody noticing, etc.

Regards. Mel.

[John Hunter]

Re: invert dictionary with list &amp;c

>>>>> "Des" == Des Small <des.small@bris tol.ac.uk> writes:

def count(l):
d = {}
[d.setdefault(w, 0) += 1 for w in l]
return d

This code raises a SyntaxError.

The standard idiom for counting all the elements in a list with a
dictionary is

def count(l):
d = {}
for w in l: d[w] = d.get(w,0) + 1
return d

I think the basic rule of thumb is to use the setdefault approach for
mutable objects (list, dict) and the get approach for immutable
elements (strings, ints, floats, tuples).

Also, many here would find using list comprehensions while ignoring
their return value, as you did in '[d.setdefault(w, 0) += 1 for w in
l]' to be an abuse of list comps. At least I've been admonished for
doing so, and so I must now admonish you to continue the cycle of
violence.

Des> Is this the pythonic way to do such things? Ideally I'd like
Des> to write them as one liners, but I can't see how.

I think that striving for one-liners is not pythonic. Most python
coders value readability over compactness. It's more important to
write an appropriately named function that works and is readable and
efficient than it is to write a one liner.

John Hunter

[Des Small]

Re: invert dictionary with list &amp;c

John Hunter <jdhunter@ace.b sd.uchicago.edu > writes:
[color=blue][color=green][color=darkred]
> >>>>> "Des" == Des Small <des.small@bris tol.ac.uk> writes:[/color][/color]
>
> def count(l):
> d = {}
> [d.setdefault(w, 0) += 1 for w in l]
> return d
>
> This code raises a SyntaxError.[/color]

So it does. Whatever it was I had working yesterday wasn't that,
then.
[color=blue]
> The standard idiom for counting all the elements in a list with a
> dictionary is
>
> def count(l):
> d = {}
> for w in l: d[w] = d.get(w,0) + 1
> return d
>
> I think the basic rule of thumb is to use the setdefault approach for
> mutable objects (list, dict) and the get approach for immutable
> elements (strings, ints, floats, tuples).[/color]

What I really want is a generic pattern that constructs dictionary
from iterators in a clean way, while allowing mutation of the
dictionary entry as necessary.
[color=blue]
> Also, many here would find using list comprehensions while ignoring
> their return value, as you did in '[d.setdefault(w, 0) += 1 for w in
> l]' to be an abuse of list comps. At least I've been admonished for
> doing so, and so I must now admonish you to continue the cycle of
> violence.[/color]

I shall be breaking the cycle by not caring, although if and when
generator comprehensions allow me to comply with your preferences I'll
probably do so.

[...]

Des
--
"[T]he structural trend in linguistics which took root with the
International Congresses of the twenties and early thirties [...] had
close and effective connections with phenomenology in its Husserlian
and Hegelian versions." -- Roman Jakobson

[Des Small]

Re: invert dictionary with list &amp;c

anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:
[color=blue]
> Des Small wrote:
>[color=green]
> > Lately I have found myself using a pattern to make new dictionaries
> > quite often, by which I mean twice:
> > def invert(d):
> > nd = {}
> > [nd.setdefault(v al, []).append(key) for k, v in d]
> > return nd
> > def count(l):
> > d = {}
> > [d.setdefault(w, 0) += 1 for w in l]
> > return d[/color][/color]

As another poster noted, this latter is broken.
[color=blue][color=green]
> > Is this the pythonic way to do such things? Ideally I'd like to
> > write
> > them as one liners, but I can't see how.
> > Des[/color]
>
> Most pythonic way IMHO would be:
>
> def invert(d):
> nd = {}
> for k, v in d.iteritems():
> nd[v] = nd.get(k, []) + [k]
> return nd
>
> def count(l):
> d = {}
> for e in l:
> d[e] = d.get(e, 0) + 1
> return d[/color]

I was anxious to extract the common pattern, if possible, but these
are clean enough that it seems slightly perverse.

[...]
[color=blue]
> However, if you insist on one-liners, I can suggest some ;):
>
> def count(l):
> return reduce(
> lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
> l, {}
> )
>
> def invert(d):
> return reduce(
> lambda d, (k, v): (d.update(dict([(v, d.get(v, []) + [k])])), d)[1],
> d.iteritems(), {}
> )
>
> (given in several lines, but can be written in one)[/color]

This, however, has given me ideas. It was never concision I wanted
but rather redundancy elimination, and the pattern I wanted _can_ be
written:

def dict_cons(iter, func, default):
def process_element (d, (k, v)):
val = d.get(k, default)
d.update(dict([[k, func(v, val)]]))
return d
return reduce(process_ element, iter, {})

Which is not to say that it should be, of course.
Whereupon, we can say:

def count(l):
def pair_pad(l): return [(e, ()) for e in l]
return dict_cons(pair_ pad(l), lambda k,d: d+1, 0)

def invert(d):
def invertitems(l): for k,v in l: yield v,k
def addtolist(k, l): return l+[k]
return dict_cons(inver titems(d.iterit ems()),
addtolist, [])

Perhaps I'm terminally unpythonic, but I quite like these.
[color=blue]
> count = lambda l: reduce(
> lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
> l, {}
> )
>
> :)[/color]

If Python's lambda wasn't so broken I'd use it all the time, for sure.

Des
--
"[T]he structural trend in linguistics which took root with the
International Congresses of the twenties and early thirties [...] had
close and effective connections with phenomenology in its Husserlian
and Hegelian versions." -- Roman Jakobson

[Des Small]

Re: invert dictionary with list &amp;c

Des Small <des.small@bris tol.ac.uk> writes:
[color=blue]
> anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:[/color]
[...]
[color=blue]
> This, however, has given me ideas. It was never concision I wanted
> but rather redundancy elimination, and the pattern I wanted _can_ be
> written:
>
> def dict_cons(iter, func, default):
> def process_element (d, (k, v)):
> val = d.get(k, default)
> d.update(dict([[k, func(v, val)]]))
> return d
> return reduce(process_ element, iter, {})[/color]

Or rather:

def dict_cons(iter, func, default):
def process_element (d, (k, v)):
d[k] = func(v, d.get(k, default))
return d
return reduce(process_ element, iter, {})

def count(l):
def pair_pad(l): return [(e, ()) for e in l]
return dict_cons(pair_ pad(l), lambda k,d: d+1, 0)

def invert(d):
def invertitems(l):
for k,v in l: yield v,k
def addtolist(k, l): return l+[k]
return dict_cons(inver titems(d.iterit ems()),
addtolist, [])
[color=blue]
> Which is not to say that it should be, of course.
> Whereupon, we can say:
>
> def count(l):
> def pair_pad(l): return [(e, ()) for e in l]
> return dict_cons(pair_ pad(l), lambda k,d: d+1, 0)
>
> def invert(d):
> def invertitems(l): for k,v in l: yield v,k
> def addtolist(k, l): return l+[k]
> return dict_cons(inver titems(d.iterit ems()),
> addtolist, [])
>
> Perhaps I'm terminally unpythonic, but I quite like these.[/color]

[...]

--
"[T]he structural trend in linguistics which took root with the
International Congresses of the twenties and early thirties [...] had
close and effective connections with phenomenology in its Husserlian
and Hegelian versions." -- Roman Jakobson

[anton muhin]

Re: invert dictionary with list &amp;c

Des Small wrote:[color=blue]
> Des Small <des.small@bris tol.ac.uk> writes:
>
>[color=green]
>>anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:[/color]
>
> [...]
>
>[color=green]
>>This, however, has given me ideas. It was never concision I wanted
>>but rather redundancy elimination, and the pattern I wanted _can_ be
>>written:
>>
>>def dict_cons(iter, func, default):
>> def process_element (d, (k, v)):
>> val = d.get(k, default)
>> d.update(dict([[k, func(v, val)]]))
>> return d
>> return reduce(process_ element, iter, {})[/color]
>
>
> Or rather:
>
> def dict_cons(iter, func, default):
> def process_element (d, (k, v)):
> d[k] = func(v, d.get(k, default))
> return d
> return reduce(process_ element, iter, {})
>
> def count(l):
> def pair_pad(l): return [(e, ()) for e in l]
> return dict_cons(pair_ pad(l), lambda k,d: d+1, 0)
>
> def invert(d):
> def invertitems(l):
> for k,v in l: yield v,k
> def addtolist(k, l): return l+[k]
> return dict_cons(inver titems(d.iterit ems()),
> addtolist, [])
>
>[color=green]
>>Which is not to say that it should be, of course.
>>Whereupon, we can say:
>>
>>def count(l):
>> def pair_pad(l): return [(e, ()) for e in l]
>> return dict_cons(pair_ pad(l), lambda k,d: d+1, 0)
>>
>>def invert(d):
>> def invertitems(l): for k,v in l: yield v,k
>> def addtolist(k, l): return l+[k]
>> return dict_cons(inver titems(d.iterit ems()),
>> addtolist, [])
>>
>>Perhaps I'm terminally unpythonic, but I quite like these.[/color]
>
>
> [...]
>[/color]

Or like this:

def dict_update(ite r, func, default, d):
def process_element (d, e):
d[e[0]] = func(d.get(e[0], default), *e[1:])
return d

return reduce(process_ element, iter, d)

def count(l):
return dict_update(l, lambda x: x + 1, 0, {})

def invert(d):
return dict_update(
[(v, k) for k, v in d.iteritems()],
# In the future (I hope): ((v, k) for k, v in d.iteritems()),
lambda l, e: l + [e], [], {}
)

print count(list('aab bbbcc'))

print invert({'A': 'a', 'B': 'b', 'C': 'a'})

regards,
anton.

[Des Small]

Re: invert dictionary with list &amp;c

anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:
[color=blue]
> Des Small wrote:[color=green]
> > Des Small <des.small@bris tol.ac.uk> writes:
> >[color=darkred]
> >>anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:[/color]
> > [...][/color]
>
> Or like this:
>
> def dict_update(ite r, func, default, d):
> def process_element (d, e):
> d[e[0]] = func(d.get(e[0], default), *e[1:])
> return d
>
> return reduce(process_ element, iter, d)
>
> def count(l):
> return dict_update(l, lambda x: x + 1, 0, {})[/color]

With these I get:
[color=blue][color=green][color=darkred]
>>> count(["yes", "yes", "no"])[/color][/color][/color]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in count
File "<stdin>", line 5, in dict_update
File "<stdin>", line 3, in process_element
TypeError: <lambda>() takes exactly 1 argument (3 given)

I tried to write something that would work on more generic arguments,
but I couldn't do it without explicit type checking, so I gave up.

[...]

Des
has now embalmed versions in his utils collection.

--
"[T]he structural trend in linguistics which took root with the
International Congresses of the twenties and early thirties [...] had
close and effective connections with phenomenology in its Husserlian
and Hegelian versions." -- Roman Jakobson

[anton muhin]

Re: invert dictionary with list &amp;c

Des Small wrote:
[color=blue]
> anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:
>
>[color=green]
>>Des Small wrote:
>>[color=darkred]
>>>Des Small <des.small@bris tol.ac.uk> writes:
>>>
>>>
>>>>anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> writes:
>>>
>>>[...][/color]
>>
>>Or like this:
>>
>>def dict_update(ite r, func, default, d):
>> def process_element (d, e):
>> d[e[0]] = func(d.get(e[0], default), *e[1:])
>> return d
>>
>> return reduce(process_ element, iter, d)
>>
>>def count(l):
>> return dict_update(l, lambda x: x + 1, 0, {})[/color]
>
>
> With these I get:
>
>[color=green][color=darkred]
>>>>count(["yes", "yes", "no"])[/color][/color]
>
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> File "<stdin>", line 2, in count
> File "<stdin>", line 5, in dict_update
> File "<stdin>", line 3, in process_element
> TypeError: <lambda>() takes exactly 1 argument (3 given)
>
> I tried to write something that would work on more generic arguments,
> but I couldn't do it without explicit type checking, so I gave up.
>
> [...]
>
> Des
> has now embalmed versions in his utils collection.
>[/color]

My fault :( A small patch:

def count(l):
return dict_update([(e,) for e in l], lambda x: x + 1, 0, {})

now

print count(['yes', 'yes', 'no'])

print count('aabbbc')

prints

{'yes': 2, 'no': 1}
{'a': 2, 'c': 1, 'b': 3}

regards,
anton.

[Anton Vredegoor]

Re: invert dictionary with list &amp;c

anton muhin <antonmuhin.REM OVE.ME.FOR.REAL .MAIL@rambler.r u> wrote:
[color=blue]
>However, if you insist on one-liners, I can suggest some ;):
>
>def count(l):
> return reduce(
> lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
> l, {}
> )
>
>def invert(d):
> return reduce(
> lambda d, (k, v): (d.update(dict([(v, d.get(v, []) + [k])])), d)[1],
> d.iteritems(), {}
> )
>
>(given in several lines, but can be written in one)
>
>or even
>
>count = lambda l: reduce(
> lambda d, e: (d.update(dict([(e, d.get(e, 0) + 1)])), d)[1],
> l, {}
>)
>
>:)[/color]

Is big bot not watching? Then:

def invert(D):
return reduce(lambda d,(k,v):
d.setdefault(v,[]).append(k) or d,D.iteritems() ,{})

def count(L):
return reduce(lambda d,v:
d.__setitem__(v ,d.get(v,0)+1) or d,L,{})

def test():
print count('aabbbbcc ')
print invert({'A': 'a', 'B': 'b', 'C': 'a'})
print count("yes yes no".split())

if __name__=='__ma in__':
test()

"""
{'a': 2, 'c': 2, 'b': 4}
{'a': ['A', 'C'], 'b': ['B']}
{'yes': 2, 'no': 1} """

Anton