I thought it might be useful to put the recent lambda threads into
perspective a bit. I was wondering what lambda gets used for in "real"
code, so I grepped my Python Lib directory. Here are some of the ones I
looked, classified by how I would rewrite them (if I could):
* Rewritable as def statements (<name> = lambda <args>: <expr> usage)
These are lambdas used when a lambda wasn't needed -- an anonymous
function was created with lambda and then immediately bound to a name.
Since this is essentially what def does, using lambdas here is (IMHO) silly.
pickletools.py: getpos = lambda: None
def getpos(): return None
tarfile.py: normpath = lambda path:
os.path.normpat h(path).replace (os.sep, "/")
def normpath(path): os.path.normpat h(path).replace (os.sep, "/")
urllib2.py: H = lambda x: md5.new(x).hexd igest()
def H(x): md5.new(x).hexd igest()
urllib2.py: H = lambda x: sha.new(x).hexd igest()
def H(x): sha.new(x).hexd igest()
* Rewritable with existing functions
Mainly these are examples of code that can benefit from using the
functions available in the operator module, especially
operator.itemge tter and operator.attrge tter (available in 2.4)
cgi.py: return map(lambda v: v.value, value)
return map(operator.at trgetter('value '), value)
CGIHTTPServer.p y: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()) )
nobody = 1 + max(map(operato r.itemgetter(2) , pwd.getpwall()) )
SimpleXMLRPCSer ver.py: server.register _function(lambd a x,y: x+y, 'add')
server.register _function(opera tor.add, 'add')
SimpleXMLRPCSer ver.py: server.register _function(lambd a x,y: x+y, 'add')
server.register _function(opera tor.add, 'add')
sre_constants.p y: items.sort(key= lambda a: a[1])
items.sort(key= operator.itemge tter(1))
tarfile.py: return map(lambda m: m.name, self.infolist() )
return map(operator.at trgetter('name' ), self.infolist() )
* Rewritable with list comprehensions/generator expressions
Lambdas in map or filter expressions can often be replaced by an
appropriate list comprehension or generator expression (in Python 2.3/2.4)
cgi.py: plist = map(lambda x: x.strip(), line.split(';') )
plist = [x.strip() for x in line.split(';')
cgi.py: return map(lambda v: v.value, value)
return [v.value for v in value]
CGIHTTPServer.p y: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()) )
nobody = 1 + max(x[2] for x in pwd.getpwall())
glob.py: names=filter(la mbda x: x[0]!='.',names)
names=[x for x in names if x[0] != '.']
hmac.py: return "".join(map(lam bda x, y: chr(ord(x) ^ ord(y)),
s1, s2))
return "".join(chr(ord (x) ^ ord(y)) for x, y in zip(s1, s2))
imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and
'" "'.join(x[1]) or ''), l)
l = ['%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '')
for x in l]
inspect.py: suffixes = map(lambda (suffix, mode, mtype):
(-len(suffix), suffix, mode, mtype),
imp.get_suffixe s())
suffixes = [(-len(suffix), suffix, mode, mtype)
for suffix, mode, mtype in imp.get_suffixe s()
inspect.py: return join(map(lambda o, c=convert, j=join:
strseq(o, c, j), object))
return join([strseq(o, convert, join) for o in object])
mailcap.py: entries = filter(lambda e,key=key: key in e, entries)
entries = [e for e in entries if key in e]
poplib.py: digest = ''.join(map(lam bda x:'%02x'%ord(x) , digest))
digest = ''.join('%02x' % ord(x) for x in digest)
pstats.py: if line and not filter(lambda x,a=abbrevs:
x not in a,line.split()) :
if line and not [x for x in line.split() if x not in abbrevs]:
tabnanny.py: firsts = map(lambda tup: str(tup[0]), w)
firsts = [str(tup[0]) for tup in w]
tarfile.py: return map(lambda m: m.name, self.infolist() )
return [m.name for m in self.infolist()]
tarfile.py: return filter(lambda m: m.type in REGULAR_TYPES,
self.tarfile.ge tmembers())
return [m for m in self.tarfile.ge tmembers()
if m.type in REGULAR_TYPES]
urllib2.py: return map(lambda x: x.strip(), list)
return [x.strip() for x in list]
webbrowser.py: _tryorder = filter(lambda x: x.lower() in _browsers
or x.find("%s") > -1, _tryorder
_tryorder = [x for x in _tryorder
if x.lower() in _browsers or x.find("%s") > -1]
* Functions I don't know how to rewrite
Some functions I looked at, I couldn't figure out a way to rewrite them
without introducing a new name or adding new statements. (Warning: I
have trouble following code that uses 'reduce', so I only glossed over
lambdas in reduce calls.)
calendar.py: _months.insert( 0, lambda x: "")
cgitb.py: inspect.formata rgvalues(args, varargs, varkw, locals,
formatvalue=lam bda value: '=' + pydoc.html.repr (value))
cgitb.py: inspect.formata rgvalues(args, varargs, varkw, locals,
formatvalue=lam bda value: '=' + pydoc.text.repr (value))
csv.py: quotechar = reduce(lambda a, b, quotes = quotes:
(quotes[a] > quotes[b]) and a or b, quotes.keys())
csv.py: delim = reduce(lambda a, b, delims = delims:
(delims[a] > delims[b]) and a or b, delims.keys())
difflib.py: matches = reduce(lambda sum, triple: sum + triple[-1],
self.get_matchi ng_blocks(), 0)
gettext.py: return eval('lambda n: int(%s)' % plural)
gettext.py: self.plural = lambda n: int(n != 1)
inspect.py: classes.sort(ke y=lambda c: (c.__module__, c.__name__))
inspect.py: def formatargspec(a rgs, varargs=None, varkw=None,
...
formatvarargs=l ambda name: '*' + name,
formatvarkw=lam bda name: '**' + name,
formatvalue=lam bda value: '=' + repr(value),
inspect.py: def formatargvalues (args, varargs, varkw, locals,
...
formatvarargs=l ambda name: '*' + name,
formatvarkw=lam bda name: '**' + name,
formatvalue=lam bda value: '=' + repr(value),
pyclbr.py: objs.sort(lambd a a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
SimpleHTTPServe r.py: list.sort(key=l ambda a: a.lower())
subprocess.py: p = Popen(["id"], preexec_fn=lamb da: os.setuid(100))
symtable.py: self.__params = self.__idents_m atching(lambda x:
x & DEF_PARAM)
symtable.py: self.__locals = self.__idents_m atching(lambda x:
x & DEF_BOUND)
symtable.py: self.__globals = self.__idents_m atching(lambda x:
x & glob)
urllib2.py:seta ttr(self, '%s_open' % type,
lambda r, proxy=url, type=type, meth=self.proxy _open:
meth(r, proxy, type))
xdrlib.py: unpacktest = [
(up.unpack_uint , (), lambda x: x == 9),
(up.unpack_bool , (), lambda x: not x),
(up.unpack_bool , (), lambda x: x),
(up.unpack_uhyp er, (), lambda x: x == 45L),
(up.unpack_floa t, (), lambda x: 1.89 < x < 1.91),
(up.unpack_doub le, (), lambda x: 1.89 < x < 1.91),
(up.unpack_stri ng, (), lambda x: x == 'hello world'),
(up.unpack_list , (up.unpack_uint ,), lambda x: x == range(5)),
(up.unpack_arra y, (up.unpack_stri ng,),
lambda x: x == ['what', 'is', 'hapnin', 'doctor']),
]
Of the functions that I don't know how to rewrite, I think there are a
few interesting cases:
(1) lambda x: ""
This is the kind of parameter adaptation that I think Jeff Shannon was
talking about in another lambda thread. Using the ignoreargs function I
suggested there[1], you could rewrite this as:
ignoreargs(str, 1)
(2) lambda a: a.lower()
My first thought here was to use str.lower instead of the lambda, but of
course that doesn't work if 'a' is a unicode object:
py> str.lower(u'a')
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: descriptor 'lower' requires a 'str' object but received a
'unicode'
It's too bad I can't do something like:
basestring.lowe r
(3) self.plural = lambda n: int(n != 1)
Note that this is *almost* writable with def syntax. If only we could do:
def self.plural(n):
int(n != 1)
(4) objs.sort(lambd a a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
My first intuition here was to try something like:
objs.sort(key=o perator.attrget ter('lineno'))
but this doesn't work because then we don't get the default value of 0
if the attribute doesn't exist. I wonder if operator.attrge tter should
take an optional "default" parameter like getattr does:
Help on built-in function getattr in module __builtin__:
getattr(...)
getattr(object, name[, default]) -> value
(5) lambda x: x & DEF_PARAM
This could probably be written as:
functional.part ial(operator.an d_, DEF_PARAM)
if PEP 309[2] was accepted, thought I'm not claiming that's any clearer...
So, those are my thoughts on how lambdas are "really" used. If others
out there have real-life code that uses lambdas in interesting ways,
feel free to share them here!
Steve
[1]http://mail.python.org/pipermail/python-list/2004-December/257982.html
[2]http://python.fyxm.net/peps/pep-0309.html
perspective a bit. I was wondering what lambda gets used for in "real"
code, so I grepped my Python Lib directory. Here are some of the ones I
looked, classified by how I would rewrite them (if I could):
* Rewritable as def statements (<name> = lambda <args>: <expr> usage)
These are lambdas used when a lambda wasn't needed -- an anonymous
function was created with lambda and then immediately bound to a name.
Since this is essentially what def does, using lambdas here is (IMHO) silly.
pickletools.py: getpos = lambda: None
def getpos(): return None
tarfile.py: normpath = lambda path:
os.path.normpat h(path).replace (os.sep, "/")
def normpath(path): os.path.normpat h(path).replace (os.sep, "/")
urllib2.py: H = lambda x: md5.new(x).hexd igest()
def H(x): md5.new(x).hexd igest()
urllib2.py: H = lambda x: sha.new(x).hexd igest()
def H(x): sha.new(x).hexd igest()
* Rewritable with existing functions
Mainly these are examples of code that can benefit from using the
functions available in the operator module, especially
operator.itemge tter and operator.attrge tter (available in 2.4)
cgi.py: return map(lambda v: v.value, value)
return map(operator.at trgetter('value '), value)
CGIHTTPServer.p y: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()) )
nobody = 1 + max(map(operato r.itemgetter(2) , pwd.getpwall()) )
SimpleXMLRPCSer ver.py: server.register _function(lambd a x,y: x+y, 'add')
server.register _function(opera tor.add, 'add')
SimpleXMLRPCSer ver.py: server.register _function(lambd a x,y: x+y, 'add')
server.register _function(opera tor.add, 'add')
sre_constants.p y: items.sort(key= lambda a: a[1])
items.sort(key= operator.itemge tter(1))
tarfile.py: return map(lambda m: m.name, self.infolist() )
return map(operator.at trgetter('name' ), self.infolist() )
* Rewritable with list comprehensions/generator expressions
Lambdas in map or filter expressions can often be replaced by an
appropriate list comprehension or generator expression (in Python 2.3/2.4)
cgi.py: plist = map(lambda x: x.strip(), line.split(';') )
plist = [x.strip() for x in line.split(';')
cgi.py: return map(lambda v: v.value, value)
return [v.value for v in value]
CGIHTTPServer.p y: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()) )
nobody = 1 + max(x[2] for x in pwd.getpwall())
glob.py: names=filter(la mbda x: x[0]!='.',names)
names=[x for x in names if x[0] != '.']
hmac.py: return "".join(map(lam bda x, y: chr(ord(x) ^ ord(y)),
s1, s2))
return "".join(chr(ord (x) ^ ord(y)) for x, y in zip(s1, s2))
imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and
'" "'.join(x[1]) or ''), l)
l = ['%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '')
for x in l]
inspect.py: suffixes = map(lambda (suffix, mode, mtype):
(-len(suffix), suffix, mode, mtype),
imp.get_suffixe s())
suffixes = [(-len(suffix), suffix, mode, mtype)
for suffix, mode, mtype in imp.get_suffixe s()
inspect.py: return join(map(lambda o, c=convert, j=join:
strseq(o, c, j), object))
return join([strseq(o, convert, join) for o in object])
mailcap.py: entries = filter(lambda e,key=key: key in e, entries)
entries = [e for e in entries if key in e]
poplib.py: digest = ''.join(map(lam bda x:'%02x'%ord(x) , digest))
digest = ''.join('%02x' % ord(x) for x in digest)
pstats.py: if line and not filter(lambda x,a=abbrevs:
x not in a,line.split()) :
if line and not [x for x in line.split() if x not in abbrevs]:
tabnanny.py: firsts = map(lambda tup: str(tup[0]), w)
firsts = [str(tup[0]) for tup in w]
tarfile.py: return map(lambda m: m.name, self.infolist() )
return [m.name for m in self.infolist()]
tarfile.py: return filter(lambda m: m.type in REGULAR_TYPES,
self.tarfile.ge tmembers())
return [m for m in self.tarfile.ge tmembers()
if m.type in REGULAR_TYPES]
urllib2.py: return map(lambda x: x.strip(), list)
return [x.strip() for x in list]
webbrowser.py: _tryorder = filter(lambda x: x.lower() in _browsers
or x.find("%s") > -1, _tryorder
_tryorder = [x for x in _tryorder
if x.lower() in _browsers or x.find("%s") > -1]
* Functions I don't know how to rewrite
Some functions I looked at, I couldn't figure out a way to rewrite them
without introducing a new name or adding new statements. (Warning: I
have trouble following code that uses 'reduce', so I only glossed over
lambdas in reduce calls.)
calendar.py: _months.insert( 0, lambda x: "")
cgitb.py: inspect.formata rgvalues(args, varargs, varkw, locals,
formatvalue=lam bda value: '=' + pydoc.html.repr (value))
cgitb.py: inspect.formata rgvalues(args, varargs, varkw, locals,
formatvalue=lam bda value: '=' + pydoc.text.repr (value))
csv.py: quotechar = reduce(lambda a, b, quotes = quotes:
(quotes[a] > quotes[b]) and a or b, quotes.keys())
csv.py: delim = reduce(lambda a, b, delims = delims:
(delims[a] > delims[b]) and a or b, delims.keys())
difflib.py: matches = reduce(lambda sum, triple: sum + triple[-1],
self.get_matchi ng_blocks(), 0)
gettext.py: return eval('lambda n: int(%s)' % plural)
gettext.py: self.plural = lambda n: int(n != 1)
inspect.py: classes.sort(ke y=lambda c: (c.__module__, c.__name__))
inspect.py: def formatargspec(a rgs, varargs=None, varkw=None,
...
formatvarargs=l ambda name: '*' + name,
formatvarkw=lam bda name: '**' + name,
formatvalue=lam bda value: '=' + repr(value),
inspect.py: def formatargvalues (args, varargs, varkw, locals,
...
formatvarargs=l ambda name: '*' + name,
formatvarkw=lam bda name: '**' + name,
formatvalue=lam bda value: '=' + repr(value),
pyclbr.py: objs.sort(lambd a a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
SimpleHTTPServe r.py: list.sort(key=l ambda a: a.lower())
subprocess.py: p = Popen(["id"], preexec_fn=lamb da: os.setuid(100))
symtable.py: self.__params = self.__idents_m atching(lambda x:
x & DEF_PARAM)
symtable.py: self.__locals = self.__idents_m atching(lambda x:
x & DEF_BOUND)
symtable.py: self.__globals = self.__idents_m atching(lambda x:
x & glob)
urllib2.py:seta ttr(self, '%s_open' % type,
lambda r, proxy=url, type=type, meth=self.proxy _open:
meth(r, proxy, type))
xdrlib.py: unpacktest = [
(up.unpack_uint , (), lambda x: x == 9),
(up.unpack_bool , (), lambda x: not x),
(up.unpack_bool , (), lambda x: x),
(up.unpack_uhyp er, (), lambda x: x == 45L),
(up.unpack_floa t, (), lambda x: 1.89 < x < 1.91),
(up.unpack_doub le, (), lambda x: 1.89 < x < 1.91),
(up.unpack_stri ng, (), lambda x: x == 'hello world'),
(up.unpack_list , (up.unpack_uint ,), lambda x: x == range(5)),
(up.unpack_arra y, (up.unpack_stri ng,),
lambda x: x == ['what', 'is', 'hapnin', 'doctor']),
]
Of the functions that I don't know how to rewrite, I think there are a
few interesting cases:
(1) lambda x: ""
This is the kind of parameter adaptation that I think Jeff Shannon was
talking about in another lambda thread. Using the ignoreargs function I
suggested there[1], you could rewrite this as:
ignoreargs(str, 1)
(2) lambda a: a.lower()
My first thought here was to use str.lower instead of the lambda, but of
course that doesn't work if 'a' is a unicode object:
py> str.lower(u'a')
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
TypeError: descriptor 'lower' requires a 'str' object but received a
'unicode'
It's too bad I can't do something like:
basestring.lowe r
(3) self.plural = lambda n: int(n != 1)
Note that this is *almost* writable with def syntax. If only we could do:
def self.plural(n):
int(n != 1)
(4) objs.sort(lambd a a, b: cmp(getattr(a, 'lineno', 0),
getattr(b, 'lineno', 0)))
My first intuition here was to try something like:
objs.sort(key=o perator.attrget ter('lineno'))
but this doesn't work because then we don't get the default value of 0
if the attribute doesn't exist. I wonder if operator.attrge tter should
take an optional "default" parameter like getattr does:
Help on built-in function getattr in module __builtin__:
getattr(...)
getattr(object, name[, default]) -> value
(5) lambda x: x & DEF_PARAM
This could probably be written as:
functional.part ial(operator.an d_, DEF_PARAM)
if PEP 309[2] was accepted, thought I'm not claiming that's any clearer...
So, those are my thoughts on how lambdas are "really" used. If others
out there have real-life code that uses lambdas in interesting ways,
feel free to share them here!
Steve
[1]http://mail.python.org/pipermail/python-list/2004-December/257982.html
[2]http://python.fyxm.net/peps/pep-0309.html
Comment