Storing objects required by functions.

[David M. Wilson]

Further to my last post here, I was playing some more with building a
regex object and storing it somewhere for use internally by a
function. I'm not happy with any of my solutions:


# I don't like this, but the fact that you can modify the procedure's
# function via a named argument seems neat in a hacky sort of way.

def uses_default_pa rm_yuck(x, r = re.compile("... ")):
pass


g = re.compile('... ')

def uses_global_yuc k(x):
global g
pass


# This is horrible and probably slow.

class is_hex:
def __init__(self):
self.r = re.compile('... ')

def __call__(self, x):
r = self.r
pass

is_hex = is_hex()


# This mucks up scoping so that your procedure can't access it's
# parent scope like it could normally. Since I never do this,
# it's my favourite.

def is_hex():
r = re.compile('... ')
def is_hex(s):
return r.match(s) is not None
return is_hex

is_hex = is_hex()



Am I missing something? Is there a nicer way of doing this? On a day
to day basis I find myself in this situation quite regularly, and now
I come to think of it today, it is something that I would like to
improve.

It is funny that in private it has never bothered me much, but when
posting on comp.lang.pytho n I find that the code is unacceptable.
Maybe I have two modes of programming, idealist and practical? *shrug*
:)


David.

[John Roth]

Re: Storing objects required by functions.


"David M. Wilson" <dw-google.com@bota nicus.net> wrote in message
news:99dce321.0 312300655.14c5a 8db@posting.goo gle.com...[color=blue]
> Further to my last post here, I was playing some more with building a
> regex object and storing it somewhere for use internally by a
> function. I'm not happy with any of my solutions:
>
>
> # I don't like this, but the fact that you can modify the procedure's
> # function via a named argument seems neat in a hacky sort of way.
>
> def uses_default_pa rm_yuck(x, r = re.compile("... ")):
> pass
>
>
> g = re.compile('... ')
>
> def uses_global_yuc k(x):
> global g
> pass
>
>
> # This is horrible and probably slow.
>
> class is_hex:
> def __init__(self):
> self.r = re.compile('... ')
>
> def __call__(self, x):
> r = self.r
> pass
>
> is_hex = is_hex()
>
>
> # This mucks up scoping so that your procedure can't access it's
> # parent scope like it could normally. Since I never do this,
> # it's my favourite.
>
> def is_hex():
> r = re.compile('... ')
> def is_hex(s):
> return r.match(s) is not None
> return is_hex
>
> is_hex = is_hex()
>
>
>
> Am I missing something? Is there a nicer way of doing this? On a day
> to day basis I find myself in this situation quite regularly, and now
> I come to think of it today, it is something that I would like to
> improve.
>
> It is funny that in private it has never bothered me much, but when
> posting on comp.lang.pytho n I find that the code is unacceptable.
> Maybe I have two modes of programming, idealist and practical? *shrug*
> :)[/color]

Your second solution (the one you labeled "horrible and probably
slow") is a classic example of a Function Object, and it's described
that way in "Design Patterns". AFAIK, it's got the same call overhead
as any other way of doing the job. I'd call it the cleanest way of doing
a parameterized function.

John Roth[color=blue]
>
>
> David.[/color]

[Aahz]

Re: Storing objects required by functions.

In article <99dce321.03123 00655.14c5a8db@ posting.google. com>,
David M. Wilson <dw-google.com@bota nicus.net> wrote:[color=blue]
>
>g = re.compile('... ')
>
>def uses_global_yuc k(x):
> global g
> pass[/color]

Why not just use the global? Without the ``global`` statement, that is.
Don't like that? How's it any different from using a class instance?
--
Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote
programs, then the first woodpecker that came along would destroy civilization.

[John Roth]

Re: Storing objects required by functions.


"Aahz" <aahz@pythoncra ft.com> wrote in message
news:bss5t7$1tf $1@panix1.panix .com...[color=blue]
> In article <99dce321.03123 00655.14c5a8db@ posting.google. com>,
> David M. Wilson <dw-google.com@bota nicus.net> wrote:[color=green]
> >
> >g = re.compile('... ')
> >
> >def uses_global_yuc k(x):
> > global g
> > pass[/color]
>
> Why not just use the global? Without the ``global`` statement, that is.
> Don't like that? How's it any different from using a class instance?[/color]

You can't parameterize it. You've got one global, while
you can put different parameters in different class instances.
He isn't doing that here, but that's the major place where the
Function Object design pattern shines.

John Roth
[color=blue]
> --
> Aahz (aahz@pythoncra ft.com) <*>[/color]

http://www.pythoncraft.com/

[Matt Goodall]

Re: Storing objects required by functions.

David M. Wilson wrote:
[color=blue]
>Further to my last post here, I was playing some more with building a
>regex object and storing it somewhere for use internally by a
>function. I'm not happy with any of my solutions:
>
>
># I don't like this, but the fact that you can modify the procedure's
># function via a named argument seems neat in a hacky sort of way.
>
>def uses_default_pa rm_yuck(x, r = re.compile("... ")):
> pass
>
>
>g = re.compile('... ')
>
>def uses_global_yuc k(x):
> global g
> pass
>
>[/color]
There is no need to define g as global unless you actually need to
rebind g inside the function.
[color=blue]
>
># This is horrible and probably slow.
>
>class is_hex:
> def __init__(self):
> self.r = re.compile('... ')
>
> def __call__(self, x):
> r = self.r
> pass
>
>is_hex = is_hex()
>
>[/color]
I doubt it's that much slower (if at all). You should profile it to check.
[color=blue]
>
># This mucks up scoping so that your procedure can't access it's
># parent scope like it could normally. Since I never do this,
># it's my favourite.
>
>def is_hex():
> r = re.compile('... ')
> def is_hex(s):
> return r.match(s) is not None
> return is_hex
>
>is_hex = is_hex()
>
>
>
>Am I missing something? Is there a nicer way of doing this? On a day
>to day basis I find myself in this situation quite regularly, and now
>I come to think of it today, it is something that I would like to
>improve.
>
>It is funny that in private it has never bothered me much, but when
>posting on comp.lang.pytho n I find that the code is unacceptable.
>Maybe I have two modes of programming, idealist and practical? *shrug*
>:)
>
>
>David.
>
>[/color]

Another alternative relies on the fact that functions themselves are
objects so you could do this:

def is_hex(s):
return is_hex.r.match( s) is not None

is_hex.r = re.compile(...)


Although, for simplicity, I would probably just make the compiled regex
a module scope variable with a sensible name, maybe even using the _
prefix to hint that noone should touch it:

_is_hex_regex = re.compile(...)

def is_hex(s):
return _is_hex_regex.m atch(s) is not None


Cheers, Matt

--
Matt Goodall, Pollenation Internet Ltd
w: http://www.pollenationinternet.com
e: matt@pollenatio n.net

[Terry Reedy]

Re: Storing objects required by functions.


"David M. Wilson" <dw-google.com@bota nicus.net> wrote in message
news:99dce321.0 312300655.14c5a 8db@posting.goo gle.com...[color=blue]
> Further to my last post here, I was playing some more with building a
> regex object and storing it somewhere for use internally by a
> function. I'm not happy with any of my solutions:[/color]
[color=blue]
> # I don't like this, but the fact that you can modify the procedure's
> # function via a named argument seems neat in a hacky sort of way.
>
> def uses_default_pa rm_yuck(x, r = re.compile("... ")):
> pass[/color]

I default args less bothersome than some people. The only problem for me
is that a caller may accidentally give nonesense second param. If this can
silently give a junk answer, this is not very acceptable. In this case,
random objects without a match() method would raise an exception.
[color=blue]
> g = re.compile('... ')[/color]

Use _ to indicate 'private' or 'local-use-only' status.

_hex = re.compile()
[color=blue]
> def uses_global_yuc k(x):
> global g
> pass[/color]

The global declaration is not needed for read-only access and is therefore
misleading. So delete and just use _hex in the body. Python functions
constantly use globals and builtins, both functions and other values, in
read-only mode. So I see no problem with this standard Python idiom.
[color=blue]
> # This is horrible and probably slow.
>
> class is_hex:
> def __init__(self):
> self.r = re.compile('... ')
>
> def __call__(self, x):
> r = self.r
> pass[/color]

One should at least generalize this to re_matcher and pass specific re to
init. OO purists might prefer this but I agree that it is overkill unless,
possibly, one has lots of instances. OO was made for man, not man for OO
purity.
[color=blue]
> is_hex = is_hex()[/color]

is_hex = re_matcher(hex_ re)
[color=blue]
> # This mucks up scoping so that your procedure can't access it's
> # parent scope like it could normally.[/color]

Funny, you just objected above to having a function access is parent scope
for the re.
[color=blue]
> def is_hex():
> r = re.compile('... ')
> def is_hex(s):
> return r.match(s) is not None
> return is_hex[/color]
[color=blue]
> is_hex = is_hex()[/color]

Same comment: generalize

def re_matcher(some _re):
r = re.compile(some _re)
def is_some(s):
return r.match(s) is not None
return is_some

is_hex = re_matcher(hex_ re)

I would use this in preverence to class version for making multiple
matchers. I think closures are great as one-method instances (or as
callable no-method instances, if one prefers).

Terry J. Reedy

[Jp Calderone]

Re: Storing objects required by functions.

On Tue, Dec 30, 2003 at 12:43:13PM -0500, Terry Reedy wrote:[color=blue]
>
> I default args less bothersome than some people. The only problem for me
> is that a caller may accidentally give nonesense second param. If this can
> silently give a junk answer, this is not very acceptable. In this case,
> random objects without a match() method would raise an exception.
> [color=green]
> > g = re.compile('... ')[/color]
>
> Use _ to indicate 'private' or 'local-use-only' status.
>
> _hex = re.compile()
> [/color]

Why should it be local-use-only?

Jp

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/8dMYedcO2BJA+4Y RAgWbAJwNC9zfer duRJarhPQbRBltn JWw9gCglSJA
c/gKhA2op4Y7Sg43u gSdG+A=
=ku9c
-----END PGP SIGNATURE-----

[John Roth]

Re: Storing objects required by functions.

Could you please *not* add whatever signature or business
card or whatever you're adding to your messages? My anti-virus
blocks access to your entire message when you do that,
and it's a royal pain to save the message and then disassemble
it to find you made one line of comment.

John Roth

"Jp Calderone" <exarkun@intarw eb.us> wrote in message
news:mailman.51 .1072812951.813 4.python-list@python.org ...

[Jp Calderone]

Re: Storing objects required by functions.

On Tue, Dec 30, 2003 at 03:22:09PM -0500, John Roth wrote:[color=blue]
> Could you please *not* add whatever signature or business
> card or whatever you're adding to your messages? My anti-virus
> blocks access to your entire message when you do that,
> and it's a royal pain to save the message and then disassemble
> it to find you made one line of comment.
>
> John Roth
>
> "Jp Calderone" <exarkun@intarw eb.us> wrote in message
> news:mailman.51 .1072812951.813 4.python-list@python.org ...
>[/color]

It's a PGP signature. If the mail client you're using can't extricate the
plaintext part, I'd say it isn't a very good mail client. (Even mailman's
archiver deals with them the right way these days ;)

Jp

[Bengt Richter]

Re: Storing objects required by functions.

On 30 Dec 2003 06:55:27 -0800, dw-google.com@bota nicus.net (David M. Wilson) wrote:
[color=blue]
>Further to my last post here, I was playing some more with building a
>regex object and storing it somewhere for use internally by a
>function. I'm not happy with any of my solutions:
>
>
># I don't like this, but the fact that you can modify the procedure's
># function via a named argument seems neat in a hacky sort of way.
>
>def uses_default_pa rm_yuck(x, r = re.compile("... ")):
> pass
>[/color]
There is another way, but IMO it would be nice to be able to have an optional
second parenthesized list to make bindings the same way which wouldn't be part
of the call signature. I.e.,

def uses_default_pa rm_yuck(x)(r = re.compile("... ")):
pass

or, same thing more prominently:

def uses_default_pa rm_yuck(x)(
# pre-bound locals
r = re.compile("... ")
):
pass

ISTM most of the implementation pieces for that should already be there.

The other way is to take advantage of functions' roles as decriptors and the mechanism that
makes bound methods with a self as the first arg, but the rest apparently normal. I.e,
we can put the r parameter in the place of self (not specifically tested)

def uses_self(r, x):
pass
uses_self = uses_self.__get __(re.compile(" ..."))

then (the rebound) uses_self will look like a bound method and want exactly one parameter x
(or whatever signature you program) and be able to refer to r like self.
[color=blue]
>
>g = re.compile('... ')
>
>def uses_global_yuc k(x):
> global g
> pass
>
>
># This is horrible and probably slow.
>
>class is_hex:
> def __init__(self):
> self.r = re.compile('... ')
>
> def __call__(self, x):
> r = self.r
> pass
>
>is_hex = is_hex()
>
>
># This mucks up scoping so that your procedure can't access it's
># parent scope like it could normally. Since I never do this,
># it's my favourite.
>
>def is_hex():
> r = re.compile('... ')
> def is_hex(s):
> return r.match(s) is not None
> return is_hex
>
>is_hex = is_hex()
>
>
>
>Am I missing something? Is there a nicer way of doing this? On a day
>to day basis I find myself in this situation quite regularly, and now
>I come to think of it today, it is something that I would like to
>improve.
>
>It is funny that in private it has never bothered me much, but when
>posting on comp.lang.pytho n I find that the code is unacceptable.
>Maybe I have two modes of programming, idealist and practical? *shrug*
>:)
>
>
>David.[/color]

Regards,
Bengt Richter

[John Roth]

Re: Storing objects required by functions.


"Jp Calderone" <exarkun@intarw eb.us> wrote in message
news:mailman.58 .1072824331.813 4.python-list@python.org ...[color=blue]
> On Tue, Dec 30, 2003 at 03:22:09PM -0500, John Roth wrote:[color=green]
> > Could you please *not* add whatever signature or business
> > card or whatever you're adding to your messages? My anti-virus
> > blocks access to your entire message when you do that,
> > and it's a royal pain to save the message and then disassemble
> > it to find you made one line of comment.
> >
> > John Roth
> >
> > "Jp Calderone" <exarkun@intarw eb.us> wrote in message
> > news:mailman.51 .1072812951.813 4.python-list@python.org ...
> >[/color]
>
> It's a PGP signature. If the mail client you're using can't extricate[/color]
the[color=blue]
> plaintext part, I'd say it isn't a very good mail client. (Even mailman's
> archiver deals with them the right way these days ;)[/color]

Actually, whatever is doing it simply sequesters almost anything,
including perfectly innocuous graphics. The difficulty is that the
mail client still displays most of the sequestered attachments, but
for some reason if it can't figure out one attachment, it doesn't
display any of them.

That means I've got to save the entire mail and use a command line
tool to break it into pieces. Python's mail tools do a reasonable
(not excellent) job of this, so nothing is lost, but it's more work
than I usually want to go through for newsgroup messages.

John Roth

[color=blue]
>
> Jp
>[/color]

[Christos TZOTZIOY Georgiou]

Re: Storing objects required by functions.

On 31 Dec 2003 00:08:47 GMT, rumours say that bokr@oz.net (Bengt
Richter) might have written:
[color=blue]
>The other way is to take advantage of functions' roles as decriptors and the mechanism that
>makes bound methods with a self as the first arg, but the rest apparently normal. I.e,
>we can put the r parameter in the place of self (not specifically tested)
>
> def uses_self(r, x):
> pass
> uses_self = uses_self.__get __(re.compile(" ..."))[/color]

It's much more general to use new.instancemet hod. See:

def voodoo(function , *its_arguments) :
from new import instancemethod
def child(function, first_argument, *rest_of_argume nts):
if rest_of_argumen ts:
return child(
instancemethod( function, first_argument, object),
*rest_of_argume nts
)
else:
return instancemethod( function, first_argument, object)
return child(function, *its_arguments)

The import statement is in the voodoo just for completeness including it
here.
The function above recurses in order to allow stuff like:

getter = voodoo(getattr, my_object, "its_attribute" )

or the more modern

getter = voodoo(operator .itemgetter("it s_attribute"), my_object)

and similarly

setter = voodoo(operator , my_object, "its_attribute" )

allowing

setter(value)

at good speeds.


I have a module predicates.py defining All and Any classes for
iterables, and the trick above plus itertools allows *some* operations
to run faster than correspondent python code...
--
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix

[Bengt Richter]

Re: Storing objects required by functions.

On Wed, 31 Dec 2003 04:04:47 +0200, Christos "TZOTZIOY" Georgiou <tzot@sil-tec.gr> wrote:
[color=blue]
>On 31 Dec 2003 00:08:47 GMT, rumours say that bokr@oz.net (Bengt
>Richter) might have written:
>[color=green]
>>The other way is to take advantage of functions' roles as decriptors and the mechanism that
>>makes bound methods with a self as the first arg, but the rest apparently normal. I.e,
>>we can put the r parameter in the place of self (not specifically tested)
>>
>> def uses_self(r, x):
>> pass
>> uses_self = uses_self.__get __(re.compile(" ..."))[/color]
>
>It's much more general to use new.instancemet hod. See:[/color]
I think I did use new.instancemet hod, through another door ;-)
[color=blue][color=green][color=darkred]
>>> def uses_self(r, x):[/color][/color][/color]
... pass
...[color=blue][color=green][color=darkred]
>>> import re
>>> uses_self = uses_self.__get __(re.compile(" ..."))
>>> type(uses_self)[/color][/color][/color]
<type 'instancemethod '>
[color=blue][color=green][color=darkred]
>>> myim = type(uses_self)
>>> myim[/color][/color][/color]
<type 'instancemethod '>
[color=blue][color=green][color=darkred]
>>> import new
>>> myim is new.instancemet hod[/color][/color][/color]
True
;-)
[color=blue][color=green][color=darkred]
>>> def foo(self, *args, **kw): print 'self=%r, args=%r, kw=%r'%(self, args, kw)[/color][/color][/color]
...[color=blue][color=green][color=darkred]
>>> myim(foo, 'dummy self', object) # object per your usage[/color][/color][/color]
<bound method object.foo of 'dummy self'>[color=blue][color=green][color=darkred]
>>> foom = myim(foo, 'dummy self', object) # object per your usage
>>> foom(1,2,3,hi=' Hello')[/color][/color][/color]
self='dummy self', args=(1, 2, 3), kw={'hi': 'Hello'}

I didn't make voodoo out of it though. Interesting, but all that nested
calling at call-time seems like it would make for a performance hit? Unless maybe it is
all packed up in slots that C can get to very fast??
[color=blue]
>
>def voodoo(function , *its_arguments) :
> from new import instancemethod
> def child(function, first_argument, *rest_of_argume nts):
> if rest_of_argumen ts:
> return child(
> instancemethod( function, first_argument, object),
> *rest_of_argume nts
> )
> else:
> return instancemethod( function, first_argument, object)
> return child(function, *its_arguments)
>
>The import statement is in the voodoo just for completeness including it
>here.
>The function above recurses in order to allow stuff like:
>
>getter = voodoo(getattr, my_object, "its_attribute" )
>
>or the more modern
>
>getter = voodoo(operator .itemgetter("it s_attribute"), my_object)
>
>and similarly
>
>setter = voodoo(operator , my_object, "its_attribute" )
>
>allowing
>
>setter(value )
>
>at good speeds.
>
>
>I have a module predicates.py defining All and Any classes for
>iterables, and the trick above plus itertools allows *some* operations
>to run faster than correspondent python code...[/color]
Interesting. Gotta go.

Regards,
Bengt Richter

[Christos TZOTZIOY Georgiou]

Re: Storing objects required by functions.

On 31 Dec 2003 04:07:01 GMT, rumours say that bokr@oz.net (Bengt
Richter) might have written:
[color=blue][color=green]
>>It's much more general to use new.instancemet hod. See:[/color][/color]
[color=blue]
>I think I did use new.instancemet hod, through another door ;-)[/color]

No doubt :) . *I* wasn't clear: new.instancemet hod accepts any callable
I throw to it, while not all callables have a __get__ method...
[color=blue][color=green][color=darkred]
>>>> foom(1,2,3,hi=' Hello')[/color][/color]
> self='dummy self', args=(1, 2, 3), kw={'hi': 'Hello'}
>
>I didn't make voodoo out of it though. Interesting, but all that nested
>calling at call-time seems like it would make for a performance hit? Unless maybe it is
>all packed up in slots that C can get to very fast??[/color]

Actually, I think that most overhead lies in creating a frame object for
the execution of a python callable. Calling C functions is fastest. I
didn't spot any delays, but I didn't search thoroughly ;)
--
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix

[Aahz]

Re: Storing objects required by functions.

In article <mailman.58.107 2824331.8134.py thon-list@python.org >,
Jp Calderone <exarkun@intarw eb.us> wrote:[color=blue]
>
>It's a PGP signature. If the mail client you're using can't extricate the
>plaintext part, I'd say it isn't a very good mail client. (Even mailman's
>archiver deals with them the right way these days ;)[/color]

I'm using netnews, not e-mail. PGP doesn't belong in netnews posts, IMO.
--
Aahz (aahz@pythoncra ft.com) <*> http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote
programs, then the first woodpecker that came along would destroy civilization.