Loading just in time

[D'Arcy J.M. Cain]

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything. I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)

This works the first time I call utility.calc_ta x but if I call it
again I get a "TypeError: 'module' object is not callable" error. Is
there any way to do what I want or do I have to put everything back
into a single file. Of course, I can simply change all my calls to
utility.calc_ta x.calc_tax(...) but I have a lot of code to change if I
do that.

Thanks.

--
D'Arcy J.M. Cain <darcy@druid.ne t | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

[Larry Bates]

Re: Loading just in time

D'Arcy J.M. Cain wrote:

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything. I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:
>
def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)
>
This works the first time I call utility.calc_ta x but if I call it
again I get a "TypeError: 'module' object is not callable" error. Is
there any way to do what I want or do I have to put everything back
into a single file. Of course, I can simply change all my calls to
utility.calc_ta x.calc_tax(...) but I have a lot of code to change if I
do that.
>
Thanks.
>

You are stuck in a futile battle called "premature optimization". I would
suggest that you stop worrying about any performance you would gain from doing
something like this. Python has been "highly" optimized to handle imports in a
very efficient way. Just put your functions in a file and import them.

from myfunctions import calc_tax, ...

Then you don't have to preface the function name with the module name.

-Larry

[D'Arcy J.M. Cain]

Re: Loading just in time

On Thu, 10 Jul 2008 11:03:10 -0500
Larry Bates <larry.bates@we bsafe.com`wrote :

D'Arcy J.M. Cain wrote:

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)

You are stuck in a futile battle called "premature optimization". I would

I thought that I might be.

suggest that you stop worrying about any performance you would gain from doing
something like this. Python has been "highly" optimized to handle imports in a
very efficient way. Just put your functions in a file and import them.

Performance optimization wasn't really my goal here. What I was
looking for was the ability to spread the functions around different
files to manage them better from a proggrammer's POV.

Oh well. I guess I will just combine them all again. At least I found
some dead code, duplicate functions and just plain bad code doing this
little exercise. :-)

--
D'Arcy J.M. Cain <darcy@druid.ne t | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

[samwyse]

Re: Loading just in time

On Jul 10, 9:45 am, "D'Arcy J.M. Cain" <da...@druid.ne twrote:

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything.  I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:
>
def calc_tax(*arg, **name):
    from calc_tax import calc_tax as _func_
    calc_tax = _func_
    return _func_(*arg, **name)

This doesn't do what you think. The line "calc_tax = _func_" is
probably modifying a local variable that is then thrown away. I've
got a slightly different (simpler) version to illustrate:

=== main.py ===

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
#global calc_tax
calc_tax = _func_
print '_func_ is', repr(_func_)
print 'calc_tax is', repr(calc_tax)
return _func_(*arg, **name)

print 'before: calc_tax is', repr(calc_tax)
result = calc_tax()
print 'after: calc_tax is', repr(calc_tax)

=== calc_tax.py ===

def calc_tax(*arg, **name):
return 42

=== end of files ===

Running main.py gives this:

before: calc_tax is <function calc_tax at 0x015049F0>
_func_ is <function calc_tax at 0x014950B0>
calc_tax is <function calc_tax at 0x014950B0>
after: calc_tax is <function calc_tax at 0x015049F0>

Note that the value of calc_test is the same in the first and last
lines.

If you uncomment the line "#global calc_tax" and run it again, you get
this:

before: calc_tax is <function calc_tax at 0x01504A30>
_func_ is <function calc_tax at 0x014950B0>
calc_tax is <function calc_tax at 0x014950B0>
after: calc_tax is <function calc_tax at 0x014950B0>

Interestingly, neither version gives me a TypeError, no matter how
many times I call calc_tax.

(BTW, you might want to look up Memoization; it's very similar to what
you want to do, and might give you a way to more efficiently code
things.)

[Ben Finney]

Re: Loading just in time

"D'Arcy J.M. Cain" <darcy@druid.ne twrites:

Performance optimization wasn't really my goal here. What I was
looking for was the ability to spread the functions around different
files to manage them better from a proggrammer's POV.

Why not do that, then? Your functions should be organised into
conceptually sensible module groupings.

--
\ “My doctor told me to stop having intimate dinners for four. |
`\ Unless there are three other people.” —Orson Welles |
_o__) |
Ben Finney

[Scott David Daniels]

Re: Loading just in time

D'Arcy J.M. Cain wrote:

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything. I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:
>
def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)
...

Well, you may well be "mired in premature optimization," but you _could_
do the following (you'll need to evaluate whether it addresses your
issues).

def calc_tax(*arg, **kwargs):
from calc_tax import calc_tax
globals()['calc_tax'] = calc_tax
return calc_tax(*arg, **kwargs)

But if you are terrifyingly clever, (or have enough motive*), you can
define a function, '_remote', So that your __init__.py contains:
<some magic>
...
_remote('some_t ax')
_remote('other' , 'frogs')
...

* The pythonic "enough motive" is how redundant your "__init__.p y"
looks. The "DRY" (Don't Repeat Yourself) principle comes into effect.
The more your source looks like boilerplate, the more you should itch
to replace the boilerplate.

--Scott David Daniels
Scott.Daniels@A cm.Org

[Ross Ridge]

Re: Loading just in time

D'Arcy J.M. Cain <darcy@druid.ne twrote:

>def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)

This should do what you want:

def calc_tax(*arg, **name):
global calc_tax
from calc_tax import calc_tax
return calc_tax(*arg, **name)

I suspect though that the cost of importing a lot of little modules
won't be as bad as you might think.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.u waterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //