Proposal: Inline Import

[Alex Martelli]

Re: Proposal: Inline Import

Erik Max Francis <max@alcyone.co m> wrote:
[color=blue]
> Shane Hathaway wrote:
>[color=green]
> > Let me fully elaborate the heresy I'm suggesting: I am talking about
> > inline imports on every other line of code. The obvious implementation
> > would drop performance by a double digit percentage.[/color]
>
> Module importing is already idempotent. If you try to import an
> already-imported module, inline or not, the second (or subsequent)
> imports are no-operations.[/color]

Hmmm, yes, but they're rather SLOW no-operations...:

Helen:~ alex$ python -mtimeit -s'import sys' 'import sys'
100000 loops, best of 3: 3.52 usec per loop

Now this is just a humble ultralight laptop, to be sure, but still, to
put the number in perspective...:

Helen:~ alex$ python -mtimeit -s'import sys' 'sys=23'
10000000 loops, best of 3: 0.119 usec per loop

....we ARE talking about a factor of 30 or so slower than elementary
assignments (I'm wondering whether this may depend on import hooks, or,
what else...).


Alex

[Bengt Richter]

Re: Proposal: Inline Import

On Fri, 09 Dec 2005 12:24:59 -0700, Shane Hathaway <shane@hathaway mix.org> wrote:
[color=blue]
>Here's a heretical idea.
>
>I'd like a way to import modules at the point where I need the
>functionalit y, rather than remember to import ahead of time. This might
>eliminate a step in my coding process. Currently, my process is I
>change code and later scan my changes to make matching changes to the
>import statements. The scan step is error prone and time consuming.
>By importing inline, I'd be able to change code without the extra scan step.
>
>Furthermore, I propose that the syntax for importing inline should be an
>expression containing a dot followed by an optionally dotted name. For
>example:
>
> name_expr = .re.compile('[a-zA-Z]+')
>
>The expression on the right causes "re.compile " to be imported before
>calling the compile function. It is similar to:
>
> from re import compile as __hidden_re_com pile
> name_expr = __hidden_re_com pile('[a-zA-Z]+')
>
>The example expression can be present in any module, regardless of
>whether the module imports the "re" module or assigns a different
>meaning to the names "re" or "compile".
>
>I also propose that inline import expressions should have no effect on
>local or global namespaces, nor should inline import be affected by
>local or global namespaces. If users want to affect a namespace, they
>must do so with additional syntax that explicitly assigns a name, such as:
>
> compile = .re.compile[/color]

Are you willing to type a one-letter prefix to your .re ? E.g.,
[color=blue][color=green][color=darkred]
>>> class I(object):[/color][/color][/color]
... def __getattr__(sel f, attr):
... return __import__(attr )
...[color=blue][color=green][color=darkred]
>>> I = I()
>>> name_expr = I.re.compile('[a-zA-Z+]')
>>> name_expr[/color][/color][/color]
<_sre.SRE_Patte rn object at 0x02EF4AC0>[color=blue][color=green][color=darkred]
>>> compile = I.re.compile
>>> compile[/color][/color][/color]
<function compile at 0x02EFE144>[color=blue][color=green][color=darkred]
>>> pi = I.math.pi
>>> pi[/color][/color][/color]
3.1415926535897 931[color=blue][color=green][color=darkred]
>>> I.math.sin(pi/6)[/color][/color][/color]
0.4999999999999 9994

Of course it does cost you some overhead that you could avoid.
[color=blue]
>In the interest of catching errors early, it would be useful for the
>Python parser to produce byte code that performs the actual import upon
>loading modules containing inline import expressions. This would catch
>misspelled module names early. If the module also caches the imported
>names in a dictionary, there would be no speed penalty for importing
>inline rather than importing at the top of the module.
>
>I believe this could help many aspects of the language:
>
>- The coding workflow will improve, as I mentioned.
>
>- Code will become more self-contained. Self-contained code is easier
>to move around or post as a recipe.
>
>- There will be less desire for new builtins, since modules will be just
>as accessible as builtins.
>
>Thoughts?
>[/color]
There are special caveats re imports in threads, but otherwise
I don't know of any significant downsides to importing at various
points of need in the code. The actual import is only done the first time,
so it's effectively just a lookup in sys.modules from there on.
Am I missing something?

Regards,
Bengt Richter

[Robert Kern]

Re: Proposal: Inline Import

Bengt Richter wrote:
[color=blue]
> Are you willing to type a one-letter prefix to your .re ? E.g.,
>[color=green][color=darkred]
> >>> class I(object):[/color][/color]
> ... def __getattr__(sel f, attr):
> ... return __import__(attr )[/color]

[snip]
[color=blue]
> There are special caveats re imports in threads, but otherwise
> I don't know of any significant downsides to importing at various
> points of need in the code. The actual import is only done the first time,
> so it's effectively just a lookup in sys.modules from there on.
> Am I missing something?[/color]

Packages.

--
Robert Kern
robert.kern@gma il.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Bengt Richter]

Re: Proposal: Inline Import

On Sat, 10 Dec 2005 19:40:08 -0800, Robert Kern <robert.kern@gm ail.com> wrote:
[color=blue]
>Bengt Richter wrote:
>[color=green]
>> Are you willing to type a one-letter prefix to your .re ? E.g.,
>>[color=darkred]
>> >>> class I(object):[/color]
>> ... def __getattr__(sel f, attr):
>> ... return __import__(attr )[/color]
>
>[snip]
>[color=green]
>> There are special caveats re imports in threads, but otherwise
>> I don't know of any significant downsides to importing at various
>> points of need in the code. The actual import is only done the first time,
>> so it's effectively just a lookup in sys.modules from there on.
>> Am I missing something?[/color]
>
>Packages.
>[/color]
Ok, if you're willing to add a trailing '._' to indicate the end of a package path,
and start it with a P instead of an I, you could try the following (just a hack, not tested beyond
what you see, (again ;-) )

----< impexpr.py >--------------------
class I(object):
__cache = {}
def __getattr__(sel f, attr, cache = __cache):
try: return cache[attr]
except KeyError:
cache[attr] = ret = __import__(attr )
return ret
getdotted = __getattr__

class P(I):
def __init__(self):
self.elems = []
def __getattr__(sel f, attr):
if attr == '_':
dotted = '.'.join(self.e lems)
mod = self.getdotted( dotted)
for attr in self.elems[1:]:
mod = getattr(mod, attr)
self.elems = []
return mod
else:
self.elems.appe nd(attr)
return self

P, I = P(), I()
--------------------------------------
[color=blue][color=green][color=darkred]
>>> from ut.impexpr import I, P
>>> I.math.pi[/color][/color][/color]
3.1415926535897 931[color=blue][color=green][color=darkred]
>>> I.os.path.isfil e[/color][/color][/color]
<function isfile at 0x02EB5534>[color=blue][color=green][color=darkred]
>>> P.ut.miscutil._ .prb[/color][/color][/color]
<function prb at 0x02F1EBC4>[color=blue][color=green][color=darkred]
>>> type(I)._I__cac he.keys()[/color][/color][/color]
['ut.miscutil', 'os', 'math'][color=blue][color=green][color=darkred]
>>> P.ut.miscutil._ .disex[/color][/color][/color]
<function disex at 0x02F1EDBC>[color=blue][color=green][color=darkred]
>>> type(I)._I__cac he.keys()[/color][/color][/color]
['ut.miscutil', 'os', 'math']
[color=blue][color=green][color=darkred]
>>> I.ut.miscutil[/color][/color][/color]
<module 'ut.miscutil' from 'c:\pywk\ut\mis cutil.pyc'>[color=blue][color=green][color=darkred]
>>> I.ut[/color][/color][/color]
<module 'ut' from 'c:\pywk\ut\__i nit__.pyc'>

I am not recommending this particularly. I just like to see how close
python already is to allowing the spelling folks initially think requires
a language change ;-)

Regards,
Bengt Richter

[bonono@gmail.com]

Re: Proposal: Inline Import


Shane Hathaway wrote:[color=blue]
> Mike Meyer wrote:[color=green]
> > Shane Hathaway <shane@hathaway mix.org> writes:[color=darkred]
> >>I'd like a way to import modules at the point where I need the
> >>functionality , rather than remember to import ahead of time. This
> >>might eliminate a step in my coding process. Currently, my process is
> >>I change code and later scan my changes to make matching changes to
> >>the import statements. The scan step is error prone and time
> >>consuming. By importing inline, I'd be able to change code without the
> >>extra scan step.[/color]
> >
> >
> > As others have pointed out, you can fix your process. That your
> > process doesn't work well with Python isn't a good reason for changing
> > Python.[/color]
>
> Do you have any ideas on how to improve the process of maintaining
> imports? Benji's suggestion of jumping around doesn't work for moving
> code and it interrupts my train of thought. Sprinkling the code with
> import statements causes a speed penalty and a lot of clutter.
>[/color]
Is using import near where you need it really such a hit to speed ? May
be it is critical inside a function(may be called in a loop) but what
about something like this :

import <my needed modules for this_func>
def this_func:
use it here

In this way, it is no different than the @decorator in terms of "moving
codes around", i.e. just copy the extra import/@decorator lines if you
want to move this to a new module. In terms of speed, it won't be part
of the function but a module so the speed penalty would be only when
other import this module(one time or multiple time but still not the
same as calling a function in a loop), no different than putting
everything at the top of the file(well a bit but should be
neglecgible).

[Martin v. Löwis]

Re: Proposal: Inline Import

Shane Hathaway wrote:[color=blue]
> Do you have any ideas on how to improve the process of maintaining
> imports? Benji's suggestion of jumping around doesn't work for moving
> code and it interrupts my train of thought. Sprinkling the code with
> import statements causes a speed penalty and a lot of clutter.[/color]

You didn't detail your implementation strategy for that feature very
much, but the most natural way of implementing it would also cause the
same speed penalty as sprinkled import statements.

Regards,
Martin

[Martin v. Löwis]

Re: Proposal: Inline Import

Shane Hathaway wrote:[color=blue]
> Thoughts?[/color]

I have two reasons to dislike it:
1. It's a language change. Others have pointed out that you can achieve
the same without a language change; it would be easy to write

name_expr = _import.re.comp ile('[a-zA-Z]+')

2. In the form in which you have written it, I believe it is
unimplementable (or, else, an implementation would have
counter-intuitive border cases). For example, what would

.xml.sax.expatr eader.ExpatPars er.__init__(sel f)

mean? From the description, it appears that it would be

from xml.sax.expatre ader.ExpatParse r import __init__ as \
__hidden_xml_sa x_expatreader_E xpatParser___in it__

which, of course, wouldn't work, because ExpatParser is
not a module.
3. As in any good list of two items, I have a third complaint:
for packages, this would be tedious to type (as the sax
example illustrates)

Regards,
Martin

[bruno at modulix]

Re: Proposal: Inline Import

Mike Meyer wrote:[color=blue]
> Shane Hathaway <shane@hathaway mix.org> writes:
>[/color]
(snip)[color=blue]
>[color=green]
>>What's really got me down is the level of effort required to move code
>>between modules. After I cut 100 lines from a 500 line module and
>>paste them to a different 500 line module, I have to examine every
>>import in both modules as well as examine the code I moved for missing
>>imports.[/color]
>
>
> Has it ever occured to you that if you're cutting and pasting 500 line
> blocks, you're doing something fundamentally wrong? One of the points
> of modules and OO is that you don't *have* to do things like
> that. Cut-n-paste means you wind up with two copies of the code to
> maintain,[/color]

Mike, has it ever occured to you that this could be refactoring, not
copy_paste ?-)

(for the record, I too frequently *move* - not 'copy_paste' - big chunks
of code, at least at the beginning of a project, or before a major
evolution)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[adam]

Re: Proposal: Inline Import

When I'm feeling too lazy to do imports while moving large blocks of
code, I use this little hack. It lets me proceed with checking whether
the move does what I wanted and at the end I fix the imports and remove
the try/except wrapper. I think it would achieve your desired result
and not have an impact on the language itself.

try:
#your main code here
print string.upper("b lah")
except NameError, error_value:
mod_name = error_value.arg s[0][error_value.arg s[0].find("'") +
1:error_value.a rgs[0].rfind("'")]
try:
__import__(mod_ name)
print "imported %s" % mod_name
except:
print "NameError: %s" % error_value.arg s[0]
pass

-adam