import keyword behaviour - performance impact if used multipletimes?

[Andrew James]

Hi,
I've been looking around on Google for the answer to this question, and
it's beginning to really bug me. I'm making some design decisions for
some code I'm writing, and I'm wondering whether (Good Design Decisions
apart), there's a performance impact in importing the same module in two
different files. For example, with the following:

fileA.py
-----------
import psycopg
class A:
....

fileB.py
import psycopg
class B:
a = A()
....

If I run fileB, will this import the psycopg twice, or once only? Is
this something to do with system modules being singletons?

If someone could help me out with this it would be much appreciated.

Regards,
Andrew

--
Andrew James <drew@gremlinho sting.com>

[Diez B. Roggisch]

Re: import keyword behaviour - performance impact if used multiple times?

> If I run fileB, will this import the psycopg twice, or once only? Is[color=blue]
> this something to do with system modules being singletons?[/color]

It will be imported only once. Of course the statement will be read twice -
but as this only happens while reading the *.py the first time, you needn't
bother about performance issues.
--
Regards,

Diez B. Roggisch

[Nick Coghlan]

Re: import keyword behaviour - performance impact if used multipletimes?

Andrew James wrote:[color=blue]
> Hi,
> I've been looking around on Google for the answer to this question, and
> it's beginning to really bug me. I'm making some design decisions for
> some code I'm writing, and I'm wondering whether (Good Design Decisions
> apart), there's a performance impact in importing the same module in two
> different files. For example, with the following:
>
> fileA.py
> -----------
> import psycopg
> class A:
> ....
>
> fileB.py
> import psycopg
> class B:
> a = A()
> ....
>
> If I run fileB, will this import the psycopg twice, or once only?[/color]

I'm guessing fileB.py should read:
import psycopg
import A
...etc

Anyway, when Python imports modules it stores a reference to them in
sys.modules. Any attempts to import the module again are satisfied using the
cached version already stored in sys.modules.

So, in your example, psycopg is imported once only, and both fileA and fileB
would be referring to the same version of psycopg.
[color=blue]
> Is
> this something to do with system modules being singletons?[/color]

They aren't singletons in the GoF design pattern sense. However, Python's import
machinery operates in such a way that it takes effort to get multiple version of
the same module into memory at the same time (it *can* be done, but you have to
work at it).

Cheers,
Nick.

[neophyte]

Re: import keyword behaviour - performance impact if used multiple times?

Nick Coghlan wrote:[color=blue][color=green]
> > Is
> > this something to do with system modules being singletons?[/color]
>
> They aren't singletons in the GoF design pattern sense. However,[/color]
Python's import[color=blue]
> machinery operates in such a way that it takes effort to get multiple[/color]
version of[color=blue]
> the same module into memory at the same time (it *can* be done, but[/color]
you have to[color=blue]
> work at it).[/color]
Given that this is exactly what I want, how can you do it?

[Nick Coghlan]

Re: import keyword behaviour - performance impact if used multipletimes?

neophyte wrote:[color=blue]
> Nick Coghlan wrote:
>[color=green][color=darkred]
>> > Is
>> > this something to do with system modules being singletons?[/color]
>>
>>They aren't singletons in the GoF design pattern sense. However,[/color]
>
> Python's import
>[color=green]
>>machinery operates in such a way that it takes effort to get multiple[/color]
>
> version of
>[color=green]
>>the same module into memory at the same time (it *can* be done, but[/color]
>
> you have to
>[color=green]
>>work at it).[/color]
>
> Given that this is exactly what I want, how can you do it?
>[/color]

If you just want to reload an existing module, use the builtin "reload" function.

Getting multiple versions of a module into sys.modules at the same time isn't
something I've ever actually wanted to do, but the following will do it:

Py> import sys
Py> sys.modules["sys1"] = sys
Py> del sys.modules["sys"]
Py> import sys
Py> import sys1
Py> sys is sys1
False

However:
1. Doing this at all is probably a bad idea (since you may end up duplicating
objects that are meant to be unique within the process, and any C-extension code
will still be shared between the multiple versions of the module)
2. Doing it to 'sys' like I just did is an even worse idea, since you
*definitely* end up doing 1 :)

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net