We have zipimport, how about dllimport?

[vivainio@gmail.com]

I may be on particularly potent crack, but I was wondering whether it
would make sense to distribute python code in DLLs so that the memory
occupied by the bytecode would be consumed only once even if there were
multiple processes using the same bytecode. Or is there another way to
accomplish code sharing?

Of course I don't need the functionality right this week, but
capability of running shared code is one of the advantages of natively
compiled languages and someone must already have thought of this and
knows why it would never work. ;-)

[Méta-MCI]

Re: We have zipimport, how about dllimport?

ctypes ?

[Martin v. Löwis]

Re: We have zipimport, how about dllimport?

vivainio@gmail. com wrote:[color=blue]
> I may be on particularly potent crack, but I was wondering whether it
> would make sense to distribute python code in DLLs so that the memory
> occupied by the bytecode would be consumed only once even if there were
> multiple processes using the same bytecode. Or is there another way to
> accomplish code sharing?[/color]

That wouldn't help much: as soon as a .pyc file is imported, it is
passed to the marshal module, which will create a code object out
of it (copying the actual data in the pyc file). Then, this code
object gets executed, containing import, def, and other statements.
The def statements, in turn, result in yet other code objects being
created, copying fragments of the original code object. Then, the
original code object is discarded (and its memory released).

So typically, you have two levels of copying, each involving interaction
with the per-process memory management. There is little hope for sharing
here.
[color=blue]
> Of course I don't need the functionality right this week, but
> capability of running shared code is one of the advantages of natively
> compiled languages and someone must already have thought of this and
> knows why it would never work. ;-)[/color]

It's an advantage only if the code occupies a signifcant portion of the
memory. In bytecode languages, byte code is much more compact than
machine code, so there is comparatively little opportunity for savings.

OTOH, in *JIT* languages, the JIT code again counts for something,
and it typically has to be generated for each process. That's why
Microsoft's .NET has ngen.exe, the ahead-of-time (traditional)
compiler. The primary rationale is not sharing, though, but reducing
the startup time. In .NET 2.0, there is even an OS service that
creates compiled images of IL byte code.

LISP and Smalltalk implementations have yet another approach: the image.
They dump everything in memory into a disk file which can then be mapped
into memory. This image is created after all the loading indirections
and copies, and so constitutes the final form. So on the plus side,
this allows for sharing of code across multiple instances of the same
application. On the minus side, each Smalltalk application needs its
own image (even if they share a lot of code).

Regards,
Martin