compiling to python byte codes

**Martin v. Löwis** · Jul 18 '05, 02:49 PM

Re: compiling to python byte codes

Maurice LING wrote:[color=blue]
> I remembered reading a MSc thesis about compiling Perl to Java bytecodes
> (as in java class files).[/color]

You don't have to look that far. Jython compiles Python code into Java
bytecode; IronPython compiles Python code into Microsoft intermediate
language.
[color=blue]
> I'm wondering if
> something of such had been attempted in python, as in compiling X
> language into .pyc.[/color]

The easiest way to create a .pyc file is to create a Python file,
and then compile that. There are various tools that compile X to
..pyc. For example, Fnorb compiles OMG IDL into .pyc files.
[color=blue]
> I do not understand the schematics of .pyc files but
> I assume that they are the so called python bytecode files.[/color]

That's correct.
[color=blue]
> Or is there any documentation or books that is the python equivalent of
> "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]

There is the dis module and its documentation. However, as I said, in
Python, you don't really *need* to create .pyc files directly, as
the Python compiler is always available through the compile() builtin
function. This is unlike Java or .NET, where the compiler is not
available in the JRE, or the .NET commercial framework.

Regards,
Martin

**Leif K-Brooks** · Jul 18 '05, 02:49 PM

Re: compiling to python byte codes

Maurice LING wrote:[color=blue]
> Or is there any documentation or books that is the python equivalent of
> "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]

Python's byte code isn't very stable, so you might have to recreate your
entire code base with every new Python version. I would suggest
generating Python code (not byte code) instead and compiling that.

**Michael Foord** · Jul 18 '05, 02:49 PM

Re: compiling to python byte codes

"Martin v. Löwis" <martin@v.loewi s.de> wrote in message news:<4136bbf3$ 0$197$9b622d9e@ news.freenet.de >...[color=blue]
> Maurice LING wrote:[color=green]
> > I remembered reading a MSc thesis about compiling Perl to Java bytecodes
> > (as in java class files).[/color]
>
> You don't have to look that far. Jython compiles Python code into Java
> bytecode; IronPython compiles Python code into Microsoft intermediate
> language.
>[color=green]
> > I'm wondering if
> > something of such had been attempted in python, as in compiling X
> > language into .pyc.[/color]
>
> The easiest way to create a .pyc file is to create a Python file,
> and then compile that. There are various tools that compile X to
> .pyc. For example, Fnorb compiles OMG IDL into .pyc files.
>[color=green]
> > I do not understand the schematics of .pyc files but
> > I assume that they are the so called python bytecode files.[/color]
>
> That's correct.
>[color=green]
> > Or is there any documentation or books that is the python equivalent of
> > "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]
>
> There is the dis module and its documentation. However, as I said, in
> Python, you don't really *need* to create .pyc files directly, as
> the Python compiler is always available through the compile() builtin
> function. This is unlike Java or .NET, where the compiler is not
> available in the JRE, or the .NET commercial framework.
>
> Regards,
> Martin[/color]

But that still doesn't answer the OPs question which is about writing
code in another language to generate python bytecode....

Which is interesting.. but not that interesting I suppose.
Is python bytecode *that* different to Java bytecode (not in detail
but in concept ?). There's no reason why another compiler couldn't
emit python bytecode to run on the 'python virtual machine' ? (plenty
of reasons not to do it I suppose just no reasons why it shouldn't be
possible).

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html

**Michael Hudson** · Jul 18 '05, 02:49 PM

Re: compiling to python byte codes

Maurice LING <mauriceling@ac m.org> writes:
[color=blue]
> Hi,
>
> I remembered reading a MSc thesis about compiling Perl to Java
> bytecodes (as in java class files). At least, it seems that someone
> had compiled scheme to java class files quite successfully. I'm
> wondering if something of such had been attempted in python, as in
> compiling X language into .pyc.[/color]

Not to my knowledge. It wouldn't be very interesting: the Python
bytecode is pretty Python specific.
[color=blue]
> I do not understand the schematics of .pyc files but I assume that
> they are the so called python bytecode files.
>
> Or is there any documentation or books that is the python equivalent
> of "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]

Nope. As others point out, the details tend to change each (major)
version of Python. The documentation for the standard library module
'dis' might help. You could also look at the 'bytecodehacks' package
(google, and make sure you get the CVS version).

Cheers,
mwh

--
39. Re graphics: A picture is worth 10K words - but only those
to describe the picture. Hardly any sets of 10K words can be
adequately described with pictures.
-- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

**Martin v. Löwis** · Jul 18 '05, 02:50 PM

Re: compiling to python byte codes

Michael Foord wrote:[color=blue]
> But that still doesn't answer the OPs question which is about writing
> code in another language to generate python bytecode....[/color]

I did. I told him about the compile() function, and indeed,

compile("2+4"," <string>","eval ")

generates Python bytecode.
[color=blue]
> Is python bytecode *that* different to Java bytecode (not in detail
> but in concept ?).[/color]

Yes. Java bytecode is typed; Python bytecode is not.
[color=blue]
> There's no reason why another compiler couldn't
> emit python bytecode to run on the 'python virtual machine' ?[/color]

It is certainly possible. Indeed, the Python compiler does generate
Python bytecode from source code, so it must be possible :-)

Regards,
Martin

**Maurice LING** · Jul 18 '05, 02:51 PM

Re: compiling to python byte codes

Hi Martin,

Martin v. Löwis wrote:[color=blue]
> Michael Foord wrote:
>[color=green]
>> But that still doesn't answer the OPs question which is about writing
>> code in another language to generate python bytecode....[/color]
>
>
> I did. I told him about the compile() function, and indeed,
>
> compile("2+4"," <string>","eval ")
>
> generates Python bytecode.[/color]

Can I feed a python source file into compile(), line by line, and expect
it to generate a working .pyc file? I suppose my intended use is to be
able to handle python codes written at run time, to execute python codes
line by line, in a python program. It is somewhat like a tracer routine
that can interpret a line of python code, read out the variables, before
going to the next line of python code. Can compile() do this, or do I
have to use pexpect to run an instance of python?
[color=blue]
>[color=green]
>> Is python bytecode *that* different to Java bytecode (not in detail
>> but in concept ?).[/color]
>
>
> Yes. Java bytecode is typed; Python bytecode is not.
>[color=green]
>> There's no reason why another compiler couldn't
>> emit python bytecode to run on the 'python virtual machine' ?[/color][/color]

I was thinking that it may be simpler to say, write a PHP-to-Python
compiler which compiles PHP into an intermediate form, which is then
converted into python bytecodes, rather than trying to automate source
code conversion from PHP to Python. Well, PHP is just an off-hand
example, it may be COBOL or Pascal. Any ideas?

Regards
Maurice
[color=blue]
>
>
> It is certainly possible. Indeed, the Python compiler does generate
> Python bytecode from source code, so it must be possible :-)
>
> Regards,
> Martin
>[/color]

**Maurice LING** · Jul 18 '05, 02:51 PM

Re: compiling to python byte codes

Hi Leif,

Leif K-Brooks wrote:
[color=blue]
> Maurice LING wrote:
>[color=green]
>> Or is there any documentation or books that is the python equivalent
>> of "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]
>
>
> Python's byte code isn't very stable, so you might have to recreate your
> entire code base with every new Python version. I would suggest
> generating Python code (not byte code) instead and compiling that.[/color]

So are you suggesting that say I want to write a compiler to compile
Pascal to Python Virtual Machine, it will be wiser to do source code
conversion from Pascal to Python?

maurice

**Jason Lai** · Jul 18 '05, 02:51 PM

Re: compiling to python byte codes

Maurice LING wrote:[color=blue]
> Hi Leif,
>
> Leif K-Brooks wrote:
>[color=green]
>> Maurice LING wrote:
>>[color=darkred]
>>> Or is there any documentation or books that is the python equivalent
>>> of "Programmin g for the Java Virtual Machine" by Joshua Engel?[/color]
>>
>>
>>
>> Python's byte code isn't very stable, so you might have to recreate
>> your entire code base with every new Python version. I would suggest
>> generating Python code (not byte code) instead and compiling that.[/color]
>
>
> So are you suggesting that say I want to write a compiler to compile
> Pascal to Python Virtual Machine, it will be wiser to do source code
> conversion from Pascal to Python?
>
> maurice[/color]

What about generating an Abstract Syntax Tree (compiler.ast) and using
the compiler module (compiler.pycod egen) to write the bytecode?

**Jeremy Bowers** · Jul 18 '05, 02:51 PM

Re: compiling to python byte codes

On Thu, 02 Sep 2004 23:10:11 +0000, Maurice LING wrote:[color=blue]
> Can I feed a python source file into compile(), line by line, and expect
> it to generate a working .pyc file? I suppose my intended use is to be
> able to handle python codes written at run time, to execute python codes
> line by line, in a python program. It is somewhat like a tracer routine
> that can interpret a line of python code, read out the variables, before
> going to the next line of python code. Can compile() do this, or do I
> have to use pexpect to run an instance of python?[/color]

Why don't you clearly spell out your intended use and ask about that,
instead?

If, based on your use of "I suppose" and "somewhat", you are still unclear
on your intended use, figuring that out would be step #1. :-)

Many good modules exist for many things already; if you're trying to trace
for instance, there is a module for that. Let's start at the beginning:
What are you trying to do?

**Maurice LING** · Jul 18 '05, 02:51 PM

Re: compiling to python byte codes

Jeremy Bowers wrote:[color=blue]
> On Thu, 02 Sep 2004 23:10:11 +0000, Maurice LING wrote:
>[color=green]
>>Can I feed a python source file into compile(), line by line, and expect
>>it to generate a working .pyc file? I suppose my intended use is to be
>>able to handle python codes written at run time, to execute python codes
>>line by line, in a python program. It is somewhat like a tracer routine
>>that can interpret a line of python code, read out the variables, before
>>going to the next line of python code. Can compile() do this, or do I
>>have to use pexpect to run an instance of python?[/color]
>
>
> Why don't you clearly spell out your intended use and ask about that,
> instead?
>
> If, based on your use of "I suppose" and "somewhat", you are still unclear
> on your intended use, figuring that out would be step #1. :-)
>
> Many good modules exist for many things already; if you're trying to trace
> for instance, there is a module for that. Let's start at the beginning:
> What are you trying to do?
>[/color]

I am using SBML (system biology markup language) as a front-end
modelling language for my project. And for ease of further maintenance
of the model and interoperabilit y purposes, my project requires me to
convert it into an intermediate form (MA), which is somewhat assembly is
structure, as in, each instruction takes the form of <opcode>
<operand>*. Here I am, attempting to write a virtual machine that can
run MA, using python. So, it becomes a MA virtual machine running on
python virtual machine.

My concern is, is it simpler to convert MA to python codes or python
bytecodes. What are the pros and cons? Assuming that to convert to
python source code is a choice, I'm thinking that MA virtual machine can
then read a MA instruction and output the corresponding python source
codes, but are there facilities in python to run python codes, line by
line, as it is being thrown out by MA virtual machine?

As a side note, does anyone think that this project might be suitable
enough to apply for PSF Grant?

Thanks
Maurice

--
Maurice Han Tong LING, BSc(Hons)(MCB), AdvDipComp, SSN, FIFA
Doctor of Philosophy (Science) Candidate, The University of Melbourne
mobile: +61 4 22781753
+65 96669233
mailing address: Department of Zoology, The University of Melbourne
Royal Parade, Parkville, Victoria 3010, Australia
residential address: 9/41 Dover Street
Flemington, Victoria 3031, Australia
email: mauriceling@acm .org
resume: http://maurice.vodien.com/maurice_resume.pdf
www: http://www.geocities.com/beldin79/

The information contained in this message, including its attachment(s),
is CONFIDENTIAL and solely intended to its addressee(s) only. The
content of this message, including its attachment(s), may be subjected
to copyright and privacy laws. If you have received this email in error,
please let me know by returning this email, and then destroy all copies.

"I cannot discover anyone knows enough to say definitely what is
and what is not possible" -Henry Ford
"The difference between the impossible and the possible lies
in a person's determination" -Tommy Charles Lasorda

**Jeremy Bowers** · Jul 18 '05, 02:52 PM

Re: compiling to python byte codes

On Fri, 03 Sep 2004 00:48:55 +0000, Maurice LING wrote:[color=blue]
> I am using SBML (system biology markup language) as a front-end
> modelling language for my project. And for ease of further maintenance
> of the model and interoperabilit y purposes, my project requires me to
> convert it into an intermediate form (MA), which is somewhat assembly is
> structure, as in, each instruction takes the form of <opcode>
> <operand>*. Here I am, attempting to write a virtual machine that can
> run MA, using python. So, it becomes a MA virtual machine running on
> python virtual machine.[/color]

Hmmm, could you post an example of this assembly-like code? It might be
easiest to implement a Python interpreter directly; if the assembly-like
code is simple enough it isn't even worth a true parser.

Without knowing about your code, I can't be sure, but I would be surprised
if MA is similar enough to Python to make it worth running MA on the
Python machine directly.

Assembly language is right up there with LISP (without macros) in terms of
ease of parsing, if no opcode ever crosses multiple lines.

**Maurice LING** · Jul 18 '05, 02:52 PM

Re: compiling to python byte codes

Jeremy Bowers wrote:[color=blue]
>
> Hmmm, could you post an example of this assembly-like code? It might be
> easiest to implement a Python interpreter directly; if the assembly-like
> code is simple enough it isn't even worth a true parser.[/color]

What do you mean by implementing a Python interpreter directly? Sorry, I
am unable to provide an example of this assembly-like code. This is
currently still unpublished work, so I'm not able to disclose much,
especially in a public forum.
[color=blue]
> Without knowing about your code, I can't be sure, but I would be surprised
> if MA is similar enough to Python to make it worth running MA on the
> Python machine directly.[/color]

Do you think that there is very slight chance that it is worthwhile
converting MA directly to python bytecodes? This is how I read it.
Please tell me if I've misunderstood you.
[color=blue]
>
> Assembly language is right up there with LISP (without macros) in terms of
> ease of parsing, if no opcode ever crosses multiple lines.[/color]

Some parts of MA is still undergoing development and cleaning up but I
certainly do not see why any opcode should cross multiple lines. As far
as I can see, 70% of the opcodes are able to be represented by multiple
lines of python codes. I've not thought hard enough on this yet.

All I can say is that MA looks similar to any assembly is structure,
with directives.

Sorry that I am not able to disclose much, but hope to get some opinions
based on what I can say.

Thank you,
Maurice

**Leif K-Brooks** · Jul 18 '05, 02:52 PM

Re: compiling to python byte codes

Jason Lai wrote:
[talking about compiling some language besides Python to Python bytecode][color=blue]
> What about generating an Abstract Syntax Tree (compiler.ast) and using
> the compiler module (compiler.pycod egen) to write the bytecode?[/color]

That would certainly be possible, but it seems to me like it might be
easier to generate Python code. You're using Python logic if you use its
AST, after all.

**Jason Lai** · Jul 18 '05, 02:52 PM

Re: compiling to python byte codes

Maurice LING wrote:[color=blue]
>
> Some parts of MA is still undergoing development and cleaning up but I
> certainly do not see why any opcode should cross multiple lines. As far
> as I can see, 70% of the opcodes are able to be represented by multiple
> lines of python codes. I've not thought hard enough on this yet.
>
> All I can say is that MA looks similar to any assembly is structure,
> with directives.[/color]

Well, x86/PPC assembly operates on registers. Python uses a stack.
See http://docs.python.org/lib/bytecodes.html

If you're using registers, I guess you'd have to store the values in
variables, and load/store them through the stack whenever you do an
operation -- maybe with some optimization if you can keep the result on
the stack.

Python pretty much only lets you run a block of code at once (using exec
or eval). So if you compile it line by line on the fly, your VM would
have to ask Python to run each line, and take care of unstructured jumps
itself. Python doesn't really like arbitrary gotos anyway. I assume if
you were translating to Python code, you'd have to have the whole block
for if, while, etc, ahead of time. Or only jump backwards, since you
can't jump to something that hasn't been written yet.

- Jason

compiling to python byte codes

compiling to python byte codes

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment