Compiling

[Ravi Teja]

Re: Compiling

Actually optimizations are not what concern me. I am pretty happy with
Pyrex/Swig etc for that. What I want is the ability to make a native
DLL/SO. A whole lot easier to integrate/embed to other languages.

[Martin v. Löwis]

Re: Compiling

Ravi Teja wrote:[color=blue]
> This is a standard response to a rather frequent question here. But I
> am not sure I ever understood. Scheme / Lisp are about as dynamic as
> Python. Yet they have quite efficient native compilers. Ex: Bigloo
> Scheme.[/color]

You might be missing two details here:
1. those compilers are less efficient than you might think
2. Scheme/Lisp are indeed less dynamic than Python

Consider the following Scheme example:

(module a)

(define (add a b)
(+ a b))

(print (add 3.0 4))

Bigloo generates the following code (which I have cleaned up
quite a bit):

/* toplevel-init */
obj_t BGl_toplevelzd2 initzd2zzaz00()
{
AN_OBJECT;
obj_t BgL_arg1078z00_ 10;
BgL_arg1078z00_ 10 = BGl_addz00zzaz0 0(((double)3.0) , ((long)4));
obj_t BgL_list1080z00 _11;
BgL_list1080z00 _11 = MAKE_PAIR(BgL_a rg1078z00_10, BNIL);
return BGl_printz00zz_ _r4_output_6_10 _3z00(BgL_list1 080z00_11);
}

/* add */
obj_t BGl_addz00zzaz0 0(double BgL_az00_1, long BgL_bz00_2)
{
AN_OBJECT;
obj_t BgL_list1084z00 _12;
obj_t BgL_arg1087z00_ 13;
BgL_arg1087z00_ 13 = MAKE_PAIR(BINT( BgL_bz00_2), BNIL);
BgL_list1084z00 _12 = MAKE_PAIR(DOUBL E_TO_REAL(BgL_a z00_1),
BgL_arg1087z00_ 13);
return BGl_zb2zb2zz__r 4_numbers_6_5z0 0(BgL_list1084z 00_12);
}


You can see several things from that:
1. The compiler was not able to/did not chose to implement
the add operation using native C. Instead, it allocates
two cons cells, and one object for the double; it then
calls a generic implementation of +.
2. The compiler *did* infer that the procedure add is
always called with (double long). This is because I
didn't export it. If I exported the function, it
would become

obj_t BGl_zc3anonymou sza31077ze3z83z zaz00(obj_t BgL_envz00_16, obj_t
BgL_az00_17, obj_t BgL_bz00_18)
{
AN_OBJECT;
obj_t BgL_az00_8;obj_ t BgL_bz00_9;
BgL_az00_8 = BgL_az00_17;
BgL_bz00_9 = BgL_bz00_18;
obj_t BgL_list1079z00 _11;
obj_t BgL_arg1081z00_ 12;
BgL_arg1081z00_ 12 = MAKE_PAIR(BgL_b z00_9, BNIL);
BgL_list1079z00 _11 = MAKE_PAIR(BgL_a z00_8, BgL_arg1081z00_ 12);
return BGl_zb2zb2zz__r 4_numbers_6_5z0 0(BgL_list1079z 00_11);
}

In this case, all parameters are of type obj_t (which is a pointer
type).

Furthermore, looking at the actual implementation of 2+, it is
defined as

(define (2+ x y)
(2op + x y))

and then 2op is defined as

(define-macro (2op op x y)
(let ((opfx (symbol-append op 'fx))
(opfl (symbol-append op 'fl))
(opelong (symbol-append op 'elong))
(opllong (symbol-append op 'llong)))
`(cond
((fixnum? ,x)
(cond
((fixnum? ,y)
(,opfx ,x ,y))
((flonum? ,y)
(,opfl (fixnum->flonum ,x) ,y))
((elong? ,y)
(,opelong (fixnum->elong ,x) ,y))
((llong? y)
(,opllong (fixnum->llong ,x) ,y))
(else
(error ,op "not a number" ,y))))
; more cases enumerating all possible
; combinations of fixnum, flonum, elong, and llong

Now, compare this with Python:

- In the general case of the add definition, the code Bigloo
generates is roughly equivalent to the sequence of function calls
the Python interpreter performs. For Python,

def add(a,b):
return a+b

translates into

2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 BINARY_ADD
7 RETURN_VALUE
8 LOAD_CONST 0 (None)
11 RETURN_VALUE

The entire work is done in BINARY_ADD: It allocates a tuple with
the two arguments, then dispatches to the actual __add__
implementation. Compared to the code Bigloo generates, this might
be more efficient (a single memory allocation instead of two).

- the approach of collecting all implementations of + in a single
place of Bigloo cannot be transfered to Python. Python's
implementation of + is dynamically extensible.

- the approach of directly calling add() in the toplevel init
cannot be applied to Python, either. The meaning of the name
"add" can change between the time of definition and the actual
call. Therefore, the simple function call must go through a
table lookup.

So while it would be possible to apply the same strategy to
Python, it likely wouldn't gain any performance increase over
the interpreter.

Regards,
Martin

[Ravi Teja]

Re: Compiling

> So while it would be possible to apply the same strategy to[color=blue]
> Python, it likely wouldn't gain any performance increase over
> the interpreter.[/color]

Thanks,
That was quite illustrative. But as I posted elsewhere, I am looking at
the other advantages of native compilation rather than speed. Python's
ability to interface with C code is quite good and I am happy to simply
do the performance critical parts in it although I rarely end up
needing to do that. But more often I am looking to use Python libraries
in other languages since I am more familiar with them and they
typically tend to be more high level (the way I like it) than the
standard libraries of others.

Perhaps someone already has a tool that simplifies / automates creating
DLLs that embed and export Python functions as C prototypes. This was
discussed in Pyrex mailing list a while ago.

[Martin v. Löwis]

Re: Compiling

Ravi Teja wrote:[color=blue]
> But more often I am looking to use Python libraries
> in other languages since I am more familiar with them and they
> typically tend to be more high level (the way I like it) than the
> standard libraries of others.[/color]

Ah. In that case, the normal C API should work fine to call into Python
in most cases, no? If not, have you tried the CXX package, or
Boost::Python?

Regards,
Martin

[Ravi Teja]

Re: Compiling


Martin v. Löwis wrote:[color=blue]
> Ravi Teja wrote:[color=green]
> > But more often I am looking to use Python libraries
> > in other languages since I am more familiar with them and they
> > typically tend to be more high level (the way I like it) than the
> > standard libraries of others.[/color]
>
> Ah. In that case, the normal C API should work fine to call into Python
> in most cases, no? If not, have you tried the CXX package, or
> Boost::Python?[/color]

I am aware of those. But my "other" languages are not C/C++. Sure I
could first wrap them in a DLL written in C/C++ using CXX/Boost and
then call the DLL from the language in question. But that is more
trouble than it is worth.