Python syntax in Lisp and Scheme

[Terry Reedy]

Re: Python syntax in Lisp and Scheme


"Peter Seibel" <peter@javamonk ey.com> wrote in message
news:m34qyfibc8 .fsf@javamonkey .com...[color=blue]
> Note that it's the ability, at macro expansion time, to treat the[/color]
code[color=blue]
> as data that allows me to generate test failure messages that[/color]
contain[color=blue]
> the literal code of the test case *and* the value that it evaluated
> to. I could certainly write a HOF version of CHECK that accepts a[/color]
list[color=blue]
>of test-case-functions:[/color]
....[color=blue]
> But since each test case would be an opaque function object by the
> time CHECK sees it, there'd be no good option for nice reporting[/color]
from[color=blue]
> the test framework.[/color]

But can't you explicitly quote the test cases for input to the HOF and
eval them within the HOF, so you again have both the literal code and
value generated? Not as pretty, admittedly, and perhaps less
efficient, but workable?

Terry J. Reedy

[Pascal Costanza]

Re: Python syntax in Lisp and Scheme

Andrew Dalke wrote:
[color=blue]
> In retrospect I should have given a more obvious possibility.
> As some point I hope to have computer systems I can program
> by voice in English, as in "House? Could you wake me up
> at 7?" That is definitely a type of programming, but Lisp is
> a language designed for text, not speed.[/color]

I don't understand that last sentence. Could you clarify this a bit? You
don't want to say that there is an inherent dichotomy between text and
speed, do you?!?
[color=blue]
> Pascal Costanza:
>[color=green]
>>I believe it is an accepted fact that uniformity in GUI design is a good
>>thing because users don't need to learn arbitrarily different ways of
>>using different programs. You only need different ways of interaction
>>when a program actually requires it for its specific domain.[/color]
>
>
> My spreadsheet program looks different from my word processor
> looks different from my chemical structure editor looks different from
> my biosequence display program looks different from my image
> editor looks different from my MP3 player looks different from my
> email reader looks different from Return to Castle Wolfinstein ....
>
> There are a few bits of commonality; they can all open files. But
> not much more.[/color]

....but you probably know from the start where to find the menus, what
the shortcuts are for opening and saving files, how to find the online
help, and so forth.

Lisp also has this to a certain degree: It's always clear what
constitutes the meaning of an s-expression, namely its car, no matter
what language "paradigm" you are currently using.
[color=blue]
> Toss out the MP3 player and RtCW and there
> is more in common. Still, the phrase "practicali ty beats purity" is
> seems appropriate here.
>
>[color=green][color=darkred]
>>>I firmly believe people can in general easily handle much more
>>>complicate d syntax than Lisp has. There's plenty of room to
>>>spare in people's heads for this subject.[/color]
>>
>>Sure, but is it worth it?[/color]
>
>
> Do you have any doubt to my answer? :)[/color]

No, not really. :)
[color=blue][color=green]
>>Convenience is what matters. If you are able to conveniently express
>>solutions for hard problems, then you win. In the long run, it doesn't
>>matter much how things behave in the background, only at first.[/color]
>
>
> Personally, I would love to write equations on a screen like I
> would on paper, with integral signs, radicals, powers, etc. and
> not have to change my notation to meet the limitations of computer
> input systems.[/color]

I know people who have even started to use s-expression for mathematical
notation (on paper), because they find it more convenient.
[color=blue]
> For Lisp is a language tuned to keyboard input and not the full
> range of human expression. (As with speech.)[/color]

There is some research going on to extend Lisp even in this regard
(incorporating more ways of expression).
[color=blue]
> (I know, there are people who can write equations in TeX as
> fast as they can on paper. But I'm talking about lazy ol' me
> who wants the covenience.)
>
> Or, will there ever be a computer/robot combination I can
> teach to dance? Will I do so in Lisp?[/color]

?!?
[color=blue][color=green]
>>It seems to me that in Python, just as in most other languages, you
>>always have to be aware that you are dealing with classes and objects.
>>Why should one care? Why does the language force me to see that when it
>>really doesn't contribute to the solution?[/color]
>
>
> Hmmm.. Is the number '1' an object? Is a function an object?
> What about a module? A list? A class?
>
>[color=green][color=darkred]
>>>>print sum(range(100))[/color][/color]
>
> 4950
>
>
> Where in that example are you aware that you are dealing with classes
> and objects?[/color]

Well, maybe I am wrong. However, in a recent example, a unit test
expressed in Python apparently needed to say something like
"self.assertEqu al ...". Who is this "self", and what does it have to do
with testing? ;)
[color=blue][color=green][color=darkred]
>>>>If it's only a syntactical issue, then it's a safe bet that you can add
>>>>that to the language. Syntax is boring.
>>>
>>>Umm... Sure. C++ can be expressed as a parse tree, and that
>>>parse tree converted to an s-exp, which can be claimed to be
>>>a Lisp; perhaps with the right set of macros.[/color]
>>
>>That's computational equivalence, and that's not interesting.[/color]
>
>
> Which is why I didn't see the point of original statement. My
> conjecture is that additional syntax can make some things easier.
> That a problem can be solved without new syntax does not
> contradict my conjecture.[/color]

If additional syntax makes specific things easier, then in god's name
just add it! The loop macro in Common Lisp is an example of how you can
add syntax to make certain things easier. This is not rocket science.

The point here is that for most languages, if you want to add some
syntax, you have to change the definition of the language, extend the
grammar, write a parser, extended a compiler and/or interpreter, maybe
even the internal bytecode representation, have wars with other users of
the language whether it's a good idea to change the language that way,
and so forth. In Lisp, you just write a bunch of macros and you're done.
No problems with syntax except if you want them, most of the time no
problems with changes to the language (far less than in other
languages), no messing around with grammars and related tools, no need
to know about compiler/interpreter internals and internal
representation, no wars with other language users, and so forth.

Syntax is boring. ;)
[color=blue][color=green]
>>If it's a good Lisp library I would expect it to work like this:
>>
>>(with-allocation-from :shared-memory
>> ...)
>>
>>;)
>>
>>Any more questions?[/color]
>
>
> Yes. Got a URL for documentation on a Lisp providing access
> to shared memory?[/color]

OK, I am sorry that I have lost focus here. You have given this example
as one that shows what probably cannot not be done in Lisp out of the
box. However, most Lisp implementations provide a way to access native
code and in that way deal with specific features of the operating
system. And there is a de-facto standard for so-called foreign function
calls called UFFI that you can use if you are interested in a
considerable degree of portability.

I don't know a lot about the specifics of shared memory, so I can't
comment on your specific questions.
[color=blue]
> This service is straight-forward to support in C/C++. It
> sounds like for Lisp you are dependent on the implementation,
> in that if the implementation doesn't support access to its
> memory allocator/gc subsystem then it's very hard to
> write code for this hardware on your own. It may be
> possible to use an extension (written in C? ;) to read/write
> to that persistent memory using some sort of serialization,
> but that's the best you can do -- you don't have live objects
> running from nonvolatile store -- which is worse than C++.[/color]

This should be possible as a combination of a FFI/UFFI and the CLOS MOP.
AFAIK, you can define the memory layout and the allocation of memory for
specific metaclasses. However, I really don't know the details.

The paper at
http://www-db.stanford.edu/~paepcke/...ts/mopintro.ps might
be interesting. For UFFI, see http://uffi.b9.com/

As Paul Graham put it, yes, there is some advantage when you use the
language the operating system is developed in, or it ++.

Pascal

[Pascal Costanza]

Re: Python syntax in Lisp and Scheme

Terry Reedy wrote:
[color=blue]
> "Pascal Costanza" <costanza@web.d e> wrote in message
> news:bm9it8$6j5 $1@newsreader2. netcologne.de.. .
>[color=green]
>>Does Python allow local function definitions?[/color]
>
>
> Module level functions are local to the module unless imported by
> another module. Nested functions are local to the function they are
> nested within unless explicitly returned. Methods are local to
> classes and subclasses. Lambda expressions are very local unless
> somehow passed around.
>
> I am not sure which best meets your intention.
>
>[color=green]
>>Can they shadow predefined functions?[/color]
>
>
> Yes, named objects, including functions can (locally) shadow
> (override) builtins. It is considered a bad habit/practice unless
> done intentionally with a functional reason.[/color]

Well, this proves that Python has a language feature that is as
dangerous as many people seem to think macros are.

<irony>
What you say is that local function definitions can obscure the meaning
of the Python language and/or its standard library, and this has the
potential to split the language community and make it impossible to read
each other's code. Heck, you really can't rely on the fact that sum(...)
sums its arguments? This means that a local function definition is
really a dangerous language feature, isn't it? Shouldn't it better be
abandoned?
</irony>

That last paragraph sounds as non-sensical to your ears as the arguments
against the inclusion of macros into a language because of their
expressive power sound to our ears.

BTW, do you know the talk "Growing a Language" by Guy Steele? See
http://www.research.avayalabs.com/us...e-oopsla98.pdf - read
it, it's very insightful (even though it talks about Java. ;)

Pascal

[prunesquallor@comcast.net]

Re: Python syntax in Lisp and Scheme

"Andrew Dalke" <adalke@mindspr ing.com> writes:
[color=blue]
> prunesquallor@c omcast.net:[color=green]
>> The smartest programmers I know all prefer Lisp (in some form or
>> another). Given that they agree on very little else, that's saying
>> a lot.[/color]
>
> Guess you don't know Knuth.[/color]

Never met him.
[color=blue]
> The smartest people I know aren't programmers. What does
> that say?[/color]

You hang out with dumb programmers?

[Alex Martelli]

Re: Python syntax in Lisp and Scheme

Andrew Dalke wrote:
[color=blue]
> Pascal Costanza:[color=green]
>> [quantum programming][/color]
>
> While an interesting topic, it's something I'm not going to worry about.[/color]

Me neither, for now.
[color=blue]
> And if I did, it would be in Python ;)[/color]

I suspect no existing language would be anywhere near adequate.
But if any current programming concept could stretch there, it might
be that of "unrealized until looked-into set of things", as in, Haskell's
"lazy" (nonstrict) lists. Now lists are sequential and thus quantumly
inappropriate, but perhaps it's a start.
[color=blue]
> I bring it up as a counter-example to the idea that all modes of
> programming have been and can be explored in a current Lisp.
> I conjectured one interesting possibility -- that of handling ensembles
> of possible solutions to a given problem.[/color]

I suspect we may have to map the 'ensembles' down to sets of
items, just as we generally map concurrency down to sets of
sequential actions, in order to be able to reason about them (though
I have no proof of that conjecture). IF you have to map more
complicated intrinsics down to sequential, deterministic, univocal
"things", I'm sure you could do worse than Lisp. As to whether
that makes more sense than dreaming up completely different
languages having (e.g.) nondeterminism or multivocity as more
intrinsic concepts, I pass: it depends mostly on what human beings
will find they need to use in order to reason most effectively in
this new realm -- and quite likely different humans will find they
have different needs in this matter.

[color=blue]
> In retrospect I should have given a more obvious possibility.
> As some point I hope to have computer systems I can program
> by voice in English, as in "House? Could you wake me up
> at 7?" That is definitely a type of programming, but Lisp is[/color]

Yeah, well, I fear the answer will be yes (it could), but it won't
do so since you haven't _asked_ it to wake you up, only if it
could. ME, I definitely don't want to use natural language with
all of its ambiguity for anything exept communicating with
other human beings, thankyouverymuc h.
[color=blue]
> a language designed for text, not speed.[/color]

*blink* what does THAT doubtful assertion have to do with anything
else we were discussing just now...? I think lisp was designed for
lists (as opposed to, say, snobol, which WAS "designed for text") and
that they're a general enough data structure (and supplemented in
today's lisps with other good data structures) that they'll be quite good
for all kinds of 'normal' (deterministic &c) programming. As for speed,
I'm sure it's easier to get it out of lisp than out of python right now.
So what's your point, and its relation to the above...?

[color=blue]
> Pascal Costanza:[color=green]
>> I believe it is an accepted fact that uniformity in GUI design is a good
>> thing because users don't need to learn arbitrarily different ways of
>> using different programs. You only need different ways of interaction
>> when a program actually requires it for its specific domain.[/color][/color]

Yes, I agree this IS generally accepted (with, of course, some dissenters,
but in a minority).
[color=blue]
> My spreadsheet program looks different from my word processor[/color]

Sure, so do mine, but their menus are quite similar -- in as much as
it makes sense for them to have similar operations -- and ditto ditto
for their toolbars, keyboard shortcuts, etc etc. I.e. the differences
only come "when needed for a specific domain" just as Pascal just
said. So I don't know what you're intending with this answer.
[color=blue]
> is more in common. Still, the phrase "practicali ty beats purity" is
> seems appropriate here.[/color]

Uniformity is more practical than diversity: e.g. ctrl-c as the Copy
operation everywhere means my fingers, even more than my brain, get
used to it. If you assign ctrl-c to some totally different operation in
your gui app "because you think it's more practical" you're gonna
drive me crazy, assuming I have to use your app. (That already
happens to me with the -- gnome based i think -- programs using
ctrl-z for minimize instead of undo -- I'm starting to have frayed
nerves about that even for GVIM, one of the programs I use most
often...:-).
[color=blue][color=green][color=darkred]
>> > I firmly believe people can in general easily handle much more
>> > complicated syntax than Lisp has. There's plenty of room to
>> > spare in people's heads for this subject.[/color]
>>
>> Sure, but is it worth it?[/color]
>
> Do you have any doubt to my answer? :)[/color]

Given the difficulty I'm having understanding your stance[s] in
this post, I do. My own answer would be that syntax sugar is
in people's head anyway because of different contexts -- infix
arithmetic taught since primary school, indentation in outlines
and pseudocode, etc etc -- so, following that already-ingrained
set of conventions takes no "room to spare in people's heads" --
indeed, the contrary -- it saves them effort. If people's head
started as "tabula rasa" it might be different, but they don't, so
that's a moot issue.

That much being said, I _do_ like prefix syntax. In some cases
I need to sum a+b+c+d and repeating that silly plus rather than
writing (+ a b c d) grates. Or I need to check a<b<c<d and
again I wish I could more summarily write (< a b c d). When I
designed my own language for bridge-hands evaluation, BBL, I
used prefix notation, though in the form operator ( operands )
[which I thought would have been easier for other bridge players
to use], e.g.:

& ( # weak NT opener requires AND of two things:
s ( 1 g 4 3 3 3 # shape 4333 (any), or
2 g 4 4 3 2 # 4432 (any), or
3 3- 3- 3- 5 # 5332 with 5 clubs, or
4 3- 3- 5 3- # 5332 with 5 diamonds
)
< ( 12 # as well as, 13-15 range for
\+ SHDC c( 4 3 2 1 0) # normal Milton-Work pointcount
16
)
)

Maybe readers are starting to understand why I don't WANT to
use a language I design myself;-). Anyway, the language was
NOT enthusiasticall y taken up, until I wrote code generators with
a GUI accepting conditions in more "ordinary looking" notations
and building this, ahem, intrinsically "better" one;-) -- then, but only
then, did other players start using this to customize hand generators
and the like. (Yes, I did have macros, but puny enough that they
still required operator(operan ds) syntax -- they basically served only
to reduce duplication, or provide some little abstraction, not to
drastically change the language syntax at all). Ah well -- maybe I
should just put the BBL (Turbo Pascal) implementation and (Italian-
language) reference manual online -- it still moves nostalgia in me!-)

[color=blue][color=green]
>> Convenience is what matters. If you are able to conveniently express
>> solutions for hard problems, then you win. In the long run, it doesn't[/color][/color]

My APL experience tells me this is false: conveniently expressing
solutions is HALF the problem -- you (and others!) have to be
able to read them back and maintain and alter them later too.
[color=blue][color=green]
>> matter much how things behave in the background, only at first.[/color]
>
> Personally, I would love to write equations on a screen like I
> would on paper, with integral signs, radicals, powers, etc. and
> not have to change my notation to meet the limitations of computer
> input systems.[/color]

So jot your equations on a tablet-screen and look for a good
enriched text recognition system. What's programming gotta
do with it?
[color=blue]
> For Lisp is a language tuned to keyboard input and not the full
> range of human expression. (As with speech.)[/color]

Python even more so on the output side -- try getting a screen-reader to
do a halfway decent job with it. But what does this matter here?

[color=blue]
> (I know, there are people who can write equations in TeX as
> fast as they can on paper. But I'm talking about lazy ol' me
> who wants the covenience.)
>
> Or, will there ever be a computer/robot combination I can
> teach to dance? Will I do so in Lisp?[/color]

You may want to teach by showing and having the computer
infer more general rules from example. Whether the inference
engine will be best built in lisp, prolog, ocaml, mozart, whatever,
I dunno. I don't think it will be optimally done in Python, though.
"Horses for courses" is my philosophy in programming.

[color=blue][color=green]
>> It seems to me that in Python, just as in most other languages, you
>> always have to be aware that you are dealing with classes and objects.[/color][/color]

Given the "everything is an object" (classes included) and every object
belongs to a class, you could indeed say that -- in much the same sense
as you may be said to always be aware that you're breathing air in
everyday life. Such awareness is typically very subconscious, of course.
[color=blue][color=green]
>> Why should one care? Why does the language force me to see that when it
>> really doesn't contribute to the solution?[/color][/color]

I'm not sure in what sense "python forces you to see" that, e.g.,
the number 2 is an object -- or how can that fail to "contribute to
the solution". Care to exemplify?
[color=blue]
> Hmmm.. Is the number '1' an object? Is a function an object?
> What about a module? A list? A class?[/color]

Yes to all of the above, in Python. I don't get your point.
[color=blue][color=green][color=darkred]
>>>> print sum(range(100))[/color][/color]
> 4950
>
> Where in that example are you aware that you are dealing with classes
> and objects?[/color]

Me? At every step -- I know 'sum' names a builtin object that is a
function (belonging to the class of builtin functions) taking one argument
which is a sequence, 'range' names another builtin object returning
a list object, etc. I'm not directly dealing with any of their classes --
I know they belong to classes, like any object does, but I have no need
to think about them in this specific statement (in fact, I hardly ever do;
signature-based polymorphism is what I usually care about, not class
membership, far more often than not).

But I don't get your points -- neither Andrew's nor Pascal's. How does
this differ from the awareness I might have in some macro-enhanced
lisp where I would type (print (sum (range 100))) or the like?
[color=blue]
> conjecture is that additional syntax can make some things easier.
> That a problem can be solved without new syntax does not
> contradict my conjecture.[/color]

But even if we provisionally concede your conjecture we are still
left wondering: is the degree of easing so high that it overcomes
the inevitable increase in complication, needed for a language to
have N+1 syntax forms where previously it only had N? I.e., it's
in general a difficult engineering tradeoff, like many in language
design -- which is why I'd rather delegate the decisions on these
tradeoffs to individuals, groups and processes with a proven track
record for making a lot of them with complexive results that I find
delightful, rather than disperse them to myself & many others
(creating lots of not-quite-congruent language dialects).


Alex

[prunesquallor@comcast.net]

Re: Python syntax in Lisp and Scheme

"Andrew Dalke" <adalke@mindspr ing.com> writes:
[color=blue]
> Pascal Costanza:[color=green]
>> [quantum programming][/color]
>
> While an interesting topic, it's something I'm not going to worry about.
> And if I did, it would be in Python ;)
>
> I bring it up as a counter-example to the idea that all modes of
> programming have been and can be explored in a current Lisp.
> I conjectured one interesting possibility -- that of handling ensembles
> of possible solutions to a given problem.[/color]

Oops, try again.

http://hampshire.edu/lspector/qgame.html

http://www.het.brown.edu/people/andre/qlambda/

http://mitpress.mit.edu/sicp/full-text/sicp/book/node88.html

[color=blue]
> In retrospect I should have given a more obvious possibility.
> As some point I hope to have computer systems I can program
> by voice in English, as in "House? Could you wake me up
> at 7?" That is definitely a type of programming, but Lisp is
> a language designed for text, not speed.[/color]

Oops, try again.

http://www.hpl.hp.com/techreports/94/HPL-94-30.html

http://dynamo.ecn.purdue.edu/~qobi/software.html

http://citeseer.nj.nec.com/siskind93screamer.html

If you were to select a language that has been used for *more* different
kinds of programming paradigms, you'd be hard pressed to find something
better than lisp.
[color=blue]
> Personally, I would love to write equations on a screen like I
> would on paper, with integral signs, radicals, powers, etc. and
> not have to change my notation to meet the limitations of computer
> input systems.
>
> For Lisp is a language tuned to keyboard input and not the full
> range of human expression. (As with speech.)[/color]

Oops, it turns out that Lisp is used for handwriting recognition
in the banking industry. I've heard rumors of Lisp being used
for voice recognition.
[color=blue]
> Or, will there ever be a computer/robot combination I can
> teach to dance? Will I do so in Lisp?[/color]

You want to teach a robot to dance? I would prefer a cute woman,
myself. Suit yourself, but you can use Lisp:

http://citeseer.nj.nec.com/lee95programming.html

[color=blue]
> Yes. Got a URL for documentation on a Lisp providing access
> to shared memory? My guess is that the Lisp runtime needs
> to be told about the arenas and that the multiple instances of
> Lisp sharing the arena must use some extra IPC to handle
> the distributed gc.[/color]

http://www.double.co.nz/creatures/developer/sharedmemory.htm

[color=blue]
> It gets worse if program X forks copies Y and Z, with shared
> memory XY between X and Y (but not Z) and XZ between
> X and Z (but not Y). X needs to be very careful on which
> data is copied, and it isn't immediately obvious what happens
> when some object from XZ is inserted into a list accessible
> to Y via XY.[/color]

Actually, it is obvious with a little thought. Objects sharable
between X and Y must reside in the XY address space. Objects
sharable between X and Z must reside in the XZ address space.

Since neither Y nor Z share their entire address space with X, there
exist outgoing edges in the heap of Y that do not refer to objects
within Y. Presumably the author of the system thought of that.
(If not, he's in trouble before Z even exists.) An object common
with X and Z is as opaque to Y as any other object in X.
[color=blue]
> Consider also a "persistent memory" server running in C
> (or hardware access to some sort of non-volatile memory.)
> You can use standard IPC to get an initially zeroed memory
> block and are free to use that memory without restrictions.
> It's persistent after program exit so when the program restarts
> it can reconnect to shared memory and get the data as it
> was at exit.
>
> This service is straight-forward to support in C/C++. It
> sounds like for Lisp you are dependent on the implementation,
> in that if the implementation doesn't support access to its
> memory allocator/gc subsystem then it's very hard to
> write code for this hardware on your own.[/color]

Why do you think this? The api would be straighforward. a
shared persistent memory would at least have these calls:

allocate, which takes an allocation amount and returns
some name for the object allocated,

retrieve, which takes some name for an object and recovers
the object itself (for use upon restarting)

dereference, which takes an object and a field and returns
or assigns the field.

An obvious way to integrate this into lisp is to make a
displaced array.

[Alexander Schmolck]

Re: Python syntax in Lisp and Scheme

"Andrew Dalke" <adalke@mindspr ing.com> writes:[color=blue]
> The smartest people I know aren't programmers. What does
> that say?[/color]

I think this is vital point. CL's inaccessibility is painted as a feature of
CL by many c.l.l denizens (keeps the unwashed masses out), but IMO the CL
community stunts and starves itself intellectually big time because CL is (I
strongly suspect) an *extremely* unattractive language for smart people
(unless they happen to be computer geeks).

Apart from the fact that this yields a positive feedback loop, I'd think that
even the smart computer geeks are likely to suffer from this incestuousness in
the midrun.

'as

[Peter Seibel]

Re: Python syntax in Lisp and Scheme

"Terry Reedy" <tjreedy@udel.e du> writes:
[color=blue]
> "Peter Seibel" <peter@javamonk ey.com> wrote in message
> news:m34qyfibc8 .fsf@javamonkey .com...[/color]
[color=blue][color=green]
> > Note that it's the ability, at macro expansion time, to treat the
> > code as data that allows me to generate test failure messages that
> > contain the literal code of the test case *and* the value that it
> > evaluated to. I could certainly write a HOF version of CHECK that
> > accepts a list of test-case-functions:[/color]
> ...[color=green]
> > But since each test case would be an opaque function object by the
> > time CHECK sees it, there'd be no good option for nice reporting
> > from the test framework.[/color]
>
> But can't you explicitly quote the test cases for input to the HOF and
> eval them within the HOF, so you again have both the literal code and
> value generated? Not as pretty, admittedly, and perhaps less
> efficient, but workable?[/color]

Well, if you eval the test cases, they are evaled in what's known as
the "null" lexical environment. So if you do something like:

(let ((x 10))
(check '(= (foo x) (* 2 x))))

where check is a function that evals its argument, rather than a
macro, then the evaluation will not be able to "see" the local binding
of the variable x so is unlikely to do what you think.

Python's eval--as I understand it--handles this differently. Common
Lisp's EVAL may be the way it is partially because it is not needed
for things like this given the existence of macros. There's are also
some semantic difficulties of what lexical environment the EVAL should
occur in. Given that the quoted expression above is just a piece of
data there's no particular way to attach the lexical environment to it
so that a subsequent EVAL can use it.

-Peter

--
Peter Seibel peter@javamonke y.com

Lisp is the red pill. -- John Fraser, comp.lang.lisp

[Alexander Schmolck]

Re: Python syntax in Lisp and Scheme

Peter Seibel <peter@javamonk ey.com> writes:
[color=blue]
> If for some reason you believe that macros will have a different
> effect--perhaps decreasing simplicity, clarity, and directness then
> I'm not surprised you disapprove of them. But I'm not sure why you'd
> think they have that effect.[/color]

Well, maybe he's seen things like IF*, MVB, RECEIVE, AIF, (or as far as
simplicity is concerned LOOP)...?

I'm not saying that macros always have ill-effects, but the actual examples
above demonstrate that they *are* clearly used to by people to create
idiosyncratic versions of standard functionality. Do you really think clarity,
interoperabilit y or expressiveness is served if person A writes
MULTIPLE-VALUE-BIND, person B MVB and person C RECEIVE?
[color=blue]
> (deftest foo-tests ()
> (check
> (= (foo 1 2 3) 42)
> (= (foo 4 5 6) 99)))
>
> Note that this is all about the problem domain, namely testing.[/color]

I think the example isn't a bad one, in principle, in practice however I guess
you could handle this superiorly in python.

I develop my testing code like this:

# like python's unittest.TestCa se, only that it doesn't "disarm"
# exceptions
TestCase = awmstest.Permea bleTestCase
#TestCase = unittest.TestCa se

class BarTest(TestCas e):
...
def test_foos(self) :
assert foo(1,2,3) = 42
assert foo(4,5,6) = 99

Now if you run this in emacs/ipython with '@pdb on' a failure will raise an
Exception, the debugger is entered and emacs automatically will jump to the
right source file and line of code (I am not mistaken in thinking that you
can't achieve this using emacs/CL, right?) and I can interactively inspect the
stackframes and objects that were involved in the failure.

I find this *very* handy (much handier than having the wrong result printed
out, because in many cases I'm dealing with objects such as large arrays wich
are not easily visualized).

Once the code and test code works I can easily switch to mere reporting
behavior (as described by andrew dalke) by uncommenting unittest.TestCa se back
in.


'as

[Peter Seibel]

Re: Python syntax in Lisp and Scheme

"Andrew Dalke" <adalke@mindspr ing.com> writes:
[color=blue]
> Peter Seibel:[color=green]
> >So, to write a new test function, here's what I
> > write:
> >
> > (deftest foo-tests ()
> > (check
> > (= (foo 1 2 3) 42)
> > (= (foo 4 5 6) 99)))[/color]
>
> Python bases its unit tests on introspection. Including the
> full scaffolding, the equivalent for Python would be
>
> import unittest
> import foo_module # I'm assuming 'foo' is in some other module
>
> class FooTestCase(uni ttest.TestCase) :
> def testFoo(self):
> self.assertEqua ls(foo_module.f oo(1, 2, 3), 42)
> self.assertEqua ls(foo_module.f oo(4, 5, 6), 99)
>
> if __name__ == '__main__':
> unittest.main()
>[/color]

Yup. I'd certainly be loath to claim that there's anything that *only*
be done using macros. And it *is* the case that dynamic and
introspective features make lots of things that might otherwise be
done with macros less painful.

So here's another example of using macros to define a domain specific
language. In this case the domain is lexing and parsing. In this case
I wrote a parser generator (like ANTLR in Java on YACC in C) by
defining the macros DEFPROD, DEFCHARTYPE, and DEFLEXE macros that
allow me to define the gramatical rules for a lexer in an s-expression
notation and then stitch them together into a procedure that
implements a tokenizer based on the grammar rules. The notation may
look a bit funny if you're not used to Lisp syntax but you can
probably observe that what I have hear is pretty much a translation
from the BNF in the Java language standard to my own s-exp notation.
(I'll give just a sample of the production rules, the rest are
similar. The code required to implement the lexer proper is 91 lines
of actual code, not counting a preprocessor function that translates
Java's special unicode escape syntax.)

In other languages parser generators usually have to define their own
grammar language and write a program that parsers *that* format in
order to generate the parsing code. Macros let me do the same thing
with much less work because Lisp does parsing for me--what follows
*is* a Lisp program given that I've defined the macros it uses.

As with DEFTEST, the point is not that this is the only way to do it
but rather that macros give me a way to concisely and directly express
the stuff I *care* about (i.e. what is the grammar I'm trying to
parse) while abstracting away the mechanims by which it gets
translated into efficient code that actually does the work.


(A sample of the productions from the code that generates a Java lexer.)

;; 3.4 Line terminators

(defprod line-terminator () (/ #\newline (#\return (? #\newline))))

(defchartype input-character
'(and character (not (member #\newline #\return))))

;; 3.5 Input Elements and Tokens

(defprod input () ((* input-element) (? #\Sub)))

(defprod input-element () (/ white-space comment token))

(defprod token () (/ identifier java-keyword literal separator operator))

;; 3.6 White space

(defprod white-space () (/ #\space #\tab #\page line-terminator))

;; 3.9 Keywords
(defprod java-keyword ()
(/
"abstract" "boolean" "break" "byte" "case" "catch" "char" "class" "const"
"continue" "default" ("do" (? "uble")) "else" "extends" ("final" (? "ly"))
"float" "for" "goto" "if" "implements " "import" "instanceof "
("int" (? "erface")) "long" "native" "new" "package" "private" "protected"
"public" "return" "short" "static" "strictfp" "super" "switch"
"synchroniz ed" "this" ("throw" (? "s")) "transient" "try" "void"
"volatile" "while"))

;; 3.10.2 Floating-Point Literals

(defprod floating-point-literal ()
(/
((+ digit)
(/
(#\. (* digit) (? exponent-part) (? float-type-suffix))
(exponent-part (? float-type-suffix))
float-type-suffix))
(#\. (+ digit) (? exponent-part) (? float-type-suffix))))

(defprod exponent-part () (exponent-indicator signed-integer))
(defchartype exponent-indicator '(member #\e #\E))
(defprod signed-integer () ((? sign) (+ digit)))
(defchartype sign '(member #\+ #\-))
(defchartype float-type-suffix '(member #\f #\F #\d #\D))

;; 3.12 Operators

(defprod operator ()
(/
":"
"?"
"~"
("!" (? "="))
("%" (? "="))
("&" (? (/ "=" "&")))
("*" (? "="))
("+" (? (/ "=" "+")))
("-" (? (/ "=" "-")))
("/" (? "="))
("<" (? "<") (? "="))
("=" (? "="))
(">" (? ">" (? ">")) (? "="))
("^" (? "="))
("|" (? (/ "=" "|")))))


;; This macro actually expands into a fuction java-lexer that takes
;; a string of text and tokenizes it according to the rules defined
;; above, starting from the "input" rule and returning as tokens
;; objects that represent identifiers, java-keywords, literals,
;; separators, and operators. Comments and whitespace are, thus,
;; discarded.

(deflexer java-lexer
((:start-rule input)
(:tokens identifier java-keyword literal separator operator)))
[color=blue]
> The Python code is more verbose in that regard because == isn't a
> way to write a function. I assume you also have tests for things
> like "should throw exception of type X" and "should not throw
> expection" and "floating point within epsilon of expected value"?[/color]

Sure, you can put any boolean expression in a check and have it
treated as a test case. The macro code can figure out whether it's a
function call and if it is arranges to evaluate the arguments itself
so it can see what their values are and then passes the values to the
function. Because all the comparators such as =, EQL (object
equality), <, >, string<, etc. are functions, this works great.

[snip]
[color=blue]
> I expect the usefulness of showing the full expression to be smaller
> when the expression is large, because it could be an intermediate in
> the expression which has the problem, and you don't display those
> intermediates.[/color]

Yeah. Though that's just because I didn't get around to implementing
that. As long as the expression consists of a tree of function calls,
I could do the same thing to sub-expressions I do to the top-level
expressions in the CHECK, and display them as well. If I really wanted
I could write a GUI to browse through the whole tree of function
calls. But I decided that the 80/20 rule applies and if I can't figure
out from the top-level expressions what's going on then I can always
resort to normal debugging. (Also the test framework can dump you into
the debugger at the point of a test failure so you can poke around
with the debugger to see any values you care about.)
[color=blue][color=green]
> > So what is the equivalent non-macro code? Well the equivalent code
> > to the DEFTEST form (i.e. the macro expansion) is not *that* much
> > more complex--it just has to do the stuff I mentioned; binding the
> > test name variable and registering the test function. But it's
> > complex enough that I sure wouldn't want to have to type it over
> > and over again each time I write a test:[/color]
>
> Python's introspection approach works by looking for classes of a
> given type (yes, classes, not instances), then looking for methods
> in that class which have a given prefix. These methods become the
> test cases. I imagine Lisp could work the same way, except that
> because other solutions exist (like macros), there's a prefered
> reason to choose another style.[/color]

The other difference is that runtime introspection happens at runtime.
That's probably fine for a test framework. I've written several Java
test frameworks that use reflection in similar ways so I know about
that approach too. But for other tasks, such as my parsing example
above, even if they could be done using introspection, there's a big
advantage for doing a lot of the work once, at compile time.

[snip]
[color=blue]
> A solution which would get what you want without macros is the
> addition of more parse tree information, like the start/end
> positions of each expression. In that way the function could look up
> the stack, find the context from which it was called, then get the
> full text of the call. This gets at the code "from the other
> direction", that is, from looking at the code after it was parsed
> rather than before.[/color]

Heh. That's just a sort of, pardon my French, kludgy reimplementatio n
of macros. Macros are nothing more (or less) than a mechanism whereby
you can--at compile time--get a hold of the parse tree in a form that
your code can manipulate it to generate other code. (Your's is still
at runtime which makes it somewhat less useful, though it does work
for the test framework case.)
[color=blue]
> I'll let you decide if Lisp's introspection abilities provide an
> alternate non-macro way to handle building test cases which is just
> as short.[/color]

Sure. I could use Lisp's meta object protocol (MOP) to define all
sorts of crazy things. But I'd probably still end up wrapping them in
some nice syntactic abstractions (macros) to make them as expressive
as possible. The thing is, I'm not looking for a "non-macro way" to do
these things because macros (in Lisp) are no more problematic than
functions--they're just another way of definining abstractions. Like
any mechanism for defining abstractions they need to be used with good
design sense because bad abstractions can be worse than no
abstractions at all--at least if things are concrete you can see how
they work, even if you do have to wade through pages of code to see
it. But good abstractions--whether functions, classes, or macros--make
it even easier to understand the code.

-Peter

P.S. Pythonistas--I was originally following this thread in c.l.lisp.
Somewhere along the lines the follow-ups got set to c.l.python which
is fine with me as the Lisp guys (should) already know this. But feel
free let me know if you want me to shut up about this.

--
Peter Seibel peter@javamonke y.com

Lisp is the red pill. -- John Fraser, comp.lang.lisp

[Alex Martelli]

Re: Python syntax in Lisp and Scheme

Pascal Costanza wrote:
...[color=blue][color=green][color=darkred]
>>>Does Python allow local function definitions?[/color][/color][/color]
...[color=blue][color=green][color=darkred]
>>>Can they shadow predefined functions?[/color][/color][/color]
...[color=blue][color=green]
>> Yes, named objects, including functions can (locally) shadow
>> (override) builtins. It is considered a bad habit/practice unless
>> done intentionally with a functional reason.[/color]
>
> Well, this proves that Python has a language feature that is as
> dangerous as many people seem to think macros are.[/color]

Indeed, a chorus of "don't do that" is the typical comment each
and every time a newbie falls into that particular mis-use. Currently,
the --shadow option of PyChecker only warns about shadowing of
_variables_, not shadowing of _functions_, but there's really no
reason why it shouldn't warn about both. Logilab's pylint does
diagnose "redefining built-in" with a warning (I think they mean
_shadowing_, not actually _redefining_, but this may be an issue
of preferred usage of terms).

"Nailing down" built-ins (at first with a built-in warning for overriding
them, later in stronger ways -- slowly and gradually, like always, to
maintain backwards compatibility and allow slow, gradual migration of the
large existing codebase) is under active consideration for the next version
of Python, expected (roughly -- no firm plans yet) in early 2005.

So, yes, Python is not perfect today (or else, we wouldn't be planning a
2.4 release...:-). While it never went out of its way to give the user "as
much rope as needed to shoot oneself in the foot", neither did it ever
spend enormous energy in trying to help the user avoid many possible errors
and dubious usage. Such tools as PyChecker and pylint are a start, and
some of their functionality should eventually be folded back into the
core, just as tabnanny's was in the past with the -t switch. I don't think
the fundamental Python will ever nag you for missing comments or
docstrings, too-short names, etc, the way pylint does by default (at
least, I sure hope not...!-), but there's quite a bit I _would_ like to have
it do in terms of warnings and, eventually, error messages for
"feechurs" that only exist because it was once simple to allow than
to forbid them, not by a deliberate design decision to have them there.

Note that SOME built-ins exist SPECIFICALLY for the purpose of
letting you override them. Consider, for example, __import__ -- this
built-in function just exposes the inner mechanics of the import
statement (and friends) to let you get modules from some other
place (e.g., when your program must run off a relational database
rather than off a filesystem). In other word, it's a rudimentary hook
in a "Template Method" design pattern (it's also occasionally handy
to let you import a module whose name is in a string, without
going to the bother of an 'exec', so it will surely stay for that purpose
even though we now have a shiny brand-new architecture for
import hooks -- but that's another story). Having a single hook of
global effect has all the usual downsides, of course (which is exactly
why we DO have that new architecture;-): two or more complicated
packages doing import-hooks can't necessarily coexist within the
same Python application program (the only saving grace which let
us live with that simplistic hook for so many years is that importing
from strange places is typically a need of a certain deployment of
an overall application, not of a package -- still, such packages DO
exist, so the previous solution was far from perfect).

Anyway, back to your contention: I do not think that the fact that
the user can, within his functions, choose very debatable names,
such as those which shadow built-ins, is anywhere as powerful,
and therefore as dangerous, as macros. My own functions using
'sum' will get the built-in one even if yours do weird things with
that same name as a local variable of their own. The downsides
of shadowing are essentially as follows...

a newbie posts some fragment of his code asking for guidance,
and among other things that fragment has
for i in range(length(th enumbers)):
total = total + thenumbers[i]
he will receive many suggestions on how to make it better,
including the ideal one:
total = sum(thenumbers, total)
But then he tries it out and reports "it breaks" (newbies rarely
are clueful enough to just copy and paste error messages). And
we all waste lots of time finding out that this is because... the
hapless newbie had named HIS OWN FUNCTION 'sum', so
this was causing runaway recursion. Having met similar issues
over and over, one starts to warn newbies against shadowing
and get sympathetic with the idea of forbidding it:-).

That doesn't really compare to an extra feature in the language
that is deliberately designed to let reasonably-clueful users do
their thing, isn't deprecated nor warned against by anybody at
all (with a few isolated voices speaking about "abuse" of macros
in this thread, but still with an appreciation for macros when
_well_ used), and is MEANT to do what newbies _accidentally_
do with shadowing & much more besides;-).


Alex

[Andrew Dalke]

Re: Python syntax in Lisp and Scheme

Alex:[color=blue]
> Yeah, well, I fear the answer will be yes (it could), but it won't
> do so since you haven't _asked_ it to wake you up, only if it
> could.[/color]

Pshaw. My hypothetical house of the 2050s or so will know
that "could" in this context is a command. :)
[color=blue]
> ME, I definitely don't want to use natural language with
> all of its ambiguity for anything exept communicating with
> other human beings, thankyouverymuc h.[/color]

But what if computers someday become equally capable
as humans in understanding uncontrained speech? It
can be a dream, yes?
[color=blue][color=green]
> > a language designed for text, not speed.[/color]
>
> *blink* what does THAT doubtful assertion have to do with anything
> else we were discussing just now...?[/color]

An unfortunate typo. I meant "speech" instead of "speed" but
my fingers are too used to typing the latter. Here I would like
a computer to ask "um, did you really mean that?" -- so long as
the false positive rate was low enough.
[color=blue][color=green]
> > My spreadsheet program looks different from my word processor[/color]
>
> Sure, so do mine, but their menus are quite similar --[/color]
...[color=blue]
> So I don't know what you're intending with this answer.[/color]

Loosing ... focus ... mind .. numb .. must stop answering thread.

It's a knee-jerk reaction and an example that I'm no longer
thinking before I start to reply.
[color=blue][color=green]
> > For Lisp is a language tuned to keyboard input and not the full
> > range of human expression. (As with speech.)[/color]
>
> Python even more so on the output side -- try getting a screen-reader to
> do a halfway decent job with it. But what does this matter here?[/color]

The conjecture that computer programming languages are
contrained by the form of I/O and that other languages, based
on speech, free-form 2D writing, or other forms of input may
be more appropriate, at least for some domain.

This was in response to the idea that Lisp is the most appropriate
language for all forms of programming.
[color=blue][color=green]
> > Hmmm.. Is the number '1' an object? Is a function an object?
> > What about a module? A list? A class?[/color]
>
> Yes to all of the above, in Python. I don't get your point.[/color]

It was a rhetorical set of questions, to see what Pascal meant
about all the time being aware that we are dealing with classes
and object. When I wrote it, I didn't see the ambiguity that I
could be pointing out that these aren't classes/objects; in part
because I just didn't think about that alternative.
[color=blue]
> But I don't get your points -- neither Andrew's nor Pascal's. How does
> this differ from the awareness I might have in some macro-enhanced
> lisp where I would type (print (sum (range 100))) or the like?[/color]

That was my point.
[color=blue]
> But even if we provisionally concede your conjecture we are still
> left wondering: is the degree of easing so high that it overcomes
> the inevitable increase in complication, needed for a language to
> have N+1 syntax forms where previously it only had N?[/color]

Good point. And as I no longer feel like following up on this
thread, I'll leave it at that.

Andrew
dalke@dalkescie ntific.com

[Paolo Amoroso]

Re: Python syntax in Lisp and Scheme

Andrew Dalke writes:
[color=blue]
> Still doesn't answer my question on how nicely Lisp handles
> the 'unexpected' need of allocating objects from different
> memory arenas.[/color]

If I understand things correctly, ITA Software's Orbitz system does
that. See the slides of Rodney Daughtrey's ILC 2002 talk "ITA Software
and Orbitz: Lisp in the Online Travel World" (International Lisp
Conference 2002 Proceedings, page 606).

A toy example is contained in Paul Graham's book "ANSI Common
Lisp". See section 13.5 "Example: Pools" on page 226.


Paolo
--
Paolo Amoroso <amoroso@mclink .it>

[Raffael Cavallaro]

Re: Python syntax in Lisp and Scheme

Pascal Costanza <costanza@web.d e> wrote in message news:<bm9g9m$28 n$1@newsreader2 .netcologne.de> ...
[color=blue]
> Lispniks are driven by the assumption that there is always the
> unexpected. No matter what happens, it's a safe bet that you can make
> Lisp behave the way you want it to behave, even in the unlikely event
> that something happens that no language designer has ever thought of
> before. And even if you cannot find a perfect solution in some cases,
> you will at least be able to find a good approximation for hard
> problems.[/color]

This I believe is the very crux of the matter. The problem domain to
which lisp has historically been applied, artificial intelligence,
more or less guaranteed that lisp hackers would run up against the
sorts of problems that no one had ever seen before. The language
therefore evolved into a "programmab le programming language," to quote
John Foderaro (or whoever first said or wrote this now famous line).

Lisp gives the programmer who knows he will be working in a domain
that is not completely cut and dried, the assurance that his language
will not prevent him for doing something that has never been done
before. Python gives me the distinct impression that I might very well
run up against the limitations of the language when dealing with very
complex problems.

For 90% of tasks, even large projects, Python will certainly have
enough in its ever expanding bag of tricks to provide a clean,
maintainable solution. But that other 10% keeps lisp hackers from
using Python for exploratory programming - seeking solutions in
problem domains that have not been solved before.

[Pascal Costanza]

Re: Python syntax in Lisp and Scheme

Alex Martelli wrote:
[color=blue]
> Pascal Costanza wrote:
> ...
>[color=green][color=darkred]
>>>>Does Python allow local function definitions?[/color][/color]
>
> ...
>[color=green][color=darkred]
>>>>Can they shadow predefined functions?[/color][/color]
>
> ...
>[color=green][color=darkred]
>>>Yes, named objects, including functions can (locally) shadow
>>>(override) builtins. It is considered a bad habit/practice unless
>>>done intentionally with a functional reason.[/color]
>>
>>Well, this proves that Python has a language feature that is as
>>dangerous as many people seem to think macros are.[/color]
>
>
> Indeed, a chorus of "don't do that" is the typical comment each
> and every time a newbie falls into that particular mis-use. Currently,
> the --shadow option of PyChecker only warns about shadowing of
> _variables_, not shadowing of _functions_, but there's really no
> reason why it shouldn't warn about both. Logilab's pylint does
> diagnose "redefining built-in" with a warning (I think they mean
> _shadowing_, not actually _redefining_, but this may be an issue
> of preferred usage of terms).
>
> "Nailing down" built-ins (at first with a built-in warning for overriding
> them, later in stronger ways -- slowly and gradually, like always, to
> maintain backwards compatibility and allow slow, gradual migration of the
> large existing codebase) is under active consideration for the next version
> of Python, expected (roughly -- no firm plans yet) in early 2005.[/color]

OK, I understand that the Python mindset is really _a lot_ different
than the Lisp mindset in this regard.
[color=blue]
> Note that SOME built-ins exist SPECIFICALLY for the purpose of
> letting you override them. Consider, for example, __import__ -- this
> built-in function just exposes the inner mechanics of the import
> statement (and friends) to let you get modules from some other
> place (e.g., when your program must run off a relational database
> rather than off a filesystem). In other word, it's a rudimentary hook
> in a "Template Method" design pattern (it's also occasionally handy
> to let you import a module whose name is in a string, without
> going to the bother of an 'exec', so it will surely stay for that purpose
> even though we now have a shiny brand-new architecture for
> import hooks -- but that's another story).[/color]

Ah, you want something like final methods in Java, or better probably
final implicitly as the default and means to make select methods
non-final, right?
[color=blue]
> Anyway, back to your contention: I do not think that the fact that
> the user can, within his functions, choose very debatable names,
> such as those which shadow built-ins, is anywhere as powerful,
> and therefore as dangerous, as macros. My own functions using
> 'sum' will get the built-in one even if yours do weird things with
> that same name as a local variable of their own. The downsides
> of shadowing are essentially as follows...[/color]

What makes you think that macros have farther reaching effects in this
regard than functions? If I call a method and pass it a function object,
I also don't know what the method will do with it.

Overriding methods can also be problematic when they break contracts.
(Are you also considering to add DBC to Python? I would expect that by
now given your reply above.)

Can you give an example for the presumably dangerous things macros
supposedly can do that you have in mind?


Pascal