Boost Workshop at OOPSLA 2004

[David Abrahams]

Re: Boost Workshop at OOPSLA 2004

"Paul Mensonides" <leavings@comca st.net> writes:
[color=blue]
> "David Abrahams" <dave@boost-consulting.com> wrote in message
> news:u1xiciifa. fsf@boost-consulting.com. ..
>[color=green]
> > Yes, but as I mentioned none of that is required in std C++.
> > http://sourceforge.net/projects/chaos-pp/ doesn't use any
> > program-generated macros.[/color]
>
> It does use some, but not for algorithmic constructs. E.g. the closest
> equivalent (i.e. as feature-lacking as possible) to BOOST_PP_REPEAT under Chaos
> is:
>
> #include <chaos/preprocessor/arithmetic/dec.h>
> #include <chaos/preprocessor/control/when.h>
> #include <chaos/preprocessor/recursion/basic.h>
> #include <chaos/preprocessor/recursion/expr.h>
>
> #define REPEAT(count, macro, data) \
> REPEAT_S(CHAOS_ PP_STATE(), count, macro, data) \
> /**/
> #define REPEAT_S(s, count, macro, data) \
> REPEAT_I( \
> CHAOS_PP_OBSTRU CT(), CHAOS_PP_NEXT(s ), \
> count, macro, data \
> ) \
> /**/
> #define REPEAT_INDIRECT () REPEAT_I
> #define REPEAT_I(_, s, count, macro, data) \
> CHAOS_PP_WHEN _(count)( \
> CHAOS_PP_EXPR_S _(s)(REPEAT_IND IRECT _()( \
> CHAOS_PP_OBSTRU CT _(), CHAOS_PP_NEXT(s ), \
> CHAOS_PP_DEC(co unt), macro, data \
> )) \
> macro _(s, CHAOS_PP_DEC(co unt), data) \
> ) \
> /**/[/color]

Confused. I don't see anything here that looks like a
program-generated macro.

--
Dave Abrahams
Boost Consulting

http://www.boost-consulting.com

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Paul Mensonides]

Re: Boost Workshop at OOPSLA 2004

David Abrahams wrote:[color=blue][color=green][color=darkred]
>> > Yes, but as I mentioned none of that is required in std C++.
>> > http://sourceforge.net/projects/chaos-pp/ doesn't use any
>> > program-generated macros.[/color]
>>
>> It does use some, but not for algorithmic constructs. E.g. the
>> closest equivalent (i.e. as feature-lacking as possible) to
>> BOOST_PP_REPEAT under Chaos is:
>>
>> #include <chaos/preprocessor/arithmetic/dec.h>
>> #include <chaos/preprocessor/control/when.h>
>> #include <chaos/preprocessor/recursion/basic.h>
>> #include <chaos/preprocessor/recursion/expr.h>
>>
>> #define REPEAT(count, macro, data) \
>> REPEAT_S(CHAOS_ PP_STATE(), count, macro, data) \
>> /**/
>> #define REPEAT_S(s, count, macro, data) \
>> REPEAT_I( \
>> CHAOS_PP_OBSTRU CT(), CHAOS_PP_NEXT(s ), \
>> count, macro, data \
>> ) \
>> /**/
>> #define REPEAT_INDIRECT () REPEAT_I
>> #define REPEAT_I(_, s, count, macro, data) \
>> CHAOS_PP_WHEN _(count)( \
>> CHAOS_PP_EXPR_S _(s)(REPEAT_IND IRECT _()( \
>> CHAOS_PP_OBSTRU CT _(), CHAOS_PP_NEXT(s ), \
>> CHAOS_PP_DEC(co unt), macro, data \
>> )) \
>> macro _(s, CHAOS_PP_DEC(co unt), data) \
>> ) \
>> /**/[/color]
>
> Confused. I don't see anything here that looks like a
> program-generated macro.[/color]

That was the point. REPEAT is an algorithmic construct that uses recursion, but
it doesn't require macro repetition. However, some lower-level abstractions,
like recursion itself (e.g. EXPR_S) and saturation arithmetic (e.g. DEC),
require macro repetition. For something like recursion, which is not naturally
present in macro expansion, some form of macro repetition will always be
necessary. The difference is that that repetition is hidden behind an
abstraction and the relationship of N macros need not imply on N steps. As
Andrei mentioned, BOOST_PP_REPEAT requires (at least) N macros to repeat N
things. Similarly, BOOST_PP_FOR, BOOST_PP_WHILE, etc., all require (at least) N
macros to perform N steps. That is not the case with Chaos.

#include <chaos/preprocessor/control/inline_when.h>
#include <chaos/preprocessor/recursion/basic.h>
#include <chaos/preprocessor/recursion/expr.h>

#define FOR(pred, op, macro, state) \
FOR_S(CHAOS_PP_ STATE(), pred, op, macro, state) \
/**/
#define FOR_S(s, pred, op, macro, state) \
FOR_I( \
CHAOS_PP_OBSTRU CT(), CHAOS_PP_NEXT(s ), \
pred, op, macro, state\
) \
/**/
#define FOR_INDIRECT() FOR_I
#define FOR_I(_, s, pred, op, macro, state) \
CHAOS_PP_INLINE _WHEN _(pred _(s, state))( \
macro _(s, state) \
CHAOS_PP_EXPR_S _(s)(FOR_INDIRE CT _()( \
CHAOS_PP_OBSTRU CT _(), CHAOS_PP_NEXT(s ), \
pred, op, macro, op _(s, state) \
)) \
) \
/**/

#include <chaos/preprocessor/control/iif.h>
#include <chaos/preprocessor/recursion/basic.h>
#include <chaos/preprocessor/recursion/expr.h>

#define WHILE(pred, op, state) \
WHILE_S(CHAOS_P P_STATE(), pred, op, state) \
/**/
#define WHILE_S(s, pred, op, state) \
WHILE_I( \
CHAOS_PP_OBSTRU CT(), CHAOS_PP_NEXT(s ), \
pred, op, state \
) \
/**/
#define WHILE_INDIRECT( ) WHILE_I
#define WHILE_I(_, s, pred, op, state) \
CHAOS_PP_IIF _(pred _(s, state))( \
CHAOS_PP_EXPR_S _(s)(WHILE_INDI RECT _()( \
CHAOS_PP_OBSTRU CT _(), CHAOS_PP_NEXT(s ), \
pred, op, op _(s, state) \
)), \
state
) \
/**/

Regards,
Paul Mensonides



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Paul Mensonides]

Re: Boost Workshop at OOPSLA 2004

"Daniel R. James" <daniel@calamit y.org.uk> wrote in message[color=blue][color=green]
> > (http://sourceforge.net/projects/chaos-pp/) that has almost no
> > boilerplate, but it only works on a few compilers (GCC among them).[/color]
>
> Unless I'm missing something, that link goes to an empty sourceforge
> project. Which is a pity, because I remember seeing some old chaos
> code somewhere, and it looked ace.[/color]

The project is definitely not empty, it just hasn't made any "official"
releases.

Regards,
Paul Mensonides



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Andrei Alexandrescu \(See Website for Email\)]

Re: The C++ Preprocessor [Re: Boost Workshop at OOPSLA 2004]

"Paul Mensonides" <leavings@comca st.net> wrote in message
news:IuydnfKYUP wWHoDcRVn-tQ@comcast.com. ..[color=blue]
> Regarding actual code examples, here's a Chaos-based version of the old
> TYPELIST_1, TYPELIST_2, etc., macros. Note that this example uses
> variadics
> which are likely to be added with C++0x. (It is also a case-in-point of
> why
> variadics are important.)[/color]

Cool. Before continuing the discussion, I have a simple question - how does
your implementation cope with commas in template types, for example:

TYPELIST(vector <int, my_allocator<in t> >, vector<float>)

would correctly create a typelist of two elements? If not, what steps do I
need to take to creat such a typelists (aside from a typedef)?


Andrei



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[galathaea]

Re: Boost Workshop at OOPSLA 2004

"Andrei Alexandrescu \(See Website for Email\)" <SeeWebsiteForE mail@moderncppd esign.com> wrote in message news:<2nqfupF3f jgkU1@uni-berlin.de>...[color=blue]
> "Jeremy Siek" <jeremy.siek@gm ail.com> wrote in message
> news:21925601.0 408090845.4a6d6 a7c@posting.goo gle.com...[color=green]
> > CALL FOR PAPERS/PARTICIPATION
> >
> > C++, Boost, and the Future of C++ Libraries
> > Workshop at OOPSLA
> > October 24-28, 2004
> > Vancouver, British Columbia, Canada
> > http://tinyurl.com/4n5pf[/color]
> [snip]
>
> I wonder if the submitters follow this post's trail or I should email
> them... anyway, here goes.
>
> I am not sure if I'll ever get around to writing it, so I said I'd post the
> idea here, maybe someone will pursue it. In short, I think a proposal for a
> replacement of C++'s preprocessor would be, I think, welcome.
>[/color]

There is only one replacement for the c++ preprocessor which I would
consider truly up to c++'s potential as a competitive language into
the near future: metacode. Full metacode capabilities, not just a
minor update to template capabilities.

By this I mean the capability to walk the parse tree at compile time
and perform transformations in a meta-type safe manner (analagous to
full second-order lambda capability such as System F). Vandevoorde's
metacode seems a good step in this direction, but I really think that
such a proposal must be as complete as possible (and not a library
proposal but a full language extension).

Consider some of the things I've seen talked about recently on the
newsgroups. Injecting a function call after all constructors have
executed is certainly one of those things the language should allow,
but currently we must work 'around' the language by forcing the use of
factories in such cases or leaving the two-stage use to clients (never
a good idea). In terms of functional relationships, though (even in
the presence of exceptions), such a task is a simple injection in
terms of functional orderings of the ctors and we are currently made
to fight with the language definitions enforced by the compiler.

Or consider the place where I currently use the Boost preprocessing
library the most: serialisation of classes. If we were allowed to
walk the member list for a class, walk its inheritance graph, and
stringise the class names (or produce better unique identifiers),
serialisation would be a cakewalk. Unfortunately, it is made much
more difficult and requires a more difficult object definition model
if any of the tasks of serialisation are to be automated by a library.

And of course there is all that control over the exception path
process, pattern generation, and general aspect functionsl
relationship injection that programmers have been crying about for
years.
[color=blue]
> Today Boost uses a "preprocess or library", which in turn (please correct me
> if my understanding is wrong) relies on a program to generate some many big
> macros up to a fixed "maximum" to overcome preprocessor's incapability to
> deal with variable number of arguments.[/color]

Others have pointed out that this is much more a nonconformance issue
than it is an inherent preprocessor limitation, but I'd like to stress
that a fully recursive code generation system in c++ would not present
such problems.
[color=blue]
> Also, please correct me if I'm wrong (because I haven't really looked deep
> into it), but my understanding is that people around Boost see the PP
> library as a necessary but unpleasantly-smelling beast that makes things
> around it smelly as well. [Reminds me of the Romanian story: there was a guy
> called Pepelea (pronounced Peh-Peh-leah) who was poor but had inherited a
> beautiful house. A rich man wanted to buy it, and Pepelea sold it on one
> condition: that Pepelea owns a nail in the living room's wall, in which he
> can hang whatever he wanted. Now when the rich man was having guests and
> whatnot, Pepelea would drop by and embarraisingly hang a dirty old coat. Of
> course in the end the rich man got so exasperated that he gave Pepelea the
> house back for free. Ever since that story, "Pepelea's nail" is referred to
> as something like... like what the preprocessor is to the C++ language.][/color]

You are certainly a storyteller, but I'd give the c++ preprocessor
more credit. It has been the only method to acheive certain
limitations (shortfalls in complete second-order lambda
expressiveness) when needed by the coder. Indeed, if you take one of
the coders duties as minimising the updates needed by future feature
revisions of other coders, the preprocessor has always had its place
secure from typed language features. Again, serialisation has been my
major use, but other reasons for, for instance, type to string
conversion include interception of API's through lookup in the
import/export lists of the modules. A full metacode capability would
make that obsolete (walking the symbol table should be as easy as
walking the parse tree itself).
[color=blue]
> That would be reason one to create a new C++ preprocessor. (And when I say
> "new," that's not like in "yet another standard C++ preprocessor". I have
> been happy to see my suggestion on the Boost mailing list followed in that
> the WAVE preprocessor was built using Boost's own parser generator library,
> Spirit.) What I am talking now is "a backwards-INcompatible C++ preprocessor
> aimed at displacing the existing preprocessor forever and replacing it with
> a better one".[/color]

Certainly full metacoding capabilities would make this obsolete...
[color=blue]
> If backed by the large Boost community, the new preprocessor could easily
> gain popularity and be used in new projects instead of the old one. To avoid
> inheriting past's mistakes, the new preprocessor doesn't need to be
> syntax-compatible in any way with the old preprocessor, but only
> functionally compatible, in that it can do all that can be done with the
> existing preprocessor, only that it has new means to do things safer and
> better.
>
> I think that would be great. Because it we all stop coding for a second and
> think of it, what's the ugliest scar on C++'s face - what is Pepelea's nail?
> Maybe "export" which is so broken and so useless and so abusive that its
> implementers have developed Stockholm syndrome during the long years that
> took them to implement it? Maybe namespaces that are so badly designed,
> you'd think they are inherited from C? I'd say they are good contenders
> against each other, but none of them holds a candle to the preprocessor.[/color]

If we extend the idea of metacode to all of the translation process,
in other words if the programmer were to have control points inserted
into all parts of the code generation process, then export would never
have been a problem to begin with. Unfortunately, the c++
standardisation community feels that processes like linking are
sacrilege and not to be touched by regulation. If a full parse tree
walk were to include the ability to load other translation units and
manipulate their trees, then we wouldn't find export a 'scar' or in
any way difficile.

You know, if the c++ committee had the cojones to make standardisation
over the full translation process, we might even see dynamic linking a
possibility for the next language revision.
[color=blue]
> So, a proposal for a new preprocessor would be great. Here's a short wish
> list:
>
> * Does what the existing one does (although some of those coding patterns
> will be unrecommended);
>
> * Supports one-time file inclusion and multiple file inclusion, without the
> need for guards (yes, there are subtle issues related to that... let's at
> least handle a well-defined subset of the cases);
>
> * Allows defining "hygienic" macros - macros that expand to the same text
> independent on the context in which they are expanded;
>
> * Allows defining scoped macros - macros visible only within the current
> scope;
>
> * Has recursion and possibly iteration;
>
> * Has a simple, clear expansion model (negative examples abound - NOT like
> m4, NOT like tex... :o))
>
> * Supports variable number of arguments. I won't venture into thinking of
> more cool support a la scheme or dylan ofr Java Extender macros.[/color]

I'm not a big fan of textual macros when typed completion gives the
same computational capability. I really think that metacoding,
architecture generation, and all of those great things we come to look
for in AOP and generative intentional programming is what the c++
standards commitee should focus on. With that type of capability, a
"pre"-processor is superfluous.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

galathaea: prankster, fablist, magician, liar

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Paul Mensonides]

Re: The C++ Preprocessor [Re: Boost Workshop at OOPSLA 2004]

"Andrei Alexandrescu (See Website for Email)"
<SeeWebsiteForE mail@moderncppd esign.com> wrote in message
news:2o7khqF7p3 paU1@uni-berlin.de...[color=blue]
> "Paul Mensonides" <leavings@comca st.net> wrote in message
> news:IuydnfKYUP wWHoDcRVn-tQ@comcast.com. ..[color=green]
> > Regarding actual code examples, here's a Chaos-based version of the old
> > TYPELIST_1, TYPELIST_2, etc., macros. Note that this example uses
> > variadics
> > which are likely to be added with C++0x. (It is also a case-in-point of
> > why
> > variadics are important.)[/color]
>
> Cool. Before continuing the discussion, I have a simple question - how does
> your implementation cope with commas in template types, for example:
>
> TYPELIST(vector <int, my_allocator<in t> >, vector<float>)
>
> would correctly create a typelist of two elements? If not, what steps do I
> need to take to creat such a typelists (aside from a typedef)?[/color]

You'd just have to parenthesize types that contain open commas:

TYPELIST((vecto r<int, my_allocator<in t> >), vector<float>)

The DECODE macro in the example removes parentheses if they exist. E.g.

CHAOS_PP_DECODE (int) // int
CHAOS_PP_DECODE ((int)) // int

Using parentheses is, of course, only necessary for types that contain open
commas. (There is also a ENCODE macro that completes the symmetry, but it is
unnecessary.)

There are also a several other alternatives. It is possible to pass a type
through a system of macros without the system of macros being intrusively
modified (with DECODE or similar). This, in Chaos terminology, is a "rail". It
is a macro invocation that effectively won't expand until some context is
introduced. E.g.

#include <chaos/preprocessor/punctuation/comma.h>

#define A(x) B(x)
#define B(x) C(x)
#define C(x) D(x)
#define D(x) x

A(CHAOS_PP_COMM A())

This will error with too many arguments to B. However, the following disables
evaluation of COMMA() until after the system "returns" from A:

#include <chaos/preprocessor/punctuation/comma.h>
#include <chaos/preprocessor/recursion/rail.h>

#define A(x) B(x)
#define B(x) C(x)
#define C(x) D(x)
#define D(x) x

CHAOS_PP_WALL(A (
CHAOS_PP_UNSAFE _RAIL(CHAOS_PP_ COMMA)()
))

(There is also CHAOS_PP_RAIL that is similar, but getting into the difference
here is too complex a subject.)

In any case, the expansion of COMMA is inhibited until it reaches the context
established by WALL. The same thing can be achieved for types non-intrusively.
Chaos has two rail macros designed for this purpose, TYPE and TYPE_II. The
first, TYPE, is the most syntactically clean, but is only available with
variadics:

#include <chaos/preprocessor/facilities/type.h>
#include <chaos/preprocessor/recursion/rail.h>

#define A(x) B(x)
#define B(x) C(x)
#define C(x) D(x)
#define D(x) x

CHAOS_PP_WALL(A (
CHAOS_PP_TYPE(s td::pair<int, int>)
))
// std::pair<int, int>

The second, TYPE_II, is more syntactically verbose, but it works even without
variadics without counting commas:

#include <chaos/preprocessor/facilities/type.h>
#include <chaos/preprocessor/recursion/rail.h>

#define A(x) B(x)
#define B(x) C(x)
#define C(x) D(x)
#define D(x) x

CHAOS_PP_WALL(A (
CHAOS_PP_TYPE_I I(CHAOS_PP_BEGI N std::pair<int, int> CHAOS_PP_END)
))
// std::pair<int, int>

Thus, you *could* make a typelist using rails such as this to protect
open-comma'ed types, but for typelists (which inherently deal with types), it
would be pointless. Rails are more useful when some arbitrary data that you
need to pass around happens to be a type, but doesn't necessarily have to be.

Regards,
Paul Mensonides



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Andrei Alexandrescu \(See Website for Email\)]

Re: The C++ Preprocessor [Re: Boost Workshop at OOPSLA 2004]

"Paul Mensonides" <leavings@comca st.net> wrote in message
news:IuydnfKYUP wWHoDcRVn-tQ@comcast.com. ..
(from another post)[color=blue]
> The DECODE macro in the example removes parentheses if they exist. E.g.
>
> CHAOS_PP_DECODE (int) // int
> CHAOS_PP_DECODE ((int)) // int[/color]

That's what I was hoping for; thanks.

(back to this other post)[color=blue]
> Regarding actual code examples, here's a Chaos-based version of the old
> TYPELIST_1, TYPELIST_2, etc., macros. Note that this example uses
> variadics
> which are likely to be added with C++0x. (It is also a case-in-point of
> why
> variadics are important.)
>
> #include <chaos/preprocessor/control/iif.h>
> #include <chaos/preprocessor/detection/is_empty.h>
> #include <chaos/preprocessor/facilities/encode.h>
> #include <chaos/preprocessor/facilities/split.h>
> #include <chaos/preprocessor/limits.h>
> #include <chaos/preprocessor/recursion/basic.h>
> #include <chaos/preprocessor/recursion/expr.h>
>
> #define TYPELIST(...) TYPELIST_BYPASS (CHAOS_PP_LIMIT _EXPR, __VA_ARGS__)
> #define TYPELIST_BYPASS (s, ...) \
> CHAOS_PP_EXPR_S (s)(TYPELIST_I( \
> CHAOS_PP_OBSTRU CT(), CHAOS_PP_PREV(s ), __VA_ARGS__, \
> )) \
> /**/
> #define TYPELIST_INDIRE CT() TYPELIST_I
> #define TYPELIST_I(_, s, ...) \
> CHAOS_PP_IIF _(CHAOS_PP_IS_E MPTY_NON_FUNCTI ON(__VA_ARGS__) )( \
> Loki::NilType, \
> Loki::TypeList< \
> CHAOS_PP_DECODE _(CHAOS_PP_SPLI T _(0, __VA_ARGS__)), \
> CHAOS_PP_EXPR_S _(s)(TYPELIST_I NDIRECT _()( \
> CHAOS_PP_OBSTRU CT _(), CHAOS_PP_PREV(s ), \
> CHAOS_PP_SPLIT _(1, __VA_ARGS__) \
> )) \[color=green]
> > \[/color]
> ) \
> /**/[/color]

I am sure I will develop a lot more appreciation for this solution once I
will fully understand all of the clever techniques and idioms used.

For now, I hope you will agree with me that the above fosters learning yet
another programming style, which is different than straight programming,
template programming, or MPL-based programming.

I would also like to compare the solution above with the "imaginary" one
that I have in mind as a reference. It uses LISP-macros-like artifacts and a
few syntactic accoutrements.

$define TYPELIST() { Loki::NullType }
$define TYPELIST(head $rest more) {
Loki::Typelist< head, TYPELIST(more) >
}

About all that needs to be explained is that "$rest name" binds name to
whatever other comma-separated arguments follow, if any, and that the
top-level { and } are removed when creating the macro.

If you would argue that your version above is more or as elegant as this
one, we have irreducible opinions. I consider your version drowning in
details that have nothing to do with the task at hand, but with handling the
ways in which the preprocessor is inadequate for the task at hand. Same
opinion goes for the other version below:
[color=blue]
> #include <chaos/preprocessor/facilities/encode.h>
> #include <chaos/preprocessor/lambda/ops.h>
> #include <chaos/preprocessor/punctuation/comma.h>
> #include <chaos/preprocessor/recursion/expr.h>
> #include <chaos/preprocessor/tuple/for_each.h>
>
> #define TYPELIST(...) \
> CHAOS_PP_EXPR( \
> CHAOS_PP_TUPLE_ FOR_EACH( \
> CHAOS_PP_LAMBDA (Loki::TypeList <) \
> CHAOS_PP_DECODE _(CHAOS_PP_ARG( 1)) CHAOS_PP_COMMA_ (), \
> (__VA_ARGS__) \
> ) \
> Loki::NilType \
> CHAOS_PP_TUPLE_ FOR_EACH( \
> CHAOS_PP_LAMBDA (>), (__VA_ARGS__) \
> ) \
> ) \
> /**/[/color]
[color=blue]
> This implementation can process up to ~5000 types and there is no list of
> 5000
> macros anywhere in Chaos. (There are also other, more advanced methods
> capable
> of processing trillions upon trillions of types.)[/color]

I guess you have something that increases with the logarithm if that number,
is that correct?
[color=blue][color=green]
> > Let's think wishes. You've done a great many good things with the
> > preprocessor, so you are definitely the one to be asked. What features
> > do
> > you think would have made it easier for you and your library's clients?[/color]
>
> The most fundamental thing would be the ability to separate the first
> arbitrary
> preprocessing token (or whitespace separation) from those that follow it
> in a
> sequence of tokens and be able to classify it in some way (i.e. determine
> what
> kind of token it is and what its value is). The second thing would be the
> ability to take a single preprocessing token and deconstruct it into
> characters.
> I can do everything else, but can only do those things in a limited ways.[/color]

I understand the first desideratum, but not the second. What would be the
second thing beneficial for?


Andrei



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Paul Mensonides]

Re: The C++ Preprocessor [Re: Boost Workshop at OOPSLA 2004]

Andrei Alexandrescu (See Website for Email) wrote:
[color=blue]
> I am sure I will develop a lot more appreciation for this solution
> once I will fully understand all of the clever techniques and idioms
> used.[/color]

Yes.
[color=blue]
> For now, I hope you will agree with me that the above fosters
> learning yet another programming style, which is different than
> straight programming, template programming, or MPL-based programming.[/color]

Definitely. It is a wholly different language.
[color=blue]
> I would also like to compare the solution above with the "imaginary"
> one that I have in mind as a reference. It uses LISP-macros-like
> artifacts and a few syntactic accoutrements.
>
> $define TYPELIST() { Loki::NullType }
> $define TYPELIST(head $rest more) {
> Loki::Typelist< head, TYPELIST(more) >
> }
>
> About all that needs to be explained is that "$rest name" binds name
> to whatever other comma-separated arguments follow, if any, and that
> the top-level { and } are removed when creating the macro.[/color]

So, for the above you need (basically) two things: overloading on number of
arguments and recursion. Both of those things are already indirectly possible
(with the qualification that overloading on number of arguments is only possible
with variadics). That isn't to say that those facilities wouldn't be useful
features of the preprocessor, because they would. I'm merely referring to those
things which can be done versus those things which cannot with the preprocessor
as it currently exists. I'm concerned more with functionality than I am with
syntactic cleanliness.
[color=blue]
> If you would argue that your version above is more or as elegant as
> this one, we have irreducible opinions.[/color]

The direct "imaginary" version is obviously more elegant.
[color=blue]
> I consider your version
> drowning in details that have nothing to do with the task at hand,
> but with handling the ways in which the preprocessor is inadequate
> for the task at hand. Same opinion goes for the other version below:[/color]

But the preprocessor *is* adequate for the task. It just isn't as syntactically
clean as you'd like it to be.
[color=blue][color=green]
>> This implementation can process up to ~5000 types and there is no
>> list of 5000
>> macros anywhere in Chaos. (There are also other, more advanced
>> methods capable
>> of processing trillions upon trillions of types.)[/color]
>
> I guess you have something that increases with the logarithm if that
> number, is that correct?[/color]

Exponential structure, yes. A super-reduction of the idea is this:

#define A(x) B(B(x))
#define B(x) C(C(x))
#define C(x) x

Here, the x argument gets scanned for expansion with a base-2 exponential. With
about 25 macros you're already into millions of scans. Each of those scans can
be an arbitrary computational step.
[color=blue][color=green]
>> The most fundamental thing would be the ability to separate the first
>> arbitrary
>> preprocessing token (or whitespace separation) from those that
>> follow it in a
>> sequence of tokens and be able to classify it in some way (i.e.
>> determine what
>> kind of token it is and what its value is). The second thing would
>> be the ability to take a single preprocessing token and deconstruct
>> it into characters.
>> I can do everything else, but can only do those things in a limited
>> ways.[/color]
>
> I understand the first desideratum, but not the second. What would be
> the second thing beneficial for?[/color]

Identifier and number processing primarily, but also string and character
literals. Given those two things alone you could write a C++ interpreter with
the preprocessor--or, much more simply, you could trivially write your imaginary
example above. Speaking of which, you can already write interpreters that get
close. The one thing that you cannot do is get beyond of arbitrary
preprocessing tokens. They would have to be quoted in some way.

Regards,
Paul Mensonides



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Arkadiy Vertleyb]

Re: Boost Workshop at OOPSLA 2004

"Paul Mensonides" <leavings@comca st.net> wrote in message news:<CNCdnas4r M_hEoDcRVn-qQ@comcast.com> ...[color=blue]
> "Andrei Alexandrescu (See Website for Email)"
> <SeeWebsiteForE mail@moderncppd esign.com> wrote in message
> news:2o4citF6im fdU1@uni-berlin.de...[color=green]
> > "Paul Mensonides" <leavings@comca st.net> wrote in message[color=darkred]
> > > (I believe that the link Dave posted was to Chaos--which is distinct from
> > > Boost
> > > Preprocessor.)[/color]
> >
> > I've looked at the existing PP library, not at Chaos.[/color]
>
> In that case, I agree. Internally, Boost PP is a mess--but a mess caused by
> lackluster conformance.[/color]

I think the actual value of a library can be determined as a
difference between the "mess" it gets as its input, and the "mess" (if
some still remains) its user gets on its output. In this regard, IMO,
it's difficult to overestimate the value of the Boost PP library, no
matter how messy its implementation might be.

(Just a thought from one of recently converted former PP-haters)

Regards,
Arkadiy

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Daveed Vandevoorde]

Re: Boost Workshop at OOPSLA 2004

"Andrei Alexandrescu wrote:
[...][color=blue]
> Maybe "export" which is so broken and so useless and so abusive that its
> implementers have developed Stockholm syndrome during the long years that
> took them to implement it?[/color]

How is "export" useless and broken?

Have you used it for any project? I find it very pleasant
to work with in practice.

Daveed

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Andrei Alexandrescu \(See Website for Email\)]

Re: The C++ Preprocessor [Re: Boost Workshop at OOPSLA 2004]

"Paul Mensonides" <leavings@comca st.net> wrote in message
news:ct6dnYwsFu ESgbzcRVn-pw@comcast.com. ..[color=blue][color=green]
> > For now, I hope you will agree with me that the above fosters
> > learning yet another programming style, which is different than
> > straight programming, template programming, or MPL-based programming.[/color]
>
> Definitely. It is a wholly different language.[/color]

Not it only remains for me to convince you that that's a disadvantage :o).
[color=blue]
> So, for the above you need (basically) two things: overloading on number
> of
> arguments and recursion. Both of those things are already indirectly
> possible
> (with the qualification that overloading on number of arguments is only
> possible
> with variadics). That isn't to say that those facilities wouldn't be
> useful
> features of the preprocessor, because they would. I'm merely referring to
> those
> things which can be done versus those things which cannot with the
> preprocessor
> as it currently exists. I'm concerned more with functionality than I am
> with
> syntactic cleanliness.[/color]

I disagree it's only syntactic cleanliness. Lack of syntactic cleanliness is
the CHAOS_PP_ that you need to prepend to most of your library's symbols.
But let me pull the code again:

#define REPEAT(count, macro, data) \
REPEAT_S(CHAOS_ PP_STATE(), count, macro, data) \
/**/
#define REPEAT_S(s, count, macro, data) \
REPEAT_I( \
CHAOS_PP_OBSTRU CT(), CHAOS_PP_NEXT(s ), \
count, macro, data \
) \
/**/
#define REPEAT_INDIRECT () REPEAT_I
#define REPEAT_I(_, s, count, macro, data) \
CHAOS_PP_WHEN _(count)( \
CHAOS_PP_EXPR_S _(s)(REPEAT_IND IRECT _()( \
CHAOS_PP_OBSTRU CT _(), CHAOS_PP_NEXT(s ), \
CHAOS_PP_DEC(co unt), macro, data \
)) \
macro _(s, CHAOS_PP_DEC(co unt), data) \
) \
/**/

As far as I understand, REPEAT, REPEAT_S, REPEAT_INDIRECT , REPEAT_I, and the
out-of-sight CHAOS_PP_STATE, CHAOS_PP_OBSTRU CT, CHAOS_PP_EXPR_S are dealing
with the preprocessor alone and have zero relevance to the task. The others
implement an idiom for looping that I'm sure one can learn, but is far from
familiar to a C++ programmer. To say that that's just a syntactic
cleanliness thing is a bit of a stretch IMHO. By the same argument, any
Turing complete language will do at the cost of "some" syntactic
cleanliness.
[color=blue][color=green]
> > I consider your version
> > drowning in details that have nothing to do with the task at hand,
> > but with handling the ways in which the preprocessor is inadequate
> > for the task at hand. Same opinion goes for the other version below:[/color]
>
> But the preprocessor *is* adequate for the task. It just isn't as
> syntactically
> clean as you'd like it to be.[/color]

I maintain my opinion that we're talking about more than syntactic
cleanliness here. I didn't say the preprocessor is "incapable" for the task.
But I do believe (and your code strengthened my belief) that it is
"inadequate ". Now I looked on www.m-w.com and I saw that inadequate means "
: not adequate : INSUFFICIENT; also : not capable " and that adequate means
"sufficient for a specific requirement" and "lawfully and reasonably
sufficient". I guess I meant it as a negation of the last meaning, and even
that is a bit too strong. Obviously the preprocessor is "capable", because
hey, there's the code, but it's not, let me rephrase - very "fit" for the
task.
[color=blue][color=green]
> > I guess you have something that increases with the logarithm if that
> > number, is that correct?[/color]
>
> Exponential structure, yes. A super-reduction of the idea is this:
>
> #define A(x) B(B(x))
> #define B(x) C(C(x))
> #define C(x) x
>
> Here, the x argument gets scanned for expansion with a base-2 exponential.
> With
> about 25 macros you're already into millions of scans. Each of those
> scans can
> be an arbitrary computational step.[/color]

Wouldn't it be nicer if you just had one mechanism (true recursion or
iteration) that does it all in one shot?


Andrei



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Walter]

Is export &quot;useles s and broken&quot;?


"Daveed Vandevoorde" <google@vandevo orde.com> wrote in message
news:52f2f9cd.0 408161420.56e0d 49f@posting.goo gle.com...[color=blue]
> "Andrei Alexandrescu wrote:
> [...][color=green]
> > Maybe "export" which is so broken and so useless and so abusive that[/color][/color]
its[color=blue][color=green]
> > implementers have developed Stockholm syndrome during the long years[/color][/color]
that[color=blue][color=green]
> > took them to implement it?[/color]
>
> How is "export" useless and broken?
>
> Have you used it for any project? I find it very pleasant
> to work with in practice.[/color]
[color=blue]
>From my readings on export, the benefits are supposed to be:[/color]

1) avoid pollution of the name space with the names involved in the
implementation details of the template, sometimes called "code hygene"

2) template source code hiding

3) faster compilation

Examining each of these in turn:

1) Isn't this what C++ namespaces are for?

2) Given the ease with which Java .class files are "decompiled " back into
source code, and the fact that precompiled templates will necessarilly
contain even more semantic info than .class files, it is hard to see how
exported templates offer secure hiding of template implementations . It is
not analagous to the problem of "turning hamburger back into a cow" that
object file decompilers have. While some "security through obscurity" may be
achieved by not documenting the file format, if the particular compiler
implementation is popular, there surely will appear some tool to do it.

3) The faster compilation is theoretically based on the idea that the
template implementation doesn't need to be rescanned and reparsed every time
it is #include'd. However, many modern C++ compilers already support
"precompile d headers", which already provide just that capability without
export at all. I'd be happy to accept a compilation speed benchmark
challenge of Digital Mars DMC++ with precompiled headers vs an export
implementation.

I look at export as a cost/benefit issue. What are the benefits, and what
are the costs? The benefits, as discussed above, are not demonstrated to be
significant. The cost, however, is enormous - 2 to 3 man years of
implementation effort, which means that other, more desirable, features
would necessarilly get deferred/delayed.

What export does is attempt to graft some import model semantics onto the
inclusion model semantics. The two are fundamentally at odds, hence all the
complicated rules and implementation effort. The D Programming Language
simply abandons the inclusion model semantics completely, and goes instead
with true imported modules. This means that exported templates in D are
there "for free", i.e. they involve no extra implementation effort and no
strange rules. And after using them for a while, yes it is very pleasant to
be able to do:

----- foo.d ----
template Foo(T) { T x; }
----- bar.d ----
import foo;

foo.Foo!(int); // instantiate template Foo with 'int' type
----------------

-Walter
www.digitalmars.com free C/C++/D compilers


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Hyman Rosen]

Re: Is export &quot;useles s and broken&quot;?

Walter wrote:[color=blue]
>From my readings on export, the benefits are supposed to be:[/color]

The benefits of export are the same as the benefits of
separate compilation of non-templated code. That is, for
normal code, I can write

In joe.h:
struct joe { void frob(); double gargle(double); };
In joe.c:
namespace { void fiddle() { } double grok() { return 3.7; } }
void joe::frob() { fiddle(); }
double joe::gargle(dou ble d) { return d + grok(); }

And users of the joe class only include joe.h, and never
have to worry about joe.c. Without export, if joe were a
template class, then every compilation unit which uses a
method of joe would have to include the implementation of
those methods bodily. This constrains the implementation;
for example, that anonymous namespace wouldn't work. With
export, users include the header file and they are done.

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[tom_usenet]

Re: Is export &quot;useles s and broken&quot;?

On 17 Aug 2004 18:03:15 -0400, "Walter"
<walter@digital mars.nospamm.co m> wrote:
[color=blue]
>
>"Daveed Vandevoorde" <google@vandevo orde.com> wrote in message
>news:52f2f9cd. 0408161420.56e0 d49f@posting.go ogle.com...[color=green]
>> "Andrei Alexandrescu wrote:
>> [...][color=darkred]
>> > Maybe "export" which is so broken and so useless and so abusive that[/color][/color]
>its[color=green][color=darkred]
>> > implementers have developed Stockholm syndrome during the long years[/color][/color]
>that[color=green][color=darkred]
>> > took them to implement it?[/color]
>>
>> How is "export" useless and broken?
>>
>> Have you used it for any project? I find it very pleasant
>> to work with in practice.[/color]
>[color=green]
>>From my readings on export, the benefits are supposed to be:[/color]
>
>1) avoid pollution of the name space with the names involved in the
>implementati on details of the template, sometimes called "code hygene"
>
>2) template source code hiding
>
>3) faster compilation[/color]

4) avoid pollution of the lookup context in which the template
definition exists with names from the instantiation context, except
where these are required (e.g. dependent names).

5) reduce dependencies
[color=blue]
>Examining each of these in turn:[/color]

Hmm, this is all rehashing I think.
[color=blue]
>1) Isn't this what C++ namespaces are for?[/color]

That ignores macros and argument dependent lookup, which transcend
namespaces (or rather operate in a slightly unpredictable set of
namespaces in the case of the latter).
[color=blue]
>2) Given the ease with which Java .class files are "decompiled " back into
>source code, and the fact that precompiled templates will necessarilly
>contain even more semantic info than .class files, it is hard to see how
>exported templates offer secure hiding of template implementations . It is
>not analagous to the problem of "turning hamburger back into a cow" that
>object file decompilers have. While some "security through obscurity" may be
>achieved by not documenting the file format, if the particular compiler
>implementati on is popular, there surely will appear some tool to do it.[/color]

Precompiled templates don't need more semantic information than class
files. In particular, all code involving non-dependent names can be
fully compiled, or at the very least, the names can be removed from
the precompiled template file. In other words, the file format might
consist of a combination of ordinary object code intermingled with
other more detailed stuff.

I believe that EDG may be working on something related to this, but
they're keeping fairly schtum about it so I don't know the details.
[color=blue]
>3) The faster compilation is theoretically based on the idea that the
>template implementation doesn't need to be rescanned and reparsed every time
>it is #include'd. However, many modern C++ compilers already support
>"precompile d headers", which already provide just that capability without
>export at all. I'd be happy to accept a compilation speed benchmark
>challenge of Digital Mars DMC++ with precompiled headers vs an export
>implementation .[/color]

The compilation speed advantages also come from dependency reductions.
If template definitions are modified, only the template
specializations need to be recompiled. If the instantiation context
(which I believe only consists of all extern names) is saved in each
case, then the instantiation can be recompiled without having to
recompile the whole TU containing the implicit template instantiation.
[color=blue]
>I look at export as a cost/benefit issue. What are the benefits, and what
>are the costs? The benefits, as discussed above, are not demonstrated to be
>significant. The cost, however, is enormous - 2 to 3 man years of
>implementati on effort, which means that other, more desirable, features
>would necessarilly get deferred/delayed.
>
>What export does is attempt to graft some import model semantics onto the
>inclusion model semantics. The two are fundamentally at odds, hence all the
>complicated rules and implementation effort. The D Programming Language
>simply abandons the inclusion model semantics completely, and goes instead
>with true imported modules. This means that exported templates in D are
>there "for free", i.e. they involve no extra implementation effort and no
>strange rules. And after using them for a while, yes it is very pleasant to
>be able to do:
>
>----- foo.d ----
>template Foo(T) { T x; }
>----- bar.d ----
>import foo;
>
>foo.Foo!(int ); // instantiate template Foo with 'int' type
>----------------[/color]

It seems to me that there were three alternatives in C++.

1. Don't support any kind of model except the inclusion one. If this
is done, two-phase name lookup should have been dropped as well, since
it is confusing at best, and only really necessary if export is to be
supported. It catches some errors earlier, but at some expense to
programmers. The "typename" and "template" disambiguators could also
blissfully be dropped.

2. Add module support to C++. This obviously is as large a proposal as
export, but clearly #include is unhelpful in a language as complex as
C++; really you just want to import extern names from particular
namespaces, not textually include a whole file.

3. Support separate compilation of templates in some form, without
modules. If you do this, I think you pretty much end up with two phase
name lookup and export (indicating that it isn't broken).

The committee rejected 1, I doubt anyone suggested 2, so 3 was the
remaining choice.

Personally, I might well have gone with 1; templates are complicated
enough, and two-phase name lookup and export have unnecessarily made
them much more complex. On the other hand, 1 doesn't provide the
benefits that export does provide.

So, ignoring implementation difficultly, I think export does just win
as a useful feature. With the implementation difficultly, it's not so
clear.

Tom

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Andrei Alexandrescu \(See Website for Email\)]

export (WAS: Boost Workshop at OOPSLA 2004) (WILL BE: Save Andrei)

"Daveed Vandevoorde" <google@vandevo orde.com> wrote in message
news:52f2f9cd.0 408161420.56e0d 49f@posting.goo gle.com...[color=blue]
> "Andrei Alexandrescu wrote:
> [...][color=green]
> > Maybe "export" which is so broken and so useless and so abusive that its
> > implementers have developed Stockholm syndrome during the long years
> > that
> > took them to implement it?[/color]
>
> How is "export" useless and broken?
>
> Have you used it for any project? I find it very pleasant
> to work with in practice.[/color]

Haven't used export, and not because I didn't wanna.

[Whom do you think I referred to when mentioning the Stockholm syndrome?
:o)]

I'd say, a good feature, like a good business idea, can be explained in a
few words. What is the few-words good explanation of export? (I *am*
interested.)

In addition, a good programming language feature does what it was intended
to do (plus some other neat things :o)), and can be reasonably implemented.

I think we all agree "export" fell the last test.

Does it do what it was intended to do? (Again, I *am* interested.)

A summary of what's the deal with export would be of great help to at least
myself, so I'd be indebted to anyone who'd give me one. For full disclosure,
my current perception is:

1. It's hard to give a good account of what export does in a few words, at
least an account that's impressive.

2. export failed horribly at doing what was initially supposed to do. I
believe what it was supposed to do was true (not "when"s and "cough"s and
"um"s) separate compilation of templates. Admittedly, gaffes in other areas
of language design are at fault for that failure. Correct me if I'm wrong.

3. export failed miserably at being reasonably easy to implement.

Combined with 1 and 2, I can only say: at the very best, export is a Pyrrhic
victory.


Andrei



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]