Building extensibility into an API

[Francis Johnson]

I've been having a discussion with some folks here about the wisdom or
otherwise of "building extensibility into an API".

Consider your favorite function in one of your libraries, perhaps
void foo(int bar, char *baz);

Now maybe you think of some cool improvement to foo that you could make,
but this relies on being able to pass an additional parameter to foo.
You now have some unattractive choices:
1) change the signature of foo and break API compatability between
versions of the library
2) have a new function
foo_v2(int bar, char *baz, long double newparam);
and make foo a wrapper for foo_v2.
3) don't make the improvement.

Now, this dilemma would never have arisen if you'd "built extensibility
into the API" in the following way.

Each function you write takes an unused void * parameter. So foo starts
out as
void foo(int bar, char *baz, void *p);
and the last parameter should be NULL.

Now, consider extending the parameters of foo. You now use a
foo_v2_extra_pa rams structure. Its first field encodes the additional
parameters, and the remaining fields are these parameters. Then version
2 of foo can do this:

void foo(int bar, char *baz, void *p)
{
/* define additional params as auto variables */
if(p)
/* process *p, set up extra args */
else
/* set additional params to defaults */
/* do stuff */
}

If foo improves again, the struct just gets enlarged, and the function
casts p appropriately before proceeding.

Have people here implemented something like this in the past? What do
you think of it as a solution to the "API extension" problem?

--
Red danger: http://www.mindszenty.org

[cr88192]

Re: Building extensibility into an API


"Francis Johnson" <spamtrap@garba ge.invalidwrote in message
news:slrnfi77pv .flp.nospam@nos pam.com...

I've been having a discussion with some folks here about the wisdom or
otherwise of "building extensibility into an API".
>
Consider your favorite function in one of your libraries, perhaps
void foo(int bar, char *baz);
>
Now maybe you think of some cool improvement to foo that you could make,
but this relies on being able to pass an additional parameter to foo.
You now have some unattractive choices:
1) change the signature of foo and break API compatability between
versions of the library
2) have a new function
foo_v2(int bar, char *baz, long double newparam);
and make foo a wrapper for foo_v2.
3) don't make the improvement.
>

2.
VirtualAllocEx anyone?...

you know, maybe in the future, this too will be extended:
VirtualAllocXXX
and:
VirtualAllocHar dCore
VirtualAllocExt reme
VirtualAllocToo TheMax

or maybe:
VirtualAllocRel oaded
VirtualAllocRes surection
....

now, they have to merge some of these together into a more powerful
function:
VirtualAllocHar dCoreExtremeToo TheMax

or something...

Now, this dilemma would never have arisen if you'd "built extensibility
into the API" in the following way.
>
Each function you write takes an unused void * parameter. So foo starts
out as
void foo(int bar, char *baz, void *p);
and the last parameter should be NULL.
>
Now, consider extending the parameters of foo. You now use a
foo_v2_extra_pa rams structure. Its first field encodes the additional
parameters, and the remaining fields are these parameters. Then version
2 of foo can do this:
>

<snip>

>
If foo improves again, the struct just gets enlarged, and the function
casts p appropriately before proceeding.
>
Have people here implemented something like this in the past? What do
you think of it as a solution to the "API extension" problem?
>

syscall, ioctl, ...

4. or, how about, we encode all our arguments and data as a big string?...
then, on call, the API parses the string and figures out what is going on.

5. or, even more spiffy, how about we make each argument its own function
call.
so, we call a function telling the API we are sending something, and a call
for each arg, and then a call to indicate when we are done.

....

hmm, was trying to come up with ideas which were bizarre, but oddly enough,
these are actually useful (and I have made good use of them in the past).

4. works good when the other end of the API is some elaborate piece of
machinery (such as a compiler or interpreter).
5. works well when you are building something, such as in a physics library
or GUI framework.

--
Red danger: http://www.mindszenty.org

[Tor Rustad]

Re: Building extensibility into an API

Francis Johnson wrote:

I've been having a discussion with some folks here about the wisdom or
otherwise of "building extensibility into an API".
>
Consider your favorite function in one of your libraries, perhaps
void foo(int bar, char *baz);
>
Now maybe you think of some cool improvement to foo that you could make,
but this relies on being able to pass an additional parameter to foo.
You now have some unattractive choices:
1) change the signature of foo and break API compatability between
versions of the library

Many times, backward compability is important, but in some cases the API
is simply broken, and there is no way to avoid fixing existing code,
unless you change the API. gets() is a prime example here.

2) have a new function
foo_v2(int bar, char *baz, long double newparam);
and make foo a wrapper for foo_v2.

If the existing code isn't broken, adding a new API, is the bets way
IMO, but instead of using a wrapper, the usual case involve duplicating
some code and providing a new independent function to ensure you don't
break existing code.

3) don't make the improvement.

On a complex system, it might not be an improvement... :)


--
Tor <bwzcab@wvtqvm. vw | tr i-za-h a-z>

My C page: http://www.pg2.moo.no/C/index.html
-include Win32 build of splint 3.1.2

[ymuntyan@gmail.com]

Re: Building extensibility into an API

On Oct 27, 3:27 pm, Francis Johnson <spamt...@garba ge.invalidwrote :

I've been having a discussion with some folks here about the wisdom or
otherwise of "building extensibility into an API".
>
Consider your favorite function in one of your libraries, perhaps
void foo(int bar, char *baz);
>
Now maybe you think of some cool improvement to foo that you could make,
but this relies on being able to pass an additional parameter to foo.
You now have some unattractive choices:
1) change the signature of foo and break API compatability between
versions of the library
2) have a new function
foo_v2(int bar, char *baz, long double newparam);
and make foo a wrapper for foo_v2.
3) don't make the improvement.
>
Now, this dilemma would never have arisen if you'd "built extensibility
into the API" in the following way.
>
Each function you write takes an unused void * parameter. So foo starts
out as
void foo(int bar, char *baz, void *p);
and the last parameter should be NULL.
>
Now, consider extending the parameters of foo. You now use a
foo_v2_extra_pa rams structure. Its first field encodes the additional
parameters, and the remaining fields are these parameters. Then version
2 of foo can do this:
>
void foo(int bar, char *baz, void *p)
{
/* define additional params as auto variables */
if(p)
/* process *p, set up extra args */
else
/* set additional params to defaults */
/* do stuff */
>
}
>
If foo improves again, the struct just gets enlarged, and the function
casts p appropriately before proceeding.
>
Have people here implemented something like this in the past? What do
you think of it as a solution to the "API extension" problem?

Microsoft likes to put the size of a structure argument into itself
as a version indicator. This has an advantage (for poor programmer)
of not having to have/use magic constants.

SOMESTUFF stuff;
ZeroMemory (&stuff, sizeof stuff);
stuff.cbSize = sizeof stuff;
/* ... */
CallFunctionExS oWhatIfItsExAlr eady (&stuff);

Because Microsoft cares.

Best regards,
Yevgen

[Eric Sosman]

Re: Building extensibility into an API

ymuntyan@gmail. com wrote:

>
Microsoft likes to put the size of a structure argument into itself
as a version indicator. This has an advantage (for poor programmer)
of not having to have/use magic constants.

DEC used to do that in OpenVMS (so I imagine H-P still does).
It didn't mesh well with C, though, and I once had to swat a bug
that the practice caused.

As I recall, the structure in question was an RMS ("file
system") control block of some kind. C didn't and doesn't have
a good construct for initializing such things in a way that's
"opaque" to the programmer, so VMS' library provided `const'
versions of the various control blocks, initialized with assorted
defaults. Your program set up its own control block by copying
the library's version and then filling in extra fields:

struct RMSthing cb = preInitializedR MSthing;
cb.this = 42;
cb.that = "foobar.txt ";

The control block contained its own self-describing size field,
which got copied from the library to the program's version -- and
since both of them were `struct RMSthing' instances, all was well,
right?

Wrong. DEC introduced a new feature (I think it was in
support of what was then called the "High Sierra" CD-ROM format)
that involved expanding the RMSthing by a few bytes -- let's say
it went from 40 bytes to 48, although I don't recall the actual
numbers.

Alas, preInitializedR MSthing lived in a shared library (like
a "DLL" or ".so" in other systems). Our code, already compiled
and running in the field, had allocated 40 bytes for `cb' above,
and copied only 40 bytes into it from preInitializedR MSthing.
So far, so good -- but among those 40 bytes was the size field,
which advertised the control block as being 48 bytes long, not 40.
So when we used the block to tell VMS to do something, VMS looked
at the size field, said "Oh, goodie: it's a new-style RMSthing!"
and started messing around with the eight bytes past the end of
our control block ...

--
Eric Sosman
esosman@ieee-dot-org.invalid

[Malcolm McLean]

Re: Building extensibility into an API

<ymuntyan@gmail .comwrote in message

On Oct 27, 3:27 pm, Francis Johnson <spamt...@garba ge.invalidwrote :

>I've been having a discussion with some folks here about the wisdom or
>otherwise of "building extensibility into an API".
>>

Microsoft likes to put the size of a structure argument into itself
as a version indicator. This has an advantage (for poor programmer)
of not having to have/use magic constants.
>
SOMESTUFF stuff;
ZeroMemory (&stuff, sizeof stuff);
stuff.cbSize = sizeof stuff;
/* ... */
CallFunctionExS oWhatIfItsExAlr eady (&stuff);
>
Because Microsoft cares.
>

That's the way to do it. If you need make a function extensible, wrap the
parameters into a structure and pass a size in so you can extend it. The
fact that MS do it means the paradigm will be familiar to a large base of
programmers.

However it's not a good way of writing software. Filling out the struct will
be a grief to your caller. It's the best that can be done with C, and in
fact with most languages.

--
Free games and programming goodies.

http://www.personal.leeds.ac.uk/~bgy1mm

[Charlie Gordon]

Re: Building extensibility into an API

"Malcolm McLean" <regniztar@btin ternet.coma écrit dans le message de news:
cYudnWS3GKDFV7n aRVnyvAA@bt.com...

<ymuntyan@gmail .comwrote in message

>On Oct 27, 3:27 pm, Francis Johnson <spamt...@garba ge.invalidwrote :

>>I've been having a discussion with some folks here about the wisdom or
>>otherwise of "building extensibility into an API".
>>>

>Microsoft likes to put the size of a structure argument into itself
>as a version indicator. This has an advantage (for poor programmer)
>of not having to have/use magic constants.
>>
>SOMESTUFF stuff;
>ZeroMemory (&stuff, sizeof stuff);
>stuff.cbSize = sizeof stuff;
>/* ... */
>CallFunctionEx SoWhatIfItsExAl ready (&stuff);
>>
>Because Microsoft cares.
>>

That's the way to do it. If you need make a function extensible, wrap the
parameters into a structure and pass a size in so you can extend it. The
fact that MS do it means the paradigm will be familiar to a large base of
programmers.
>
However it's not a good way of writing software. Filling out the struct
will be a grief to your caller. It's the best that can be done with C, and
in fact with most languages.

Unix has even worse solutions for similar but related issues:

int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen);

--
Chqrlie.

[Roland Pibinger]

Re: Building extensibility into an API

On Sun, 28 Oct 2007 17:43:57 -0000, "Malcolm McLean" wrote:

>That's the way to do it. If you need make a function extensible, wrap the
>parameters into a structure and pass a size in so you can extend it. The
>fact that MS do it means the paradigm will be familiar to a large base of
>programmers.

More often MS uses different names, e.g. CreateWindowEx vs.
CreateWindow. BTW, from a C programmer's point of view the Windows API
is a relatively good API.


--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch

[Charlie Gordon]

Re: Building extensibility into an API

"Roland Pibinger" <rpbg123@yahoo. coma écrit dans le message de news:
47251246.148572 33@news.utanet. at...

On Sun, 28 Oct 2007 17:43:57 -0000, "Malcolm McLean" wrote:

>>That's the way to do it. If you need make a function extensible, wrap the
>>parameters into a structure and pass a size in so you can extend it. The
>>fact that MS do it means the paradigm will be familiar to a large base of
>>programmers .

>
More often MS uses different names, e.g. CreateWindowEx vs.
CreateWindow. BTW, from a C programmer's point of view the Windows API
is a relatively good API.

That's a case of a glas half full or half empty. I as a C programmer
consider the Windows API to a relatively bad API.

--
Chqrlie

[Richard Heathfield]

Re: Building extensibility into an API

Charlie Gordon said:

<snip>

I as a C programmer
consider the Windows API to a relatively bad API.

Which just goes to show that it takes all sorts, since I think the Win32 C
API is not only very good indeed, but in fact the only part of Windows I
actually enjoy using.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999