xmalloc

[Malcolm McLean]

Continuing the error-handling thread.

I am floating

void *xmalloc(int sz)
{
void *answer;

assert(sz >= 0);
if(sz == 0)
sz = 1;
answer = malloc(sz);
if(!answer)
{
fprintf(stderr, "Can't allocate %d byte%c\n", sz, sz == 1 ? ' ', 's');
exit(EXIT_FAILU RE);
}
return answer;
}

as a solution to the malloc() problem.
(Chuck Falconer's suggestion)
You call it for trivial allocations on the basis that if the computer won't
give a few bytes of memory, not much can be done.
However you wouldn't call it to allocate an image, for example, because
legitmate images can be quite large in relation to computer memories.


--
Free games and programming goodies.

http://www.personal.leeds.ac.uk/~bgy1mm

[Richard Heathfield]

Re: xmalloc

Malcolm McLean said:

Continuing the error-handling thread.
>
I am floating
>
void *xmalloc(int sz)
{
void *answer;
>
assert(sz >= 0);
if(sz == 0)
sz = 1;
answer = malloc(sz);
if(!answer)
{
fprintf(stderr, "Can't allocate %d byte%c\n", sz, sz == 1 ? ' ',
's'); exit(EXIT_FAILU RE);
}
return answer;
}
>
as a solution to the malloc() problem.

What malloc() problem? And this "solution" suffers from the same problem
as any other "solution" of the same kind - it's no earthly use in a
library, and libraries are where malloc calls belong.

There are two reasons why it's no earthly use in a library.

For one thing, it displays a message on stderr, which isn't much cop in
a system where stderr messages are ignored - e.g. a Win32 GUI app. But
that's trivial to fix - either remove the message completely, or add a
callback or a pointer to an error code, so that the application can
worry about how to display the message.

The second problem is also quite easy to fix, and it's this - the
function calls exit(), thus taking the decision to quit the program out
of the hands of the programmer. Therefore, the programmer won't call
this function himself, and won't be able to call any other function in
the library that /does/ call it.

Here's my suggested replacement, which incorporates all the fixes I see
as being necessary:

#include <stdlib.h>

void *xmalloc(size_t sz)
{
return malloc(sz);
}

You call it for trivial allocations on the basis that if the computer
won't give a few bytes of memory, not much can be done.

Whether that is true depends on the problem you're trying to solve. For
example, it may well be possible to switch to a static solution which
is perhaps not as fast as the dynamic solution but which will
nevertheless do the Right Thing in a reasonable time.

<snip>

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Malcolm McLean]

Re: xmalloc


"Richard Heathfield" <rjh@see.sig.in validwrote in message
news:kbednX2M0N uh9-DbnZ2dnUVZ8qOtn Z2d@bt.com...

Malcolm McLean said:
>

>Continuing the error-handling thread.
>>
>I am floating
>>
>void *xmalloc(int sz)
>{
> void *answer;
>>
> assert(sz >= 0);
> if(sz == 0)
> sz = 1;
> answer = malloc(sz);
> if(!answer)
> {
> fprintf(stderr, "Can't allocate %d byte%c\n", sz, sz == 1 ? ' ',
> 's'); exit(EXIT_FAILU RE);
> }
> return answer;
>}
>>
>as a solution to the malloc() problem.

>
What malloc() problem? And this "solution" suffers from the same problem
as any other "solution" of the same kind - it's no earthly use in a
library, and libraries are where malloc calls belong.
>
There are two reasons why it's no earthly use in a library.
>
For one thing, it displays a message on stderr, which isn't much cop in
a system where stderr messages are ignored - e.g. a Win32 GUI app. But
that's trivial to fix - either remove the message completely, or add a
callback or a pointer to an error code, so that the application can
worry about how to display the message.
>
The second problem is also quite easy to fix, and it's this - the
function calls exit(), thus taking the decision to quit the program out
of the hands of the programmer. Therefore, the programmer won't call
this function himself, and won't be able to call any other function in
the library that /does/ call it.
>
Here's my suggested replacement, which incorporates all the fixes I see
as being necessary:
>
#include <stdlib.h>
>
void *xmalloc(size_t sz)
{
return malloc(sz);
}
>
>

>You call it for trivial allocations on the basis that if the computer
>won't give a few bytes of memory, not much can be done.

>
Whether that is true depends on the problem you're trying to solve. For
example, it may well be possible to switch to a static solution which
is perhaps not as fast as the dynamic solution but which will
nevertheless do the Right Thing in a reasonable time.
>

In which case would call regular malloc()
However you are doubling the cost of the software with that strategy, all
because a 2GB machine might refuse to give a few hundred bytes of memory. If
it is running a life-support machine, fair enough, but most software
doesn't.

On the other hand you could argue that xmalloc() is a bad idea, because no
library that uses it could ever find its way into a life-support machine.
On the other hand, you could argue that the life support machine is more
likely to blow a fuse than to exit in xmalloc(), so you need a backup
anyway.

I take your point about vandalised stderrs.
Maybe we should put an assert fail in there as well / instead.
--
Free games and programming goodies.

http://www.personal.leeds.ac.uk/~bgy1mm

[Richard Heathfield]

Re: xmalloc

Malcolm McLean said:

>
"Richard Heathfield" <rjh@see.sig.in validwrote in message
news:kbednX2M0N uh9-DbnZ2dnUVZ8qOtn Z2d@bt.com...

>Malcolm McLean said:
>>

<snip>

>>You call it for trivial allocations on the basis that if the
>>computer won't give a few bytes of memory, not much can be done.

>>
>Whether that is true depends on the problem you're trying to solve.
>For example, it may well be possible to switch to a static solution
>which is perhaps not as fast as the dynamic solution but which will
>nevertheless do the Right Thing in a reasonable time.
>>

In which case would call regular malloc()
However you are doubling the cost of the software with that strategy,
all because a 2GB machine might refuse to give a few hundred bytes of
memory.

The way I see it is this: by writing an inferior program that is
prepared to bomb out at the drop of a hat, I can *reduce* the cost of
writing the program, but it *would be* an inferior program. And then
I'd have to market the program to the kind of people who used to buy
Cortinas, or Edsels. I prefer to deal with Maserati types. :-)

<snip>

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Roland Pibinger]

Re: xmalloc

On Sat, 23 Jun 2007 18:47:44 +0000, Richard Heathfield wrote:

>The second problem is also quite easy to fix, and it's this - the
>function calls exit(), thus taking the decision to quit the program out
>of the hands of the programmer. Therefore, the programmer won't call
>this function himself, and won't be able to call any other function in
>the library that /does/ call it.

You can often see programs that check the return value of malloc but
you hardly (practically never) see any code that actually handles the
OOM (out-of-memory) condition (e.g. frees some pre-allocated memory).
Strange, isn't it?
Moreover, C libraries usually crash when the caller provides
insufficient resources or invalid arguments. In that sense abort() is
preferable to exit() for OOM.

>For
>example, it may well be possible to switch to a static solution which
>is perhaps not as fast as the dynamic solution but which will
>nevertheless do the Right Thing in a reasonable time.

The 'static solution' may cause a 'stack' overflow. BTW, how would you
handle the second type of OOM, 'stack' overflow? The same way as OOM
for dynamic memory?


--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch

[Eric Sosman]

Re: xmalloc

Malcolm McLean wrote:

[...]
However you are doubling the cost of the software with that strategy,
all because a 2GB machine might refuse to give a few hundred bytes of
memory. [...]

A 2GB machine *will* refuse to allocate a few hundred
bytes of memory -- immediately after a successful allocation
of 2GB. Maybe the program could choose to discard the 2GB
allocation that's just holding a bunch of cache, and try again
on the couple hundred bytes it Really Needs? Too late, Malcolm
has already euthanized it. Requesciat in pacem.

--
Eric Sosman
esosman@acm-dot-org.invalid

[Richard Heathfield]

Re: xmalloc

Roland Pibinger said:

<snip>

You can often see programs that check the return value of malloc but
you hardly (practically never) see any code that actually handles the
OOM (out-of-memory) condition (e.g. frees some pre-allocated memory).
Strange, isn't it?

Yes. It's a great shame.

Moreover, C libraries usually crash when the caller provides
insufficient resources or invalid arguments. In that sense abort() is
preferable to exit() for OOM.

Well, I do see your point, but personally I'd prefer for the program to
make sensible choices that minimise inconvenience to the user.

>>For
>>example, it may well be possible to switch to a static solution which
>>is perhaps not as fast as the dynamic solution but which will
>>nevertheles s do the Right Thing in a reasonable time.

>
The 'static solution' may cause a 'stack' overflow.

So make sure it doesn't, by avoiding stress to your 'stack'.

BTW, how would you
handle the second type of OOM, 'stack' overflow? The same way as OOM
for dynamic memory?

One of the reasons dynamic memory is so useful is that there's a heck of
a lot of it to go around, compared to a typical 'stack' - but the other
great advantage thereof is that you /can/ detect an OOM condition with
malloc, whereas you can't with 'stack' overflow - at least, not
portably. Nevertheless, you can still do quite a lot with a 'stack'
that doesn't involve a significant risk of overflow (the adjective
being necessary only because the C Standard gives us very little in the
way of guarantees about 'stack' capacity - in practice, nowadays it is
only a problem if you treat the 'stack' rather recklessly).

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Eric Sosman]

Re: xmalloc

Roland Pibinger wrote:

On Sat, 23 Jun 2007 18:47:44 +0000, Richard Heathfield wrote:

>The second problem is also quite easy to fix, and it's this - the
>function calls exit(), thus taking the decision to quit the program out
>of the hands of the programmer. Therefore, the programmer won't call
>this function himself, and won't be able to call any other function in
>the library that /does/ call it.

>
You can often see programs that check the return value of malloc but
you hardly (practically never) see any code that actually handles the
OOM (out-of-memory) condition (e.g. frees some pre-allocated memory).
Strange, isn't it?

Not all that strange. Half the programmers in the world
are below-average.

But if you follow Malcolm's Maxims, even the above-average
programmers will be helpless. Programs that deal with malloc()
failure in a significant way may be a minority, but those that
do it *need* to to do it. Malcolm argues, in essence, that
such programs should not be written in C.

Moreover, C libraries usually crash when the caller provides
insufficient resources or invalid arguments. In that sense abort() is
preferable to exit() for OOM.

Oh, heaven spare us! Not only do you want to kill off
the program, but you also want to prevent it from running
its carefully-registered atexit() handlers? And you don't
care whether buffers are flushed, streams are closed, and
temporary files are removed? What do you do for amusement,
juggle flasks of nitroglycerin while fire-walking?

--
Eric Sosman
esosman@acm-dot-org.invalid

[Malcolm McLean]

Re: xmalloc


"Roland Pibinger" <rpbg123@yahoo. comwrote in message
news:467d734b.3 793825@news.uta net.at...

On Sat, 23 Jun 2007 18:47:44 +0000, Richard Heathfield wrote:

>>The second problem is also quite easy to fix, and it's this - the
>>function calls exit(), thus taking the decision to quit the program out
>>of the hands of the programmer. Therefore, the programmer won't call
>>this function himself, and won't be able to call any other function in
>>the library that /does/ call it.

>
You can often see programs that check the return value of malloc but
you hardly (practically never) see any code that actually handles the
OOM (out-of-memory) condition (e.g. frees some pre-allocated memory).
Strange, isn't it?
Moreover, C libraries usually crash when the caller provides
insufficient resources or invalid arguments. In that sense abort() is
preferable to exit() for OOM.
>

Normally if you've written the program in such a way that it makes a lot of
trival allocations, there is nothing you can do on failure, short of calling
a back-up routine that uses fixed buffers.
So you handle fail conditions for the few large legitimate allocations,
which might run the computer out of memory, and abort on the rest.

You can declare a global "world" pointer, and then in atexit try to save the
state of the program, but there are two problems. Firstly if the OS won't
give you a few hundred bytes, it might well not open files for you either.
Secondly, all the code has to be very carefully written so that an
allocation failure never causes the state to lose coherence. Otherwise you
will crash when you try to write a null pointer, or worse, write an image
that looks right but is in fact subtly corrupted.

So there really isn't an easy answer.

>>For
>>example, it may well be possible to switch to a static solution which
>>is perhaps not as fast as the dynamic solution but which will
>>nevertheles s do the Right Thing in a reasonable time.

>
The 'static solution' may cause a 'stack' overflow. BTW, how would you
handle the second type of OOM, 'stack' overflow? The same way as
OOM for dynamic memory?
>

So if you use recursive subroutines you've already got one potential source
of crashes on malicious or excessive input. That reduces the motivation to
fix malloc(), because you can't guarantee correct operation anyway.

--
Free games and programming goodies.

http://www.personal.leeds.ac.uk/~bgy1mm

[Ian Collins]

Re: xmalloc

Richard Heathfield wrote:

Malcolm McLean said:
>

>Continuing the error-handling thread.
>>
>I am floating
>>
>void *xmalloc(int sz)
>{

Should be size_t.

> void *answer;
>>
> assert(sz >= 0);
> if(sz == 0)
> sz = 1;
> answer = malloc(sz);
> if(!answer)
> {
> fprintf(stderr, "Can't allocate %d byte%c\n", sz, sz == 1 ? ' ',
> 's'); exit(EXIT_FAILU RE);
> }
> return answer;
>}
>>
>as a solution to the malloc() problem.

>
What malloc() problem? And this "solution" suffers from the same problem
as any other "solution" of the same kind - it's no earthly use in a
library, and libraries are where malloc calls belong.
>

It could be enhanced by providing a (cough) global failure function, say
xmalloc_failure , that gets called when malloc returns NULL:

void* default_xmalloc _failure(size_t s) {
assert(!"Can't allocate memory"); return NULL;}

void* (*xmalloc_failu re)(size_t) = &default_xmallo c_failure;

The xmalloc becomes

void *xmalloc(size_t sz)
{
void *answer = malloc( sz == 0 ? 1 : sz );
if(answer == NULL)
{
xmalloc_failure (sz);
}
return answer;
}

Which gives the user control over what happens when malloc fails.

--
Ian Collins.

[Richard Heathfield]

Re: xmalloc

Ian Collins said:

<snip>

It could be enhanced by providing a (cough) global failure function,
say xmalloc_failure , that gets called when malloc returns NULL:
>
void* default_xmalloc _failure(size_t s) {
assert(!"Can't allocate memory"); return NULL;}
>
void* (*xmalloc_failu re)(size_t) = &default_xmallo c_failure;
>
The xmalloc becomes
>
void *xmalloc(size_t sz)
{
void *answer = malloc( sz == 0 ? 1 : sz );
if(answer == NULL)
{
xmalloc_failure (sz);
}
return answer;
}
>
Which gives the user control over what happens when malloc fails.

It's an improvement, but it's still not good enough as a general
solution. Consider this code:

foo *foo_create(siz e_t n)
{
static const foo fblank;
foo *new = malloc(sizeof *new); /* A */
if(new != NULL)
{
*new = fblank;
new->bar = malloc(n * sizeof *new->bar); /* B */
if(new->bar == NULL)
{
free(new);
new = NULL;
}
else
{
static const t_bar bblank;
while(n--)
{
new->bar[n] = bblank;
}
}
}
return new;
}

If we were to rewrite this to use your suggestion, that would be fine
for the malloc at A, but if the malloc at B fails, we have to clean up,
and that means that the cleanup code needs access to new. Your solution
doesn't address that problem.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Malcolm McLean]

Re: xmalloc


"Richard Heathfield" <rjh@see.sig.in validwrote in message
news:4dudnXd1d8 IMPODbnZ2dnUVZ8 seinZ2d@bt.com. ..

Ian Collins said:
>
<snip>
>

>It could be enhanced by providing a (cough) global failure function,
>say xmalloc_failure , that gets called when malloc returns NULL:
>>
>void* default_xmalloc _failure(size_t s) {
> assert(!"Can't allocate memory"); return NULL;}
>>
>void* (*xmalloc_failu re)(size_t) = &default_xmallo c_failure;
>>
>The xmalloc becomes
>>
>void *xmalloc(size_t sz)
>{
> void *answer = malloc( sz == 0 ? 1 : sz );
> if(answer == NULL)
> {
> xmalloc_failure (sz);
> }
> return answer;
>}
>>
>Which gives the user control over what happens when malloc fails.

>
It's an improvement, but it's still not good enough as a general
solution. Consider this code:
>
foo *foo_create(siz e_t n)
{
static const foo fblank;
foo *new = malloc(sizeof *new); /* A */
if(new != NULL)
{
*new = fblank;
new->bar = malloc(n * sizeof *new->bar); /* B */
if(new->bar == NULL)
{
free(new);
new = NULL;
}
else
{
static const t_bar bblank;
while(n--)
{
new->bar[n] = bblank;
}
}
}
return new;
}
>
If we were to rewrite this to use your suggestion, that would be fine
for the malloc at A, but if the malloc at B fails, we have to clean up,
and that means that the cleanup code needs access to new. Your solution
doesn't address that problem.
>

How about this

void *xmalloc(size_t sz)
{
void *answer;

if(sz == 0)
sz = 1;
while( !(answer = malloc(sz))
(*malloc_failur e(sz));

return answer;
}
--
Free games and programming goodies.

http://www.personal.leeds.ac.uk/~bgy1mm

[Ian Collins]

Re: xmalloc

Richard Heathfield wrote:

Ian Collins said:
>
<snip>
>

>It could be enhanced by providing a (cough) global failure function,
>say xmalloc_failure , that gets called when malloc returns NULL:
>>
>void* default_xmalloc _failure(size_t s) {
> assert(!"Can't allocate memory"); return NULL;}
>>
>void* (*xmalloc_failu re)(size_t) = &default_xmallo c_failure;
>>
>The xmalloc becomes
>>
>void *xmalloc(size_t sz)
>{
> void *answer = malloc( sz == 0 ? 1 : sz );
> if(answer == NULL)
> {
> xmalloc_failure (sz);
> }
> return answer;
>}
>>
>Which gives the user control over what happens when malloc fails.

>
It's an improvement, but it's still not good enough as a general
solution. Consider this code:
>
foo *foo_create(siz e_t n)
{
static const foo fblank;
foo *new = malloc(sizeof *new); /* A */
if(new != NULL)
{
*new = fblank;
new->bar = malloc(n * sizeof *new->bar); /* B */
if(new->bar == NULL)
{
free(new);
new = NULL;
}
else
{
static const t_bar bblank;
while(n--)
{
new->bar[n] = bblank;
}
}
}
return new;
}
>
If we were to rewrite this to use your suggestion, that would be fine
for the malloc at A, but if the malloc at B fails, we have to clean up,
and that means that the cleanup code needs access to new. Your solution
doesn't address that problem.
>

Then you have to turn up the complexity another notch an introduce a
cleanup handler stack, which just about takes us back to checking each
return of malloc()!

This situation is one of the reasons I like that unmentionable in these
parts cousin of C with its exceptions and destructors!

--
Ian Collins.

[Roland Pibinger]

Re: xmalloc

On Sat, 23 Jun 2007 16:30:12 -0400, Eric Sosman wrote:

Not all that strange. Half the programmers in the world
>are below-average.

which is at least half-true.

But if you follow Malcolm's Maxims, even the above-average
>programmers will be helpless. Programs that deal with malloc()
>failure in a significant way may be a minority, but those that
>do it *need* to to do it. Malcolm argues, in essence, that
>such programs should not be written in C.

The discussion basically boils down to one question: Is OOM an error
that reasonably can and should be handled by the application or is it
a fatal error?
The 'fatal error' advocates have already shown how they tackle the
problem. Now it's time for the other camp to demonstrate how OOM can
be consistently _handled_ throughout the program ('return NULL;' is
not enough).


--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch

[Roland Pibinger]

Re: xmalloc

On Sat, 23 Jun 2007 22:16:11 +0100, "Malcolm McLean" wrote:

>Normally if you've written the program in such a way that it makes a lot of
>trival allocations, there is nothing you can do on failure, short of calling
>a back-up routine that uses fixed buffers.
>So you handle fail conditions for the few large legitimate allocations,
>which might run the computer out of memory, and abort on the rest.

The main cause of OOM are memory leaks, i.e. bugs, that cannot be
handled anyway. abort() is more dramatic than exit(). It indicates
that there is a problem and some action ought to be performed now.


--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch