snprint rationale?

[Michael B Allen]

What is the rationale for snprintf to "return the number of characters
(excluding the trailing '\0') which would have been written to the final
string if enough space had been available"?

This just twists my noodle in a knot every time! What is the proper way
to test the return value for overflow?

What is the name and address of the person responsible for this?

Mike

[Keith Thompson]

Re: snprint rationale?

Michael B Allen <mba2000@ioplex .com> writes:[color=blue]
> What is the rationale for snprintf to "return the number of characters
> (excluding the trailing '\0') which would have been written to the final
> string if enough space had been available"?
>
> This just twists my noodle in a knot every time! What is the proper way
> to test the return value for overflow?[/color]

The declaration of snprintf() is:

int snprintf(char * restrict s, size_t n,
const char * restrict format, ...);

The number of characters available is passed in as the second argument.
To test for overflow, check whether the result exceeds n.

If you call snprintf() with n==0; it won't write any characters, but
it will return the number of characters that would have been written.
You can then allocate the appropriate space and call snprintf() again
with the same arguments, but with a non-zero n.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Chris Torek]

Re: snprint rationale?

In article <pan.2004.11.22 .03.43.11.23569 5.512@ioplex.co m>
Michael B Allen <mba2000@ioplex .com> wrote:[color=blue]
>What is the rationale for snprintf to "return the number of characters
>(excluding the trailing '\0') which would have been written to the final
>string if enough space had been available"?[/color]

This lets you allocate a buffer that is big enough, without having
to do many passes:

needed = snprintf(NULL, 0, fmt, arg1, arg2);
if (needed < 0) ... handle error ...
mem = malloc(needed + 1);
if (mem == NULL) ... handle error ...
result = snprintf(mem, needed + 1, fmt, arg1, arg2);

It is also consistent with fprintf(), which returns the number of
characters printed.
[color=blue]
>This just twists my noodle in a knot every time! What is the proper way
>to test the return value for overflow?[/color]

Given a buffer "buf" of size "size":

result = snprintf(buf, size, fmt, arg);

if (result >= 0 && result < size)
all_is_well();
else
needed_more_spa ce();
[color=blue]
>What is the name and address of the person responsible for this?[/color]

That would be me. :-)
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: forget about it http://web.torek.net/torek/index.html
Reading email is like searching for food in the garbage, thanks to spammers.

[Erik de Castro Lopo]

Re: snprint rationale?

Michael B Allen wrote:[color=blue]
>
> What is the rationale for snprintf to "return the number of characters
> (excluding the trailing '\0') which would have been written to the final
> string if enough space had been available"?
>
> This just twists my noodle in a knot every time! What is the proper way
> to test the return value for overflow?[/color]

Snprintf is guaranteed not to overflow.

This works:

if (snprintf (buf, buflen, "...", ....) < buflen)
puts ("No overflow occurred");
else
puts ("Overflow might have occurred");

This also works:

buflen = snprintf (NULL, 0, "...", ....);
buf = malloc (buflen + 1) ;
snprintf (buf, buflen, "...", ....);



Erik
--
+-----------------------------------------------------------+
Erik de Castro Lopo nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
Seen on comp.lang.pytho n:
Q : If someone has the code in python for a buffer overflow,
please post it.
A : Python does not support buffer overflows, sorry.

[Michael Mair]

Re: snprint rationale?



Michael B Allen wrote:[color=blue]
> What is the rationale for snprintf to "return the number of characters
> (excluding the trailing '\0') which would have been written to the final
> string if enough space had been available"?[/color]

It allows you to find out by how much to enlarge your final string
in order to fit it in. What is troubling me about this is that snprintf
takes a size_t parameter and returns an int, which is broken by design.
Returning 0 on error and the hypothetical number of written characters
including the string terminator with return type size_t would IMO have
been better.

[color=blue]
> This just twists my noodle in a knot every time! What is the proper way
> to test the return value for overflow?[/color]

From the C99 standard:
"7.19.6.5 The snprintf function
Synopsis
1
#include <stdio.h> int snprintf(char * restrict s, size_t n,
const char * restrict format, ...);

Description
2 The snprintf function is equivalent to fprintf, except that the output
is written into an array (specified by argument s) rather than to a
stream. If n is zero, nothing is written, and s may be a null pointer.
Otherwise, output characters beyond the n-1st are discarded rather than
being written to the array, and a null character is written at the end
of the characters actually written into the array. If copying takes
place between objects that overlap, the behavior is undefined.

Returns
3 The snprintf function returns the number of characters that would have
been written had n been sufficiently large, not counting the terminating
null character, or a neg ative value if an encoding error occurred.
Thus, the null-terminated output has been completely written if and only
if the returned value is nonnegative and less than n.
"

So, I would use allocated buffers and do something along the lines

char *buf;
size_t buf_size;
int retval;

buf = NULL;
buf_size = 0;

while (1) {
retval = snprintf(buf, buf_size, "Test with buf of size %zu\n",
buf_size);
if (retval < 0) {
/* Treat encoding error or die */
}
else if (retval<buf_siz e) {
break; /* We finally made it */
}
else {
char *tmp;
if ( (tmp=realloc(bu f, (size_t) retval + 1)) == NULL ) {
/* Give up trying to write this string or die */
}
buf = tmp;
buf_size = (size_t) retval + 1;
}
}

I did not test it but you see that it deals with the problem
that, depending on buf_size, the length of the output varies
so we need to adjust the size a second time.

[color=blue]
> What is the name and address of the person responsible for this?[/color]

I think this is slighty OT here. Try comp.std.c but I guess
they won't tell you either.


-Michael
--
E-Mail: Mine is a gmx dot de address.

[pete]

Re: snprint rationale?

Michael Mair wrote:[color=blue]
>
> Michael B Allen wrote:[color=green]
> > What is the rationale for snprintf to "return the number of characters
> > (excluding the trailing '\0') which would have been written to the final
> > string if enough space had been available"?[/color]
>
> It allows you to find out by how much to enlarge your final string
> in order to fit it in. What is troubling me about this
> is that snprintf takes a size_t parameter and returns an int,
> which is broken by design.[/color]

There's also an environmental limit, which is the minimum value for
the maximum number of characters produced by any single conversion:
509 in C89,
4095 in C99.

--
pete

[Michael Mair]

Re: snprint rationale?



pete wrote:[color=blue]
> Michael Mair wrote:
>[color=green]
>>Michael B Allen wrote:
>>[color=darkred]
>>>What is the rationale for snprintf to "return the number of characters
>>>(excluding the trailing '\0') which would have been written to the final
>>>string if enough space had been available"?[/color]
>>
>>It allows you to find out by how much to enlarge your final string
>>in order to fit it in. What is troubling me about this
>>is that snprintf takes a size_t parameter and returns an int,
>>which is broken by design.[/color]
>
>
> There's also an environmental limit, which is the minimum value for
> the maximum number of characters produced by any single conversion:
> 509 in C89,
> 4095 in C99.[/color]

Thank you :-)
I was completely unaware of this.
However, this does not really affect that this switching of types
in between is ugly.

Cheers
Michael
--
E-Mail: Mine is a gmx dot de address.

[pete]

Re: snprint rationale?

Michael Mair wrote:[color=blue]
>
> pete wrote:[color=green]
> > Michael Mair wrote:
> >[color=darkred]
> >>Michael B Allen wrote:
> >>
> >>>What is the rationale for snprintf to
> >>>"return the number of characters
> >>>(excluding the trailing '\0')
> >>>which would have been written to the final
> >>>string if enough space had been available"?
> >>
> >>It allows you to find out by how much to enlarge your final string
> >>in order to fit it in. What is troubling me about this
> >>is that snprintf takes a size_t parameter and returns an int,
> >>which is broken by design.[/color]
> >
> >
> > There's also an environmental limit, which is the minimum value for
> > the maximum number of characters produced by any single conversion:
> > 509 in C89,
> > 4095 in C99.[/color]
>
> Thank you :-)
> I was completely unaware of this.
> However, this does not really affect that this switching of types
> in between is ugly.[/color]

I think it has to do with snprintf being based on the
functionality of fprintf and with fprintf being older than size_t.

--
pete

[Richard Bos]

Re: snprint rationale?

Michael B Allen <mba2000@ioplex .com> wrote:
[color=blue]
> What is the rationale for snprintf to "return the number of characters
> (excluding the trailing '\0') which would have been written to the final
> string if enough space had been available"?
>
> This just twists my noodle in a knot every time! What is the proper way
> to test the return value for overflow?[/color]

So what else would you have it return? The number of characters it
actually did write? That's almost always useless information, since it's
easily found using strlen(). The number of characters it would've
written had it had the space, however, is very useful.

Richard

[Keith Thompson]

Re: snprint rationale?

Erik de Castro Lopo <nospam@mega-nerd.com> writes:
[...][color=blue]
> Snprintf is guaranteed not to overflow.[/color]

Well, sort of; it will overflow if you tell it to.

For example,

char buf[5];
snprintf(buf, 30, "%s", "This string is too big");

But assuming the arguments are consistent, yes, it's guaranteed not to
overflow.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Keith Thompson]

Re: snprint rationale?

Michael Mair <Michael.Mair@i nvalid.invalid> writes:[color=blue]
> Michael B Allen wrote:[color=green]
>> What is the rationale for snprintf to "return the number of characters
>> (excluding the trailing '\0') which would have been written to the final
>> string if enough space had been available"?[/color]
>
> It allows you to find out by how much to enlarge your final string
> in order to fit it in. What is troubling me about this is that snprintf
> takes a size_t parameter and returns an int, which is broken by design.
> Returning 0 on error and the hypothetical number of written characters
> including the string terminator with return type size_t would IMO have
> been better.[/color]
[...]

The following:

snprintf(buf, buf_size, "");

is a legitimate call to snprintf; it returns 0 but doesn't indicate an
error.

If ISO C had a "ssize_t" type (a signed equivalent of size_t), this
would be a good place to use it. (POSIX defines ssize_t; ISO C
doesn't.)

An alternative might be to have the return value just indicate success
or failure, and return the number of bytes via a separate size_t*
argument, but that would make the function more difficult to use.

In practice, returning int is only going to be a problem if the length
of the string would exceed INT_MAX characters. This is unlikely on
systems with 16-bit int, and even more unlikely on systems with 32-bit
or larger int. I agree that it's a wart, but I'm not sure there's a
good way to fix it.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Michael Mair]

Re: snprint rationale?

Keith Thompson wrote:
[color=blue]
> Michael Mair <Michael.Mair@i nvalid.invalid> writes:
>[color=green]
>>Michael B Allen wrote:
>>[color=darkred]
>>>What is the rationale for snprintf to "return the number of characters
>>>(excluding the trailing '\0') which would have been written to the final
>>>string if enough space had been available"?[/color]
>>
>>It allows you to find out by how much to enlarge your final string
>>in order to fit it in. What is troubling me about this is that snprintf
>>takes a size_t parameter and returns an int, which is broken by design.
>>Returning 0 on error and the hypothetical number of written characters
>>including the string terminator with return type size_t would IMO have
>>been better.[/color]
>
> [...]
>
> The following:
>
> snprintf(buf, buf_size, "");
>
> is a legitimate call to snprintf; it returns 0 but doesn't indicate an
> error.[/color]

With my suggestion, this would have returned 1 ('\0') which is distinct
from 0 :-)

[color=blue]
> If ISO C had a "ssize_t" type (a signed equivalent of size_t), this
> would be a good place to use it. (POSIX defines ssize_t; ISO C
> doesn't.)[/color]

Yep, I really do not understand why we were not given that toy by
C99... especially since at other places the standard goes to a length
avoiding to say ssize_t (for example when describing the *printf/*scanf
length modifier z, referring to size_t or the corresponding signed
type...).
Losing half the positive range of size_t is certainly better than
a potential int/size_t problem.

[color=blue]
> An alternative might be to have the return value just indicate success
> or failure, and return the number of bytes via a separate size_t*
> argument, but that would make the function more difficult to use.[/color]

Indeed.

[color=blue]
> In practice, returning int is only going to be a problem if the length
> of the string would exceed INT_MAX characters. This is unlikely on
> systems with 16-bit int, and even more unlikely on systems with 32-bit
> or larger int. I agree that it's a wart, but I'm not sure there's a
> good way to fix it.[/color]

Well, apart from the differences to fprintf() which will lead to
problems with people too lazy to look up snprintf(), I still hold
that returning -- as size_t value -- the numbers of characters to
be written _including_ the string terminator or zero on error would
have been the easiest and probably best way.
However, this is purely academical as we already have the wart.


Cheers
Michael
--
E-Mail: Mine is an /at/ gmx /dot/ de address.

[Michael B Allen]

Re: snprint rationale?

On Mon, 22 Nov 2004 04:15:02 -0500, Chris Torek wrote:
[color=blue]
> In article <pan.2004.11.22 .03.43.11.23569 5.512@ioplex.co m> Michael B
> Allen <mba2000@ioplex .com> wrote:[color=green]
>>What is the rationale for snprintf to "return the number of characters
>>(excluding the trailing '\0') which would have been written to the final
>>string if enough space had been available"?[/color]
>
> This lets you allocate a buffer that is big enough, without having to do
> many passes:
>
> needed = snprintf(NULL, 0, fmt, arg1, arg2); if (needed < 0) ...
> handle error ...
> mem = malloc(needed + 1);
> if (mem == NULL) ... handle error ... result = snprintf(mem, needed
> + 1, fmt, arg1, arg2);[/color]

I see. This is reasonable. I was wondering why it didn't just return -1
but I prefer this behavior. If I want something dumber I can wrap it.

Thanks,
Mike

[Keith Thompson]

Re: snprint rationale?

Michael Mair <Michael.Mair@i nvalid.invalid> writes:[color=blue]
> Keith Thompson wrote:[color=green]
>> Michael Mair <Michael.Mair@i nvalid.invalid> writes:[color=darkred]
>>>Michael B Allen wrote:
>>>
>>>>What is the rationale for snprintf to "return the number of characters
>>>>(excludin g the trailing '\0') which would have been written to the final
>>>>string if enough space had been available"?
>>>
>>>It allows you to find out by how much to enlarge your final string
>>>in order to fit it in. What is troubling me about this is that snprintf
>>>takes a size_t parameter and returns an int, which is broken by design.
>>>Returning 0 on error and the hypothetical number of written characters
>>>including the string terminator with return type size_t would IMO have
>>>been better.[/color]
>> [...]
>> The following:
>> snprintf(buf, buf_size, "");
>> is a legitimate call to snprintf; it returns 0 but doesn't indicate
>> an
>> error.[/color]
>
> With my suggestion, this would have returned 1 ('\0') which is distinct
> from 0 :-)[/color]

Right, I missed the "including the string terminator" clause. I think
that would be counterintuitiv e, since most similar functions return
the length (strlen()) of the string excluding the terminator. But in
any case we're stuck with the current behavior.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[pete]

Re: snprint rationale?

Michael Mair wrote:[color=blue]
>
> Keith Thompson wrote:[/color]
[color=blue][color=green]
> > If ISO C had a "ssize_t" type (a signed equivalent of size_t),[/color][/color]

That's how I think ptrdiff_t should have been defined.
[color=blue]
> Losing half the positive range of size_t is certainly better than
> a potential int/size_t problem.[/color]

--
pete