secure integer library

[Keith Thompson]

Re: secure integer library

jacob navia <jacob@jacob.re mcomp.frwrites:
[...]

But any improvement of C in the "more safety"
direction is too much for certain people apparently.
>
Any "argument" counts:
>
"... the end of the earth as we know it"
>
As if anybody caring to ensure the correctness of the software
was half mad or not a "C" programmer.

jacob, will you *please* stop making up this utter crap about people's
motivations?

Nobody other than you has said anything about "the end of the earth as
we know it", or anything similar.

Most of us here are not implementers. As such we have little or no
ability to control what language features are available to us. We
work with compilers that conform to the C standard as it actually
exists (usually C90 with some extensions, sometimes most of C99,
rarely all of C99). This newsgroup is a place for us to discuss
programming in the C language as it is defined by the standard(s) in
the real world.

If you want to discuss techniques for writing safer code *in standard
C*, this is the right place.

If you want to advocate changes to the standard, or extensions outside
the standard, take it somewhere else. (I've told you many times that
comp.std.c is the most appropriate newsgroup for advocating changes to
the standard; if you don't get a favorable reception there, that
doesn't make your ideas topical here.) And you've practically got
your own newsgroup, after all.

And if you want to accuse either the standards committee or C
programmers in general of not caring about safety, or of wanting to
destroy the C language or keep it stagnant, or whatever paranoid
fantasy you've come up with, please do us all a favor and keep it to
yourself.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Mark McIntyre]

Re: secure integer library

On Sat, 19 Aug 2006 21:36:56 GMT, in comp.lang.c , Keith Thompson
<kst-u@mib.orgwrote:

>Nobody other than you has said anything about "the end of the earth as
>we know it", or anything similar.

Actually, Tom St Denis did, though you may have him killfiled and
Jacob chose to quote him in a totally nonstandard way.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

[Mark McIntyre]

Re: secure integer library

On Sat, 19 Aug 2006 20:48:40 +0200, in comp.lang.c , jacob navia
<jacob@jacob.re mcomp.frwrote:

>
>All the problem he has disappear with a C compiler that supports
>operator overloading.

If I wanted a truck, I'd drive a truck. I don't transport cows in my
sportscar. Similarly if I wanted C++ I'd use it.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

[Clark S. Cox III]

Re: secure integer library

jacob navia wrote:

Clark S. Cox III wrote:

>Tom St Denis wrote:
>>

>>CBFalconer wrote:
>>>
>>>There is no such thing. Once an integer overflow occurs the action
>>>is undefined, and the program is invalid. Just because most
>>>systems do ignore them and perform a 2's complement peculiar action
>>>is no excuse.
>>>
>>You mean
>>>
>>unsigned x = 1UL << 31;
>>>
>>x = x + x;
>>>
>>That's going to cause the end of the earth as we know it?
>>>
>>Damn. Ok. Good to know.

>>
>>
>No, simply put, that isn't integer overflow.
>>
>>

>
That is obvious. Besides, nobody is speaking about
overflow for unsigned numbers. Only signed overflow
is undefined behavior according to the standard.

Actually, this is exactly what Mr. St Denis was talking about. I was
simply pointing out that his example was unrelated to the actual
discussion taking place in the thread.

But any improvement of C in the "more safety"
direction is too much for certain people apparently.
>
Any "argument" counts:
>
"... the end of the earth as we know it"
>
As if anybody caring to ensure the correctness of the software
was half mad or not a "C" programmer.

Ensuring correctness is not worth it if it makes my already correct
programs slower.

Ensuring correctness is not worth it if it makes my legacy code
uncompilable.

I don't remember anyone claiming that "more safety" would cause "the end
of the Earth as we know it."

--
Clark S. Cox III
clarkcox3@gmail .com

[Keith Thompson]

Re: secure integer library

Mark McIntyre <markmcintyre@s pamcop.netwrite s:

On Sat, 19 Aug 2006 21:36:56 GMT, in comp.lang.c , Keith Thompson
<kst-u@mib.orgwrote:
>

>>Nobody other than you has said anything about "the end of the earth as
>>we know it", or anything similar.

>
Actually, Tom St Denis did, though you may have him killfiled and
Jacob chose to quote him in a totally nonstandard way.

Actually, I not only read Tom St Denis's use of the phrase "the end of
the earth as we know it", it was also quoted in jacob's article to
which I responded. I just missed it somehow. He used it
sarcastically, but I think I misinterpreted jacob's response to it.

Looking back at the thread, and particularly at jacob's article to
which I was responding, I'm no longer quite sure what point jacob was
trying to make.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[CBFalconer]

Re: secure integer library

Tom St Denis wrote:

CBFalconer wrote:
>

>There is no such thing. Once an integer overflow occurs the action
>is undefined, and the program is invalid. Just because most
>systems do ignore them and perform a 2's complement peculiar action
>is no excuse.

>
You mean
>
unsigned x = 1UL << 31;
>
x = x + x;
>
That's going to cause the end of the earth as we know it?
>
Damn. Ok. Good to know.

That's not an integer overflow, that's an unsigned overflow, and
the action is specified by the standard. Change unsigned to int
(assuming 32 bit ints) and yes, then you have an integer overflow
and UB.

--
Chuck F (cbfalconer@yah oo.com) (cbfalconer@mai neline.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home .att.netUSE maineline address!

[stdazi@gmail.com]

Re: secure integer library


Robert Seacord wrote:

The CERT/CC has released a beta version of a secure integer library for
the C Programming Language. The library is available for download from
the CERT/CC Secure Coding Initiative web page at:

http://www.cert.org/secure-coding/

>
The purpose of this library is to provide a collection of utility
functions that can assist software developers in writing C programs that
are free from common integer problems such as integer overflow, integer
truncation, and sign errors that are a common source of software
vulnerabilities .
>
Functions have been provided for all integer operations subject to
overflow such as addition, subtraction, multiplication, division, unary
negation, etc.) for int, long, long long, and size_t integers. The
following example illustrates how the library can be used to add two
signed long integer values:
>
long retsl, xsl, ysl;
xsl = LONG_MAX;
ysl = 0;
retsl = addsl(xsl,ysl);
>
For short integer types (char and short) it is necessary to truncate the
result of the addition using one of the safe conversion functions
provided, for example:
>
char retsc, xsc, ysc;
xsc = SCHAR_MAX;
ysc = 0;
retsc = si2sc(addsi(xsc , ysc));
>
For error handling, the secure integer library uses the mechanism for
Runtime-constraint handling defined by TR 24731 "Specificat ion for
Safer, More Secure C Library Functions" available at:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1135.pdf

>
The implementation uses the high performance algorithms defined by Henry
S. Warren in the book "Hacker's Delight".
>
For more information on vulnerabilities and other problems resulting
from the incorrect use of integers in C and C++ please read Chapter 5 of
"Secure Coding in C and C++" which is available as a free download from
the CERT web site:
>

http://www.cert.org/books/secure-coding/moreinfo.html

>
Please address any defect reports, comments and suggestions concerning
the Secure Integer Library or CERT Secure Coding Initiative to me.
Thanks to Henry and to Juan Alvarado who coded the implementation.
>
Thanks,
rCs

I was always missing a functionality to detect integer overflows
(sometimes it's a pain when you realize the "wrong answer" is due to a
overflow), but I'd never really like to use the above implementation.

It stunts performance and (most importantly) coding time. I don't want
to think how ugly the code looks in a for loop manipulating many
integers. It's not that hard to put some "limits" in regions of code
you find prone to overflows - after some time, you get used to.

The best place for that kind of utility is IMO the compiler, and IIRC,
there is a patch for gcc too!

Robert C. Seacord
Senior Vulnerability Analyst
CERT/CC
>
Work: 412-268-7608
FAX: 412-268-6989

[rCs]

Re: secure integer library

Random response to some of the ideas I've seen expressed in this
thread.

First of all, when I talk about integers I talk about "unexpected "
behavior. We all know that unsigned integers are defined by the C
standard as modulo so by definition this is the correct behavior. In
practice, however, many programmers fail to code for modulo behavior
resulting in unsigned overflow, incorrect program behavior, and
software vulnerabilities .

Second, integer overflow is not the only problem. For example, the
following code will never overflow for most implementations :

char sc1, sc2, sc_result;

sc_result = sc1 + sc2;

Let's assume for example that signed char is 8 bits and int is 32 bits.
Because of integer promotions both sc1 and sc2 are promoted to signed
integers before this addition takes place. This means overflow is
impossible, but truncation is not. Simply providing an overflow
detection mechanism in this case is insufficient.

Third, I would also like to see range checking built into compilers.
There is of course, a great deal of overhead even here (I have heard
100%-500%). I think my favorite idea is to extend the specification
to allow for a new type of range checked integers, for example:

int i 25:100 = 35;

This is similar to the range clause in Ada and would in this example
declare an integer with a range of 25 to 100, initialized to 35. The
compiler would be required to make sure the range is not violated.

I don't think performance is an issue in this case because you won't
get the overhead unless you use the feature and in that case you are
only paying for what you get.

Error handling is more of a problem. How do you report a range error
given the lack of an error handling mechanism in C? My best solution
would be to implement the runtime constraint mechanism used in TR 24731
and the secure integer library that started this thread but I am open
to other ideas.

If you expect your existing compiler to provide integer range checking,
good luck! I know that gcc provides an -ftrapv flag that provides
overflow protection for a small list of signed integer operations.
Their implementation handles errors by calling abort(). As far as I
know, Visual Studio offers no overflow protection although they have
tried to tighten up their compiler warnings for integer truncation.

Fourth, I agree that integer overflow protection is not necessary
everywhere. Protecting each operation is sort of a sloppy approach to
limiting the ranges of values at input and making sure that all the
intermediate representations of integer values have sufficient
precision. However, this can be difficult to guarantee when you have
multiple integer inputs that are combined in a variety of ways. When
no such guarantees are possible, I recommend the use of this library
(or similar protections) for integer values that originate from
untrusted sources (e.g., users) and then are used as sizes, lengths,
loop counters, or indicies as the incorrect use of integers in these
cases most common results in exploitable vulnerabilities .

rCs

[jacob navia]

Re: secure integer library

rCs a écrit :

Random response to some of the ideas I've seen expressed in this
thread.
>
First of all, when I talk about integers I talk about "unexpected "
behavior. We all know that unsigned integers are defined by the C
standard as modulo so by definition this is the correct behavior. In
practice, however, many programmers fail to code for modulo behavior
resulting in unsigned overflow, incorrect program behavior, and
software vulnerabilities .
>
Second, integer overflow is not the only problem. For example, the
following code will never overflow for most implementations :
>
char sc1, sc2, sc_result;
>
sc_result = sc1 + sc2;
>
Let's assume for example that signed char is 8 bits and int is 32 bits.
Because of integer promotions both sc1 and sc2 are promoted to signed
integers before this addition takes place. This means overflow is
impossible, but truncation is not. Simply providing an overflow
detection mechanism in this case is insufficient.
>

Good point, however it would be impossible to test for this at run time
since this "feature" is part of the expected behavior of
many programs

Third, I would also like to see range checking built into compilers.
There is of course, a great deal of overhead even here (I have heard
100%-500%). I think my favorite idea is to extend the specification
to allow for a new type of range checked integers, for example:
>
int i 25:100 = 35;
>
This is similar to the range clause in Ada and would in this example
declare an integer with a range of 25 to 100, initialized to 35. The
compiler would be required to make sure the range is not violated.
>

Interesting idea.

I don't think performance is an issue in this case because you won't
get the overhead unless you use the feature and in that case you are
only paying for what you get.
>

Exactly.

Error handling is more of a problem. How do you report a range error
given the lack of an error handling mechanism in C? My best solution
would be to implement the runtime constraint mechanism used in TR 24731
and the secure integer library that started this thread but I am open
to other ideas.
>

Yes, I proposed using TR 24731 error handling mechanism as a default
error handling mechanism for C in the comp.lang.c discussion group.

http://groups.google.fr/group/comp.std.c/browse_thread/thread/323acf65bf5211e9/1e5d0a124fe18c1c?hl=fr#1e5d0a124fe18c1c

Obviously that is an interesting idea since silence was the only answer.
Skarmander and Keith Thompson answered but none of the comitee guys
answered since obviously I have complained (and loudly) about gets()
being still in the standard...

If you expect your existing compiler to provide integer range checking,
good luck! I know that gcc provides an -ftrapv flag that provides
overflow protection for a small list of signed integer operations.
Their implementation handles errors by calling abort(). As far as I
know, Visual Studio offers no overflow protection although they have
tried to tighten up their compiler warnings for integer truncation.
>
Fourth, I agree that integer overflow protection is not necessary
everywhere. Protecting each operation is sort of a sloppy approach to
limiting the ranges of values at input and making sure that all the
intermediate representations of integer values have sufficient
precision. However, this can be difficult to guarantee when you have
multiple integer inputs that are combined in a variety of ways. When
no such guarantees are possible, I recommend the use of this library
(or similar protections) for integer values that originate from
untrusted sources (e.g., users) and then are used as sizes, lengths,
loop counters, or indicies as the incorrect use of integers in these
cases most common results in exploitable vulnerabilities .
>
rCs
>

[Keith Thompson]

Re: secure integer library

"rCs" <rcs@cert.orgwr ites:
[snip]

Second, integer overflow is not the only problem. For example, the
following code will never overflow for most implementations :
>
char sc1, sc2, sc_result;
>
sc_result = sc1 + sc2;
>
Let's assume for example that signed char is 8 bits and int is 32 bits.
Because of integer promotions both sc1 and sc2 are promoted to signed
integers before this addition takes place. This means overflow is
impossible, but truncation is not. Simply providing an overflow
detection mechanism in this case is insufficient.

[snip]

Truncation isn't the only possibility.

First, let's assume that int is bigger than char. If it isn't (which
requires CHAR_BIT 8), then some other odd things can happen; we'll
ignore those unlikely possibilities.

sc1 and sc2 are promoted (i.e., converted) to type int and added,
yielding a result of type int. So far, no overflow is possible. Then
the result is assigned to sc_result -- but first it must be converted
from int to char.

For an arithmetic operations on a signed integer type, overflow
invokes undefined behavior. For a *conversion* to a signed integer
type, where the target type cannot represent the value, either the
result is implementation-defined or an implementation-defined signal
is raised (the latter possibility was introduced in C99).

For signed integer overflow on both arithmetic operations and
conversions, the most common behavior is that the high-order bits are
discarded, but the standard allows for many other possibilities.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Mark McIntyre]

Re: secure integer library

On Sun, 20 Aug 2006 19:15:22 +0200, in comp.lang.c , jacob navia
<jacob@jacob.re mcomp.frwrote:

>Skarmander and Keith Thompson answered but none of the comitee guys
>answered since obviously I have complained (and loudly) about gets()
>being still in the standard...

The part from "since" onwards is unnecessary, insulting and
gratuitous, and explains exactly why people killfile you. If you want
to be listened to, stop making defamatory remarks.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

[Keith Thompson]

Re: secure integer library

jacob navia <jacob@jacob.re mcomp.frwrites:
[...]

Obviously that is an interesting idea since silence was the only answer.
Skarmander and Keith Thompson answered but none of the comitee guys
answered since obviously I have complained (and loudly) about gets()
being still in the standard...

I've also complained about gets() being in the standard, and I get
responses from committee members all the time.

jacob, it seems to me that you are *really bad* at guessing other
people's motivations. That's ok (not everyone can be good at
everything), but I suggest that you stop trying to do so, at least in
public, until you figure out how to do it right. It would greatly
improve your personal signal-to-noise ratio, and would likely cause
people to pay more serious attention to your ideas.

I admit I'm being a bit sarcastic here, but this is also intended as
serious advice.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[jacob navia]

Re: secure integer library

Mark McIntyre a écrit :

On Sun, 20 Aug 2006 19:15:22 +0200, in comp.lang.c , jacob navia
<jacob@jacob.re mcomp.frwrote:
>
>

>>Skarmander and Keith Thompson answered but none of the comitee guys
>>answered since obviously I have complained (and loudly) about gets()
>>being still in the standard...

>
>
The part from "since" onwards is unnecessary, insulting and
gratuitous, and explains exactly why people killfile you. If you want
to be listened to, stop making defamatory remarks.

Mr Gwyn, that is related to the people in the comitee
answered AT LENGTH defending gets().

You can read that thread in Google:
Thread name: Why does rewind() ignore errors?
Thread author: Keith Thompson
Date: May 26th 2006 21:34

There you will se Mr Gwyn defending gets() and treating people that
wanted that function away from the standard as "fanatics".

This was done under the thunderous SILENCE of the other comitee members.

NONE of them cared to express a different view.

I am not insulting anybody, neither did I treat them as "fanatics"
as they did to me.

jacob

[jacob navia]

Re: secure integer library

Keith Thompson a écrit :

jacob navia <jacob@jacob.re mcomp.frwrites:
[...]
>

>>Obviously that is an interesting idea since silence was the only answer.
>>Skarmander and Keith Thompson answered but none of the comitee guys
>>answered since obviously I have complained (and loudly) about gets()
>>being still in the standard...

>
>
I've also complained about gets() being in the standard, and I get
responses from committee members all the time.
>

Yes, all of them defending gets().

Maybe I am exaggerating. I hope so actually that it is just an
oversight, but really, after that discussion about gets() there is
nothing that can be done there.

The next standard is not even "in the works", and language evolution
is completely at a standstill, because, (as they have often repeated)
their job is to leave the language as it is without any change
so that legacy software doesn't get disturbed and C can die a
quite death without even a whisper.

C++ is where all development is done. The language is relatively
new, and people there accept that a language must be improved
by the comitee, that in C++ has a very positive function.

Contrast that with the C situation where getting rid of a bug
like gets() implies insults from the comitee representatives
like "fanatics".

Nothing is done, no new directions, no improvements and proposal like
getting rid of trigraphs/gets()/asctime()/ provoke only answers
like "nothing will be done".

But maybe there are people in the comitee that have another opinion,
they just do not participate to any discussion.

I spoke with the French standards comitee and they would maybe care to
review my suggestions AFTER I spend 10 000 euros in costs...

So I am stuck and neither comp.lang.c nor comp.std.c discussions will
change anything.

jacob

[Keith Thompson]

Re: secure integer library

jacob navia <jacob@jacob.re mcomp.frwrites:
[...]

The next standard is not even "in the works", and language evolution
is completely at a standstill, because, (as they have often repeated)
their job is to leave the language as it is without any change
so that legacy software doesn't get disturbed and C can die a
quite death without even a whisper.

You see, this is exactly what I meant when I said that you are *really
bad* at guessing other people's motivations.

I don't believe that *anybody* who has participated in these
discussions wants C to "die a quiet death". That's just your
interpretation, and it's absolutely wrong. I've spent far too much
time trying to help you understand that, but apparently you're
unwilling to listen.

[...]

So I am stuck and neither comp.lang.c nor comp.std.c discussions will
change anything.

So do us all a favor and stop posting here.

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.