fwrite output is not efficient (fast) ?

[Abubakar]

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

regards,

...ab

[Richard Bos]

Re: fwrite output is not efficient (fast) ?

Abubakar <abubakarm@gmai l.comwrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing.

That most probably wasn't a C programmer, but a GNU-almost-POSIX-
with-a-bit-of-C-thrown-in-for-appearance's-sake programmer.

Is that true?

As phrased, it is not true. It may be true for some systems, under some
circumstances; but you should never worry about optimisation until you
_know_ that you have to, and don't just think you might.

Richard

[Gordon Burditt]

Re: fwrite output is not efficient (fast) ?

>recently some C programmer told me that using fwrite/fopen functions

>are not efficient because the output that they do to the file is
>actually buffered and gets late in writing. Is that true?

For some programs I have benchmarked (which were essentially doing
disk-to-disk file copies), using single-character read() and write()
calls to copy files chews up a lot of CPU time. Switching to
getc()/putc()/fopen() calls reduced the CPU used to copy a given
file by a factor of 10 and the wall clock time to finish copying
the file by about 10%. In this particular situation, buffering
helped speed things up a lot.

[Richard Heathfield]

Re: fwrite output is not efficient (fast) ?

Abubakar said:

Hi,
>
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

I suggest you ask your friend:

1) why he thinks fopen outputs anything at all to a file;
2) why he thinks the buffering status of a file depends entirely on fopen
or fwrite;
3) what he thinks setbuf and setvbuf are for;
4) what he means by "efficient" .

You may well find the answers to those questions illuminating. On the other
hand, you might not.

1), 2), and 3) should all send him back-pedalling, and 4) should at least
make him stop and think about the real issues here.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Nate Eldredge]

Re: fwrite output is not efficient (fast) ?

Abubakar <abubakarm@gmai l.comwrites:

Hi,
>
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

It's true that the output is buffered (usually). Depending on your
situation, that may be good or bad. If you are calling fwrite with
large chunks of data (many kbytes), the buffering will not gain you
much, but a sensible implementation will bypass it and you'll only have
a small amount of overhead. If you are writing small chunks or single
characters at a time (for instance, with fprintf() or putc()), the
buffering can speed things up drastically, since it cuts down on calls
to the operating system, which can be expensive.

It is true in this case that some data may not be written as soon as you
call fwrite(), but this doesn't slow down the data transfer in the long
run. If it's important for some reason that a small bit of data go out
immediately, you can call fflush() or use setvbuf() to change the
buffering mode, but at a cost to overall efficiency.

[s0suk3@gmail.com]

Re: fwrite output is not efficient (fast) ?

On Oct 24, 2:03 am, Abubakar <abubakarm@gmai l.comwrote:

Hi,
>
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

The purpose of buffering is actually efficiency; transferring a large
block of data to disk at once is faster than transferring it byte by
byte (and by this I don't mean that singe-byte functions such as
getc() and putc() are inefficient; these functions also read/write to
or from a buffer in memory).

This is usually true not only for disk files, but also for I/O devices
in general such as network sockets. In some applications, however, it
can be convenient to develop a more specialized I/O layer model that
can provide more functionality and improve efficiency in specific
situations.

(If you're just worried that the data will remain in the buffer
indefinitely, just call fflush() or close the file.)

Sebastian

[Abubakar]

Re: fwrite output is not efficient (fast) ?

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

I have given him the link to this discussion and he is going to be
posting his replies soon hopefully.

On Oct 24, 2:00 pm, Richard Heathfield <r...@see.sig.i nvalidwrote:

Abubakar said:
>

Hi,

>

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

>
I suggest you ask your friend:
>
1) why he thinks fopen outputs anything at all to a file;
2) why he thinks the buffering status of a file depends entirely on fopen
or fwrite;
3) what he thinks setbuf and setvbuf are for;
4) what he means by "efficient" .
>
You may well find the answers to those questions illuminating. On the other
hand, you might not.
>
1), 2), and 3) should all send him back-pedalling, and 4) should at least
make him stop and think about the real issues here.
>
--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Richard Heathfield]

Re: fwrite output is not efficient (fast) ?

Abubakar said:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.

I suggest you ask your friend the questions that I asked you to ask him.

He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

FILE * is a pointer type. It doesn't have a speed.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Abubakar]

Re: fwrite output is not efficient (fast) ?

Well he read the link and now he says that the replies by Nate
Eldredge and the guy whose email starts with "s0s" have said what
proves what he says is right so he says there is no need to reply to
anything. ummmm , i dont know whats up. I am going to be posting more
questions to clear things that he has told me because he has much more
experience in C than me. Thanks for the replies so far, if you guys
have more comments please continue posting.

On Oct 24, 1:49 pm, gordonb.l2...@b urditt.org (Gordon Burditt) wrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

>
For some programs I have benchmarked (which were essentially doing
disk-to-disk file copies), using single-character read() and write()
calls to copy files chews up a lot of CPU time.  Switching to
getc()/putc()/fopen() calls reduced the CPU used to copy a given
file by a factor of 10 and the wall clock time to finish copying
the file by about 10%.  In this particular situation, buffering
helped speed things up a lot.

[James Kuyper]

Re: fwrite output is not efficient (fast) ?

Abubakar wrote:

Hi,
>
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

It's actually backwards, in general. It would be closer to being
accurate to say that they are more efficient because they are buffered
(but event that statement isn't quite true). The purpose of buffering
the data is to take advantage of the fact that, in most cases, it's more
efficient to transfer many bytes at a time. When transferring a large
number of bytes, using unbuffered writes gets the first byte out earlier
than using buffered writes; but gets the last byte out later.

Which approach is better depends upon your application, but in many
contexts the earlier bytes can't (or at least won't) even be used until
after the later bytes have been written, which makes buffered I/O the
clear winner.

Furthermore, C gives you the option of turning off buffering with
setvbuf(). If you do, then the behavior should be quite similar to that
you would get from using system-specific unbuffered I/O functions (such
as write() on Unix-like systems).

[James Kuyper]

Re: fwrite output is not efficient (fast) ?

Abubakar wrote:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

fwrite() buffers data only if you don't tell it not to, by calling
setvbuf(). When talking about buffered I/O, he's right about the
predictability, if he's talking about predicting when data gets written
to the file. That depends upon the details of the buffering scheme,
which are in general unknown to the user.

However, what he's saying about speed is true only if he's mainly
concerned with the speed with which the first byte reaches a file. If
he's concerned with the speed with which the last byte reaches the file,
buffering is generally faster, at least with sufficiently large files.

[Richard Bos]

Re: fwrite output is not efficient (fast) ?

James Kuyper <jameskuyper@ve rizon.netwrote:

Abubakar wrote:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

>
fwrite() buffers data only if you don't tell it not to, by calling
setvbuf(). When talking about buffered I/O, he's right about the
predictability, if he's talking about predicting when data gets written
to the file.

Not even that. The OS may have its own buffers, and so may the drive
firmware. Bottom line, if you want absolute file security, you have to
nail it down to the hardware level. If you don't need that, 99+% of the
time, ISO C <stdio.hfunctio ns are good enough.

Richard

[Richard Heathfield]

Re: fwrite output is not efficient (fast) ?

Abubakar said:

Well he read the link and now he says that the replies by Nate
Eldredge and the guy whose email starts with "s0s" have said what
proves what he says is right

They have both argued the opposite point of view. They may or may not have
proved /their/ case, but they certainly haven't proved /his/.

so he says there is no need to reply to anything.

Of course not. If he wishes to continue in blissful ignorance, that's his
choice! :-)

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[William Pursell]

Re: fwrite output is not efficient (fast) ?

On 24 Oct, 08:03, Abubakar <abubak...@gmai l.comwrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

No. Sometimes using fwrite() is less efficient than using
(the non-standard function) write(), sometimes it is
more efficient. For example:

fwrite( buf, 1, 8192, fp );

is likely to be less efficient than

write( fd, buf, 8192 );

but

for( i = 0; i < 8192; i++ )
fwrite( buf + i, 1, 1, fp );

is likely to be more efficient than

for( i = 0; i < 8192; i++ )
write( fd, buf + i, 1 );

In the loop, fwrite() is likely to be more efficient
**because of** the buffering. In the first example, fwrite()
is likely to be less efficient because there is (usually)
an extra data move that doesn't occur with write().

However, the portability of fwrite() is a significant
feature to be considered, as is its ease of use. Unless
you have a verified, measured need to replace fwrite()
with write(), don't bother.

[CBFalconer]

Re: fwrite output is not efficient (fast) ?

Abubakar wrote:

>
well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.
>
I have given him the link to this discussion and he is going to be
posting his replies soon hopefully.

Please do not top-post. Your answer belongs after (or intermixed
with) the quoted material to which you reply, after snipping all
irrelevant material. Your top-posting has lost all continuity from
the thread. See the following links:

<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html >
<http://www.netmeister. org/news/learn2quote.htm l>
<http://cfaj.freeshell. org/google/ (taming google)
<http://members.fortune city.com/nnqweb/ (newusers)

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home .att.net>
Try the download section.