Reading from a stream til EOF

[Jonathan Turkanis]

Re: Reading from a stream til EOF


"Hendrik Schober" <SpamTrap@gmx.d e> wrote in message
news:c1l3es$6fo $1@news1.transm edia.de...[color=blue]
> tom_usenet <tom_usenet@hot mail.com> wrote:[color=green]
> > [...][/color][/color]
[color=blue]
> I didn't think of seeking through a
> stream to get its size! Of all the
> reasons I wanted to do this I did
> manage to eliminate all except that
> I need the size of the data to be
> read from the stream. Since you just
> showed me how to get this, I won't
> even need to read the whole thing
> into a string anymore![/color]

This is fine depending on the stream type. As I'm sure you know, an
arbitrary stream deosn't have to be arbitrarily-positional. If you
know that the streams you will be using are arbitrarily-positional,
you're all set.

You could try seeking, and then testing whether the result is a valid
stream poosition. If it's not, you could then use another method.
However, I'm not sure its guaranteed that a stream will be in a valid
state after a failed seek.

Jonathan

[Hendrik Schober]

Re: Reading from a stream til EOF

Jonathan Turkanis <technews@kanga roologic.com> wrote:[color=blue]
> [...][color=green]
> > I didn't think of seeking through a
> > stream to get its size! Of all the
> > reasons I wanted to do this I did
> > manage to eliminate all except that
> > I need the size of the data to be
> > read from the stream. Since you just
> > showed me how to get this, I won't
> > even need to read the whole thing
> > into a string anymore![/color]
>
> This is fine depending on the stream type. As I'm sure you know, an
> arbitrary stream deosn't have to be arbitrarily-positional. If you
> know that the streams you will be using are arbitrarily-positional,
> you're all set.
>
> You could try seeking, and then testing whether the result is a valid
> stream poosition. If it's not, you could then use another method.
> However, I'm not sure its guaranteed that a stream will be in a valid
> state after a failed seek.[/color]

How do I detect a failed positioning?

Mhmm. Right now it will be file streams
and string streams only which I assume
to be positional. I think I will try
this and put an assert to be triggered
if anything fails.
[color=blue]
> Jonathan[/color]

Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Sometimes compilers are so much more reasonable than people."
Scott Meyers

[Jonathan Turkanis]

Re: Reading from a stream til EOF


"Hendrik Schober" <SpamTrap@gmx.d e> wrote in message
news:c1lgqg$a4p $1@news1.transm edia.de...[color=blue]
> Jonathan Turkanis <technews@kanga roologic.com> wrote:[color=green]
> > [...][color=darkred]
> > > I didn't think of seeking through a
> > > stream to get its size! Of all the
> > > reasons I wanted to do this I did
> > > manage to eliminate all except that
> > > I need the size of the data to be
> > > read from the stream. Since you just
> > > showed me how to get this, I won't
> > > even need to read the whole thing
> > > into a string anymore![/color]
> >
> > This is fine depending on the stream type. As I'm sure you know,[/color][/color]
an[color=blue][color=green]
> > arbitrary stream deosn't have to be arbitrarily-positional. If you
> > know that the streams you will be using are[/color][/color]
arbitrarily-positional,[color=blue][color=green]
> > you're all set.
> >
> > You could try seeking, and then testing whether the result is a[/color][/color]
valid[color=blue][color=green]
> > stream poosition. If it's not, you could then use another method.
> > However, I'm not sure its guaranteed that a stream will be in a[/color][/color]
valid[color=blue][color=green]
> > state after a failed seek.[/color]
>
> How do I detect a failed positioning?[/color]

Test it against -1.

Jonathan

[Hendrik Schober]

Re: Reading from a stream til EOF

Jonathan Turkanis <technews@kanga roologic.com> wrote:[color=blue]
> [...]
> Test it against -1.[/color]

Thanks!
[color=blue]
> Jonathan[/color]

Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Sometimes compilers are so much more reasonable than people."
Scott Meyers

[Dietmar Kuehl]

Re: Reading from a stream til EOF

Hendrik Schober wrote:[color=blue]
> Dietmar Kuehl <dietmar_kuehl@ yahoo.com> wrote:[color=green]
>> I'm unaware of any other standard C++ library shipping with a commmercial
>> compiler (ObjectSpace dropped their library and mine was never shipping
>> with anything;[/color]
>
> Warum eigentlich?[/color]

I guess you choose German because you were interested more in my personal
reasons rather than in ObjectSpace's fate. Actually, these things are
somewhat related: at some point, ObjectSpace, in persona Lois Goldthwaite,
decided to take my help and incorporate my IOStreams and locales into
their product. I invested some time into fitting things together and
shortly afterwards ObjectSpace stopped their standard C++ library effort
and I decided to focus on other things for now. My standard library
implementation is still not complete and the bits missing would take
quite some effort while nobody seems to be that interested.

My latest effort in the direction of the standard library was the
implementation of property map based algorithms: this is what I want to
get into the next round of standardization . However, even this stuff is
currently mostly idle and not entirely finished... (a definite proof of
concept can be downloaded from
<http://www.dietmar-kuehl.de/cool/cool-20030106.tar.gz >; the major
omission is documentation.. .).
[color=blue]
> But I wonder whether it is a flaw in the
> design if something like reading into a
> string cannot easily be done fast with
> the recommended approach.[/color]

There is some complexity involved. It is not that bad, actually, but it
has to go somewhere. It is just that nobody ventured to really do the
implementation for whatever bad reasons people had (eg. I'm too lazy
and I have other stuff to do). Actually, the whole generic programming
stuff is about doing optimizations in a centralized form - however, the
optimizations still have to be done. I would like to setup algorithms
in a form allowing easy experimenting with optimizations (and this is
how I tried to implement the algorithms) and have multiple people work
on the optimizations. One problem with this which I haven't resolved is
how to actually test the performance.
--
<mailto:dietmar _kuehl@yahoo.co m> <http://www.dietmar-kuehl.de/>
Phaidros eaSE - Easy Software Engineering: <http://www.phaidros.co m/>

[Hendrik Schober]

Re: Reading from a stream til EOF

Dietmar Kuehl <dietmar_kuehl@ yahoo.com> wrote:[color=blue]
> Hendrik Schober wrote:[color=green]
> > Dietmar Kuehl <dietmar_kuehl@ yahoo.com> wrote:[color=darkred]
> >> I'm unaware of any other standard C++ library shipping with a commmercial
> >> compiler (ObjectSpace dropped their library and mine was never shipping
> >> with anything;[/color]
> >
> > Warum eigentlich?[/color]
>
> I guess you choose German because you were interested more in my personal
> reasons rather than in ObjectSpace's fate. Actually, these things are
> somewhat related: at some point [...][/color]

I see. Thanks for answering this here.
[color=blue]
> [...](a definite proof of
> concept can be downloaded from [...][/color]

Um, I will look at this when we passed this
milestone... :(
[color=blue][color=green]
> > But I wonder whether it is a flaw in the
> > design if something like reading into a
> > string cannot easily be done fast with
> > the recommended approach.[/color]
>
> There is some complexity involved. It is not that bad, actually, but it
> has to go somewhere. It is just that nobody ventured to really do the
> implementation for whatever bad reasons people had (eg. I'm too lazy
> and I have other stuff to do). Actually, the whole generic programming
> stuff is about doing optimizations in a centralized form - however, the
> optimizations still have to be done. I would like to setup algorithms
> in a form allowing easy experimenting with optimizations (and this is
> how I tried to implement the algorithms) and have multiple people work
> on the optimizations. One problem with this which I haven't resolved is
> how to actually test the performance.[/color]


But this isn't what I mean. I think, much (if not
most) of what makes up the stream library's design
is driven by history. And things that seemed to
have been good back then might not seem as good
anymore today.
For example, stream buffers do two things: They
manage a buffer for buffering, and they manage the
IO to some specific device. IMO this violates the
desing principle that every class should serve
exactly one purpose. The result is that it is
unnecessarily hard to adapt streams to an IO
device that I have read/write functions for. I
would expect to be able to derive from some class
(or maybe use template magic to avoid virtual
function calls) and implement its 'read()' and
'write()' functions to use the ones for my device.
Instead I have to create a stream buffer with all
the complicated mechanics involved.
And reading strings is just another example: When
the stream idea was born, IO was done using C-
strings. Consequently, the design was fit for
C-strings. Later, 'std::string' entered the picture
and some more functions/operators were added. But
the streams design wasn't adapted to fit strings.
Thus using C-strings is often easier/better than
using C++-string.
I think that's sad for a C++ IO lib.

Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Sometimes compilers are so much more reasonable than people."
Scott Meyers

[Dietmar Kuehl]

Re: Reading from a stream til EOF

Hendrik Schober wrote:[color=blue]
> For example, stream buffers do two things: They
> manage a buffer for buffering, and they manage the
> IO to some specific device.[/color]

Well, actually this is not really the correct view, IMO.
Stream buffers abstract from an "external representation" .
Period. They do one thing. Since characterwise access to
an external representation is in almost all cases slow,
the stream buffer class offers the service of using a
buffer. Use of buffers generally improves performance of
derived stream buffers due to a variaty of reasons.

Also, the class 'std::basic_str eambuf' just does one thing:
it provides buffering capability to derived classes.
[color=blue]
> IMO this violates the
> desing principle that every class should serve
> exactly one purpose. The result is that it is
> unnecessarily hard to adapt streams to an IO
> device that I have read/write functions for. I
> would expect to be able to derive from some class
> (or maybe use template magic to avoid virtual
> function calls) and implement its 'read()' and
> 'write()' functions to use the ones for my device.[/color]

What is wrong with this assumption for the general base of
the streaming library is that not all classes benefit from
buffering and that there are external representations which
use a different abstractions than read and write functions.
For example, memory mapped files would only set up a buffer
without any read and write function at all. On the other
extreme, some interfaces are actually driven characterwise
(although this is indeed rare). The external representations
providing read and write style access are pretty common but
not the only ones supported: the abstraction for stream
buffers are conceptually infinite sequences of characters.
[color=blue]
> Instead I have to create a stream buffer with all
> the complicated mechanics involved.[/color]

You need to implement two function: 'overflow()' and
'underflow()'! Big deal! Especially as these are really
that hard to implement if you obtain characters from a
'read()' style of function and dump them to a 'write()'
style one. The whole stream buffer has something like 20
lines of code.
[color=blue]
> And reading strings is just another example: When
> the stream idea was born, IO was done using C-
> strings. Consequently, the design was fit for
> C-strings. Later, 'std::string' entered the picture
> and some more functions/operators were added. But
> the streams design wasn't adapted to fit strings.[/color]

??? There is a 'operator>>()' overload for reading words,
'std::getline() ' for reading lines, and a constructor of
strings for reading whole sequences. This seems to be
pretty good. Admittedly, you need to watch out for
performance bottlenecks when reading whole sequences but
this is exactly the problem I talked about before. What
do you want instead?
[color=blue]
> Thus using C-strings is often easier/better than
> using C++-string.[/color]

Strangely enough I'm using C-strings only very rarely
and most of the time I'm using them it is due to some
form of legacy. Note, that the use in a previous article
was not about use of strings: it was a use of a buffer.
This could have been a 'std::vector' instead but a fixed
sized built-in array was just appropriate.

There is indeed no direct interface for using strings as
buffers with IOStreams. But strings don't really make
good buffers. Fixed size built-in array seem to be much
more appropriate in this case.
--
<mailto:dietmar _kuehl@yahoo.co m> <http://www.dietmar-kuehl.de/>
Phaidros eaSE - Easy Software Engineering: <http://www.phaidros.co m/>

[Hendrik Schober]

Re: Reading from a stream til EOF

tom_usenet <tom_usenet@hot mail.com> wrote:[color=blue]
> On Wed, 25 Feb 2004 23:05:55 +0100, "Hendrik Schober"
> <SpamTrap@gmx.d e> wrote:
>[color=green]
> >Hi,
> >
> >I have a 'std::istream' and need to read
> >its whole contents into a string. How can
> >I do this?[/color]
>
> I've posted a few solutions to this in the past:
>
> http://www.google.com/groups?selm=3d....easynet.co.uk
>
> There are lots more ways, and the most efficient somewhat depends on
> the library implementation in question.[/color]

FTR, I just found another one:

const std::istream::c har_type chEof = std::istream::t raits_type::eof ();
std::string f( std::istream& is )
{
std::string tmp;
std::getline( is, tmp, chEof );
return tmp;
}

[color=blue]
> Tom[/color]


Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Sometimes compilers are so much more reasonable than people."
Scott Meyers

[Nick Hounsome]

Re: Reading from a stream til EOF


"Hendrik Schober" <SpamTrap@gmx.d e> wrote in message
news:c1skci$4jk $1@news1.transm edia.de...[color=blue]
> tom_usenet <tom_usenet@hot mail.com> wrote:[color=green]
> > On Wed, 25 Feb 2004 23:05:55 +0100, "Hendrik Schober"
> > <SpamTrap@gmx.d e> wrote:
> >[color=darkred]
> > >Hi,
> > >
> > >I have a 'std::istream' and need to read
> > >its whole contents into a string. How can
> > >I do this?[/color]
> >
> > I've posted a few solutions to this in the past:
> >
> > http://www.google.com/groups?selm=3d....easynet.co.uk
> >
> > There are lots more ways, and the most efficient somewhat depends on
> > the library implementation in question.[/color]
>
> FTR, I just found another one:
>[/color]

Sorry - this is wrong.
[color=blue]
> const std::istream::c har_type chEof =[/color]
std::istream::t raits_type::eof ();

Hopefully your compiler will warn you that this is a narrowing assignment.
eof() returns int_type where int_type is required to hold
"all of the valid characters of char_type plus the end-of-file value eof()"
[color=blue]
> std::string f( std::istream& is )
> {
> std::string tmp;
> std::getline( is, tmp, chEof );[/color]

probably gets up to a character that has the same low order bits as EOF.
If EOF is -1 then this will PROBABLY stop at a DEL (0xff) character in the
file.
The behaviour is actuallu undefined.
[color=blue]
> return tmp;
> }
>
>[color=green]
> > Tom[/color]
>
>
> Schobi
>
> --
> SpamTrap@gmx.de is never read
> I'm Schobi at suespammers dot org
>
> "Sometimes compilers are so much more reasonable than people."
> Scott Meyers
>
>[/color]

[Hendrik Schober]

Re: Reading from a stream til EOF

Nick Hounsome <nh002@blueyond er.co.uk> wrote:[color=blue]
> [...]
> Sorry - this is wrong.[/color]

Yes, you're right. I just wanted to
post that I had found this out the
hard way... :(
[color=blue]
> [...][/color]

Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Sometimes compilers are so much more reasonable than people."
Scott Meyers