Problem Using TextReader and Binaray Reader on Same File

[Guest]

I am rewriting a C++ application in C#. This file has a combination of Text
and Binary data.

I used CFile before to read the text. If I hit a certain string that
denotes the following data is binary, I used the current position in the
file and another stream to read to the binary data.

All text data is ended with a carriage return / line feed while the binary
is actually an image file listed byte by byte. Preceding the binary data is
a text value ended by a cr/lf listing the actual number of bytes of binary
data.

The problem is that the position given by textreader.base stream seems to be
actually twice the actual position in bytes. Therefore positioning a Binary
Reader based on the position in a Text Reader is incorrect.

This worked with CFile.

Any ideas?

Thank You,
Jeff

[Joshua Flanagan]

Re: Problem Using TextReader and Binaray Reader on Same File

It sounds like you are using a 16-bit encoding with the TextReader. I
wonder if specifying an 8-bit encoding (System.Text.En coding.ASCII)
would solve your problem. It's worth a try.
Otherwise, if you can consistently see that the TextReader position is
always twice the binary position... well, divide by 2 (right shift 1)!


jeffwstevens@be llsouth.net wrote:[color=blue]
> I am rewriting a C++ application in C#. This file has a combination of Text
> and Binary data.
>
> I used CFile before to read the text. If I hit a certain string that
> denotes the following data is binary, I used the current position in the
> file and another stream to read to the binary data.
>
> All text data is ended with a carriage return / line feed while the binary
> is actually an image file listed byte by byte. Preceding the binary data is
> a text value ended by a cr/lf listing the actual number of bytes of binary
> data.
>
> The problem is that the position given by textreader.base stream seems to be
> actually twice the actual position in bytes. Therefore positioning a Binary
> Reader based on the position in a Text Reader is incorrect.
>
> This worked with CFile.
>
> Any ideas?
>
> Thank You,
> Jeff
>
>[/color]

[Guest]

Re: Problem Using TextReader and Binaray Reader on Same File

Joshua,

I am considering that except I did not want to do anything that would
potentially fail in converting to a 64 bit OS.

I did try the ASCII encoding, etc and the same problem occurred. I think I
am just going to use the binary reader and create my own readline
functionality.

I also just read the entire file into memory and it did not bog down the
machine too much. I am not sure about the deployment machine yet. The
FileStream / Binary Reader classes may provide some behind the scenes
buffering to help this.

This does give me the capability to scan the contents quickly and create
some file offset arrays to quickly process the data I need.

Thanks for the input,
Jeff

"Joshua Flanagan" <josh@msnews.co m> wrote in message
news:uysF7IxOFH A.1476@TK2MSFTN GP09.phx.gbl...[color=blue]
> It sounds like you are using a 16-bit encoding with the TextReader. I
> wonder if specifying an 8-bit encoding (System.Text.En coding.ASCII) would
> solve your problem. It's worth a try.
> Otherwise, if you can consistently see that the TextReader position is
> always twice the binary position... well, divide by 2 (right shift 1)!
>
>
> jeffwstevens@be llsouth.net wrote:[color=green]
>> I am rewriting a C++ application in C#. This file has a combination of
>> Text and Binary data.
>>
>> I used CFile before to read the text. If I hit a certain string that
>> denotes the following data is binary, I used the current position in the
>> file and another stream to read to the binary data.
>>
>> All text data is ended with a carriage return / line feed while the
>> binary is actually an image file listed byte by byte. Preceding the
>> binary data is a text value ended by a cr/lf listing the actual number of
>> bytes of binary data.
>>
>> The problem is that the position given by textreader.base stream seems to
>> be actually twice the actual position in bytes. Therefore positioning a
>> Binary Reader based on the position in a Text Reader is incorrect.
>>
>> This worked with CFile.
>>
>> Any ideas?
>>
>> Thank You,
>> Jeff[/color][/color]

[Jon Skeet [C# MVP]]

Re: Problem Using TextReader and Binaray Reader on Same File

<jeffwstevens@b ellsouth.net> wrote:[color=blue]
> I am rewriting a C++ application in C#. This file has a combination of Text
> and Binary data.
>
> I used CFile before to read the text. If I hit a certain string that
> denotes the following data is binary, I used the current position in the
> file and another stream to read to the binary data.
>
> All text data is ended with a carriage return / line feed while the binary
> is actually an image file listed byte by byte. Preceding the binary data is
> a text value ended by a cr/lf listing the actual number of bytes of binary
> data.
>
> The problem is that the position given by textreader.base stream seems to be
> actually twice the actual position in bytes. Therefore positioning a Binary
> Reader based on the position in a Text Reader is incorrect.[/color]

This is an unfortunate problem with TextReader. You *may* find that
creating a StreamReader with a buffer size of 1 sorts the problem, but
it'll be very inefficient.

If your file is in ASCII, you could spot the CRLF while reading it as
binary data, and then convert the binary data to text when you find the
CRLF, all the while knowing where you are.

If you're in control of the file format, however, I'd suggest that you
prefix any text section with the number of bytes in that text section.
You can then read that amount of data and convert it to text
separately, without overrunning.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Re: Problem Using TextReader and Binaray Reader on Same File

Joshua Flanagan <josh@msnews.co m> wrote:[color=blue]
> It sounds like you are using a 16-bit encoding with the TextReader. I
> wonder if specifying an 8-bit encoding (System.Text.En coding.ASCII)
> would solve your problem. It's worth a try.
> Otherwise, if you can consistently see that the TextReader position is
> always twice the binary position... well, divide by 2 (right shift 1)![/color]

No, the problem is that StreamReader will read more than it's returned
so far into an internal buffer. The stream's position is then
"inaccurate " in terms of how much appears to have been read.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Guest]

Re: Problem Using TextReader and Binaray Reader on Same File

Jon,

I appreciate you response and it makes sense now but does not help my plight
much in migrating the code.

However is there any reason I can not just access the file via BinaryReader
and load the entire file into a byte[] buffer and parse it instead. It
seems to load the large file quickly and efficiently. The files are 35 to
60 megabytes. It is so quick it seems it is buffered or cached behind the
scenes anyway.

I am doing this way now so I just have deal with parsing the byte buffer
versus potentially dealing with network / buffering issues hitting the
BinaryStream directly.

Thank You,
Jeff
"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in message
news:MPG.1cbee2 7213e0489d98bf7 2@msnews.micros oft.com...[color=blue]
> <jeffwstevens@b ellsouth.net> wrote:[color=green]
>> I am rewriting a C++ application in C#. This file has a combination of
>> Text
>> and Binary data.
>>
>> I used CFile before to read the text. If I hit a certain string that
>> denotes the following data is binary, I used the current position in the
>> file and another stream to read to the binary data.
>>
>> All text data is ended with a carriage return / line feed while the
>> binary
>> is actually an image file listed byte by byte. Preceding the binary data
>> is
>> a text value ended by a cr/lf listing the actual number of bytes of
>> binary
>> data.
>>
>> The problem is that the position given by textreader.base stream seems to
>> be
>> actually twice the actual position in bytes. Therefore positioning a
>> Binary
>> Reader based on the position in a Text Reader is incorrect.[/color]
>
> This is an unfortunate problem with TextReader. You *may* find that
> creating a StreamReader with a buffer size of 1 sorts the problem, but
> it'll be very inefficient.
>
> If your file is in ASCII, you could spot the CRLF while reading it as
> binary data, and then convert the binary data to text when you find the
> CRLF, all the while knowing where you are.
>
> If you're in control of the file format, however, I'd suggest that you
> prefix any text section with the number of bytes in that text section.
> You can then read that amount of data and convert it to text
> separately, without overrunning.
>
> --
> Jon Skeet - <skeet@pobox.co m>
> http://www.pobox.com/~skeet
> If replying to the group, please do not mail me too[/color]

[Jon Skeet [C# MVP]]

Re: Problem Using TextReader and Binaray Reader on Same File

<jeffwstevens@b ellsouth.net> wrote:[color=blue]
> I appreciate you response and it makes sense now but does not help my plight
> much in migrating the code.
>
> However is there any reason I can not just access the file via BinaryReader
> and load the entire file into a byte[] buffer and parse it instead. It
> seems to load the large file quickly and efficiently. The files are 35 to
> 60 megabytes. It is so quick it seems it is buffered or cached behind the
> scenes anyway.[/color]

If you've got enough memory and if the text is in a simple encoding
(such as ASCII) where you don't need to worry about detecting multi-
byte characters, that may well be the easiest way of doing things.
[color=blue]
> I am doing this way now so I just have deal with parsing the byte buffer
> versus potentially dealing with network / buffering issues hitting the
> BinaryStream directly.[/color]

Right.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too