Unicode to ASCII string conversion

[Ger]

I have not been able to find a simple, straight forward Unicode to ASCII
string conversion function in VB.Net.
Is that because such a function does not exists or do I overlook it?

I found Encoding.Conver t, but that needs byte arrays.

Thanks,
/Ger

[Herfried K. Wagner [MVP]]

Re: Unicode to ASCII string conversion

* "Ger" <ger.rietman@re move-this-part-of-address.planet. nl> scripsit:[color=blue]
> I have not been able to find a simple, straight forward Unicode to ASCII
> string conversion function in VB.Net.
> Is that because such a function does not exists or do I overlook it?[/color]

'System.Text.En coding.ASCII.Ge tBytes',
'System.Text.En coding.Unicode. GetByte'.

BTW: Notice that there is no 1:1 mapping defined between Unicode and
ASCII, ASCII is 7-bit only and can thus only represent 128 characters.

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://dotnet.mvps.org/dotnet/faqs/>

[Jay B. Harlow [MVP - Outlook]]

Re: Unicode to ASCII string conversion

Ger,
In addition to Herfried's comments.

For information on Unicode, ASCII and Encoding in .NET see:

http://www.yoda.arachsys.com/csharp/unicode.html

The samples may be in C#, however they should easily be converted to VB.NET

Hope this helps
Jay

"Ger" <ger.rietman@re move-this-part-of-address.planet. nl> wrote in message
news:up$L3xWmEH A.3632@TK2MSFTN GP09.phx.gbl...[color=blue]
>I have not been able to find a simple, straight forward Unicode to ASCII
> string conversion function in VB.Net.
> Is that because such a function does not exists or do I overlook it?
>
> I found Encoding.Conver t, but that needs byte arrays.
>
> Thanks,
> /Ger
>
>[/color]

[Ger]

Re: Unicode to ASCII string conversion

Thanks Jay, a most useful document.
/Ger

"Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in bericht
news:eeYR5ZamEH A.2764@TK2MSFTN GP11.phx.gbl...[color=blue]
> Ger,
> In addition to Herfried's comments.
>
> For information on Unicode, ASCII and Encoding in .NET see:
>
> http://www.yoda.arachsys.com/csharp/unicode.html
>
> The samples may be in C#, however they should easily be converted to[/color]
VB.NET[color=blue]
>
> Hope this helps
> Jay
>
> "Ger" <ger.rietman@re move-this-part-of-address.planet. nl> wrote in message
> news:up$L3xWmEH A.3632@TK2MSFTN GP09.phx.gbl...[color=green]
> >I have not been able to find a simple, straight forward Unicode to ASCII
> > string conversion function in VB.Net.
> > Is that because such a function does not exists or do I overlook it?
> >
> > I found Encoding.Conver t, but that needs byte arrays.
> >
> > Thanks,
> > /Ger
> >
> >[/color]
>
>[/color]

[Ger]

Re: Unicode to ASCII string conversion


"Herfried K. Wagner [MVP]" <hirf-spam-me-here@gmx.at> schreef in bericht
news:%23YEYFVXm EHA.1440@TK2MSF TNGP10.phx.gbl. ..[color=blue]
> * "Ger" <ger.rietman@re move-this-part-of-address.planet. nl> scripsit:[color=green]
> > I have not been able to find a simple, straight forward Unicode to ASCII
> > string conversion function in VB.Net.
> > Is that because such a function does not exists or do I overlook it?[/color]
>
> 'System.Text.En coding.ASCII.Ge tBytes',
> 'System.Text.En coding.Unicode. GetByte'.
>[/color]
Thanks for your reply, but this returns a byte array. I ment straight
forward string-to-string conversion. It is possible ofcourse to write a
simple function to do this and using the encoding class, but I was just
wondering why the framework does not support the "direct string-to-string".
/Ger

[Cor Ligthert]

Re: Unicode to ASCII string conversion

Ger,
[color=blue]
> Thanks for your reply, but this returns a byte array. I ment straight
> forward string-to-string conversion. It is possible ofcourse to write a
> simple function to do this and using the encoding class, but I was just
> wondering why the framework does not support the "direct[/color]
string-to-string".

In the dotNet is a "String" is forever a string of unicode Chars. What you
call "ascii string" is forever a bytearray.

Therefore as an answer there is nothing more than Herfried suggested.
Although you can create an array of objects which contains bytes, however
that is no solution in my opinion.

I hope this helps to get the idea?

Cor

[Ger]

Re: Unicode to ASCII string conversion

Ah, now I think I get the idea. So when I convert a (unicode) string into an
ascii byte array, and then the byte array back into a string, I still have
Unicode, right? So that is of no use when you want to write ASCII to a
filestream.

Is the code below then writing ASCII output to my filestream?

Dim UnicodeString As String = "abcdëfg"
Dim fsOutput as New FileStream(..)
Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
wOutput.WriteLi ne(UnicodeStrin g)

Thank you for your reply.

/Ger


"Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..[color=blue]
> Ger,
>[color=green]
> > Thanks for your reply, but this returns a byte array. I ment straight
> > forward string-to-string conversion. It is possible ofcourse to write a
> > simple function to do this and using the encoding class, but I was just
> > wondering why the framework does not support the "direct[/color]
> string-to-string".
>
> In the dotNet is a "String" is forever a string of unicode Chars. What you
> call "ascii string" is forever a bytearray.
>
> Therefore as an answer there is nothing more than Herfried suggested.
> Although you can create an array of objects which contains bytes, however
> that is no solution in my opinion.
>
> I hope this helps to get the idea?
>
> Cor
>
>
>[/color]

[Cor Ligthert]

Re: Unicode to ASCII string conversion

Ger
[color=blue]
> Ah, now I think I get the idea. So when I convert a (unicode) string into[/color]
an[color=blue]
> ascii byte array, and then the byte array back into a string, I still have
> Unicode, right? So that is of no use when you want to write ASCII to a
> filestream.
>[/color]

That I do not say, however when you read it back in a String you have again
a string of Chars.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemcharclasstopic.asp

I hope this helps?

Cor

[Jay B. Harlow [MVP - Outlook]]

Re: Unicode to ASCII string conversion

Ger,[color=blue]
> Ah, now I think I get the idea. So when I convert a (Unicode) string into
> an
> ascii byte array, and then the byte array back into a string, I still have
> Unicode, right?[/color]
Correct, just remember that you will loose some characters going to & from
ASCII.
[color=blue]
> So that is of no use when you want to write ASCII to a
> filestream.[/color]
If you need an ASCII file, then use a ASCII encoding. It really depends on
what is going to read the file again.

I would recommend with an ANSI encoding (see below) or UTF8 encoding. With
ASCII you will loose all extended characters (ASCII is 7 bit encoding), with
ANSI you will loose characters that are outside of your regional ANSI code
page. UTF8 preserves all Unicode characters. I would recommend ANSI encoding
if the file was going to be opened by casual users in Notepad. I would
recommend UTF8 if full Unicode support is required. ANSI & UTF8 are both 8
bit encodings.

[color=blue]
> Is the code below then writing ASCII output to my filestream?[/color]

Yes that code is writing ASCII, as you included the ASCII encoding on the
StreamWriter constructor.

The text file itself will contain ASCII characters, when you subsequently
open that text stream and read it (with a StreamReader) it will be converted
back to Unicode strings. When reading the file back try to use the same
encoding as written. For example if you wrote ANSI, then use ANSI to read.
If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode characters
127 to 255 differently. Remember that Encoding.UTF8 is used on the stream
writer if you do not give one, if you are reading text files created by
Notepad, then you want Encoding.Defaul t.

I would recommend:
[color=blue]
> Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.Default)[/color]

Which will write the file in your current ANSI code page as defined by the
regional settings in Windows Control Panel. Which will preserve extended
characters.

Remember that ANSI is an 8 bit encoding that is dependent on region (code
page). While ASCII is a 7 bit encoding, ASCII does not support extended
characters such as ë. It will be converted into either a normal e or a ?.

Hope this helps
Jay

"Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...[color=blue]
> Ah, now I think I get the idea. So when I convert a (unicode) string into
> an
> ascii byte array, and then the byte array back into a string, I still have
> Unicode, right? So that is of no use when you want to write ASCII to a
> filestream.
>
> Is the code below then writing ASCII output to my filestream?
>
> Dim UnicodeString As String = "abcdëfg"
> Dim fsOutput as New FileStream(..)
> Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
> wOutput.WriteLi ne(UnicodeStrin g)
>
> Thank you for your reply.
>
> /Ger
>
>
> "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
> news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..[color=green]
>> Ger,
>>[color=darkred]
>> > Thanks for your reply, but this returns a byte array. I ment straight
>> > forward string-to-string conversion. It is possible ofcourse to write a
>> > simple function to do this and using the encoding class, but I was just
>> > wondering why the framework does not support the "direct[/color]
>> string-to-string".
>>
>> In the dotNet is a "String" is forever a string of unicode Chars. What
>> you
>> call "ascii string" is forever a bytearray.
>>
>> Therefore as an answer there is nothing more than Herfried suggested.
>> Although you can create an array of objects which contains bytes, however
>> that is no solution in my opinion.
>>
>> I hope this helps to get the idea?
>>
>> Cor
>>
>>
>>[/color]
>
>[/color]

[Ger]

Re: Unicode to ASCII string conversion

Thank you very much guys for your help and clearing up the issue for me.
I will go for Jay's solution and use ANSI 8-bit.
/Ger

"Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in bericht
news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..[color=blue]
> Ger,[color=green]
> > Ah, now I think I get the idea. So when I convert a (Unicode) string[/color][/color]
into[color=blue][color=green]
> > an
> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
have[color=blue][color=green]
> > Unicode, right?[/color]
> Correct, just remember that you will loose some characters going to & from
> ASCII.
>[color=green]
> > So that is of no use when you want to write ASCII to a
> > filestream.[/color]
> If you need an ASCII file, then use a ASCII encoding. It really depends on
> what is going to read the file again.
>
> I would recommend with an ANSI encoding (see below) or UTF8 encoding. With
> ASCII you will loose all extended characters (ASCII is 7 bit encoding),[/color]
with[color=blue]
> ANSI you will loose characters that are outside of your regional ANSI code
> page. UTF8 preserves all Unicode characters. I would recommend ANSI[/color]
encoding[color=blue]
> if the file was going to be opened by casual users in Notepad. I would
> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are both 8
> bit encodings.
>
>[color=green]
> > Is the code below then writing ASCII output to my filestream?[/color]
>
> Yes that code is writing ASCII, as you included the ASCII encoding on the
> StreamWriter constructor.
>
> The text file itself will contain ASCII characters, when you subsequently
> open that text stream and read it (with a StreamReader) it will be[/color]
converted[color=blue]
> back to Unicode strings. When reading the file back try to use the same
> encoding as written. For example if you wrote ANSI, then use ANSI to read.
> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode characters
> 127 to 255 differently. Remember that Encoding.UTF8 is used on the stream
> writer if you do not give one, if you are reading text files created by
> Notepad, then you want Encoding.Defaul t.
>
> I would recommend:
>[color=green]
> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.Default)[/color]
>
> Which will write the file in your current ANSI code page as defined by the
> regional settings in Windows Control Panel. Which will preserve extended
> characters.
>
> Remember that ANSI is an 8 bit encoding that is dependent on region (code
> page). While ASCII is a 7 bit encoding, ASCII does not support extended
> characters such as ë. It will be converted into either a normal e or a ?.
>
> Hope this helps
> Jay
>
> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...[color=green]
> > Ah, now I think I get the idea. So when I convert a (unicode) string[/color][/color]
into[color=blue][color=green]
> > an
> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
have[color=blue][color=green]
> > Unicode, right? So that is of no use when you want to write ASCII to a
> > filestream.
> >
> > Is the code below then writing ASCII output to my filestream?
> >
> > Dim UnicodeString As String = "abcdëfg"
> > Dim fsOutput as New FileStream(..)
> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
> > wOutput.WriteLi ne(UnicodeStrin g)
> >
> > Thank you for your reply.
> >
> > /Ger
> >
> >
> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..[color=darkred]
> >> Ger,
> >>
> >> > Thanks for your reply, but this returns a byte array. I ment straight
> >> > forward string-to-string conversion. It is possible ofcourse to write[/color][/color][/color]
a[color=blue][color=green][color=darkred]
> >> > simple function to do this and using the encoding class, but I was[/color][/color][/color]
just[color=blue][color=green][color=darkred]
> >> > wondering why the framework does not support the "direct
> >> string-to-string".
> >>
> >> In the dotNet is a "String" is forever a string of unicode Chars. What
> >> you
> >> call "ascii string" is forever a bytearray.
> >>
> >> Therefore as an answer there is nothing more than Herfried suggested.
> >> Although you can create an array of objects which contains bytes,[/color][/color][/color]
however[color=blue][color=green][color=darkred]
> >> that is no solution in my opinion.
> >>
> >> I hope this helps to get the idea?
> >>
> >> Cor
> >>
> >>
> >>[/color]
> >
> >[/color]
>
>[/color]

[Cor Ligthert]

Re: Unicode to ASCII string conversion

Jay,

Because of Ger's answer, now I become curious. I did not read it in your
message, however what is the solution, Ger told he wanted a straight string
to string conversion and explicitly no bytearray, however now I understand
he can convert Unicode to a 8 bits ANSI String in VBNet? (And I am not
talking about writing a file with 8 bits chars by decoding the char)

I showed in this thread with a link to an MSDN page that a String contains
forever 16 bits Chars.

Is that documentation wrong or do I not understand it or maybe even
something complete different..

Cor




"Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in bericht
news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..[color=blue]
> Ger,[color=green]
> > Ah, now I think I get the idea. So when I convert a (Unicode) string[/color][/color]
into[color=blue][color=green]
> > an
> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
have[color=blue][color=green]
> > Unicode, right?[/color]
> Correct, just remember that you will loose some characters going to & from
> ASCII.
>[color=green]
> > So that is of no use when you want to write ASCII to a
> > filestream.[/color]
> If you need an ASCII file, then use a ASCII encoding. It really depends on
> what is going to read the file again.
>
> I would recommend with an ANSI encoding (see below) or UTF8 encoding. With
> ASCII you will loose all extended characters (ASCII is 7 bit encoding),[/color]
with[color=blue]
> ANSI you will loose characters that are outside of your regional ANSI code
> page. UTF8 preserves all Unicode characters. I would recommend ANSI[/color]
encoding[color=blue]
> if the file was going to be opened by casual users in Notepad. I would
> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are both 8
> bit encodings.
>
>[color=green]
> > Is the code below then writing ASCII output to my filestream?[/color]
>
> Yes that code is writing ASCII, as you included the ASCII encoding on the
> StreamWriter constructor.
>
> The text file itself will contain ASCII characters, when you subsequently
> open that text stream and read it (with a StreamReader) it will be[/color]
converted[color=blue]
> back to Unicode strings. When reading the file back try to use the same
> encoding as written. For example if you wrote ANSI, then use ANSI to read.
> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode characters
> 127 to 255 differently. Remember that Encoding.UTF8 is used on the stream
> writer if you do not give one, if you are reading text files created by
> Notepad, then you want Encoding.Defaul t.
>
> I would recommend:
>[color=green]
> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.Default)[/color]
>
> Which will write the file in your current ANSI code page as defined by the
> regional settings in Windows Control Panel. Which will preserve extended
> characters.
>
> Remember that ANSI is an 8 bit encoding that is dependent on region (code
> page). While ASCII is a 7 bit encoding, ASCII does not support extended
> characters such as ë. It will be converted into either a normal e or a ?.
>
> Hope this helps
> Jay
>
> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...[color=green]
> > Ah, now I think I get the idea. So when I convert a (unicode) string[/color][/color]
into[color=blue][color=green]
> > an
> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
have[color=blue][color=green]
> > Unicode, right? So that is of no use when you want to write ASCII to a
> > filestream.
> >
> > Is the code below then writing ASCII output to my filestream?
> >
> > Dim UnicodeString As String = "abcdëfg"
> > Dim fsOutput as New FileStream(..)
> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
> > wOutput.WriteLi ne(UnicodeStrin g)
> >
> > Thank you for your reply.
> >
> > /Ger
> >
> >
> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..[color=darkred]
> >> Ger,
> >>
> >> > Thanks for your reply, but this returns a byte array. I ment straight
> >> > forward string-to-string conversion. It is possible ofcourse to write[/color][/color][/color]
a[color=blue][color=green][color=darkred]
> >> > simple function to do this and using the encoding class, but I was[/color][/color][/color]
just[color=blue][color=green][color=darkred]
> >> > wondering why the framework does not support the "direct
> >> string-to-string".
> >>
> >> In the dotNet is a "String" is forever a string of unicode Chars. What
> >> you
> >> call "ascii string" is forever a bytearray.
> >>
> >> Therefore as an answer there is nothing more than Herfried suggested.
> >> Although you can create an array of objects which contains bytes,[/color][/color][/color]
however[color=blue][color=green][color=darkred]
> >> that is no solution in my opinion.
> >>
> >> I hope this helps to get the idea?
> >>
> >> Cor
> >>
> >>
> >>[/color]
> >
> >[/color]
>
>[/color]

[Jay B. Harlow [MVP - Outlook]]

Re: Unicode to ASCII string conversion

Ger,
I should add that I would normally use Encoding.Defaul t (ANSI) for straight
text files. I would normally use UTF8 for XML files.

Hope this helps
Jay

"Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
news:OybDGbmmEH A.952@TK2MSFTNG P14.phx.gbl...[color=blue]
> Thank you very much guys for your help and clearing up the issue for me.
> I will go for Jay's solution and use ANSI 8-bit.
> /Ger
>
> "Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in
> bericht
> news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..[color=green]
>> Ger,[color=darkred]
>> > Ah, now I think I get the idea. So when I convert a (Unicode) string[/color][/color]
> into[color=green][color=darkred]
>> > an
>> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
> have[color=green][color=darkred]
>> > Unicode, right?[/color]
>> Correct, just remember that you will loose some characters going to &
>> from
>> ASCII.
>>[color=darkred]
>> > So that is of no use when you want to write ASCII to a
>> > filestream.[/color]
>> If you need an ASCII file, then use a ASCII encoding. It really depends
>> on
>> what is going to read the file again.
>>
>> I would recommend with an ANSI encoding (see below) or UTF8 encoding.
>> With
>> ASCII you will loose all extended characters (ASCII is 7 bit encoding),[/color]
> with[color=green]
>> ANSI you will loose characters that are outside of your regional ANSI
>> code
>> page. UTF8 preserves all Unicode characters. I would recommend ANSI[/color]
> encoding[color=green]
>> if the file was going to be opened by casual users in Notepad. I would
>> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are both
>> 8
>> bit encodings.
>>
>>[color=darkred]
>> > Is the code below then writing ASCII output to my filestream?[/color]
>>
>> Yes that code is writing ASCII, as you included the ASCII encoding on the
>> StreamWriter constructor.
>>
>> The text file itself will contain ASCII characters, when you subsequently
>> open that text stream and read it (with a StreamReader) it will be[/color]
> converted[color=green]
>> back to Unicode strings. When reading the file back try to use the same
>> encoding as written. For example if you wrote ANSI, then use ANSI to
>> read.
>> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode
>> characters
>> 127 to 255 differently. Remember that Encoding.UTF8 is used on the stream
>> writer if you do not give one, if you are reading text files created by
>> Notepad, then you want Encoding.Defaul t.
>>
>> I would recommend:
>>[color=darkred]
>> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.Default)[/color]
>>
>> Which will write the file in your current ANSI code page as defined by
>> the
>> regional settings in Windows Control Panel. Which will preserve extended
>> characters.
>>
>> Remember that ANSI is an 8 bit encoding that is dependent on region (code
>> page). While ASCII is a 7 bit encoding, ASCII does not support extended
>> characters such as ë. It will be converted into either a normal e or a ?.
>>
>> Hope this helps
>> Jay
>>
>> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
>> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...[color=darkred]
>> > Ah, now I think I get the idea. So when I convert a (unicode) string[/color][/color]
> into[color=green][color=darkred]
>> > an
>> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
> have[color=green][color=darkred]
>> > Unicode, right? So that is of no use when you want to write ASCII to a
>> > filestream.
>> >
>> > Is the code below then writing ASCII output to my filestream?
>> >
>> > Dim UnicodeString As String = "abcdëfg"
>> > Dim fsOutput as New FileStream(..)
>> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
>> > wOutput.WriteLi ne(UnicodeStrin g)
>> >
>> > Thank you for your reply.
>> >
>> > /Ger
>> >
>> >
>> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
>> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..
>> >> Ger,
>> >>
>> >> > Thanks for your reply, but this returns a byte array. I ment
>> >> > straight
>> >> > forward string-to-string conversion. It is possible ofcourse to
>> >> > write[/color][/color]
> a[color=green][color=darkred]
>> >> > simple function to do this and using the encoding class, but I was[/color][/color]
> just[color=green][color=darkred]
>> >> > wondering why the framework does not support the "direct
>> >> string-to-string".
>> >>
>> >> In the dotNet is a "String" is forever a string of unicode Chars. What
>> >> you
>> >> call "ascii string" is forever a bytearray.
>> >>
>> >> Therefore as an answer there is nothing more than Herfried suggested.
>> >> Although you can create an array of objects which contains bytes,[/color][/color]
> however[color=green][color=darkred]
>> >> that is no solution in my opinion.
>> >>
>> >> I hope this helps to get the idea?
>> >>
>> >> Cor
>> >>
>> >>
>> >>
>> >
>> >[/color]
>>
>>[/color]
>
>[/color]

[Jay B. Harlow [MVP - Outlook]]

Re: Unicode to ASCII string conversion

Cor,
Read my post. ;-) I only discussed reading & writing strings to ASCII, ANSI,
and UTF8 files (7 & 8 bit encodings).

You are correct System.String & System.Char are UTF-16 (16 bit Unicode),
files can be ANSI, ASCII, UTF7, UTF8, EBCDIC, UTF16 and many other
encodings.


FWIW: VS.NET 2005 (.NET 2.0, aka Whidbey, due out in 2005) appears to
support UTF-32 encoding for files.

http://msdn2.microsoft.com/library/ts575t62.aspx

Hope this helps
Jay


"Cor Ligthert" <notfirstname@p lanet.nl> wrote in message
news:%23DvHNymm EHA.3968@TK2MSF TNGP11.phx.gbl. ..[color=blue]
> Jay,
>
> Because of Ger's answer, now I become curious. I did not read it in your
> message, however what is the solution, Ger told he wanted a straight
> string
> to string conversion and explicitly no bytearray, however now I understand
> he can convert Unicode to a 8 bits ANSI String in VBNet? (And I am not
> talking about writing a file with 8 bits chars by decoding the char)
>
> I showed in this thread with a link to an MSDN page that a String contains
> forever 16 bits Chars.
>
> Is that documentation wrong or do I not understand it or maybe even
> something complete different..
>
> Cor
>
>
>
>
> "Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in
> bericht
> news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..[color=green]
>> Ger,[color=darkred]
>> > Ah, now I think I get the idea. So when I convert a (Unicode) string[/color][/color]
> into[color=green][color=darkred]
>> > an
>> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
> have[color=green][color=darkred]
>> > Unicode, right?[/color]
>> Correct, just remember that you will loose some characters going to &
>> from
>> ASCII.
>>[color=darkred]
>> > So that is of no use when you want to write ASCII to a
>> > filestream.[/color]
>> If you need an ASCII file, then use a ASCII encoding. It really depends
>> on
>> what is going to read the file again.
>>
>> I would recommend with an ANSI encoding (see below) or UTF8 encoding.
>> With
>> ASCII you will loose all extended characters (ASCII is 7 bit encoding),[/color]
> with[color=green]
>> ANSI you will loose characters that are outside of your regional ANSI
>> code
>> page. UTF8 preserves all Unicode characters. I would recommend ANSI[/color]
> encoding[color=green]
>> if the file was going to be opened by casual users in Notepad. I would
>> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are both
>> 8
>> bit encodings.
>>
>>[color=darkred]
>> > Is the code below then writing ASCII output to my filestream?[/color]
>>
>> Yes that code is writing ASCII, as you included the ASCII encoding on the
>> StreamWriter constructor.
>>
>> The text file itself will contain ASCII characters, when you subsequently
>> open that text stream and read it (with a StreamReader) it will be[/color]
> converted[color=green]
>> back to Unicode strings. When reading the file back try to use the same
>> encoding as written. For example if you wrote ANSI, then use ANSI to
>> read.
>> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode
>> characters
>> 127 to 255 differently. Remember that Encoding.UTF8 is used on the stream
>> writer if you do not give one, if you are reading text files created by
>> Notepad, then you want Encoding.Defaul t.
>>
>> I would recommend:
>>[color=darkred]
>> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.Default)[/color]
>>
>> Which will write the file in your current ANSI code page as defined by
>> the
>> regional settings in Windows Control Panel. Which will preserve extended
>> characters.
>>
>> Remember that ANSI is an 8 bit encoding that is dependent on region (code
>> page). While ASCII is a 7 bit encoding, ASCII does not support extended
>> characters such as ë. It will be converted into either a normal e or a ?.
>>
>> Hope this helps
>> Jay
>>
>> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
>> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...[color=darkred]
>> > Ah, now I think I get the idea. So when I convert a (unicode) string[/color][/color]
> into[color=green][color=darkred]
>> > an
>> > ascii byte array, and then the byte array back into a string, I still[/color][/color]
> have[color=green][color=darkred]
>> > Unicode, right? So that is of no use when you want to write ASCII to a
>> > filestream.
>> >
>> > Is the code below then writing ASCII output to my filestream?
>> >
>> > Dim UnicodeString As String = "abcdëfg"
>> > Dim fsOutput as New FileStream(..)
>> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
>> > wOutput.WriteLi ne(UnicodeStrin g)
>> >
>> > Thank you for your reply.
>> >
>> > /Ger
>> >
>> >
>> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
>> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..
>> >> Ger,
>> >>
>> >> > Thanks for your reply, but this returns a byte array. I ment
>> >> > straight
>> >> > forward string-to-string conversion. It is possible ofcourse to
>> >> > write[/color][/color]
> a[color=green][color=darkred]
>> >> > simple function to do this and using the encoding class, but I was[/color][/color]
> just[color=green][color=darkred]
>> >> > wondering why the framework does not support the "direct
>> >> string-to-string".
>> >>
>> >> In the dotNet is a "String" is forever a string of unicode Chars. What
>> >> you
>> >> call "ascii string" is forever a bytearray.
>> >>
>> >> Therefore as an answer there is nothing more than Herfried suggested.
>> >> Although you can create an array of objects which contains bytes,[/color][/color]
> however[color=green][color=darkred]
>> >> that is no solution in my opinion.
>> >>
>> >> I hope this helps to get the idea?
>> >>
>> >> Cor
>> >>
>> >>
>> >>
>> >
>> >[/color]
>>
>>[/color]
>
>[/color]

[Ger]

Re: Unicode to ASCII string conversion

Jay,
Actually the reason I bothered this group with my question was because my
app generates and writes HTML data to a file believing it would be
interpreted by the browser as ASCII by setting charset=windows-1252 in the
meta tag of the htlm head.
Then I found that certain diacritical characters were not well interpreted
by the browser, and I found that that was because of the characters in the
generated stream were delivered as Unicode. After changing the meta tag in
the HTML code to charset=UTF-8 all was fine, but it left me with the
question on how these things work, hence why there was no simple
string-to-string conversion in the framework.
Your and Cor's replies made me understand better how these things work in
..Net, and I thank you both for your help in this.
/Ger



"Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in bericht
news:uO6e5rpmEH A.3968@TK2MSFTN GP11.phx.gbl...[color=blue]
> Ger,
> I should add that I would normally use Encoding.Defaul t (ANSI) for[/color]
straight[color=blue]
> text files. I would normally use UTF8 for XML files.
>
> Hope this helps
> Jay
>
> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
> news:OybDGbmmEH A.952@TK2MSFTNG P14.phx.gbl...[color=green]
> > Thank you very much guys for your help and clearing up the issue for me.
> > I will go for Jay's solution and use ANSI 8-bit.
> > /Ger
> >
> > "Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in
> > bericht
> > news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..[color=darkred]
> >> Ger,
> >> > Ah, now I think I get the idea. So when I convert a (Unicode) string[/color]
> > into[color=darkred]
> >> > an
> >> > ascii byte array, and then the byte array back into a string, I still[/color]
> > have[color=darkred]
> >> > Unicode, right?
> >> Correct, just remember that you will loose some characters going to &
> >> from
> >> ASCII.
> >>
> >> > So that is of no use when you want to write ASCII to a
> >> > filestream.
> >> If you need an ASCII file, then use a ASCII encoding. It really depends
> >> on
> >> what is going to read the file again.
> >>
> >> I would recommend with an ANSI encoding (see below) or UTF8 encoding.
> >> With
> >> ASCII you will loose all extended characters (ASCII is 7 bit encoding),[/color]
> > with[color=darkred]
> >> ANSI you will loose characters that are outside of your regional ANSI
> >> code
> >> page. UTF8 preserves all Unicode characters. I would recommend ANSI[/color]
> > encoding[color=darkred]
> >> if the file was going to be opened by casual users in Notepad. I would
> >> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are[/color][/color][/color]
both[color=blue][color=green][color=darkred]
> >> 8
> >> bit encodings.
> >>
> >>
> >> > Is the code below then writing ASCII output to my filestream?
> >>
> >> Yes that code is writing ASCII, as you included the ASCII encoding on[/color][/color][/color]
the[color=blue][color=green][color=darkred]
> >> StreamWriter constructor.
> >>
> >> The text file itself will contain ASCII characters, when you[/color][/color][/color]
subsequently[color=blue][color=green][color=darkred]
> >> open that text stream and read it (with a StreamReader) it will be[/color]
> > converted[color=darkred]
> >> back to Unicode strings. When reading the file back try to use the same
> >> encoding as written. For example if you wrote ANSI, then use ANSI to
> >> read.
> >> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode
> >> characters
> >> 127 to 255 differently. Remember that Encoding.UTF8 is used on the[/color][/color][/color]
stream[color=blue][color=green][color=darkred]
> >> writer if you do not give one, if you are reading text files created by
> >> Notepad, then you want Encoding.Defaul t.
> >>
> >> I would recommend:
> >>
> >> > Dim wOutput As New StreamWriter(fs Output,[/color][/color][/color]
System.Text.Enc oding.Default)[color=blue][color=green][color=darkred]
> >>
> >> Which will write the file in your current ANSI code page as defined by
> >> the
> >> regional settings in Windows Control Panel. Which will preserve[/color][/color][/color]
extended[color=blue][color=green][color=darkred]
> >> characters.
> >>
> >> Remember that ANSI is an 8 bit encoding that is dependent on region[/color][/color][/color]
(code[color=blue][color=green][color=darkred]
> >> page). While ASCII is a 7 bit encoding, ASCII does not support extended
> >> characters such as ë. It will be converted into either a normal e or a[/color][/color][/color]
?.[color=blue][color=green][color=darkred]
> >>
> >> Hope this helps
> >> Jay
> >>
> >> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
> >> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...
> >> > Ah, now I think I get the idea. So when I convert a (unicode) string[/color]
> > into[color=darkred]
> >> > an
> >> > ascii byte array, and then the byte array back into a string, I still[/color]
> > have[color=darkred]
> >> > Unicode, right? So that is of no use when you want to write ASCII to[/color][/color][/color]
a[color=blue][color=green][color=darkred]
> >> > filestream.
> >> >
> >> > Is the code below then writing ASCII output to my filestream?
> >> >
> >> > Dim UnicodeString As String = "abcdëfg"
> >> > Dim fsOutput as New FileStream(..)
> >> > Dim wOutput As New StreamWriter(fs Output, System.Text.Enc oding.ASCII)
> >> > wOutput.WriteLi ne(UnicodeStrin g)
> >> >
> >> > Thank you for your reply.
> >> >
> >> > /Ger
> >> >
> >> >
> >> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
> >> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..
> >> >> Ger,
> >> >>
> >> >> > Thanks for your reply, but this returns a byte array. I ment
> >> >> > straight
> >> >> > forward string-to-string conversion. It is possible ofcourse to
> >> >> > write[/color]
> > a[color=darkred]
> >> >> > simple function to do this and using the encoding class, but I was[/color]
> > just[color=darkred]
> >> >> > wondering why the framework does not support the "direct
> >> >> string-to-string".
> >> >>
> >> >> In the dotNet is a "String" is forever a string of unicode Chars.[/color][/color][/color]
What[color=blue][color=green][color=darkred]
> >> >> you
> >> >> call "ascii string" is forever a bytearray.
> >> >>
> >> >> Therefore as an answer there is nothing more than Herfried[/color][/color][/color]
suggested.[color=blue][color=green][color=darkred]
> >> >> Although you can create an array of objects which contains bytes,[/color]
> > however[color=darkred]
> >> >> that is no solution in my opinion.
> >> >>
> >> >> I hope this helps to get the idea?
> >> >>
> >> >> Cor
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >>
> >>[/color]
> >
> >[/color]
>
>[/color]

[Jay B. Harlow [MVP - Outlook]]

Re: Unicode to ASCII string conversion

Ger,[color=blue]
> interpreted by the browser as ASCII by setting charset=windows-1252 in the[/color]
charset=windows-1252 is ANSI, not ASCII!

If you use Encoding.Defaul t you will (normally) wind up with windows-1252 in
the US and some of Europe.

Try the following:

Debug.WriteLine (System.Text.En coding.Default. EncodingName, "encoding
name")
Debug.WriteLine (System.Text.En coding.Default. BodyName, "body name")
Debug.WriteLine (System.Text.En coding.Default. HeaderName, "header
name")
Debug.WriteLine (System.Text.En coding.Default. WebName, "web name")
Debug.WriteLine (System.Text.En coding.Default. CodePage, "code page")
Debug.WriteLine (System.Text.En coding.Default. WindowsCodePage ,
"windows code page")

I would expect Encoding.Defaul t with charset = Encoding.Defaul t.WebName
should give you the effect you desire, with any regional settings.

Of course UTF8 will preserve ALL unicode characters in your output (good for
XML & HTML).

Hope this helps
Jay

"Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
news:unZSJPqmEH A.2552@TK2MSFTN GP11.phx.gbl...[color=blue]
> Jay,
> Actually the reason I bothered this group with my question was because my
> app generates and writes HTML data to a file believing it would be
> interpreted by the browser as ASCII by setting charset=windows-1252 in the
> meta tag of the htlm head.
> Then I found that certain diacritical characters were not well interpreted
> by the browser, and I found that that was because of the characters in the
> generated stream were delivered as Unicode. After changing the meta tag in
> the HTML code to charset=UTF-8 all was fine, but it left me with the
> question on how these things work, hence why there was no simple
> string-to-string conversion in the framework.
> Your and Cor's replies made me understand better how these things work in
> .Net, and I thank you both for your help in this.
> /Ger
>
>
>
> "Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in
> bericht
> news:uO6e5rpmEH A.3968@TK2MSFTN GP11.phx.gbl...[color=green]
>> Ger,
>> I should add that I would normally use Encoding.Defaul t (ANSI) for[/color]
> straight[color=green]
>> text files. I would normally use UTF8 for XML files.
>>
>> Hope this helps
>> Jay
>>
>> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
>> news:OybDGbmmEH A.952@TK2MSFTNG P14.phx.gbl...[color=darkred]
>> > Thank you very much guys for your help and clearing up the issue for
>> > me.
>> > I will go for Jay's solution and use ANSI 8-bit.
>> > /Ger
>> >
>> > "Jay B. Harlow [MVP - Outlook]" <Jay_Harlow_MVP @msn.com> schreef in
>> > bericht
>> > news:%23RwyVzlm EHA.3328@TK2MSF TNGP10.phx.gbl. ..
>> >> Ger,
>> >> > Ah, now I think I get the idea. So when I convert a (Unicode) string
>> > into
>> >> > an
>> >> > ascii byte array, and then the byte array back into a string, I
>> >> > still
>> > have
>> >> > Unicode, right?
>> >> Correct, just remember that you will loose some characters going to &
>> >> from
>> >> ASCII.
>> >>
>> >> > So that is of no use when you want to write ASCII to a
>> >> > filestream.
>> >> If you need an ASCII file, then use a ASCII encoding. It really
>> >> depends
>> >> on
>> >> what is going to read the file again.
>> >>
>> >> I would recommend with an ANSI encoding (see below) or UTF8 encoding.
>> >> With
>> >> ASCII you will loose all extended characters (ASCII is 7 bit
>> >> encoding),
>> > with
>> >> ANSI you will loose characters that are outside of your regional ANSI
>> >> code
>> >> page. UTF8 preserves all Unicode characters. I would recommend ANSI
>> > encoding
>> >> if the file was going to be opened by casual users in Notepad. I would
>> >> recommend UTF8 if full Unicode support is required. ANSI & UTF8 are[/color][/color]
> both[color=green][color=darkred]
>> >> 8
>> >> bit encodings.
>> >>
>> >>
>> >> > Is the code below then writing ASCII output to my filestream?
>> >>
>> >> Yes that code is writing ASCII, as you included the ASCII encoding on[/color][/color]
> the[color=green][color=darkred]
>> >> StreamWriter constructor.
>> >>
>> >> The text file itself will contain ASCII characters, when you[/color][/color]
> subsequently[color=green][color=darkred]
>> >> open that text stream and read it (with a StreamReader) it will be
>> > converted
>> >> back to Unicode strings. When reading the file back try to use the
>> >> same
>> >> encoding as written. For example if you wrote ANSI, then use ANSI to
>> >> read.
>> >> If you wrote UTF8, then use UTF8 to read. As ANSI & UTF8 encode
>> >> characters
>> >> 127 to 255 differently. Remember that Encoding.UTF8 is used on the[/color][/color]
> stream[color=green][color=darkred]
>> >> writer if you do not give one, if you are reading text files created
>> >> by
>> >> Notepad, then you want Encoding.Defaul t.
>> >>
>> >> I would recommend:
>> >>
>> >> > Dim wOutput As New StreamWriter(fs Output,[/color][/color]
> System.Text.Enc oding.Default)[color=green][color=darkred]
>> >>
>> >> Which will write the file in your current ANSI code page as defined by
>> >> the
>> >> regional settings in Windows Control Panel. Which will preserve[/color][/color]
> extended[color=green][color=darkred]
>> >> characters.
>> >>
>> >> Remember that ANSI is an 8 bit encoding that is dependent on region[/color][/color]
> (code[color=green][color=darkred]
>> >> page). While ASCII is a 7 bit encoding, ASCII does not support
>> >> extended
>> >> characters such as ë. It will be converted into either a normal e or a[/color][/color]
> ?.[color=green][color=darkred]
>> >>
>> >> Hope this helps
>> >> Jay
>> >>
>> >> "Ger" <ger.rietman@ra thernospam.sail soft.nl> wrote in message
>> >> news:uU3WK3kmEH A.2772@tk2msftn gp13.phx.gbl...
>> >> > Ah, now I think I get the idea. So when I convert a (unicode) string
>> > into
>> >> > an
>> >> > ascii byte array, and then the byte array back into a string, I
>> >> > still
>> > have
>> >> > Unicode, right? So that is of no use when you want to write ASCII to[/color][/color]
> a[color=green][color=darkred]
>> >> > filestream.
>> >> >
>> >> > Is the code below then writing ASCII output to my filestream?
>> >> >
>> >> > Dim UnicodeString As String = "abcdëfg"
>> >> > Dim fsOutput as New FileStream(..)
>> >> > Dim wOutput As New StreamWriter(fs Output,
>> >> > System.Text.Enc oding.ASCII)
>> >> > wOutput.WriteLi ne(UnicodeStrin g)
>> >> >
>> >> > Thank you for your reply.
>> >> >
>> >> > /Ger
>> >> >
>> >> >
>> >> > "Cor Ligthert" <notfirstname@p lanet.nl> schreef in bericht
>> >> > news:eWAgM%23im EHA.3564@tk2msf tngp13.phx.gbl. ..
>> >> >> Ger,
>> >> >>
>> >> >> > Thanks for your reply, but this returns a byte array. I ment
>> >> >> > straight
>> >> >> > forward string-to-string conversion. It is possible ofcourse to
>> >> >> > write
>> > a
>> >> >> > simple function to do this and using the encoding class, but I
>> >> >> > was
>> > just
>> >> >> > wondering why the framework does not support the "direct
>> >> >> string-to-string".
>> >> >>
>> >> >> In the dotNet is a "String" is forever a string of unicode Chars.[/color][/color]
> What[color=green][color=darkred]
>> >> >> you
>> >> >> call "ascii string" is forever a bytearray.
>> >> >>
>> >> >> Therefore as an answer there is nothing more than Herfried[/color][/color]
> suggested.[color=green][color=darkred]
>> >> >> Although you can create an array of objects which contains bytes,
>> > however
>> >> >> that is no solution in my opinion.
>> >> >>
>> >> >> I hope this helps to get the idea?
>> >> >>
>> >> >> Cor
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >>
>> >>
>> >
>> >[/color]
>>
>>[/color]
>
>[/color]