Unicode/UTF-8 decoding

**Mr. Arnold** · Jun 5 '07, 06:05 AM

Re: Unicode/UTF-8 decoding

"Bill Nguyen" <billn_nospam@j aco.comwrote in message
news:AF60801B-04D4-4A84-9063-D73A9197BD62@mi crosoft.com...

Below are sometext I extracted from a mySQL database. How can I decode
them so that I can read them in Unicode?
>

If you have VS for .Net 2003 or 2005, then you can go to Help/Index/Visual
Basic and
enter Unicode UTF-8 in the Search box. It will give you the whole section
with program examples of how to do UTF-8, UTF-16, etc, etc
encoding/decoding.

**=?UTF-8?B?R8O2cmFuIEFuZGVyc3Nvbg==?=** · Jun 5 '07, 07:35 AM

Re: Unicode/UTF-8 decoding

Bill Nguyen wrote:

Below are sometext I extracted from a mySQL database. How can I decode
them so that I can read them in Unicode?
Thanks
>
Bill
>
------------
>
Virginia Hamilton Adair / LÃƒÂ¢m ThÃ¡Â»â€¹ MÃ¡Â»Â¹ DÃ¡ÂºÂ¡
LÃ¡ÂºÂ¥p lÃƒÂ¡nh hÃ¡Â»â€œn thÃ†Â¡ ViÃ¡Â»â€¡t trÃƒÂªn sÃƒÂ¢n ga Tokyo chiÃ¡Â»Âu cuÃ¡Â»â€˜i nÃ„Æ’m

This text looks as it has been decoded with a different encoding than
was used to encode it. It might be possible to recreate the data if you
know what encodings was used to encode and decode it. Then you might be
able to encode it back to it's prevois state and use the proper encoding
to decode it. There is a great risk that some data has been lost,
though, and that you can't recreate the original data from this stage.

If you want to store unicode strings in the MySQL database, it has to be
set up to use unicode as character set.

--
GÃ¶ran Andersson
_____

guffa.com

http://www.guffa.com

Göran Anderssons privata hemsida.

**Bill Nguyen** · Jun 5 '07, 02:05 PM

Re: Unicode/UTF-8 decoding

I set UTF-8 as the default encoding in mySQL.
I don't really know how this work, but IE or Firefox browser can decode
easily.
This is the test:
I put the lines below in an HTML document and viewed it in IE, and it
worked. (make sure to set encoding to UTF-8 in VIEW).
I include the test.htm for your testing. (The text is in Vietnamese).
So I think what I need is to find a utility that has the same function that
might already be available out there. Any help is greatly appreciated.

Bill

----------------
<html>

<head></head>

<body>

Virginia Hamilton Adair / LÃƒÂ¢m ThÃ¡Â»â€¹ MÃ¡Â»Â¹ DÃ¡ÂºÂ¡
LÃ¡ÂºÂ¥p lÃƒÂ¡nh hÃ¡Â»"n thÃ†Â¡ ViÃ¡Â»â€¡t trÃƒÂªn sÃƒÂ¢n ga Tokyo chiÃ¡Â»Âu cuÃ¡Â»â€˜i nÃ„Æ’m

</body>

</html>

"GÃ¶ran Andersson" <guffa@guffa.co mwrote in message
news:%23rhR3M0p HHA.1776@TK2MSF TNGP05.phx.gbl. ..

Bill Nguyen wrote:

>Below are sometext I extracted from a mySQL database. How can I decode
>them so that I can read them in Unicode?
>Thanks
>>
>Bill
>>
>------------
>>
>Virginia Hamilton Adair / LÃƒÂ¢m ThÃ¡Â»â€¹ MÃ¡Â»Â¹ DÃ¡ÂºÂ¡
>LÃ¡ÂºÂ¥p lÃƒÂ¡nh hÃ¡Â»â€œn thÃ†Â¡ ViÃ¡Â»â€¡t trÃƒÂªn sÃƒÂ¢n ga Tokyo chiÃ¡Â»Âu cuÃ¡Â»â€˜i nÃ„Æ’m

>
This text looks as it has been decoded with a different encoding than was
used to encode it. It might be possible to recreate the data if you know
what encodings was used to encode and decode it. Then you might be able to
encode it back to it's prevois state and use the proper encoding to decode
it. There is a great risk that some data has been lost, though, and that
you can't recreate the original data from this stage.
>
If you want to store unicode strings in the MySQL database, it has to be
set up to use unicode as character set.
>
--
GÃ¶ran Andersson
_____
http://www.guffa.com

**=?UTF-8?B?R8O2cmFuIEFuZGVyc3Nvbg==?=** · Jun 5 '07, 03:25 PM

Re: Unicode/UTF-8 decoding

Bill Nguyen wrote:

I set UTF-8 as the default encoding in mySQL.
I don't really know how this work, but IE or Firefox browser can decode
easily.
This is the test:
I put the lines below in an HTML document and viewed it in IE, and it
worked. (make sure to set encoding to UTF-8 in VIEW).
I include the test.htm for your testing. (The text is in Vietnamese).
So I think what I need is to find a utility that has the same function
that might already be available out there. Any help is greatly appreciated.
>
Bill
>
----------------
<html>
>
<head></head>
>
<body>
>
>
>

>Virginia Hamilton Adair / LÃƒÂ¢m ThÃ¡Â»â€¹ MÃ¡Â»Â¹ DÃ¡ÂºÂ¡
>LÃ¡ÂºÂ¥p lÃƒÂ¡nh hÃ¡Â»"n thÃ†Â¡ ViÃ¡Â»â€¡t trÃƒÂªn sÃƒÂ¢n ga Tokyo chiÃ¡Â»Âu cuÃ¡Â»â€˜i nÃ„Æ’m

>
>
</body>
>
</html>
>
>
>
"GÃ¶ran Andersson" <guffa@guffa.co mwrote in message
news:%23rhR3M0p HHA.1776@TK2MSF TNGP05.phx.gbl. ..

>Bill Nguyen wrote:

>>Below are sometext I extracted from a mySQL database. How can I
>>decode them so that I can read them in Unicode?
>>Thanks
>>>
>>Bill
>>>
>>------------
>>>
>>Virginia Hamilton Adair / LÃƒÂ¢m ThÃ¡Â»â€¹ MÃ¡Â»Â¹ DÃ¡ÂºÂ¡
>>LÃ¡ÂºÂ¥p lÃƒÂ¡nh hÃ¡Â»â€œn thÃ†Â¡ ViÃ¡Â»â€¡t trÃƒÂªn sÃƒÂ¢n ga Tokyo chiÃ¡Â»Âu cuÃ¡Â»â€˜i nÃ„Æ’m

>>
>This text looks as it has been decoded with a different encoding than
>was used to encode it. It might be possible to recreate the data if
>you know what encodings was used to encode and decode it. Then you
>might be able to encode it back to it's prevois state and use the
>proper encoding to decode it. There is a great risk that some data has
>been lost, though, and that you can't recreate the original data from
>this stage.
>>
>If you want to store unicode strings in the MySQL database, it has to
>be set up to use unicode as character set.
>>
>--
>GÃ¶ran Andersson
>_____
>http://www.guffa.com

>
------------------------------------------------------------------------
>

Virginia Hamilton Adair / LÃ¢m Thá»‹ Má»¹ Dáº¡ Láº¥p lÃ¡nh há»“n thÆ¡ Viá»‡t trÃªn

sÃ¢n ga Tokyo chiá»u cuá»‘i nÄƒm

You are doing exactly what I was talking about. If you read the data
using the wrong encoding, then save it using the same encoding, you can
then open it using the corrent encoding, provided that the process
hasn't removed any data.

If you have set up your MySQL database to use unicode, and still get the
string out in that manner, the error is before you even saved the string
in the database in the first place. What you have done is basically:

unicode -bytes -wrong encoding -MySQL -wrong encoding -html ->
bytes -browser -unicode

While this gives the correct result for some strings, some byte codes
used in UTF-8 doesn't represent a single character by themselves, so if
you contine to store mis-decoded strings as unicode, you will sooner or
later experience corrupted strings.

--
GÃ¶ran Andersson
_____

guffa.com

http://www.guffa.com

Göran Anderssons privata hemsida.

**Bill Nguyen** · Jun 5 '07, 11:55 PM

Re: Unicode/UTF-8 decoding

Göran ;

I think you are correct. However, not much I can do since I can not change
the host server parameters.
I am using SQLyog to access mySQL remotely. What I need is to be able to
read the data in its correct format/encoding scheme. Is it possible with
..NET ?

Thanks

Bill

"Göran Andersson" <guffa@guffa.co mwrote in message
news:eWl7rU4pHH A.4100@TK2MSFTN GP04.phx.gbl...

Bill Nguyen wrote:

>I set UTF-8 as the default encoding in mySQL.
>I don't really know how this work, but IE or Firefox browser can decode
>easily.
>This is the test:
>I put the lines below in an HTML document and viewed it in IE, and it
>worked. (make sure to set encoding to UTF-8 in VIEW).
>I include the test.htm for your testing. (The text is in Vietnamese).
>So I think what I need is to find a utility that has the same function
>that might already be available out there. Any help is greatly
>appreciated.
>>
>Bill
>>
>----------------
><html>
>>
><head></head>
>>
><body>
>>
>>
>>

>>Virginia Hamilton Adair / LÃ¢m Thá»< Má»¹ Dáº¡
>>Láº¥p lÃ¡nh há»"n thÆ¡ Viá»?t trÃªn sÃ¢n ga Tokyo chiá»u cuá»'i nÄfm

>>
>>
></body>
>>
></html>
>>
>>
>>
>"Göran Andersson" <guffa@guffa.co mwrote in message
>news:%23rhR3M0 pHHA.1776@TK2MS FTNGP05.phx.gbl ...

>>Bill Nguyen wrote:
>>>Below are sometext I extracted from a mySQL database. How can I decode
>>>them so that I can read them in Unicode?
>>>Thanks
>>>>
>>>Bill
>>>>
>>>------------
>>>>
>>>Virginia Hamilton Adair / LÃ¢m Thá»< Má»¹ Dáº¡
>>>Láº¥p lÃ¡nh há»"n thÆ¡ Viá»?t trÃªn sÃ¢n ga Tokyo chiá»u cuá»'i nÄfm
>>>
>>This text looks as it has been decoded with a different encoding than
>>was used to encode it. It might be possible to recreate the data if you
>>know what encodings was used to encode and decode it. Then you might be
>>able to encode it back to it's prevois state and use the proper encoding
>>to decode it. There is a great risk that some data has been lost,
>>though, and that you can't recreate the original data from this stage.
>>>
>>If you want to store unicode strings in the MySQL database, it has to be
>>set up to use unicode as character set.
>>>
>>--
>>Göran Andersson
>>_____
>>http://www.guffa.com

>>
>------------------------------------------------------------------------
>>

> Virginia Hamilton Adair / Lâm Th? M? D? L?p lánh h?n tho Vi?t trên

>sân ga Tokyo chi?u cu?i nam

>
You are doing exactly what I was talking about. If you read the data using
the wrong encoding, then save it using the same encoding, you can then
open it using the corrent encoding, provided that the process hasn't
removed any data.
>
If you have set up your MySQL database to use unicode, and still get the
string out in that manner, the error is before you even saved the string
in the database in the first place. What you have done is basically:
>
unicode -bytes -wrong encoding -MySQL -wrong encoding -html ->
bytes -browser -unicode
>
While this gives the correct result for some strings, some byte codes used
in UTF-8 doesn't represent a single character by themselves, so if you
contine to store mis-decoded strings as unicode, you will sooner or later
experience corrupted strings.
>
--
Göran Andersson
_____
http://www.guffa.com

**=?ISO-8859-1?Q?G=F6ran_Andersson?=** · Jun 6 '07, 08:45 AM

Re: Unicode/UTF-8 decoding

Bill Nguyen wrote:

Göran ;
>
I think you are correct. However, not much I can do since I can not change
the host server parameters.
I am using SQLyog to access mySQL remotely. What I need is to be able to
read the data in its correct format/encoding scheme. Is it possible with
.NET ?
>
Thanks
>
Bill
>

Yes, it's possible in .NET.

Strictly speaking you can't read it using the correct encoding, as it's
not stored using the correct encoding. You can only read it the same way
it's stored, then you have to reverse the process by encoding it using
the same wrong encoding and decoding it using the correct encoding.

As I said earlier, this will not work for all strings, so if you want a
system that works correctly, you have to change how the data is stored
in the database.

--
Göran Andersson
_____

guffa.com

http://www.guffa.com

Göran Anderssons privata hemsida.

Unicode/UTF-8 decoding

Unicode/UTF-8 decoding

Comment

Comment

Comment

Comment

Comment

Comment