unicode and strings

[Jacob Friis]

I'm trying to learn Python via Marks Feedparser.

<snip src="http://feedparser.org/docs/character-encoding.html">
If the character encoding can not be determined, Universal Feed Parser
sets the bozo bit to 1 and sets bozo_exception to
feedparser.Char acterEncodingUn known. In this case, parsed values will be
strings, not Unicode strings.
</snip>

I guess this means that all data will be unicode, and to put in a
database I could use my mycode function. Correct?

def mycode(value):
if isinstance(valu e, unicode):
value = value.encode('u tf-8')
return value

What do I do about data that is a string?

Thanks,
Jacob

[Diez B. Roggisch]

Re: unicode and strings

Jacob Friis wrote:
[color=blue]
> I'm trying to learn Python via Marks Feedparser.
>
> <snip src="http://feedparser.org/docs/character-encoding.html">
> If the character encoding can not be determined, Universal Feed Parser
> sets the bozo bit to 1 and sets bozo_exception to
> feedparser.Char acterEncodingUn known. In this case, parsed values will be
> strings, not Unicode strings.
> </snip>
>
> I guess this means that all data will be unicode, and to put in a
> database I could use my mycode function. Correct?[/color]

No. It means that you don't get unicode objects, but strings which are
basically sequences of bytes. And there is no way to be sure what encoding
they are in.
[color=blue]
>
> def mycode(value):
> if isinstance(valu e, unicode):
> value = value.encode('u tf-8')
> return value[/color]

this will either yield a string in utf8-encoding, or a string in an unknown
encoding.

--
Regards,

Diez B. Roggisch