Missing unicode data?

[Klaus Alexander Seistrup]

Hi group,

I just came across the following exception:

#v+

$ python
Python 2.4.2 (#2, Sep 30 2005, 21:19:01)
[GCC 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu8)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.[color=blue][color=green][color=darkred]
>>> import unicodedata
>>> u'\N{LATIN LETTER SMALL CAPITAL BARRED B}'[/color][/color][/color]
UnicodeDecodeEr ror: 'unicodeescape' codec can't decode bytes in position 0-38: unknown Unicode character name[color=blue][color=green][color=darkred]
>>> unicodedata.nam e(u'\u1d03')[/color][/color][/color]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: no such name[color=blue][color=green][color=darkred]
>>> ^D[/color][/color][/color]
$

#v-

When checking unicodedata.nam e() against each uchar in the file
/usr/share/unidata/UnicodeData-4.0.1d1b.txt that came with the
console-data package on my Ubuntu Linux installation a total of
1226 unicode characters seems to be missing from the unicodedata
module (2477 missing characters when checking against the latest
database from unicode.org¹). Is this a deliberate omission?

Cheers,
Klaus.

¹) http://www.unicode.org/Public/UNIDATA/UnicodeData.txt
--
Klaus Alexander Seistrup
SubZeroNet, Copenhagen, Denmark

http://magnetic-ink.dk/

[Fredrik Lundh]

Re: Missing unicode data?

Klaus Alexander Seistrup wrote:
[color=blue]
> When checking unicodedata.nam e() against each uchar in the file
> /usr/share/unidata/UnicodeData-4.0.1d1b.txt that came with the
> console-data package on my Ubuntu Linux installation a total of
> 1226 unicode characters seems to be missing from the unicodedata
> module (2477 missing characters when checking against the latest
> database from unicode.org¹). Is this a deliberate omission?[/color]

I'm pretty sure unicodename.nam e() doesn't look in the UnicodeData file
on your machine, nor in the latest file from unicode.org. in other
words, you get whatever version that was used to create the Unicode data
set in your Python distribution.

this is usually the version that was current when that Python version
was originally released (i.e. in your case, when 2.4 was released).

iirc, 2.4 uses Unicode 3.2, and 2.5 uses Unicode 4.1. to update, use
the tools under Tools/unicode.

</F>

[Klaus Alexander Seistrup]

Re: Missing unicode data?

Fredrik Lundh skrev:
[color=blue]
> I'm pretty sure unicodename.nam e() doesn't look in the Unicode-
> Data file on your machine, nor in the latest file from unicode.org.[/color]

I am pretty sure of that, too. I was only using those files as a
reference against the unicode data that comes with my python interpreter.
[color=blue]
> in other words, you get whatever version that was used to create
> the Unicode data set in your Python distribution.[/color]

I see.
[color=blue]
> iirc, 2.4 uses Unicode 3.2, and 2.5 uses Unicode 4.1. to update,
> use the tools under Tools/unicode.[/color]

Thanks for the hint.

Mvh,

--
Klaus Alexander Seistrup
SubZeroNet, Copenhagen, Denmark

http://magnetic-ink.dk/