UTF Questions

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Fuzzyman
    #1

    UTF Questions

    I have a couple of questions about the UTF encodings.

    The codecs module has constants definded for the UTF32 encoding, yet
    this encoding isn't supported as a standard encoding. Why isn't it
    supported ?

    It possibly has something to do with my next question. I know that
    unicode has (recently?) been expanded to include new character sets.
    This means that the latest unicode standard can't be fully supported
    with 2 bytes per character. As far as I know though, Python doesn't
    (yet) support the extended version of unicode anyway ? Am I correct ?

    Best Reagrds,

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml

    Tags: None
    • Serge Orlov
      #2
      Re: UTF Questions

      Fuzzyman wrote:[color=blue]
      > I have a couple of questions about the UTF encodings.
      >
      > The codecs module has constants definded for the UTF32 encoding, yet
      > this encoding isn't supported as a standard encoding. Why isn't it
      > supported ?[/color]

      Probably because there is little demand for it. The most widespread
      unicode encodings are utf-8 and utf-16
      [color=blue]
      >
      > It possibly has something to do with my next question. I know that
      > unicode has (recently?) been expanded to include new character sets.
      > This means that the latest unicode standard can't be fully supported
      > with 2 bytes per character. As far as I know though, Python doesn't
      > (yet) support the extended version of unicode anyway ? Am I correct ?[/color]

      Python does support them. PEP 261 has the answers for your questions.

      Serge.

        Comment

        • Fuzzyman
          #3
          Re: UTF Questions

          Thanks Serge.

          Regards,

          Fuzzy
          http://www.voidspace.org.uk/python/index.shtml

            Comment

            • Serge Orlov
              #4
              Re: UTF Questions

              Fuzzyman wrote:[color=blue]
              > Thanks Serge.[/color]

              You're welcome. While we at it, iconvcodec supports utf-32 and more. I have sent
              a 2.4 windows build of iconvcodec module to the author. He promised to publish it
              soon.

              Serge.


                Comment

                • Martin v. Löwis
                  #5
                  Re: UTF Questions

                  Fuzzyman wrote:[color=blue]
                  > The codecs module has constants definded for the UTF32 encoding, yet
                  > this encoding isn't supported as a standard encoding. Why isn't it
                  > supported ?[/color]

                  Because nobody has contributed such an implementation.

                  Notice that this is really trivial to implement, with very few lines
                  of pure Python code. In fact, given a Unicode string s, the line

                  codecs.BOM_UTF3 2+array.array(" i",map(ord,s)). tostring()

                  generates UTF-32 for the string s. Creating a codec on top of this
                  approach is left as an exercise for the reader.

                  Regards,
                  Martin

                    Comment

                    Previous template Next
                    Working...