ASCII / ANSI encoding

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Mark Rae

    #1

    ASCII / ANSI encoding

    Hi,

    I'm in the process if converting the data out of an old DOS-based
    SunAccounts system (don't ask!) into SQL Server.

    The data has been sent to me as a collection of hundreds of SunAccounts
    backup files and it's a simple (yet extremely laborious!) process of opening
    each backup file in turn, reading the file line by line, splitting it up
    into its constituent parts, and then squirting it into SQL Server.

    However, I'm experiencing a problem because Sun, being a DOS-based system,
    uses ASCII encoding. Specifically, the old chestnut of the British pound
    sign, which Sun considers to be hex 9c (i.e. Alt-156). However, Windows
    using ANSI encoding sees this character as o instead of £ (Alt-163).

    If I open the files into a StreamReader object without specifying an
    encoding, C# appears to be using ASCII encoding. This has the effect of
    ignoring the character completely which, in turn, throws the rest of the
    import of of kilter because the file format is fixed-width, not delimted.

    Therefore, can anyone please tell me what encoding I should use when reading
    the file into a StreamReader object? I've tried several, but none appears to
    work correctly so far. Some render the character as a question mark or a
    blank square which, although not correct, at least doesn't stop the rest of
    the import from working...

    Any assistance gratefully received.

    Mark


  • tushar.n.patel@gmail.com

    #2
    Re: ASCII / ANSI encoding

    hi Mark,
    i've gone through such pain but my problem was little different. well
    dude i would say you can not use StreamReader in your case. you have to
    use BinaryReader to read whole file and than should use
    "System.Text.AS CIIEncoding" to retrive the data into the WINDOWS ASCII
    format.

    this should work.

    Comment

    • Larry Lard

      #3
      Re: ASCII / ANSI encoding


      Mark Rae wrote:[color=blue]
      > Hi,
      >
      > I'm in the process if converting the data out of an old DOS-based
      > SunAccounts system (don't ask!) into SQL Server.
      >
      > The data has been sent to me as a collection of hundreds of SunAccounts
      > backup files and it's a simple (yet extremely laborious!) process of opening
      > each backup file in turn, reading the file line by line, splitting it up
      > into its constituent parts, and then squirting it into SQL Server.
      >
      > However, I'm experiencing a problem because Sun, being a DOS-based system,
      > uses ASCII encoding. Specifically, the old chestnut of the British pound
      > sign, which Sun considers to be hex 9c (i.e. Alt-156).[/color]

      ASCII is a 7-bit character system, and does not define any character
      code higher than 127 (hex 7F). A quick poke at Google suggests that 9C
      is the pound sign in Microsoft's DOS codepage 437. Hmm, or maybe 850.
      Take a look at <http://en.wikipedia.or g/wiki/Category:DOS_co de_pages>
      and links therefrom to decide for yourself.
      [color=blue]
      > However, Windows
      > using ANSI encoding sees this character as o instead of £ (Alt-163).[/color]

      I'm really no expert on these matters, but I'm fairly sure that
      'Windows using ANSI encoding' isn't specific enough to be amgibuous.
      [color=blue]
      >
      > If I open the files into a StreamReader object without specifying an
      > encoding, C# appears to be using ASCII encoding. This has the effect of
      > ignoring the character completely which, in turn, throws the rest of the
      > import of of kilter because the file format is fixed-width, not delimted.
      >
      > Therefore, can anyone please tell me what encoding I should use when reading
      > the file into a StreamReader object?[/color]

      If I'm right about the original encoding being DOS codepage 437, you
      should use the Encoding created by
      System.Text.Enc oding.GetEncodi ng(437). Or try 850.
      [color=blue]
      > I've tried several, but none appears to
      > work correctly so far. Some render the character as a question mark or a
      > blank square which, although not correct, at least doesn't stop the rest of
      > the import from working...
      >
      > Any assistance gratefully received.[/color]

      Encodings are a nightmare. Best of luck!

      --
      Larry Lard
      Replies to group please

      Comment

      • Mark Rae

        #4
        Re: ASCII / ANSI encoding

        "Larry Lard" <larrylard@hotm ail.com> wrote in message
        news:1134734065 .571712.8180@g4 9g2000cwa.googl egroups.com...

        Hi Larry,
        [color=blue]
        >System.Text.En coding.GetEncod ing(437)[/color]

        Worked perfectly! Thanks very much.

        Mark


        Comment

        • Mark Rae

          #5
          Re: ASCII / ANSI encoding

          <tushar.n.patel @gmail.com> wrote in message
          news:1134733211 .393426.24230@o 13g2000cwo.goog legroups.com...
          [color=blue]
          > you can not use StreamReader in your case[/color]

          Larry Lard's suggestion of System.Text.Enc oding.GetEncodi ng(437) worked
          perfectly.


          Comment

          • Jon Skeet [C# MVP]

            #6
            Re: ASCII / ANSI encoding

            Mark Rae <mark@markN-O-S-P-A-M.co.uk> wrote:[color=blue]
            > I'm in the process if converting the data out of an old DOS-based
            > SunAccounts system (don't ask!) into SQL Server.
            >
            > The data has been sent to me as a collection of hundreds of SunAccounts
            > backup files and it's a simple (yet extremely laborious!) process of opening
            > each backup file in turn, reading the file line by line, splitting it up
            > into its constituent parts, and then squirting it into SQL Server.
            >
            > However, I'm experiencing a problem because Sun, being a DOS-based system,
            > uses ASCII encoding. Specifically, the old chestnut of the British pound
            > sign, which Sun considers to be hex 9c (i.e. Alt-156). However, Windows
            > using ANSI encoding sees this character as o instead of £ (Alt-163).[/color]

            Neither of those are ASCII characters. ASCII is 7-bit. See
            http://www.pobox.com/~skeet/csharp/unicode.html for more information.

            --
            Jon Skeet - <skeet@pobox.co m>
            http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
            If replying to the group, please do not mail me too

            Comment

            • Jon Skeet [C# MVP]

              #7
              Re: ASCII / ANSI encoding

              <tushar.n.patel @gmail.com> wrote:[color=blue]
              > i've gone through such pain but my problem was little different. well
              > dude i would say you can not use StreamReader in your case. you have to
              > use BinaryReader to read whole file and than should use
              > "System.Text.AS CIIEncoding" to retrive the data into the WINDOWS ASCII
              > format.[/color]

              There's nothing wrong with using StreamReader here, but what's required
              *isn't* ASCII. There's no such thing as "Windows ASCII" - there are
              various 8-bit "extensions " to ASCII, in various "code pages"; they
              aren't ASCII themselves, however.

              --
              Jon Skeet - <skeet@pobox.co m>
              http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
              If replying to the group, please do not mail me too

              Comment

              • Lucky

                #8
                Re: ASCII / ANSI encoding

                what i mean by WINDOWS ASCII is windows compatiable string. that what
                was your problem right? you were not able to get DOS ASCII stirng
                exactly in WINDOWS string. am i right?

                Comment

                • Joerg Jooss

                  #9
                  Re: ASCII / ANSI encoding

                  Lucky wrote:
                  [color=blue]
                  > what i mean by WINDOWS ASCII is windows compatiable string. that what
                  > was your problem right? you were not able to get DOS ASCII stirng
                  > exactly in WINDOWS string. am i right?[/color]

                  As Jon said, there's no Windows ASCII, nor a DOS ASCII. There's only
                  ASCII. There's also no such thing as a Windows compatible string. It's
                  all just a matter of using the correct encoding.

                  Cheers,
                  --

                  mailto:news-reply@joergjoos s.de

                  Comment

                  • Jon Skeet [C# MVP]

                    #10
                    Re: ASCII / ANSI encoding

                    Lucky <tushar.n.patel @gmail.com> wrote:[color=blue]
                    > what i mean by WINDOWS ASCII is windows compatiable string. that what
                    > was your problem right? you were not able to get DOS ASCII stirng
                    > exactly in WINDOWS string. am i right?[/color]

                    "DOS ASCII" and "WINDOWS ASCII" aren't precisely defined terms though.
                    DOS used lots of different default code pages depending on your region,
                    and so does Windows.

                    --
                    Jon Skeet - <skeet@pobox.co m>
                    http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
                    If replying to the group, please do not mail me too

                    Comment

                    • Mihai N.

                      #11
                      Re: ASCII / ANSI encoding

                      > "DOS ASCII" and "WINDOWS ASCII" aren't precisely defined terms though.[color=blue]
                      > DOS used lots of different default code pages depending on your region,
                      > and so does Windows.[/color]
                      Joerg is right, there is no "DOS ASCII" and "WINDOWS ASCII", and ASCII is
                      very precisely defined.

                      ASCII = everything between 0-127, and is the same in all code pages (except
                      EBCDIC, but that is not ASCII, is EBCDIC :-)

                      ANSI code page = the Windows code pages. Same as "system code page". For Win
                      9x cannot be changed, for W2K, XP and newer it can be changed.
                      Best internationalization practices, solving internationalization problems.


                      OEM code page = the DOS code pages. It is still used in Windows for console
                      applications.

                      See some info here:
                      http://blogs.msdn.com/michkap/archiv...08/369197.aspx
                      http://blogs.msdn.com/oldnewthing/ar...08/389527.aspx
                      Best internationalization practices, solving internationalization problems.


                      --
                      Mihai Nita [Microsoft MVP, Windows - SDK]
                      Best internationalization practices, solving internationalization problems.

                      ------------------------------------------
                      Replace _year_ with _ to get the real email

                      Comment

                      Working...