fwrite, fgets binary file readback problem

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Jeff

    fwrite, fgets binary file readback problem

    Im trying to figure out why I cant read back a binary file correctly.

    I have the following union:

    #define BITE_RECORD_LEN 12
    typedef union {
    unsigned char byte[BITE_RECORD_LEN];
    struct {
    unsigned char type; /* 0 Type */
    unsigned char sn[3]; /* 1-3 Serial Number */
    unsigned char date_yy; /* 4 Date - Year */
    unsigned char date_mm; /* 5 Date - Month */
    unsigned char date_dd; /* 6 Date - Day */
    unsigned char flight_leg; /* 7 Number */
    unsigned char time_hh; /* 8 Time of Day - Hours */
    unsigned char time_mm; /* 9 Time of Day - Minutes
    */
    unsigned char etime[2]; /* 10-11 Elapsed Time */

    struct { /* 12 Corrective Action
    Status */
    unsigned char spare: 6; /* bits 5-0, Spare */
    unsigned char maint_alert: 1; /* bit 6, Alert,
    1 = alert */
    unsigned char serviced: 1; /* bit 7, Status,
    1 = fault serviced */
    } caa_status;
    } field;
    } myrecord;

    I open it with: (some details ommitted)

    fp = fopen(filename, "wb");

    I write it as:

    myrecord temp_fault_reco rd;
    fwrite(.byte, BITE_RECORD_LEN , 1, fp);

    my call to ReadFile:
    fopen(file, "rb");

    m_fault_record = (myrecord *) malloc (size);
    int num = ReadFile (file_des, (unsigned char *)m_fault_recor d,
    size);

    I read it back via fgets:

    int ReadFile(FILE *fp, unsigned char *buf, int count)
    {
    if (fgets((char *)buf, count, fp) == NULL)
    {
    debug_message(" ERROR ReadFile reading data");
    return -1;
    }
    return count;
    }

    My problem is that after about 6 records (I loop over the
    'm_fault_record ')
    my data is not valid. 'size' above is calculated by stat'ing the file
    being read
    and is correct.

    At this point Ive spent several hours parsing down the code to a small
    executable and trying to find the problem with no luck, which is why
    Im posting
    this here to ask. Ive looked at google for a while too for fgets. I
    saw some people
    talking about fread() for binary files, but it seemed more for
    efficiency than a need
    in my case.

    If Ive missed any details please just reply and Ill add them. I think
    I got everything.
    I appreciate any and all comments. Someone else may see what Im doing
    wrong.

    As a sidenote, Im porting this code from c to c++ (which shouldnt be a
    problem).
    The old code used CVI/Labview for the ReadFile call, which I dont have
    and I just
    ported it to what I thought would be equivalent, and it should be
    easy, but apparently
    its not been.

    Thanks,
    Jeff
  • Walter Roberson

    #2
    Re: fwrite, fgets binary file readback problem

    In article <51e86015-d72d-4c3f-821a-924c5d50f532@a9 g2000prl.google groups.com>,
    Jeff <jeep@rahul.net wrote:
    >Im trying to figure out why I cant read back a binary file correctly.
    >I have the following union:
    >#define BITE_RECORD_LEN 12
    >typedef union {
    unsigned char byte[BITE_RECORD_LEN];
    struct {
    unsigned char type; /* 0 Type */
    unsigned char sn[3]; /* 1-3 Serial Number */
    unsigned char date_yy; /* 4 Date - Year */
    unsigned char date_mm; /* 5 Date - Month */
    unsigned char date_dd; /* 6 Date - Day */
    unsigned char flight_leg; /* 7 Number */
    unsigned char time_hh; /* 8 Time of Day - Hours */
    unsigned char time_mm; /* 9 Time of Day - Minutes
    >*/
    unsigned char etime[2]; /* 10-11 Elapsed Time */
    >
    struct { /* 12 Corrective Action
    >Status */
    unsigned char spare: 6; /* bits 5-0, Spare */
    unsigned char maint_alert: 1; /* bit 6, Alert,
    1 = alert */
    unsigned char serviced: 1; /* bit 7, Status,
    1 = fault serviced */
    } caa_status;
    } field;
    >} myrecord;
    Check that sizeof myrecord.field is the same as BITE_RECORD_LEN
    (and it wouldn't hurt to crosscheck that sizeof myrecord is BITE_RECORD_LEN )

    Compilers are allowed to put unnamed padding between structure elements
    for any purpose (though there are some restrictions on bitfield padding.)
    I would not -expect- padding between consequative unsigned char, but
    it is a possibility. In particular, I would be concerned that the
    the caa_status struct might end up with padding before or after it.


    I notice that you have used unsigned char bitfields. In C89/C90,
    the base type for a bitfield must be a (possibly qualified)
    int, signed int, or unsigned int. C99 adds _Bool to the allowed base
    type list, along with "or some other implementation-defined type".
    Thus the structure you show violates a Contraint unless you
    are using C99 and your implementation defines char as an allowed base type.
    The result *could* be that each of those unsigned char base types
    in caa_status is silently converted to unsigned int, with caa_status
    thus ending up as large as an int rather than "as large as a char"
    that is implicit in your code. This is a concrete reason why your
    field substructure might not be BITE_RECORD_LEN .
    --
    "It is surprising what a man can do when he has to, and how
    little most men will do when they don't have to."
    -- Walter Linn

    Comment

    • Ben Pfaff

      #3
      Re: fwrite, fgets binary file readback problem

      Jeff <jeep@rahul.net writes:
      Im trying to figure out why I cant read back a binary file correctly.
      [...]
      if (fgets((char *)buf, count, fp) == NULL)
      fgets is not appropriate for reading binary data. Use fread.
      --
      char a[]="\n .CJacehknorstu" ;int putchar(int);in t main(void){unsi gned long b[]
      ={0x67dffdff,0x 9aa9aa6a,0xa77f fda9,0x7da6aa6a ,0xa67f6aaa,0xa a9aa9f6,0x11f6} ,*p
      =b,i=24;for(;p+ =!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
      2:{i++;if(i)bre ak;else default:continu e;if(0)case 1:putchar(a[i&15]);break;}}}

      Comment

      • Jeff

        #4
        Re: fwrite, fgets binary file readback problem

        On Jun 5, 12:42 pm, Ben Pfaff <b...@cs.stanfo rd.eduwrote:
        >
        fgetsis not appropriate for reading binary data. Use fread.
        Yep, that fixed it. Thanks for all the info from both of you!
        Jeff

        Comment

        • Peter Nilsson

          #5
          Re: fwrite, fgets binary file readback problem

          Ben Pfaff wrote:
          Jeff <jeep@rahul.net writes:
          Im trying to figure out why I cant read back a binary file correctly.
          ...
          if (fgets((char *)buf, count, fp) == NULL)
          >
          fgets is not appropriate for reading binary data.
          To be more precise, it's not appropriate for reading binary data that
          contains zero bytes. Since it doesn't return a length of characters
          read, there is no way to distinguish between a read null byte and
          the null byte with which it terminates the buffer (unless it's the
          last
          character in the buffer.)

          It will also stop reading if it encounteres a '\n', which is often
          desired with text streams, but not so much with binary streams.

          The fact that fgets doesn't cope with null bytes in streams is often
          a good reason to ignore it for parsing text streams in programs
          that need an extra level of robustness.

          --
          Peter

          Comment

          • Joe Wright

            #6
            Re: fwrite, fgets binary file readback problem

            Peter Nilsson wrote:
            Ben Pfaff wrote:
            >Jeff <jeep@rahul.net writes:
            >>Im trying to figure out why I cant read back a binary file correctly.
            >>...
            >> if (fgets((char *)buf, count, fp) == NULL)
            >fgets is not appropriate for reading binary data.
            >
            To be more precise, it's not appropriate for reading binary data that
            contains zero bytes. Since it doesn't return a length of characters
            read, there is no way to distinguish between a read null byte and
            the null byte with which it terminates the buffer (unless it's the
            last
            character in the buffer.)
            >
            It will also stop reading if it encounteres a '\n', which is often
            desired with text streams, but not so much with binary streams.
            >
            The fact that fgets doesn't cope with null bytes in streams is often
            a good reason to ignore it for parsing text streams in programs
            that need an extra level of robustness.
            >
            I agree with Ben that fgets() is inappropriate for reading binary files.
            It is designed to read lines of text and present the line to the program
            as a string. There is no general case for a null byte in a text stream.

            --
            Joe Wright
            "Everything should be made as simple as possible, but not simpler."
            --- Albert Einstein ---

            Comment

            • CBFalconer

              #7
              Re: fwrite, fgets binary file readback problem

              Peter Nilsson wrote:
              Ben Pfaff wrote:
              >Jeff <jeep@rahul.net writes:
              >>
              >>Im trying to figure out why I cant read back a binary file correctly.
              >>...
              >> if (fgets((char *)buf, count, fp) == NULL)
              >>
              >fgets is not appropriate for reading binary data.
              >
              To be more precise, it's not appropriate for reading binary data that
              contains zero bytes. Since it doesn't return a length of characters
              read, there is no way to distinguish between a read null byte and
              the null byte with which it terminates the buffer (unless it's the
              last character in the buffer.)
              >
              It will also stop reading if it encounteres a '\n', which is often
              desired with text streams, but not so much with binary streams.
              >
              The fact that fgets doesn't cope with null bytes in streams is often
              a good reason to ignore it for parsing text streams in programs
              that need an extra level of robustness.
              However there is no restriction against using getc (or fgetc),
              which will convert the '\n's from text files correctly, and pass
              all the zero bytes. This makes the code depend only on the actual
              file type. getc, if macro implemented, will be just as fast, and
              eliminates the need for a buffer array.

              --
              [mail]: Chuck F (cbfalconer at maineline dot net)
              [page]: <http://cbfalconer.home .att.net>
              Try the download section.


              ** Posted from http://www.teranews.com **

              Comment

              • Ben Pfaff

                #8
                Re: fwrite, fgets binary file readback problem

                CBFalconer <cbfalconer@yah oo.comwrites:
                However there is no restriction against using getc (or fgetc),
                which will convert the '\n's from text files correctly,
                getc will not do any conversion on new-line characters in files
                opened in binary mode, on the implementations with which I am
                familiar.
                and pass all the zero bytes. This makes the code depend only
                on the actual file type. getc, if macro implemented, will be
                just as fast, and eliminates the need for a buffer array.
                On the implementations with which I am familiar, a sequence of
                getc calls will not be as fast as a single fread or fgets call
                that reads the same number of bytes. This is because every call
                to getc must check whether the end of the file buffer has been
                reached, but fread or fgets typically only needs to check for the
                end of the buffer once.
                --
                char a[]="\n .CJacehknorstu" ;int putchar(int);in t main(void){unsi gned long b[]
                ={0x67dffdff,0x 9aa9aa6a,0xa77f fda9,0x7da6aa6a ,0xa67f6aaa,0xa a9aa9f6,0x11f6} ,*p
                =b,i=24;for(;p+ =!*p;*p/=4)switch(0[p]&3)case 0:{return 0;for(p--;i--;i--)case+
                2:{i++;if(i)bre ak;else default:continu e;if(0)case 1:putchar(a[i&15]);break;}}}

                Comment

                • CBFalconer

                  #9
                  Re: fwrite, fgets binary file readback problem

                  Ben Pfaff wrote:
                  CBFalconer <cbfalconer@yah oo.comwrites:
                  >
                  >However there is no restriction against using getc (or fgetc),
                  >which will convert the '\n's from text files correctly,
                  >
                  getc will not do any conversion on new-line characters in files
                  opened in binary mode, on the implementations with which I am
                  familiar.
                  That's what I thought I said. :-)
                  >
                  >and pass all the zero bytes. This makes the code depend only
                  >on the actual file type. getc, if macro implemented, will be
                  >just as fast, and eliminates the need for a buffer array.
                  >
                  On the implementations with which I am familiar, a sequence of
                  getc calls will not be as fast as a single fread or fgets call
                  that reads the same number of bytes. This is because every call
                  to getc must check whether the end of the file buffer has been
                  reached, but fread or fgets typically only needs to check for the
                  end of the buffer once.
                  However, it will be close. At any rate, such differences will be
                  lost in the actual hardware access times.

                  --
                  [mail]: Chuck F (cbfalconer at maineline dot net)
                  [page]: <http://cbfalconer.home .att.net>
                  Try the download section.


                  ** Posted from http://www.teranews.com **

                  Comment

                  • Richard Tobin

                    #10
                    Re: fwrite, fgets binary file readback problem

                    In article <4848851A.B43F9 C4D@yahoo.com>,
                    CBFalconer <cbfalconer@mai neline.netwrote :
                    >However there is no restriction against using getc (or fgetc),
                    >which will convert the '\n's from text files correctly, and pass
                    >all the zero bytes.
                    All the stdio functions will convert line ends in text mode, and not
                    in binary mode. There's no difference between getc(), fgets(), and
                    fread() in this respect.

                    -- Richard
                    --
                    In the selection of the two characters immediately succeeding the numeral 9,
                    consideration shall be given to their replacement by the graphics 10 and 11 to
                    facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)

                    Comment

                    • Ben Pfaff

                      #11
                      Re: fwrite, fgets binary file readback problem

                      CBFalconer <cbfalconer@yah oo.comwrites:
                      Ben Pfaff wrote:
                      >CBFalconer <cbfalconer@yah oo.comwrites:
                      >>
                      >>However there is no restriction against using getc (or fgetc),
                      >>which will convert the '\n's from text files correctly,
                      >>
                      >getc will not do any conversion on new-line characters in files
                      >opened in binary mode, on the implementations with which I am
                      >familiar.
                      >
                      That's what I thought I said. :-)
                      No, you said that getc will convert '\n's from text file
                      correctly. I said no such conversion will happen when you open a
                      file in binary mode (which is what the OP's question is about).
                      --
                      Just another C hacker.

                      Comment

                      • CBFalconer

                        #12
                        Re: fwrite, fgets binary file readback problem

                        Richard Tobin wrote:
                        CBFalconer <cbfalconer@mai neline.netwrote :
                        >
                        >However there is no restriction against using getc (or fgetc),
                        >which will convert the '\n's from text files correctly, and pass
                        >all the zero bytes.
                        >
                        All the stdio functions will convert line ends in text mode, and
                        not in binary mode. There's no difference between getc(),
                        fgets(), and fread() in this respect.
                        On *ix, but not doze. Under *ix the files are identical, under
                        doze there are \r in the char sequence for text files, which will
                        be absorbed if followed by a \n. This doesn't happen for a binary
                        file. And there are many other file systems.

                        --
                        [mail]: Chuck F (cbfalconer at maineline dot net)
                        [page]: <http://cbfalconer.home .att.net>
                        Try the download section.


                        ** Posted from http://www.teranews.com **

                        Comment

                        • Richard

                          #13
                          Re: fwrite, fgets binary file readback problem

                          Ben Pfaff <blp@cs.stanfor d.eduwrites:
                          CBFalconer <cbfalconer@yah oo.comwrites:
                          >
                          >Ben Pfaff wrote:
                          >>CBFalconer <cbfalconer@yah oo.comwrites:
                          >>>
                          >>>However there is no restriction against using getc (or fgetc),
                          >>>which will convert the '\n's from text files correctly,
                          >>>
                          >>getc will not do any conversion on new-line characters in files
                          >>opened in binary mode, on the implementations with which I am
                          >>familiar.
                          >>
                          >That's what I thought I said. :-)
                          >
                          No, you said that getc will convert '\n's from text file
                          correctly. I said no such conversion will happen when you open a
                          file in binary mode (which is what the OP's question is about).
                          Semantics. "Correctly" can mean not translating at all depending on
                          mode. i.e Correct handling.

                          Comment

                          • Keith Thompson

                            #14
                            Re: fwrite, fgets binary file readback problem

                            CBFalconer <cbfalconer@yah oo.comwrites:
                            Richard Tobin wrote:
                            >CBFalconer <cbfalconer@mai neline.netwrote :
                            >>
                            >>However there is no restriction against using getc (or fgetc),
                            >>which will convert the '\n's from text files correctly, and pass
                            >>all the zero bytes.
                            >>
                            >All the stdio functions will convert line ends in text mode, and
                            >not in binary mode. There's no difference between getc(),
                            >fgets(), and fread() in this respect.
                            >
                            On *ix, but not doze. Under *ix the files are identical, under
                            doze there are \r in the char sequence for text files, which will
                            be absorbed if followed by a \n.
                            Yes, in principle the conversion happens on any system. On Unix,
                            it just happens that it doesn't do anything.

                            The point is that you should still use "r" when opening a file in text
                            mode, and "rb" when opening a file in binary mode, even if your code
                            happens to run on a system where they happen to do the same thing.
                            This doesn't happen for a binary
                            file.
                            You mean for a file opened in binary mode.
                            And there are many other file systems.
                            Yes.

                            --
                            Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
                            Nokia
                            "We must do something. This is something. Therefore, we must do this."
                            -- Antony Jay and Jonathan Lynn, "Yes Minister"

                            Comment

                            • Richard Tobin

                              #15
                              Re: fwrite, fgets binary file readback problem

                              In article <484941B6.C21F5 C5C@yahoo.com>,
                              CBFalconer <cbfalconer@mai neline.netwrote :
                              >All the stdio functions will convert line ends in text mode, and
                              >not in binary mode. There's no difference between getc(),
                              >fgets(), and fread() in this respect.
                              >On *ix, but not doze. Under *ix the files are identical, under
                              >doze there are \r in the char sequence for text files, which will
                              >be absorbed if followed by a \n. This doesn't happen for a binary
                              >file. And there are many other file systems.
                              I don't see the relevance of your comment. On both Unix and Windows,
                              there is no difference between getc(), fgets(), and fread() in respect
                              of how they handle line ends. They all convert them in text mode (and
                              the conversion is a no-op in Unix), and none of them convert them in
                              binary mode.

                              In particular, fread() on a text stream converts line-ends in just the
                              same way as getc(). fread() is not a "binary-only" function.

                              -- Richard


                              --
                              In the selection of the two characters immediately succeeding the numeral 9,
                              consideration shall be given to their replacement by the graphics 10 and 11 to
                              facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)

                              Comment

                              Working...