fromfile error on windows, not mac

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • jadamwil

    fromfile error on windows, not mac

    Hello,
    I am using the numpy fromfile function to read binary data from a file
    on disk. The problem is that the program runs fine on a Mac, but gives
    an error or warning on windows when trying to read the data. I use it
    like this:

    Signal = zeros((N, 16), dtype=float32)
    for sample in range(0, N):
    # this function gets the next position in the file to seek to
    s = getFilePos(samp le)

    # go to the correct location in the file; this IS checked to make
    sure it is within the file
    mFile.seek(s)

    # read the 16 float32 values from the file
    D = fromfile(mFile, dtype=numpy.flo at32, 16)

    # save D in Signal
    Signal[sample, :] = D

    This will fail when sample is ~4. If I change the range to (5,N),
    skipping the "bad" file location, it will run fine for a few samples,
    and then give another error. The message it gives is:
    "16 items requested but only 7 read"

    So D is a 7x1 vector, and the program dies when it tries to assign D
    to the slice of Signal ("ValueError : shape mismatch: objects cannot be
    broadcast to a single shape").

    On windows, the Python version is 2.5.2, and the most recent numpy and
    scipy are being used as well. I tried using Enthought, but it gave
    this error as well, in addition to a c runtime error whenever I
    imported scipy (which is another post topic...).

    Any ideas on what might be causing this? Is there a way to debug the
    fromfile function? And, remember, this works perfectly on a Mac. Would
    compiling everything (python, scipy, numpy) potentially solve this?

    Thanks!
  • Uwe Schmitt

    #2
    Re: fromfile error on windows, not mac



    jadamwil schrieb:
    Hello,
    I am using the numpy fromfile function to read binary data from a file
    on disk. The problem is that the program runs fine on a Mac, but gives
    an error or warning on windows when trying to read the data. I use it
    like this:
    >
    Signal = zeros((N, 16), dtype=float32)
    for sample in range(0, N):
    # this function gets the next position in the file to seek to
    s = getFilePos(samp le)
    >
    # go to the correct location in the file; this IS checked to make
    sure it is within the file
    mFile.seek(s)
    >
    # read the 16 float32 values from the file
    D = fromfile(mFile, dtype=numpy.flo at32, 16)
    >
    # save D in Signal
    Signal[sample, :] = D
    >
    This will fail when sample is ~4. If I change the range to (5,N),
    skipping the "bad" file location, it will run fine for a few samples,
    and then give another error. The message it gives is:
    "16 items requested but only 7 read"
    >
    So D is a 7x1 vector, and the program dies when it tries to assign D
    to the slice of Signal ("ValueError : shape mismatch: objects cannot be
    broadcast to a single shape").
    >
    On windows, the Python version is 2.5.2, and the most recent numpy and
    scipy are being used as well. I tried using Enthought, but it gave
    this error as well, in addition to a c runtime error whenever I
    imported scipy (which is another post topic...).
    >
    Any ideas on what might be causing this? Is there a way to debug the
    fromfile function? And, remember, this works perfectly on a Mac. Would
    compiling everything (python, scipy, numpy) potentially solve this?
    Did you open the file in binary mode ?

    Greetings, Uwe

    Comment

    • jadamwil

      #3
      Re: fromfile error on windows, not mac

      On Jul 22, 2:05 am, Uwe Schmitt <rocksportroc.. .@googlemail.co m>
      wrote:
      jadamwil schrieb:
      >
      >
      >
      Hello,
      I am using the numpy fromfile function to read binary data from a file
      on disk. The problem is that the program runs fine on a Mac, but gives
      an error or warning on windows when trying to read the data. I use it
      like this:
      >
      Signal = zeros((N, 16), dtype=float32)
      for sample in range(0, N):
        # this function gets the next position in the file to seek to
        s = getFilePos(samp le)
      >
        # go to the correct location in the file; this IS checked to make
      sure it is within the file
        mFile.seek(s)
      >
        # read the 16 float32 values from the file
        D = fromfile(mFile, dtype=numpy.flo at32, 16)
      >
        # save D in Signal
        Signal[sample, :] = D
      >
      This will fail when sample is ~4. If I change the range to (5,N),
      skipping the "bad" file location, it will run fine for a few samples,
      and then give another error. The message it gives is:
      "16 items requested but only 7 read"
      >
      So D is a 7x1 vector, and the program dies when it tries to assign D
      to the slice of Signal ("ValueError : shape mismatch: objects cannot be
      broadcast to a single shape").
      >
      On windows, the Python version is 2.5.2, and the most recent numpy and
      scipy are being used as well. I tried using Enthought, but it gave
      this error as well, in addition to a c runtime error whenever I
      imported scipy (which is another post topic...).
      >
      Any ideas on what might be causing this? Is there a way to debug the
      fromfile function? And, remember, this works perfectly on a Mac. Would
      compiling everything (python, scipy, numpy) potentially solve this?
      >
      Did you open the file in binary mode ?
      >
      Greetings, Uwe
      Yes I did. Would that make a difference between a mac and windows?
      Thanks

      Comment

      • jadamwil

        #4
        Re: fromfile error on windows, not mac

        On Jul 22, 8:35 am, jadamwil <jadamwils...@g mail.comwrote:
        On Jul 22, 2:05 am, Uwe Schmitt <rocksportroc.. .@googlemail.co m>
        wrote:
        >
        >
        >
        jadamwil schrieb:
        >
        Hello,
        I am using the numpy fromfile function to read binary data from a file
        on disk. The problem is that the program runs fine on a Mac, but gives
        an error or warning onwindowswhen trying to read the data. I use it
        like this:
        >
        Signal = zeros((N, 16), dtype=float32)
        for sample in range(0, N):
          # this function gets the next position in the file to seek to
          s = getFilePos(samp le)
        >
          # go to the correct location in the file; this IS checked to make
        sure it is within the file
          mFile.seek(s)
        >
          # read the 16 float32 values from the file
          D = fromfile(mFile, dtype=numpy.flo at32, 16)
        >
          # save D in Signal
          Signal[sample, :] = D
        >
        This will fail when sample is ~4. If I change the range to (5,N),
        skipping the "bad" file location, it will run fine for a few samples,
        and then give another error. The message it gives is:
        "16 items requested but only 7 read"
        >
        So D is a 7x1 vector, and the program dies when it tries to assign D
        to the slice of Signal ("ValueError : shape mismatch: objects cannot be
        broadcast to a single shape").
        >
        Onwindows, the Python version is 2.5.2, and the most recent numpy and
        scipy are being used as well. I tried using Enthought, but it gave
        this error as well, in addition to a c runtime error whenever I
        imported scipy (which is another post topic...).
        >
        Any ideas on what might be causing this? Is there a way to debug the
        fromfile function? And, remember, this works perfectly on a Mac. Would
        compiling everything (python, scipy, numpy) potentially solve this?
        >
        Did you open the file in binary mode ?
        >
        Greetings, Uwe
        >
        Yes I did. Would that make a difference between a mac andwindows?
        Thanks
        Uwe,
        Thanks for the suggestion. I passed "rb" to the open file command, not
        'rb' which made a difference. I thought it opened in binary, but it
        did not, and it seems windows is not as good unix for dealing with
        this situation.
        Adam

        Comment

        • Scott David Daniels

          #5
          Re: fromfile error on windows, not mac

          jadamwil wrote:
          On Jul 22, 2:05 am, Uwe Schmitt <rocksportroc.. .@googlemail.co m>
          wrote:
          >jadamwil schrieb:
          >>
          >>
          >>
          >>Hello,
          >>I am using the numpy fromfile function to read binary data from a file
          >>on disk. The problem is that the program runs fine on a Mac, but gives
          >>an error or warning on windows when trying to read the data. I use it
          >>like this:
          >>Signal = zeros((N, 16), dtype=float32)
          >>for sample in range(0, N):
          >> # this function gets the next position in the file to seek to
          >> s = getFilePos(samp le)
          >> # go to the correct location in the file; this IS checked to make
          >>sure it is within the file
          >> mFile.seek(s)
          >> # read the 16 float32 values from the file
          >> D = fromfile(mFile, dtype=numpy.flo at32, 16)
          >> # save D in Signal
          >> Signal[sample, :] = D
          >>This will fail when sample is ~4. If I change the range to (5,N),
          >>skipping the "bad" file location, it will run fine for a few samples,
          >>and then give another error. The message it gives is:
          >>"16 items requested but only 7 read"
          >>So D is a 7x1 vector, and the program dies when it tries to assign D
          >>to the slice of Signal ("ValueError : shape mismatch: objects cannot be
          >>broadcast to a single shape").
          >>On windows, the Python version is 2.5.2, and the most recent numpy and
          >>scipy are being used as well. I tried using Enthought, but it gave
          >>this error as well, in addition to a c runtime error whenever I
          >>imported scipy (which is another post topic...).
          >>Any ideas on what might be causing this? Is there a way to debug the
          >>fromfile function? And, remember, this works perfectly on a Mac. Would
          >>compiling everything (python, scipy, numpy) potentially solve this?
          >Did you open the file in binary mode ?
          >>
          >Greetings, Uwe
          >
          Yes I did. Would that make a difference between a mac and windows?
          Thanks
          In a word, yes.
          the values for bytes that have issues are 10 and 13.

          --Scott David Daniels
          Scott.Daniels@A cm.Org

          Comment

          • Tommy Nordgren

            #6
            Re: fromfile error on windows, not mac

            One question : Did you remember to open the file in binary mode?
            This MUST be done on windows.
            On 22 jul 2008, at 06.36, jadamwil wrote:
            Hello,
            I am using the numpy fromfile function to read binary data from a file
            on disk. The problem is that the program runs fine on a Mac, but gives
            an error or warning on windows when trying to read the data. I use it
            like this:
            >
            Signal = zeros((N, 16), dtype=float32)
            for sample in range(0, N):
            # this function gets the next position in the file to seek to
            s = getFilePos(samp le)
            >
            # go to the correct location in the file; this IS checked to make
            sure it is within the file
            mFile.seek(s)
            >
            # read the 16 float32 values from the file
            D = fromfile(mFile, dtype=numpy.flo at32, 16)
            >
            # save D in Signal
            Signal[sample, :] = D
            >
            This will fail when sample is ~4. If I change the range to (5,N),
            skipping the "bad" file location, it will run fine for a few samples,
            and then give another error. The message it gives is:
            "16 items requested but only 7 read"
            >
            So D is a 7x1 vector, and the program dies when it tries to assign D
            to the slice of Signal ("ValueError : shape mismatch: objects cannot be
            broadcast to a single shape").
            >
            On windows, the Python version is 2.5.2, and the most recent numpy and
            scipy are being used as well. I tried using Enthought, but it gave
            this error as well, in addition to a c runtime error whenever I
            imported scipy (which is another post topic...).
            >
            Any ideas on what might be causing this? Is there a way to debug the
            fromfile function? And, remember, this works perfectly on a Mac. Would
            compiling everything (python, scipy, numpy) potentially solve this?
            >
            Thanks!
            --
            http://mail.python.org/mailman/listinfo/python-list
            ----------------------------------
            Skinheads are so tired of immigration, that they are going to move to
            a country that don't accept immigrants!
            Tommy Nordgren
            tommy.nordgren@ comhem.se



            Comment

            • jadamwil

              #7
              Re: fromfile error on windows, not mac

              I found the problem: I thought it was opening in binary mode on BOTH
              windows and the mac, but on windows I passed "rb" with double quotes,
              not 'rb' with single quotes to the open file function. Changing it to
              'rb' fixed it.

              On Jul 23, 8:25 am, Tommy Nordgren <tommy.nordg... @comhem.sewrote :
              One question : Did you remember to open the file in binary mode?
              This MUST be done on windows.
              On 22 jul 2008, at 06.36, jadamwil wrote:
              >
              >
              >
              Hello,
              I am using the numpy fromfile function to read binary data from a file
              on disk. The problem is that the program runs fine on a Mac, but gives
              an error or warning on windows when trying to read the data. I use it
              like this:
              >
              Signal = zeros((N, 16), dtype=float32)
              for sample in range(0, N):
               # this function gets the next position in the file to seek to
               s = getFilePos(samp le)
              >
               # go to the correct location in the file; this IS checked to make
              sure it is within the file
               mFile.seek(s)
              >
               # read the 16 float32 values from the file
               D = fromfile(mFile, dtype=numpy.flo at32, 16)
              >
               # save D in Signal
               Signal[sample, :] = D
              >
              This will fail when sample is ~4. If I change the range to (5,N),
              skipping the "bad" file location, it will run fine for a few samples,
              and then give another error. The message it gives is:
              "16 items requested but only 7 read"
              >
              So D is a 7x1 vector, and the program dies when it tries to assign D
              to the slice of Signal ("ValueError : shape mismatch: objects cannot be
              broadcast to a single shape").
              >
              On windows, the Python version is 2.5.2, and the most recent numpy and
              scipy are being used as well. I tried using Enthought, but it gave
              this error as well, in addition to a c runtime error whenever I
              imported scipy (which is another post topic...).
              >
              Any ideas on what might be causing this? Is there a way to debug the
              fromfile function? And, remember, this works perfectly on a Mac. Would
              compiling everything (python, scipy, numpy) potentially solve this?
              >>
              ----------------------------------
              Skinheads are so tired of immigration, that they are going to move to  
              a country that don't accept immigrants!
              Tommy Nordgren
              tommy.nordg...@ comhem.se

              Comment

              • Fredrik Lundh

                #8
                Re: fromfile error on windows, not mac

                Manu Hack wrote:
                by the way, anyone could explain why changing to single quote makes a
                difference?
                >
                >>"rb" == 'rb'
                True
                There's no difference between single quotes and double quotes in Python.
                My guess is cargo cult debugging.

                </F>

                Comment

                • jadamwil

                  #9
                  Re: fromfile error on windows, not mac

                  On Jul 23, 3:30 pm, Fredrik Lundh <fred...@python ware.comwrote:
                  Manu Hack wrote:
                  by the way, anyone could explain why changing to single quote makes a
                  difference?
                  >
                   >>"rb" == 'rb'
                  True
                  >
                  There's no difference between single quotes and double quotes in Python.
                      My guess is cargo cult debugging.
                  >
                  </F>
                  Ha...you're probably not that far from the truth. I just started with
                  python about a week ago, and have had a LOT of luck getting our lab's
                  data analysis routines ported from matlab, with the exception of this
                  problem. I tried a lot of different things to get it working on
                  windows, and I think I got python quotes confused with matlab or php
                  quotes (where double and single quotes are different), tried it, and
                  it happened to work (although I probably changed something else as
                  well in desperation). Of course, as many have pointed out, this does
                  not actually matter in python. So anyway, yes, someone new to a
                  language trying lots of different things without a full understanding
                  of the language could probably be considered cargo cult
                  programming :-). Thanks for the replies though!

                  Comment

                  Working...