'error reading datastream' -- loading file only when transfer is complete?

[liuliuliu@gmail.com]

hello --

i'm running python/pygame on maemo (nokia 770). my situation is that
i'm continually scouring this one directory for incoming files. if i
see if there's a new file (coming in via wireless scp), i proceed to
load it and process it.

however, i think i am running into the issue that my program starts to
load the file after it recognises there is new data, but before the
file has completely transferred, so at unpredictable times i get a
pygame.error: Error reading from datastream.

what is the easiest way to work out this issue? easy being the key
word. :) thank you very much!

christine

[Andrew Robert]

Re: 'error reading datastream' -- loading file only when transferis complete?

liuliuliu@gmail .com wrote:[color=blue]
> hello --
>
> i'm running python/pygame on maemo (nokia 770). my situation is that
> i'm continually scouring this one directory for incoming files. if i
> see if there's a new file (coming in via wireless scp), i proceed to
> load it and process it.
>
> however, i think i am running into the issue that my program starts to
> load the file after it recognises there is new data, but before the
> file has completely transferred, so at unpredictable times i get a
> pygame.error: Error reading from datastream.
>
> what is the easiest way to work out this issue? easy being the key
> word. :) thank you very much!
>
> christine
>[/color]

You might want to test for file locking before attempting to use

[liuliuliu@gmail.com]

Re: 'error reading datastream' -- loading file only when transfer is complete?

thanks - i'm looking, but i found this as well. actually, does this
work?

import os
os.access(file, os.R_OK) # is it readable?

is this valid:

{ i have my file identified }
isFileAccessibl e = os.access(file, os.R_OK)
while !isFileAccessib le:
isFileAccessibl e = os.access(file, os.R_OK)

and then whenever it's true it can only then proceed to load the file?

thanks, christine

Andrew Robert wrote:[color=blue]
> liuliuliu@gmail .com wrote:[color=green]
> > hello --
> >
> > i'm running python/pygame on maemo (nokia 770). my situation is that
> > i'm continually scouring this one directory for incoming files. if i
> > see if there's a new file (coming in via wireless scp), i proceed to
> > load it and process it.
> >
> > however, i think i am running into the issue that my program starts to
> > load the file after it recognises there is new data, but before the
> > file has completely transferred, so at unpredictable times i get a
> > pygame.error: Error reading from datastream.
> >
> > what is the easiest way to work out this issue? easy being the key
> > word. :) thank you very much!
> >
> > christine
> >[/color]
>
> You might want to test for file locking before attempting to use[/color]

[Jim Segrave]

Re: 'error reading datastream' -- loading file only when transferis complete?

In article <126v3nsag5bcqd b@corp.supernew s.com>,
Andrew Robert <andrew.arobert @gmail.com> wrote:[color=blue]
>liuliuliu@gmai l.com wrote:[color=green]
>> hello --
>>
>> i'm running python/pygame on maemo (nokia 770). my situation is that
>> i'm continually scouring this one directory for incoming files. if i
>> see if there's a new file (coming in via wireless scp), i proceed to
>> load it and process it.
>>
>> however, i think i am running into the issue that my program starts to
>> load the file after it recognises there is new data, but before the
>> file has completely transferred, so at unpredictable times i get a
>> pygame.error: Error reading from datastream.
>>
>> what is the easiest way to work out this issue? easy being the key
>> word. :) thank you very much!
>>
>> christine
>>[/color]
>
>You might want to test for file locking before attempting to use[/color]


Or, really crude - when you see the file, record the time and the file
size, but don't attempt to process it yet. Wait for a short interval,
then check the size again. If it's changed, wait again. When it gives
the same size after a delay period, assume all the data is there.

This isn't a good method, but it's simple to implement and will reduce
the occurrence of attempts to process a file which is still in
transit/

--
Jim Segrave (jes@jes-2.demon.nl)

[liuliuliu]

Re: 'error reading datastream' -- loading file only when transfer is complete?

thanks all. this seemed to work:

import os
stat = os.stat(str(new Files[index]))
size1 = stat.st_size
pygame.time.wai t(100)
stat = os.stat(str(new Files[index]))
size2 = stat.st_size
while size2 > size1:
size1 = size2
pygame.time.wai t(500)
stat = os.stat(str(new Files[index]))
size2 = stat.st_size
# procede to load complete file

my delay is so long because somehow pygame (or file-writing) runs
relatively slow on maemo/nokia770. even a delay of 400 milliseconds was
too quick to accommodate the slothly rate of an incoming file.