Something that Perl can do that Python can't?

[Dr. Who]

So here it is: handle unbuffered output from a child process.

Here is the child process script (bufcallee.py):
import time
print 'START'
time.sleep(10)
print 'STOP'

In Perl, I do:
open(FILE, "python bufcallee.py |");
while ($line = <FILE>)
{
print "LINE: $line";
}

in which case I get
LINE: START
followed by a 10 second pause and then
LINE: STOP

The equivalent in Python:
import sys, os

FILE = os.popen('pytho n bufcallee.py')
for line in FILE:
print 'LINE:', line

yields a 10 second pause followed by
LINE: START
LINE: STOP

I have tried the subprocess module, the -u on both the original and
called script, setting bufsize=0 explicitly but to no avail. I also
get the same behavior on Windows and Linux.

If anyone can disprove me or show me what I'm doing wrong, it would be
appreciated.

Jeff

[Dr. Who]

Re: Something that Perl can do that Python can't?

Well, I finally managed to solve it myself by looking at some code.
The solution in Python is a little non-intuitive but this is how to get
it:

while 1:
line = stdout.readline ()
if not line:
break
print 'LINE:', line,

If anyone can do it the more Pythonic way with some sort of iteration
over stdout, please let me know.

Jeff

[Donn Cave]

Re: Something that Perl can do that Python can't?

In article <1122064588.174 698.153220@g43g 2000cwa.googleg roups.com>,
"Dr. Who" <google@spiceai d.com> wrote:[color=blue]
> So here it is: handle unbuffered output from a child process.[/color]

Your Perl program works the same for me, on MacOS X,
as your Python program. That's what we would expect,
of course, because the problem is with the (Python)
program on the other end - it's buffering output,
because the output device is not a terminal.

Donn Cave, donn@u.washingt on.edu
[color=blue]
> Here is the child process script (bufcallee.py):
> import time
> print 'START'
> time.sleep(10)
> print 'STOP'
>
> In Perl, I do:
> open(FILE, "python bufcallee.py |");
> while ($line = <FILE>)
> {
> print "LINE: $line";
> }
>
> in which case I get
> LINE: START
> followed by a 10 second pause and then
> LINE: STOP
>
> The equivalent in Python:
> import sys, os
>
> FILE = os.popen('pytho n bufcallee.py')
> for line in FILE:
> print 'LINE:', line
>
> yields a 10 second pause followed by
> LINE: START
> LINE: STOP
>
> I have tried the subprocess module, the -u on both the original and
> called script, setting bufsize=0 explicitly but to no avail. I also
> get the same behavior on Windows and Linux.
>
> If anyone can disprove me or show me what I'm doing wrong, it would be
> appreciated.
>
> Jeff[/color]

[George Sakkis]

Re: Something that Perl can do that Python can't?

"Dr. Who" <google@spiceai d.com> wrote in message
news:1122066437 .870669.58990@z 14g2000cwz.goog legroups.com...[color=blue]
> Well, I finally managed to solve it myself by looking at some code.
> The solution in Python is a little non-intuitive but this is how to get
> it:
>
> while 1:
> line = stdout.readline ()
> if not line:
> break
> print 'LINE:', line,
>
> If anyone can do it the more Pythonic way with some sort of iteration
> over stdout, please let me know.
>
> Jeff[/color]


You can use the sentinel form of iter():

for line in iter(stdout.rea dline, ''):
print 'LINE:', line,


George

[Andrew Dalke]

Re: Something that Perl can do that Python can't?

Dr. Who wrote:[color=blue]
> Well, I finally managed to solve it myself by looking at some code.
> The solution in Python is a little non-intuitive but this is how to get
> it:
>
> while 1:
> line = stdout.readline ()
> if not line:
> break
> print 'LINE:', line,
>
> If anyone can do it the more Pythonic way with some sort of iteration
> over stdout, please let me know.[/color]

Python supports two different but related iterators over
lines of a file. What you show here is the oldest way.
It reads up to the newline (or eof) and returns the line.

The newer way is

for line in stdout:
...

which is equivalent to

_iter = iter(stdout)
while 1:
try:
line = _iter.next()
except StopIteration:
break

...

The file.__iter__() is implemented by doing
a block read and internally breaking the block
into lines. This make the read a lot faster
because it does a single system call for the
block instead of a system call for every
character read. The downside is that the read
can block (err, a different use of "block")
waiting for enough data.

If you want to use the for idiom and have
the guaranteed "no more than a line at a time"
semantics, try this

for line in iter(stdout.rea dline, ""):
print "LINE:", line
sys.stdout.flus h()

Andrew
dalke@dalkescie ntific.com

[jitu]

Re: Something that Perl can do that Python can't?

Donn Cave wrote:[color=blue]
> In article <1122064588.174 698.153220@g43g 2000cwa.googleg roups.com>,
> "Dr. Who" <google@spiceai d.com> wrote:[color=green]
> > So here it is: handle unbuffered output from a child process.[/color]
>
> Your Perl program works the same for me, on MacOS X,
> as your Python program. That's what we would expect,
> of course, because the problem is with the (Python)
> program on the other end - it's buffering output,
> because the output device is not a terminal.
>
> Donn Cave, donn@u.washingt on.edu[/color]

Yes Donn's right , works the same for me , bufcallee.py may be should
look like
this

import time
import sys

sysout=sys.stdo ut

sysout.write("S TART\n")
sysout.flush()
time.sleep(10)
sysout.write("S TOP\n")
sysout.flush()

regards
jitu