output to console and to multiple files

[nathan.shair@gmail.com]

Hello,

I searched on Google and in this Google Group, but did not find any
solution to my problem.

I'm looking for a way to output stdout/stderr (from a subprocess or
spawn) to screen and to at least two different files.

eg.
stdout/stderr -screen
stdout -log.out
stderr -log.err

and if possible
stdout/stderr -screen and log.txt

3 files from stdout/stderr

[Matimus]

Re: output to console and to multiple files

I took a look around and I couldn't find anything either. I will be
keeping an eye on this thread to see if someone posts a more standard
solution. In the mean time though, I will offer up a potential
solution. Duck typing is your friend. If you are only using the write
method of your files, it can be pretty simple to implement a fake file
object to do what you want.

import sys

class TeeFile(object):
def __init__(self,*files):
self.files = files
def write(self,txt):
for fp in self.files:
fp.write(txt)

if __name__ == "__main__":
outf = file("log.out","w")
errf = file("log.err","w")
allf = file("log.txt","w")
sys.stdout = TeeFile(sys.__stdout__,outf,allf)
sys.stderr = TeeFile(sys.__stderr__,errf,allf)

print "hello world this is stdout"
print >sys.stderr , "hello world this is stderr"

[goodwolf]

Re: output to console and to multiple files

like this?

class Writers (object):

def __init__(self, *writers):
self.writers = writers

def write(self, string):
for w in self.writers:
w.write(string)

def flush(self):
for w in self.writers:
w.flush():

import sys

logfile = open('log.txt', 'w')
sys.stdout = Writers(aya.std out, file('log.out', 'w'), logfile)
sys.stderr = Writers(aya.std out, file('log.err', 'w'), logfile)

[Gabriel Genellina]

Re: output to console and to multiple files

En Wed, 14 Feb 2007 19:28:34 -0300, nathan.shair@gm ail.com
<nathan.shair@g mail.comescribi ó:

I'm looking for a way to output stdout/stderr (from a subprocess or
spawn) to screen and to at least two different files.

Look at the tee command. If you control the subprocess, and it's written
in Python, using the Python recipes would be easier and perhaps you have
more control.
But if you can't modify the subprocess, you'll have to use tee.

--
Gabriel Genellina

[nathan.shair@gmail.com]

Re: output to console and to multiple files

On Feb 14, 5:10 pm, "goodwolf" <Robert.Ka...@g mail.comwrote:

like this?
>
class Writers (object):
>
def __init__(self, *writers):
self.writers = writers
>
def write(self, string):
for w in self.writers:
w.write(string)
>
def flush(self):
for w in self.writers:
w.flush():
>
import sys
>
logfile = open('log.txt', 'w')
sys.stdout = Writers(aya.std out, file('log.out', 'w'), logfile)
sys.stderr = Writers(aya.std out, file('log.err', 'w'), logfile)

i've tried simliar methods to this and to what Matimus wrote. I know
it works great when using print statements.
However, I'm looking to find something that will work with the output
from a subprocess, such as from spawn, os.system, os.popen, etc.

[nathan.shair@gmail.com]

Re: output to console and to multiple files

On Feb 14, 6:52 pm, "Gabriel Genellina" <gagsl...@yahoo .com.arwrote:

En Wed, 14 Feb 2007 19:28:34 -0300, nathan.sh...@gm ail.com
<nathan.sh...@g mail.comescribi ó:
>

I'm looking for a way to output stdout/stderr (from a subprocess or
spawn) to screen and to at least two different files.

>
Look at the tee command. If you control the subprocess, and it's written
in Python, using the Python recipes would be easier and perhaps you have
more control.
But if you can't modify the subprocess, you'll have to use tee.
>
--
Gabriel Genellina

Tee, the unix function? Or is there a tee that is python?

[Matimus]

Re: output to console and to multiple files

On Feb 15, 7:53 am, "nathan.sh...@g mail.com" <nathan.sh...@g mail.com>
wrote:

On Feb 14, 5:10 pm, "goodwolf" <Robert.Ka...@g mail.comwrote:
>
>
>

like this?

>

class Writers (object):

>

def __init__(self, *writers):
self.writers = writers

>

def write(self, string):
for w in self.writers:
w.write(string)

>

def flush(self):
for w in self.writers:
w.flush():

>

import sys

>

logfile = open('log.txt', 'w')
sys.stdout = Writers(aya.std out, file('log.out', 'w'), logfile)
sys.stderr = Writers(aya.std out, file('log.err', 'w'), logfile)

>
i've tried simliar methods to this and to what Matimus wrote. I know
it works great when using print statements.
However, I'm looking to find something that will work with the output
from a subprocess, such as from spawn, os.system, os.popen, etc.

I think you should be able to use my or goodwolf's solution with the
subprocess module. Something like this (untested):

class TeeFile(object):
def __init__(self,*files):
self.files = files
def write(self,txt):
for fp in self.files:
fp.write(txt)

if __name__ == "__main__":
import sys
from subprocess import Popen

command = "whatever you want to run"
outf = file("log.out","w")
errf = file("log.err","w")
allf = file("log.txt","w")
Popen(
command,
stdout = TeeFile(sys.__stdout__,outf,allf),
stderr = TeeFile(sys.__stderr__,errf,allf)
)

[Matimus]

Re: output to console and to multiple files

On Feb 15, 8:51 am, "Matimus" <mccre...@gmail .comwrote:

On Feb 15, 7:53 am, "nathan.sh...@g mail.com" <nathan.sh...@g mail.com>
wrote:
>
>
>

On Feb 14, 5:10 pm, "goodwolf" <Robert.Ka...@g mail.comwrote:

>

like this?

>

class Writers (object):

>

def __init__(self, *writers):
self.writers = writers

>

def write(self, string):
for w in self.writers:
w.write(string)

>

def flush(self):
for w in self.writers:
w.flush():

>

import sys

>

logfile = open('log.txt', 'w')
sys.stdout = Writers(aya.std out, file('log.out', 'w'), logfile)
sys.stderr = Writers(aya.std out, file('log.err', 'w'), logfile)

>

i've tried simliar methods to this and to what Matimus wrote. I know
it works great when using print statements.
However, I'm looking to find something that will work with the output
from a subprocess, such as from spawn, os.system, os.popen, etc.

>
I think you should be able to use my or goodwolf's solution with the
subprocess module. Something like this (untested):
>

class TeeFile(object):
    def __init__(self,*files):
        self.files = files
    def write(self,txt):
        for fp in self.files:
            fp.write(txt)
>
if __name__ == "__main__":
    import sys
    from subprocess import Popen
>
    command = "whatever you want to run"
    outf = file("log.out","w")
    errf = file("log.err","w")
    allf = file("log.txt","w")
    Popen(
        command,
        stdout = TeeFile(sys.__stdout__,outf,allf),
        stderr = TeeFile(sys.__stderr__,errf,allf)
    )

I tried this at lunch and it doesn't work. Some version of this method
may work, but Popen tries to call the 'fileno' method of the TeeFile
object (at least it did on my setup) and it isn't there. This is just
a preemptive warning before someone comes back to let me know my code
doesn't work.

[Gabriel Genellina]

Re: output to console and to multiple files

En Thu, 15 Feb 2007 19:35:10 -0300, Matimus <mccredie@gmail .comescribió:

>I think you should be able to use my or goodwolf's solution with the
>subprocess module. Something like this (untested):
>>
>[code]
>class TeeFile(object) :
> def __init__(self,* files):
> self.files = files
> def write(self,txt) :
> for fp in self.files:
> fp.write(txt)
>>

>
I tried this at lunch and it doesn't work. Some version of this method
may work, but Popen tries to call the 'fileno' method of the TeeFile
object (at least it did on my setup) and it isn't there. This is just
a preemptive warning before someone comes back to let me know my code
doesn't work.

I don't think any Python only solution could work. The pipe options
available for subprocess are those of the underlying OS, and the OS knows
nothing about Python file objects.

--
Gabriel Genellina

[nathan.shair@gmail.com]

Re: output to console and to multiple files

On Feb 15, 5:48 pm, "Gabriel Genellina" <gagsl...@yahoo .com.arwrote:

En Thu, 15 Feb 2007 19:35:10 -0300, Matimus <mccre...@gmail .comescribió:
>
>
>

I think you should be able to use my or goodwolf's solution with the
subprocess module. Something like this (untested):

>

[code]
class TeeFile(object) :
def __init__(self,* files):
self.files = files
def write(self,txt) :
for fp in self.files:
fp.write(txt)

>

I tried this at lunch and it doesn't work. Some version of this method
may work, but Popen tries to call the 'fileno' method of the TeeFile
object (at least it did on my setup) and it isn't there. This is just
a preemptive warning before someone comes back to let me know my code
doesn't work.

>
I don't think any Python only solution could work. The pipe options
available for subprocess are those of the underlying OS, and the OS knows
nothing about Python file objects.
>
--
Gabriel Genellina

I've tried the subprocess method before without any luck.


Thanks for all your suggestions. I guess it's time to rethink what I
want to do.

[Bart Ogryczak]

Re: output to console and to multiple files

On Feb 14, 11:28 pm, "nathan.sh...@g mail.com" <nathan.sh...@g mail.com>
wrote:

Hello,
>
I searched on Google and in this Google Group, but did not find any
solution to my problem.
>
I'm looking for a way to output stdout/stderr (from a subprocess or
spawn) to screen and to at least two different files.
>
eg.
stdout/stderr -screen
stdout -log.out
stderr -log.err
>
and if possible
stdout/stderr -screen and log.txt
>
3 files from stdout/stderr

I'd derive a class from file, overwrite it's write() method to send a
copy to the log, and then assign sys.stdout = newFile(sys.std out).
Same for stderr.

[Gabriel Genellina]

Re: output to console and to multiple files

En Fri, 16 Feb 2007 14:04:33 -0300, Bart Ogryczak <B.Ogryczak@gma il.com>
escribió:

On Feb 14, 11:28 pm, "nathan.sh...@g mail.com" <nathan.sh...@g mail.com>
wrote:

>I'm looking for a way to output stdout/stderr (from a subprocess or
>spawn) to screen and to at least two different files.

>
I'd derive a class from file, overwrite it's write() method to send a
copy to the log, and then assign sys.stdout = newFile(sys.std out).
Same for stderr.

That's ok inside the same process, but the OP needs to use it "from a
subprocess or spawn".
You have to use something like tee, working with real file handles.

--
Gabriel Genellina

[garrickp@gmail.com]

Re: output to console and to multiple files

On Feb 16, 3:28 pm, "Gabriel Genellina" <gagsl...@yahoo .com.arwrote:

>
That's ok inside the same process, but the OP needs to use it "from a
subprocess or spawn".
You have to use something like tee, working with real file handles.
>

I'm not particularly familiar with this, but it seems to me that if
you're trying to catch stdout/stderr from a program you can call with
(say) popen2, you could just read from the returned stdout/stderr
pipe, and then write to a series of file handles (including
sys.stdout).

Or am I missing something? =)

~G

[nathan.shair@gmail.com]

Re: output to console and to multiple files

On Feb 16, 4:07 pm, garri...@gmail. com wrote:

On Feb 16, 3:28 pm, "Gabriel Genellina" <gagsl...@yahoo .com.arwrote:
>
>
>

That's ok inside the same process, but the OP needs to use it "from a
subprocess or spawn".
You have to use something like tee, working with real file handles.

>
I'm not particularly familiar with this, but it seems to me that if
you're trying to catch stdout/stderr from a program you can call with
(say) popen2, you could just read from the returned stdout/stderr
pipe, and then write to a series of file handles (including
sys.stdout).
>
Or am I missing something? =)
>
~G

That works, but it isn't live streaming of stdout/stderr. Most of the
time, if you stream both, one could lock the process, or have the
stdout/stderr printed in the wrong order.

[Fuzzyman]

Re: output to console and to multiple files

On Feb 16, 11:37 pm, "nathan.sh...@g mail.com" <nathan.sh...@g mail.com>
wrote:

On Feb 16, 4:07 pm, garri...@gmail. com wrote:
>
>
>

On Feb 16, 3:28 pm, "Gabriel Genellina" <gagsl...@yahoo .com.arwrote:

>

That's ok inside the same process, but the OP needs to use it "from a
subprocess or spawn".
You have to use something like tee, working with real file handles.

>

I'm not particularly familiar with this, but it seems to me that if
you're trying to catch stdout/stderr from a program you can call with
(say) popen2, you could just read from the returned stdout/stderr
pipe, and then write to a series of file handles (including
sys.stdout).

>

Or am I missing something? =)

>

~G

>
That works, but it isn't live streaming of stdout/stderr. Most of the
time, if you stream both, one could lock the process, or have the
stdout/stderr printed in the wrong order.

Everytime I've looked to do something like this (non-blocking read on
the stdout of a subprocess) I've always come back to the conclusion
that threads and queues are the only reasonable way (particularly on
windows). There may be a better solution using select.

Fuzzyman

http://www.voidspace.org.uk/python/articles.shtml