concatenate file-like objects -> file-like object

[kgk]

I would like to concatenate several file-like objects
to create a single file-like object. I've looked at fileinput,
however
this returns a fileinput object that is not very file-like.

something like
# a has 50 bytes, and b has 100 bytes
f = FileList (open('a'), open('b'))
f.read (100) # read 50 bytes from a and 50 from b

My interest is in passing several files to an incremental parser
as if they came from a single file. I would rather not load them
in memory using StringIO and the parser reads only from file-like
objects.

Any pointers appreciated
Kris

[Marc 'BlackJack' Rintsch]

Re: concatenate file-like objects -> file-like object

On Tue, 10 Jul 2007 17:55:52 -0700, kgk wrote:

I would like to concatenate several file-like objects
to create a single file-like object. I've looked at fileinput,
however
this returns a fileinput object that is not very file-like.
>
something like
# a has 50 bytes, and b has 100 bytes
f = FileList (open('a'), open('b'))
f.read (100) # read 50 bytes from a and 50 from b
>
My interest is in passing several files to an incremental parser
as if they came from a single file. I would rather not load them
in memory using StringIO and the parser reads only from file-like
objects.

Then program a file like object yourself. Something like this (untestet):

class FileList(object ):
def __init__(self, files):
self.files = reversed(files)
self.current_fi le = self.files.pop( )

def read(size):
result = ''
while self.files:
data = self.current_fi le.read(size)
result += data
if len(data) != size:
self.current_fi le = self.files.pop( )
size = size - len(data)
return result

Ciao,
Marc 'BlackJack' Rintsch