binary input and memory address passing

[Eric Carlson]

Hello,

I can open, read, and convert data to a numeric (double) array from a
binary file using

nc = #something given
nr = #something given
f_o=open('junk. bin','rb')
x=reshape(array ('d',f_o.read() ),(nr,nc))

Is there a way in python that gives better performance? These commands
take three to 4 times longer than loading the data into an array in (for
example) octave.

Also, is there a way to pass the address of a block of memory, and then
access the data located there (pass by reference and let the python
program know what data type to use)?

My situation: I have two programs that can not communicate directly, but
I can get memory addresses for any data in my programs. I currently save
to a file, then load the data from file into my other program. Clearly
it would be much faster if I could access the block of memory directly
rather than making a copy.

Any suggestions greatly appreciated.

Cheers,

Eric Carlson

[Scott David Daniels]

Re: binary input and memory address passing

Eric Carlson wrote:

Hello,
>
I can open, read, and convert data to a numeric (double) array from a
binary file using
>
nc = #something given
nr = #something given
f_o=open('junk. bin','rb')
x=reshape(array ('d',f_o.read() ),(nr,nc))
>
Is there a way in python that gives better performance? These commands
take three to 4 times longer than loading the data into an array in (for
example) octave.
>
Also, is there a way to pass the address of a block of memory, and then
access the data located there (pass by reference and let the python
program know what data type to use)?
>
My situation: I have two programs that can not communicate directly, but
I can get memory addresses for any data in my programs. I currently save
to a file, then load the data from file into my other program. Clearly
it would be much faster if I could access the block of memory directly
rather than making a copy.
>
Any suggestions greatly appreciated.

You might look over my (admittedly old)

http://members.dsl-only.net/~daniels/Block.html

It only has a Python 2.2 and 2.3 installer, but the sources are there
for you to compile (in which case you might toss a package back to me,
and I'll put it up). If you look it over and have questions, let me
know.

--Scott David Daniels
scott.daniels@a cm.org

[Eric Carlson]

Re: binary input and memory address passing


Thanks Dennis and Scott for both responses. Since Dennis has slam dunked
my notion of sharing memory addresses (my ignorance in computing is
pretty much unbounded), I guess I will need to continue on with sharing
through files.

Opening up and loading the binary info into a string variable is very
fast. The process bogs when changing the representation to float. So it
appears that Scott's Block module might do what I need.

My original:

nc = #something given
nr = #something given
f_o=open('junk. bin','rb')
x=reshape(array ('d',f_o.read() ),(nr,nc))

gives me any array with nr rows and nc columns. Using Block I think I
would use:

f_o=open('junk. bin','rb')
x = View('d', Block(f_o.read( )))

Okay, so now how can I use this? Is this like an array type so that

x = reshape(x,(nr,n c))

makes sense?

Regards,
Eric