Struct class random access

[castironpi]

struct.Struct lets you encode Python objects into structured memory.
It accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure. What do you think of random access
for the results?

(unproduced)

>>packer= struct.Struct( 'IIIf255p' )
>>packer.pack_i nto( buf, off, 10, 20, 30, 0.5, 'abc' )
>>packer.unpack _from( buf, off, 2 ) #reads field 2

30

Does this take a PEP, or just a patch submission?

[Marc 'BlackJack' Rintsch]

Re: Struct class random access

On Mon, 25 Aug 2008 13:03:09 -0700, castironpi wrote:

struct.Struct lets you encode Python objects into structured memory. It
accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure. What do you think of random access
for the results?
>
(unproduced)

>>>packer= struct.Struct( 'IIIf255p' )
>>>packer.pack_ into( buf, off, 10, 20, 30, 0.5, 'abc' )
>>>packer.unpac k_from( buf, off, 2 ) #reads field 2

30

I don't like it for the same reason I don't like index access on tuples
or lists that represent a "record" -- the numbers are quite meaningless.
Names for the components result in much easier to understand source code,
so I would prefer to use `ctypes` or `construct` to create such a record.

Ciao,
Marc 'BlackJack' Rintsch

[castironpi]

Re: Struct class random access

On Aug 25, 4:25 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:

On Mon, 25 Aug 2008 13:03:09 -0700, castironpi wrote:

struct.Struct lets you encode Python objects into structured memory. It
accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure.  What do you think of random access
for the results?

>

(unproduced)

>>packer= struct.Struct( 'IIIf255p' )
>>packer.pack_i nto( buf, off, 10, 20, 30, 0.5, 'abc' )
>>packer.unpack _from( buf, off, 2 ) #reads field 2

30

>
I don't like it for the same reason I don't like index access on tuples
or lists that represent a "record" -- the numbers are quite meaningless.  
Names for the components result in much easier to understand source code,
so I would prefer to use `ctypes` or `construct` to create such a record.
>
Ciao,
        Marc 'BlackJack' Rintsch

I'm interested in the speed benefit, so you don't have to reconstruct
the entire 'record' just to read/write one 'field'. How in ctypes?

[castironpi]

Re: Struct class random access

On Aug 25, 4:49 pm, castironpi <castiro...@gma il.comwrote:

On Aug 25, 4:25 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:
>
>
>

On Mon, 25 Aug 2008 13:03:09 -0700, castironpi wrote:

struct.Struct lets you encode Python objects into structured memory. It
accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure.  What do you think of random access
for the results?

>

(unproduced)
>>>packer= struct.Struct( 'IIIf255p' )
>>>packer.pack_ into( buf, off, 10, 20, 30, 0.5, 'abc' )
>>>packer.unpac k_from( buf, off, 2 ) #reads field 2
30

>

I don't like it for the same reason I don't like index access on tuples
or lists that represent a "record" -- the numbers are quite meaningless..  
Names for the components result in much easier to understand source code,
so I would prefer to use `ctypes` or `construct` to create such a record.

>

Ciao,
        Marc 'BlackJack' Rintsch

>
I'm interested in the speed benefit, so you don't have to reconstruct
the entire 'record' just to read/write one 'field'.  How in ctypes?

Model the constructor after 'namedtuple' type.

(unproduced)

>>packer= struct.Struct( 'IIIf255p', 'i1', 'i2', 'i3', 'afloat', 'name')
>>packer.pack_i nto( buf, off, 10, 20, 30, 0.5, 'abc' )
>>packer.unpack _from( buf, off, 'i3' )

30

>>packer.unpack _from( buf, off )

( 10, 20, 30, 0.5, 'abc' )

You still get marginal speed benefit in sequential access. You avoid
the construction of n-1 objects.

[Marc 'BlackJack' Rintsch]

Re: Struct class random access

On Mon, 25 Aug 2008 14:49:14 -0700, castironpi wrote:

I'm interested in the speed benefit, so you don't have to reconstruct
the entire 'record' just to read/write one 'field'. How in ctypes?

Only the field accessed is converted.

Ciao,
Marc 'BlackJack' Rintsch

[castironpi]

Re: Struct class random access

On Aug 25, 11:47 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:

On Mon, 25 Aug 2008 14:49:14 -0700, castironpi wrote:

I'm interested in the speed benefit, so you don't have to reconstruct
the entire 'record' just to read/write one 'field'.  How in ctypes?

>
Only the field accessed is converted.
>
Ciao,
        Marc 'BlackJack' Rintsch

I know that. I was asking how to write 'unpack_from( buf, off, 2 )',
when buf is a non-ctypes buffer.

[castironpi]

Re: Struct class random access

On Aug 26, 12:41 am, castironpi <castiro...@gma il.comwrote:

On Aug 25, 11:47 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:
>

On Mon, 25 Aug 2008 14:49:14 -0700, castironpi wrote:

I'm interested in the speed benefit, so you don't have to reconstruct
the entire 'record' just to read/write one 'field'.  How in ctypes?

>

Only the field accessed is converted.

>

Ciao,
        Marc 'BlackJack' Rintsch

>
I know that.  I was asking how to write 'unpack_from( buf, off, 2 )',
when buf is a non-ctypes buffer.

The code with ctypes is more elegant than I thought.

from ctypes import *
prototype= PYFUNCTYPE( c_int, py_object, POINTER(c_void_ p),
POINTER(c_uint) )
PyObject_AsWrit eBuffer= prototype( ( "PyObject_AsWri teBuffer",
pythonapi ) )
def refas( buf, offset, tp ):
''' return an instance of |tp| that refers to |offset| bytes into
buffer |buf| '''
_b, _s= c_void_p(0), c_uint(0)
PyObject_AsWrit eBuffer( buf, byref(_b), byref(_s) ) #should return
0
c= cast( _b.value+ offset, POINTER( tp ) )
return c.contents

'tp' can be any class that is derived from ctypes.Structur e. 'buf'
can be any object that supports the buffer protocol, including
'mmap'. Remember when mapping pointers to store offsets, not memory
addresses.

I'd like to know how supported this is considered to be, across
platforms and versions. Can I rely on this in the future?