struct.Struct random access

[castironpi]

I'd like to seriously nominate this idea and get a considered opinion
on it.

struct.Struct lets you encode Python objects into structured memory.
It accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure. What do you think of random access
to the results? To avoid Marc's concern about meaningless anonymous
record entries, model the constructor after 'namedtuple' type.

(unproduced)

>>packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' )
>>packer.pack_i nto( buf, off, 10, 20, 30, 0.5, 'abc' )
>>packer.unpack _from( buf, off, 'i3' )

30

>>packer.unpack _from( buf, off )

( 10, 20, 30, 0.5, 'abc' )

>>packer.pack_i nto( buf, off, 'i1', 12 )

Even in sequential access speed benefits, by avoiding the construction
of n-1 objects.

PS: If you don't have time or don't want to consider it seriously, or
want to see more specifics, just say so.

[Tim Roberts]

Re: struct.Struct random access

castironpi <castironpi@gma il.comwrote:

>
>I'd like to seriously nominate this idea and get a considered opinion
>on it.
>
>struct.Struc t lets you encode Python objects into structured memory.
>It accepts a format string, and optionally a buffer and offset to/from
>which to read/write the structure. What do you think of random access
>to the results? To avoid Marc's concern about meaningless anonymous
>record entries, model the constructor after 'namedtuple' type.
>
>(unproduced)

>>>packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' )
>>>packer.pack_ into( buf, off, 10, 20, 30, 0.5, 'abc' )
>>>packer.unpac k_from( buf, off, 'i3' )

>30

>>>packer.unpac k_from( buf, off )

>( 10, 20, 30, 0.5, 'abc' )

>>>packer.pack_ into( buf, off, 'i1', 12 )

What data type would you expect "buf" to be? Your design requires it to be
both an input and an output parameter. Today's "struct" converts into and
out of a string, and strings are immutable. So, to continue with that
standard, you would have to pass a string into "pack_into" and have the
modified result returned as an output:

buf = packer.pack_int o( None, 0, 10, 20, 30, 0.5, 'abc' )
buf = packer.pack_int o( buf, 0, 'i1', 12 )

In the end, I'm not sure you are really saving anything.

>Even in sequential access speed benefits, by avoiding the construction
>of n-1 objects.

This is a fairly major change, because it requires a "struct.Str uct" object
to maintain state, something that the "struct" module currently does not
do.

In my personal opinion, this is a rather large and confusing change, for
very little benefit.
--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[castironpi]

Re: struct.Struct random access

On Aug 28, 1:59 am, Tim Roberts <t...@probo.com wrote:

castironpi <castiro...@gma il.comwrote:
>

I'd like to seriously nominate this idea and get a considered opinion
on it.

>

struct.Struct lets you encode Python objects into structured memory.
It accepts a format string, and optionally a buffer and offset to/from
which to read/write the structure.  What do you think of random access
to the results?  To avoid Marc's concern about meaningless anonymous
record entries, model the constructor after 'namedtuple' type.

>

(unproduced)

>>packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' )
>>packer.pack_i nto( buf, off, 10, 20, 30, 0.5, 'abc' )
>>packer.unpack _from( buf, off, 'i3' )

30

>>packer.unpack _from( buf, off )

( 10, 20, 30, 0.5, 'abc' )

>>packer.pack_i nto( buf, off, 'i1', 12 )

>
What data type would you expect "buf" to be?  Your design requires it to be
both an input and an output parameter.  Today's "struct" converts into and
out of a string, and strings are immutable.  So, to continue with that
standard, you would have to pass a string into "pack_into" and have the
modified result returned as an output:
>
    buf = packer.pack_int o( None, 0, 10, 20, 30, 0.5, 'abc' )
    buf = packer.pack_int o( buf, 0, 'i1', 12 )
>
In the end, I'm not sure you are really saving anything.
>

Even in sequential access speed benefits, by avoiding the construction
of n-1 objects.

>
This is a fairly major change, because it requires a "struct.Str uct" object
to maintain state, something that the "struct" module currently does not
do.
>
In my personal opinion, this is a rather large and confusing change, for
very little benefit.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.

CMIIW correct me if I'm wrong, I don't think that pack_into returns a
value the way that pack does.

The syntax is ambiguous in the case of two-element structs: are you
packing a new struct or assigning a field? Maybe the field name needs
a keyword. A field name can be a third argument in unpack_from
regardless, however. Or use the field name as keyword:
strt.pack_into( buf, off, i1= 32 ).

I just tried an experiment with an Mmap and PyObject_AsWrit eBuffer.
Mmaps could work but you would not be able to use arbitrary strings.
You would have to specially allocate a ctypes buffer with a size to
match the packed size of the structure.

[Tim Roberts]

Re: struct.Struct random access

castironpi <castironpi@gma il.comwrote:

>
>CMIIW correct me if I'm wrong, I don't think that pack_into returns a
>value the way that pack does.

Sorry, I was not aware that struct.pack_int o and struct.unpack_f rom already
existed (they were added in Python 2.5). I thought you were proposing them
as new additions.

Because of that, the rest of my post doesn't make much sense.
--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[castironpi]

Re: struct.Struct random access

On Aug 29, 10:30 pm, Tim Roberts <t...@probo.com wrote:

castironpi <castiro...@gma il.comwrote:
>

CMIIW correct me if I'm wrong, I don't think that pack_into returns a
value the way that pack does.

>
Sorry, I was not aware that struct.pack_int o and struct.unpack_f rom already
existed (they were added in Python 2.5).  I thought you were proposing them
as new additions.
>
Because of that, the rest of my post doesn't make much sense.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.

I was a lot less enthusiastic too when I discovered
'PyObject_AsWri teBuffer'. Nevertheless I think it could be
instructive as well as a benefit. The ctypes conversion,
cast( buffer_p + offset ), is a bit of a kludge. With the
introduction of 'namedtuple' type, I think it falls right in line with
the direction Python is going in.

It might suffice to merely expose the 'offset' member of the items in
the array of 'formatcode'. Though, the addition of the dictionary to
lookup offsets by field name could offset the benefit somewhat.