struct.calcsize problem

[Chandu]

In using the following struct format I get the size as 593. The same C
struct is 590 if packed on byte boundary and 596 when using pragma
pack(4). I am using pack(4) and added 3 spares at the end to get by.
hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
64s L L B B B B B 64s 64s'

Any ideas on what I am doing wrong?

[Larry Bates]

Re: struct.calcsize problem

Chandu wrote:[color=blue]
> In using the following struct format I get the size as 593. The same C
> struct is 590 if packed on byte boundary and 596 when using pragma
> pack(4). I am using pack(4) and added 3 spares at the end to get by.
> hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
> 64s L L B B B B B 64s 64s'
>
> Any ideas on what I am doing wrong?
>[/color]

Maybe this will help:

import struct
hdrFormats = ['16s','32s', '32s','B','8s', 'H','8s','H','4 s','H','H',
'20s','64s','64 s','64s','32s', '32s','64s','L' ,'L','B',
'B','B','B','B' ,'64s','64s']

sumofcalcsize=0
i=0
for fmt in hdrFormats:
l=struct.calcsi ze(fmt)
sumofcalcsize+= l
print "fmt='%s', calcsize=%i, sumofcalcsize=% i, calcsize=%i" % \
(fmt, l, sumofcalcsize, struct.calcsize (' '.join(hdrForma ts[:i+1])))
i+=1


Outputs:

fmt='16s', calcsize=16, sumofcalcsize=1 6, calcsize=16
fmt='32s', calcsize=32, sumofcalcsize=4 8, calcsize=48
fmt='32s', calcsize=32, sumofcalcsize=8 0, calcsize=80
fmt='B', calcsize=1, sumofcalcsize=8 1, calcsize=81
fmt='8s', calcsize=8, sumofcalcsize=8 9, calcsize=89
fmt='H', calcsize=2, sumofcalcsize=9 1, calcsize=92 <====
fmt='8s', calcsize=8, sumofcalcsize=9 9, calcsize=100
fmt='H', calcsize=2, sumofcalcsize=1 01, calcsize=102
fmt='4s', calcsize=4, sumofcalcsize=1 05, calcsize=106
fmt='H', calcsize=2, sumofcalcsize=1 07, calcsize=108
fmt='H', calcsize=2, sumofcalcsize=1 09, calcsize=110
fmt='20s', calcsize=20, sumofcalcsize=1 29, calcsize=130
fmt='64s', calcsize=64, sumofcalcsize=1 93, calcsize=194
fmt='64s', calcsize=64, sumofcalcsize=2 57, calcsize=258
fmt='64s', calcsize=64, sumofcalcsize=3 21, calcsize=322
fmt='32s', calcsize=32, sumofcalcsize=3 53, calcsize=354
fmt='32s', calcsize=32, sumofcalcsize=3 85, calcsize=386
fmt='64s', calcsize=64, sumofcalcsize=4 49, calcsize=450
fmt='L', calcsize=4, sumofcalcsize=4 53, calcsize=456 <====
fmt='L', calcsize=4, sumofcalcsize=4 57, calcsize=460
fmt='B', calcsize=1, sumofcalcsize=4 58, calcsize=461
fmt='B', calcsize=1, sumofcalcsize=4 59, calcsize=462
fmt='B', calcsize=1, sumofcalcsize=4 60, calcsize=463
fmt='B', calcsize=1, sumofcalcsize=4 61, calcsize=464
fmt='B', calcsize=1, sumofcalcsize=4 62, calcsize=465
fmt='64s', calcsize=64, sumofcalcsize=5 26, calcsize=529
fmt='64s', calcsize=64, sumofcalcsize=5 90, calcsize=593

Larry Bates

[Bengt Richter]

Re: struct.calcsize problem

On 7 Nov 2005 15:27:06 -0800, "Chandu" <chandu@trillia n.us> wrote:
[color=blue]
>In using the following struct format I get the size as 593. The same C
>struct is 590 if packed on byte boundary and 596 when using pragma
>pack(4). I am using pack(4) and added 3 spares at the end to get by.
> hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
>64s L L B B B B B 64s 64s'
>
>Any ideas on what I am doing wrong?
>[/color]
Looks to me like you are getting default native byte order and _alignment_
and some pad bytes are getting added in. For native order with no padding,
try prefixing the format string with '='

[color=blue][color=green][color=darkred]
>>> hdrFormat[/color][/color][/color]
'16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'

If you add up the individual sizes, padding doesn't happen, apparently:
[color=blue][color=green][color=darkred]
>>> map(struct.calc size, hdrFormat.split ())[/color][/color][/color]
[16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64][color=blue][color=green][color=darkred]
>>> sum(map(struct. calcsize, hdrFormat.split ()))[/color][/color][/color]
590

But default:[color=blue][color=green][color=darkred]
>>> struct.calcsize (hdrFormat)[/color][/color][/color]
593
Apparently is native, with native alignment & I get the same as you:[color=blue][color=green][color=darkred]
>>> struct.calcsize ('@'+hdrFormat)[/color][/color][/color]
593
Whereas native order standard (no pad) alignment is:[color=blue][color=green][color=darkred]
>>> struct.calcsize ('='+hdrFormat)[/color][/color][/color]
590
Little endian, standard alignment:[color=blue][color=green][color=darkred]
>>> struct.calcsize ('<'+hdrFormat)[/color][/color][/color]
590
Big endian, standard alignment[color=blue][color=green][color=darkred]
>>> struct.calcsize ('>'+hdrFormat)[/color][/color][/color]
590
Network (big endian), standard alignment:[color=blue][color=green][color=darkred]
>>> struct.calcsize ('!'+hdrFormat)[/color][/color][/color]
590

I guess if you want alignment for anything non-native, you have to specify pad bytes
where you need them (with x format character).

Regards,
Bengt Richter

[Chandu]

Re: struct.calcsize problem

Thanks for the helpful feedback. I guessed it was the alignment issue,
but could not find the exact format for changing the default. It is not
mystery any more!
Bengt Richter wrote:[color=blue]
> On 7 Nov 2005 15:27:06 -0800, "Chandu" <chandu@trillia n.us> wrote:
>[color=green]
> >In using the following struct format I get the size as 593. The same C
> >struct is 590 if packed on byte boundary and 596 when using pragma
> >pack(4). I am using pack(4) and added 3 spares at the end to get by.
> > hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
> >64s L L B B B B B 64s 64s'
> >
> >Any ideas on what I am doing wrong?
> >[/color]
> Looks to me like you are getting default native byte order and _alignment_
> and some pad bytes are getting added in. For native order with no padding,
> try prefixing the format string with '='
>
>[color=green][color=darkred]
> >>> hdrFormat[/color][/color]
> '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'
>
> If you add up the individual sizes, padding doesn't happen, apparently:
>[color=green][color=darkred]
> >>> map(struct.calc size, hdrFormat.split ())[/color][/color]
> [16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64][color=green][color=darkred]
> >>> sum(map(struct. calcsize, hdrFormat.split ()))[/color][/color]
> 590
>
> But default:[color=green][color=darkred]
> >>> struct.calcsize (hdrFormat)[/color][/color]
> 593
> Apparently is native, with native alignment & I get the same as you:[color=green][color=darkred]
> >>> struct.calcsize ('@'+hdrFormat)[/color][/color]
> 593
> Whereas native order standard (no pad) alignment is:[color=green][color=darkred]
> >>> struct.calcsize ('='+hdrFormat)[/color][/color]
> 590
> Little endian, standard alignment:[color=green][color=darkred]
> >>> struct.calcsize ('<'+hdrFormat)[/color][/color]
> 590
> Big endian, standard alignment[color=green][color=darkred]
> >>> struct.calcsize ('>'+hdrFormat)[/color][/color]
> 590
> Network (big endian), standard alignment:[color=green][color=darkred]
> >>> struct.calcsize ('!'+hdrFormat)[/color][/color]
> 590
>
> I guess if you want alignment for anything non-native, you have to specify pad bytes
> where you need them (with x format character).
>
> Regards,
> Bengt Richter[/color]