Reading a binary file

[Sorin Marti]

Hi all,

I am quite new to python and very new to this list.

I've got following problem. I have a binary file which contains
information I should read. I can open the file with

f = open ('cpu1db2.dat', 'rb')

That's no problem. But now I need the hex values of the binary file.

Is there a possibility to show hex-values of the bytes of a file?


Thanks in advance
Sorin

[Peter Hansen]

Re: Reading a binary file

Sorin Marti wrote:[color=blue]
>
> I am quite new to python and very new to this list.
>
> I've got following problem. I have a binary file which contains
> information I should read. I can open the file with[/color]
[snip]

It would really be best if you could describe in more detail
what you are trying to do with this data. Bytes are bytes,
and things like hex and binary are just different _representation s_
of bytes, so whether you want binary, hex, decimal, or something
else depends entirely on the use to which you will put the info.

-Peter

[Peter Hansen]

Re: Reading a binary file

Sorin Marti wrote:[color=blue]
>
> Ok I'll try to give more details. I have a Siemens SPS. With an SPS you
> can controll machines such as pumps or motors or anything else. To
> controll you have to set Variables. If you want to see which state these
> variables have you can get a file via ftp where these values are stored.
> This is what I have done. Now I have a file (called cpu1db2.dat) and
> this file has a length of 16 bytes.
>
> Byte Number/Length Type Hex-Value
> ----------------------------------------------------------------
> Byte 1: Boolean: 01 (which is true, 00 would be false)
> Byte 2: Byte: 11 (This data type is called byte)
> Byte 3: Char: 50 (Which should be a "P")
> Byte 4,5: Word 00 00
> Byte 6,7: Integer 22 04
> Byte 8,9,10,11: DoubleWord D2 00 00 BB
> Byte 12,13,14,15,16: Real BB 42 C8 00 00[/color]

Excellent detail! (It's a pleasure to help someone who actually takes
the time to put together a question with this much care! Thank you. :-)
[color=blue]
> Then there is a function where you can call a value with a startbyte and
> an endbyte. You also have to specify the type. That means you can call
> getValue('REAL' ,12,16) and you should get back 100 because if you have
> the binary value of 'BB 42 C8 00 00' is 010000101100100 000000000000000 00
> , first digit is the Sign (which is + or - ), next 8 digits are the
> exponent, in this case 10000101 = 133dec. Now you take away 127 from 133
> then you get six, thats the exponent. The rest
> (11001000000000 0000000000) has a hex value of C80000 this is 13107200
> decimal. Now you have to multiply 13107200 with 2^6 and 2^-23 and you
> get (tataaaaaa!): 100!
>
> The different data types need different calculations, that's why I asked
> a few things about changing the representation because I only can do
> some things in binary mode or hex mode.[/color]

Okay, so clearly you understand about bytes and such.... you just need
help with the specific ways of doing such things with Python. (?)

Folks have already shown you how to do hex(abyte) if you have a single
byte out of the above string of 16 bytes... That will return a
representation starting with 0x, however, so maybe ("%02x" % byte)
is more what you would need. You can also extend that to ("%04x" % word)
or %08x for a long if you need.

More likely, the comments about using the struct module are right on
target. You could easily write a string that would convert the entire
16 byte package all at once, except for your proprietary (?) float
format, which you already have under control.

Check out struct, then if you still need help, we'll be down to
specifics.

-Peter

[Axel Bock]

Re: Reading a binary file

Am Thu, 26 Jun 2003 16:14:58 +0200 schrieb Sorin Marti:
[color=blue]
> This is what I have done. Now I have a file (called cpu1db2.dat) and
> this file has a length of 16 bytes.
>
> Byte Number/Length Type Hex-Value
> ----------------------------------------------------------------
> [... content description ...]
> So I have written a python class which makes a connection to the
> [... lots of strange calculation ...]
> 10000101 = 133dec. Now you take away 127 from 133 then you get six,
> thats the exponent. The rest (11001000000000 0000000000) has a hex value
> of C80000 this is 13107200 decimal. Now you have to multiply 13107200
> with 2^6 and 2^-23 and you get (tataaaaaa!): 100![/color]

whew. I don't get it, but anyways I think I can be useful ;-)

look at the struct-module: "This module performs conversions between
Python values and C structs represented as Python strings. It uses format
strings (explained below) as compact descriptions of the lay-out of the C
structs and the intended conversion to/from Python values. This can be
used in handling binary data stored in files or from network connections,
among other sources." (out of the python-doc).

learning by example:

to convert a 4 byte integer you'd write:
"struct.unpack( "=I",str_of_len _4)[0]"

= means native endian format - use the machine's endian format
I means unsigned int
str_of_len_4 has to be a binary string of length 4 containing the int
and the [0] at the end is neccessary cause unpack *always* returns a list,
even if only one value is converted (otherwise it'd be [2387], for example).
this is even stackable:
"struct.unpack( "=IIH", str_of_len_10)[0]"
converts two unsigned ints, one unsigned short. great, huh? :-)

Hope this is what you need. your explanations seemed rather complicated to
me ;-)


greetings,

axel.

[Peter Hansen]

Re: Reading a binary file

Sorin Marti wrote:[color=blue]
>
> Byte Number/Length Type Hex-Value
> ----------------------------------------------------------------
> Byte 12,13,14,15,16: Real BB 42 C8 00 00
>
> you can call
> getValue('REAL' ,12,16) and you should get back 100 because if you have
> the binary value of 'BB 42 C8 00 00' is 010000101100100 000000000000000 00
> , first digit is the Sign (which is + or - ), next 8 digits are the
> exponent, in this case 10000101 = 133dec. Now you take away 127 from 133
> then you get six, thats the exponent. The rest
> (11001000000000 0000000000) has a hex value of C80000 this is 13107200
> decimal. Now you have to multiply 13107200 with 2^6 and 2^-23 and you
> get (tataaaaaa!): 100![/color]

I think you might be interpreting (or explaining?) the format of that
real incorrectly.

If the first bit is the sign, and the next 8 bits are the
exponent, and the rest is mantissa, then your exponent should
be 01110110 (or h76 or d118) and your mantissa value in hex
would be all of the 42 C8 00 00, or 1120403456 in decimal.

(Basically, your binary value as shown is wrong. BB42C80000 is really
1010 1010 0100 0010 1100 1000 0000 0000 0000 0000 0000 and not your
value of 0100 0010 1100 1000 0000 0000 0000 0000 as shown above.)

-Peter

[davefancella]

look at the struct-module: "This module performs conversions between
Python values and C structs represented as Python strings. It uses format

to convert a 4 byte integer you'd write:
"struct.unpack( "=I",str_of_len _4)[0]"

= means native endian format - use the machine's endian format
I means unsigned int

This is a drive-by post. I registered and stuff just to say thank you. :) The python struct docs aren't the greatest for someone like me who's never tried to read a binary file and convert endianness on the fly, and your post directed me to exactly what I needed to read to do it. :)

So thank you.

[Axel.Bock]

cool :O)

Hi, thanks for the "thank you"! :)

I just had some bad news, and one minute later I found that "thank you" mail of you. (A little late, I admit, but it served its purpose! :))