Numeric type conversions

[John Dann]

I'm new to Python and can't readily find the appropriate function for
the following situation:

I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer.

I'm guess that there should be a function available whereby I can say
something like:

My2ByteInt = ConvertToInt(By teStream[12:13])

to take a random example of the 2 bytes occurring at positions 12 and
13 in the byte stream.

Can anyone point me in the right direction towards a suitable function
please?

NB I don't know without further checking exactly how the bytes are
encoded, but I'm just transcribing some code across from a Windows
VB.Net program and these same bytes could be read straight into a
standard 2-byte signed short int, so I can be confident that it's a
fairly standard Windows-compatible encoding.

[Diez B. Roggisch]

Re: Numeric type conversions

John Dann wrote:

I'm new to Python and can't readily find the appropriate function for
the following situation:
>
I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer.
>
I'm guess that there should be a function available whereby I can say
something like:
>
My2ByteInt = ConvertToInt(By teStream[12:13])
>
to take a random example of the 2 bytes occurring at positions 12 and
13 in the byte stream.
>
Can anyone point me in the right direction towards a suitable function
please?

see the module struct in the standard-lib.

Diez

[ershov.a_n@mail.ru]

Re: Numeric type conversions

try struct.pack

[A.T.Hofkamp]

Re: Numeric type conversions

On 2008-06-17, John Dann <news@prodata.c o.ukwrote:

I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer.

[snipperdesnip]

Can anyone point me in the right direction towards a suitable function
please?

The ctypes module should be helpful here

Sincerely,
Albert

[=?ISO-8859-1?Q?Gerhard_H=E4ring?=]

Re: Numeric type conversions

John Dann wrote:

I'm new to Python and can't readily find the appropriate function for
the following situation:
>
I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer. [...]

Use the unpack() function from the struct module.

-- Gerhard

[John Machin]

Re: Numeric type conversions

On Jun 17, 9:28 pm, John Dann <n...@prodata.c o.ukwrote:

I'm new to Python and can't readily find the appropriate function for
the following situation:
>
I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer.
>
I'm guess that there should be a function available whereby I can say
something like:
>
My2ByteInt = ConvertToInt(By teStream[12:13])
>
to take a random example of the 2 bytes occurring at positions 12 and
13 in the byte stream.
>
Can anyone point me in the right direction towards a suitable function
please?
>
NB I don't know without further checking exactly how the bytes are
encoded, but I'm just transcribing some code across from a Windows
VB.Net program and these same bytes could be read straight into a
standard 2-byte signed short int, so I can be confident that it's a
fairly standard Windows-compatible encoding.

You need the unpack function of the struct module. Supposing you have
a four-byte string containing two such short ints, first + 1 then -1
then this will work:

>>import struct
>>two_shorts = '\x01\x00\xff\x ff'
>>struct.unpack ('<hh', two_shorts)

(1, -1)

>>>

In the format string, '<' means little-endian (almost universal on PCs
running Windows), and 'h' means a signed 'half-word'. See the struct
manual section for more options.

You can write a function called convert_to_int (take a hint on naming
conventions) and use it a slice at a time, or you can use
struct.unpack to grab a whole record at once. Here's a real live
example of that:

(
f.height, option_flags, f.colour_index, f.weight,
f.escapement_ty pe, f.underline_typ e, f.family,
f.character_set ,
) = unpack('<HHHHHB BB', data[0:13])

HTH,
John

[MRAB]

Re: Numeric type conversions

On Jun 17, 12:28 pm, John Dann <n...@prodata.c o.ukwrote:

I'm new to Python and can't readily find the appropriate function for
the following situation:
>
I'm reading in a byte stream from a serial port (which I've got
working OK with pyserial) and which contains numeric data in a packed
binary format. Much of the data content occurs as integers encoded as
2 consecutive bytes, ie a 2-byte integer.
>
I'm guess that there should be a function available whereby I can say
something like:
>
My2ByteInt = ConvertToInt(By teStream[12:13])
>
to take a random example of the 2 bytes occurring at positions 12 and
13 in the byte stream.
>

[snip]
Please note that in slicing the start position is included and the end
position is excluded, so that should be ByteStream[12:14].

[John Dann]

Re: Numeric type conversions

On Tue, 17 Jun 2008 08:58:11 -0700 (PDT), MRAB
<google@mrabarn ett.plus.comwro te:

>[snip]
>Please note that in slicing the start position is included and the end
>position is excluded, so that should be ByteStream[12:14].

Yes, I just tripped over that, in fact, hence the error in my original
post. I suppose there must be some logic in including the start
position but excluding the end position, though it does escape me for
now. I can understand making a range inclusive or exclusive but not a
mixture of the two. Suppose it's just something you have to get used
to with Python and, no doubt, much commented on in the past.

JGD

[Peter Otten]

Re: Numeric type conversions

John Dann wrote:

On Tue, 17 Jun 2008 08:58:11 -0700 (PDT), MRAB
<google@mrabarn ett.plus.comwro te:
>

>>[snip]
>>Please note that in slicing the start position is included and the end
>>position is excluded, so that should be ByteStream[12:14].

>
Yes, I just tripped over that, in fact, hence the error in my original
post. I suppose there must be some logic in including the start
position but excluding the end position, though it does escape me for
now. I can understand making a range inclusive or exclusive but not a
mixture of the two.

http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

Peter

[Lie]

Re: Numeric type conversions

On Jun 18, 12:23 am, John Dann <n...@prodata.c o.ukwrote:

On Tue, 17 Jun 2008 08:58:11 -0700 (PDT), MRAB
>
<goo...@mrabarn ett.plus.comwro te:

[snip]
Please note that in slicing the start position is included and the end
position is excluded, so that should be ByteStream[12:14].

>
Yes, I just tripped over that, in fact, hence the error in my original
post. I suppose there must be some logic in including the start
position but excluding the end position, though it does escape me for
now. I can understand making a range inclusive or exclusive but not a
mixture of the two. Suppose it's just something you have to get used
to with Python and, no doubt, much commented on in the past.
>
JGD

There is actually a logic to that. It's explained in the help/tutorial
file, that you should think about the index as a cursor, the index
itself doesn't point to the item itself, but to the interval between
the item.

(read this on a monospace font, or it'll look screwed up)
0 1 2 3 4 5
+-------------------+
| A | B | C | D | E |
+-------------------+
-5 -4 -3 -2 -1 0

So to get BCD, you say [1:4]

[Hrvoje Niksic]

Re: Numeric type conversions

John Dann <news@prodata.c o.ukwrites:

I suppose there must be some logic in including the start position
but excluding the end position, though it does escape me for now. I
can understand making a range inclusive or exclusive but not a
mixture of the two. Suppose it's just something you have to get used
to with Python and, no doubt, much commented on in the past.

Half-open ranges are not specific to Python: for example, they're used
in Java (String.substri ng), Lisp (start and stop arguments to sequence
functions), and C++ (STL algorithms accept begin and end iterators
that generalize from pointers to the beginning and one-past-the-end of
an array).

Ranges so expressed have several nice properties. The size of the
range is expressed simply as b-a, you don't need to remember to add 1.
Empty range is easily expressed as [a, a>, rather than the much less
intuitive [a, a-1] for all-inclusive ranges. Consecutive ranges are
easily concatenated without missing or duplicating an element, so
[a, b+ [b, cis exactly equivalent to [a, c>.