Write bits in file

[Monica Leko]

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

[Ken Starks]

Re: Write bits in file

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you? is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file, or is the whole file regularly subdivided
at 10-bit intervals?


Monica Leko wrote:

Hi
>
I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

[Monica Leko]

Re: Write bits in file

On May 18, 2:20 pm, Ken Starks <stra...@lampsa cos.demon.co.uk wrote:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

Yes.

is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

[Gabriel Genellina]

Re: Write bits in file

En Sun, 18 May 2008 10:36:28 -0300, Monica Leko <monica.leko@gm ail.comescribió :

On May 18, 2:20 pm, Ken Starks <stra...@lampsa cos.demon.co.uk wrote:

>You want your file considered as a sequence of bits rather
>than a sequence of 8-bit bytes, do you?

>
Yes.
>

>is the 10-bit
>bit-pattern to be stored at an arbitrary bit-position in
>the file

>
Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

If you really need arbitrary bit sequences that aren't synchonized into bytes, I think there is a BitVector o BitArray package somewhere.
But if you mostly have bytes and sparsely a different size, I think the struct module and some gymnastics involving bitwise operations would be enough.

--
Gabriel Genellina

[Ken Starks]

Re: Write bits in file

I admit that I was mostly just interested in getting your
question clarified, rather than having any great experise.

But a bit of Googling took me to the 'Bit vector' module,
[I googled: 'python ("bit array" OR "bit vector")']
which might be what you are after. I have no experience
with it, myself:

http://cobweb.ecn.purdue.edu/~kak/dist/BitVector-1.4.1.html

Monica Leko wrote:

On May 18, 2:20 pm, Ken Starks <stra...@lampsa cos.demon.co.uk wrote:

>You want your file considered as a sequence of bits rather
>than a sequence of 8-bit bytes, do you?

>
Yes.
>

>is the 10-bit
>bit-pattern to be stored at an arbitrary bit-position in
>the file

>
Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.
>
>

[John Nagle]

Re: Write bits in file

Monica Leko wrote:

Hi
>
I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

The "struct" module will let you format Python data as a binary
object of specified format. But it doesn't support arbitrary bit-width
fields; integers can be 1, 2, 4, or 8 bytes.

If you have a space problem, you might use the "struct" module to
convert to a reasonably concise binary representation, then use the
"gzip" module to compress the file down further.

John Nagle

[Tim Roberts]

Re: Write bits in file

Monica Leko <monica.leko@gm ail.comwrote:

>
>I have a specific format and I need binary representation. Does
>Python have some built-in function which will, for instance, represent
>number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.
--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[pataphor]

Re: Write bits in file

On Sun, 18 May 2008 06:36:28 -0700 (PDT)
Monica Leko <monica.leko@gm ail.comwrote:

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

Here's something to get you started. No guarantees, but I managed to
write four 10 bit numbers to a file producing 5 bytes, saying hello. I
was just looking for an excuse to use send.

P.

def gen(f):
L = []

def flush(L):
L.reverse()
s = ''.join(map(str ,L))
j = int(s,2)
f.write(chr(j))

while 1:
x = yield
if x in [0,1]:
L.append(x)
else:
break
if len(L) == 8:
flush(L)
L = []
if L:
while len(L) < 8:
L.append(0)
flush(L)
yield

def tenbits(i):
for j in range(9,-1,-1):
yield i >j & 1

def charbits(s):
for c in s:
i = ord(c)
for j in range(8):
yield i >j &1

def byten(L):
while L:
yield L[:10]
L = L[10:]

def test():
f = file('out.dat', 'w')
g = gen(f)
g.send(None)
for x in [90,611,397,758]:
for bit in tenbits(x):
g.send(bit)
g.send('stop')
f.close()

def test1():
bits = list(charbits(' hello'))
L = []
for x in byten(bits):
L.append(int('' .join(map(str,x )),2))
print L

if __name__=='__ma in__':
test()

[Nick Craig-Wood]

Re: Write bits in file

Monica Leko <monica.leko@gm ail.comwrote:

On May 18, 2:20?pm, Ken Starks <stra...@lampsa cos.demon.co.uk wrote:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

>
Yes.
>

is the 10-bit bit-pattern to be stored at an arbitrary
bit-position in the file

>
Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

You could try

http://construct.wikispaces.com/

which could well do exactly what you want.

--
Nick Craig-Wood <nick@craig-wood.com-- http://www.craig-wood.com/nick

[castironpi]

Re: Write bits in file

On May 20, 10:30 am, Nick Craig-Wood <n...@craig-wood.comwrote:

Monica Leko <monica.l...@gm ail.comwrote:

 On May 18, 2:20?pm, Ken Starks <stra...@lampsa cos.demon.co.uk wrote:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

>

 Yes.

>

is the 10-bit bit-pattern to be stored at an arbitrary
bit-position in the file

>

 Yes.  I need arbitrary, 8bits, than 10 bits for something else, than
 sequence of bytes, than 10 bits again, etc.

>
You could try
>
 http://construct.wikispaces.com/
>
which could well do exactly what you want.
>
--
Nick Craig-Wood <n...@craig-wood.com--http://www.craig-wood.com/nick

Do you have space to waste? If so, how much? Worst case, you could
be looking at a factor of 32 or higher, if each bit took an entire
system word.

[sjdevnull@yahoo.com]

Re: Write bits in file

On May 20, 12:14 am, Tim Roberts <t...@probo.com wrote:

Monica Leko <monica.l...@gm ail.comwrote:
>

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

>
For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Technically specifying 8-bits isn't quite accurate, as C allows for 9-
bit bytes and other variations depending on the architecture. But
that may be overly pedantic unless you have a PDP-10 laying around
that you're writing C code on or something like that.

[Tim Roberts]

Re: Write bits in file

"sjdevnull@yaho o.com" <sjdevnull@yaho o.comwrote:

>On May 20, 12:14 am, Tim Roberts <t...@probo.com wrote:

>Monica Leko <monica.l...@gm ail.comwrote:
>>

>I have a specific format and I need binary representation. Does
>Python have some built-in function which will, for instance, represent
>number 15 in exactly 10 bits?

>>
>For the record, I'd like to point out that even C cannot do this. You need
>to use shifting and masking to produce a stream of 8-bit bytes, or to
>extract your values from a stream of 8-bit bytes.

>
>Technically specifying 8-bits isn't quite accurate, as C allows for 9-
>bit bytes and other variations depending on the architecture. But
>that may be overly pedantic unless you have a PDP-10 laying around
>that you're writing C code on or something like that.

As long as we are being pedantic, and I don't mind that, I would point out
that I didn't actually say that C worked in 8-bit bytes. I was very
careful to say merely that, assuming you wanted a stream of 8-bit bytes,
you need to use shifting and masking to produce it.
--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[Andrew Lee]

Re: Write bits in file

Tim Roberts wrote:

Monica Leko <monica.leko@gm ail.comwrote:

>I have a specific format and I need binary representation. Does
>Python have some built-in function which will, for instance, represent
>number 15 in exactly 10 bits?

>
For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Hmmmm, bitfields are exactly non-aligned bits in a structure and are
part of ANSI C. Although C gives no guarantee of the ordering of fields
within machine words ... so bitfieds are of limited use in portable
programs unless they are 0-initialized.

IIRC, Huffman code uses arbitrary length bit strings and is the basis of
many compression algorithms.