flock seems very unsafe, python fcntl bug?

[xucs007@gmail.com]

I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .

Is this a bug of python fcntl module ? See 2 programs I ran:



#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(fil e, fcntl.LOCK_EX)
file.write('123 456')
time.sleep(10)
file.close()


#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(fil e, fcntl.LOCK_EX)
file.write('222 ')
time.sleep(10)
file.close()

[Jens Henrik Leonhard Jensen]

Re: flock seems very unsafe, python fcntl bug?

Your problem is that open(...,'w') is not locked.

Use something like:
lockf = open('aaa', 'a')
fnctl.flock(loc kf,fnctl.LOCK_E X)
file = open('aaa', 'w')
file.write('asd f')
file.close()
lockf.close()

xucs007@gmail.c om wrote:

I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .
>
Is this a bug of python fcntl module ? See 2 programs I ran:
>
>
>
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(fil e, fcntl.LOCK_EX)
file.write('123 456')
time.sleep(10)
file.close()
>
>
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(fil e, fcntl.LOCK_EX)
file.write('222 ')
time.sleep(10)
file.close()

[Nick Craig-Wood]

Re: flock seems very unsafe, python fcntl bug?

Jens Henrik Leonhard Jensen <jhlj@statsbibl ioteket.dkwrote :

Your problem is that open(...,'w') is not locked.
>
Use something like:
lockf = open('aaa', 'a')
fnctl.flock(loc kf,fnctl.LOCK_E X)
file = open('aaa', 'w')
file.write('asd f')
file.close()
lockf.close()

I've not seen that trick before - it is a good one. It wouldn't work
for mandatory locking though...

The basic problem is getting the file open for read/write without
destroying the contents before you have the lock. None of the
arguments to open() do the right thing....

Using an append open is a nice solution to that, apart from the fact
that you can only append data onto the file hence the other open.

Here are some other possible solutions and a test harness! The low
level open is probably the most unixy solution. If you could specify
os.O_RDWR|os.O_ CREAT to the python open() function then it would be
the best, but none of the documented open strings do that

$ strace python -c 'open("aaa", "r+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_LARGEF ILE) = 3
$ strace python -c 'open("aaa", "w+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT| O_TRUNC|O_LARGE FILE, 0666) = 3
$ strace python -c 'open("aaa", "wr+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT| O_TRUNC|O_LARGE FILE, 0666) = 3
$ strace python -c 'open("aaa", "a+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT| O_APPEND|O_LARG EFILE, 0666) = 3


------------------------------------------------------------
"""
Testing writing stuff to a file with locking
"""

import os
import sys
import fcntl

def method_0(data):
"""
Use an normal open

This doesn't work because it truncates the data before it has the
lock
"""
fd = open('aaa', 'w')
fcntl.flock(fd, fcntl.LOCK_EX)
fd.write(data)
fd.close()

def method_1(data):
"""
Use an additional append open to lock
"""
lock_fd = open('aaa', 'a')
fcntl.flock(loc k_fd,fcntl.LOCK _EX)
fd = open('aaa', 'w')
fd.write(data)
fd.close()
lock_fd.close()

def method_2(data):
"""
Use a low level open
"""
fd = os.fdopen(os.op en('aaa', os.O_CREAT|os.O _RDWR), "r+")
fcntl.flock(fd, fcntl.LOCK_EX)
fd.truncate()
fd.write(data)
fd.close()

def method_3(data):
"""
Using high level functions only

Possibly racy on open when file doesn't exist
"""
if not os.path.exists( 'aaa'):
fd = open('aaa', 'w+')
else:
fd = open('aaa', 'r+')
fcntl.flock(fd, fcntl.LOCK_EX)
fd.truncate()
fd.write(data)
fd.close()

def check():
fd = open('aaa', 'r')
fcntl.flock(fd, fcntl.LOCK_EX)
data = fd.read()
fd.close()
if data not in ("sausage", "potato"):
raise AssertionError( "Wrong data %r" % data)

if os.path.exists( "aaa"):
os.unlink("aaa" )

if len(sys.argv) < 2:
print "Syntax: %s <method number 0..3>" % sys.argv[0]
raise SystemExit(1)
method = globals()["method_"+sys.a rgv[1]]
if os.fork():
while 1:
method("sausage ")
check()
else:
while 1:
method("potato" )
check()
------------------------------------------------------------


--
Nick Craig-Wood <nick@craig-wood.com-- http://www.craig-wood.com/nick