Receive data from socket stream

[Nick Craig-Wood]

Re: Receive data from socket stream

Hrvoje Niksic <hniksic@xemacs .orgwrote:

Nick Craig-Wood <nick@craig-wood.comwrites:
>

Note that appending to a string is almost never a good idea, since it
can result in quadratic allocation.

My aim was clear exposition rather than the ultimate performance!

>
That would normally be fine. My post wasn't supposed to pick
performance nits, but to point out potentially quadratic behavior.
>

Anyway str += was optimised in python 2.4 or 2.5 (forget which) wasn't
it?

>
That optimization works only in certain cases, when working with
uninterned strings with a reference count of 1, and then only when the
strings are in stored local variables, rather than in global vars or
in slots. And then, it only works in CPython, not in other
implementations . The optimization works by "cheating" -- breaking the
immutable string abstraction in the specific cases in which it is
provably safe to do so.

http://utcc.utoronto.ca/~cks/space/blog/python/ExaminingStringConcatOpt

examines it in some detail.

Ah, I didn't realise that - thanks for the interesting link.

For the example I gave, just a simple local variable the optimisation
kicks in. I can see how you could easily migrate that to an instance
variable and the optimisation would no longer work, eg

$ python -m timeit -s 's=""' 'for i in xrange(10000): s+="x"'
1000 loops, best of 3: 1.04 msec per loop

$ python -m timeit -s 'class A: pass' -s 'a=A(); a.s=""' 'for i in xrange(10000): a.s+="x"'
10 loops, best of 3: 160 msec per loop

Guido was reluctant to accept the patch that implements the
optimization because he thought it would "change the way people write
code", a sentiment expressed in

http://mail.python.org/pipermail/python-dev/2004-August/046702.html

This discussion shows that he was quite right in retrospect. (I'm not
saying that the optimization is a bad thing, just that it is changing
the "recommende d" way of writing Python in a way that other
implementations cannot follow.)

Certainly something I wasn't aware of before - thanks!

--
Nick Craig-Wood <nick@craig-wood.com-- http://www.craig-wood.com/nick

[Nick Craig-Wood]

Re: Receive data from socket stream

s0suk3@gmail.co m <s0suk3@gmail.c omwrote:

But as I said in my first post, it's simple when you know the amount
of data that you're going to receive, or when you'll receive data
until the remote peer closes the connection. But what about receiving
a message with undetermined length in a connection that you don't want
to close?

You obviously need some sort of protocol. Here is some code (taken
from a real project and modified a bit) which returns \r\n seperated
lines from a socket as they arrive which is a very simple (but
widespread) protocol.

self.rx_buf is set to "" in the initialisation
self.sock is the socket

def rx_line(self):
message = None
while 1:
pos = self.rx_buf.fin d("\r\n")
if pos >= 0:
message = self.rx_buf[:pos]
self.rx_buf = self.rx_buf[pos+2:]
break
try:
rx = self.sock.recv( 4096)
except socket.error, e:
self.sock = None
raise ServerNetworkEx ception(e)
if len(rx) == 0:
self.sock = None
raise ServerDisconnec tedException()
self.rx_buf += rx
return message

Sorry I mis-understood your original post!

--
Nick Craig-Wood <nick@craig-wood.com-- http://www.craig-wood.com/nick