Limitate speed of a socket-based data transferring

[billie]

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloade d.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write( buf)
file_obj.close( )
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

Best regards

[Diez B. Roggisch]

Re: Limitate speed of a socket-based data transferring

billie schrieb:

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:
>
file_obj = open('downloade d.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write( buf)
file_obj.close( )
sock.close()
>
I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.

But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.


Diez

[Tim Williams]

Re: Limitate speed of a socket-based data transferring

On 14/09/06, Diez B. Roggisch <deets@nospam.w eb.dewrote:

billie schrieb:

Hi all. I'm writing a TCP-based application that I will use to trasfer
binary files through the network. This piece of code represents how do
I get a file from a remote peer and save it on my local hard drive:

file_obj = open('downloade d.ext', 'wb')
while 1:
buf = sock.recv(2048)
if len(buf) == 0:
break
file_obj.write( buf)
file_obj.close( )
sock.close()

I would like to know how could be possible to limit the file transfer
speed (for example: don't write more than 50 Kb/sec).
Some ideas?

>
If you are on unix, use trickle. If you must limit it from within your
own code, I can only assume that computing the transfer rate so far and
introducing timeouts might help - but I never did such a thing, nor do I
know what that means for example for the network stack.
>
But maybe even then trickle may help you to get an idea, as it is a
user-space program AFAIK. So they might have some information (at least
the code... ) out there that could be of use.

You could wrap buf = sock.recv(xxx) in a data counter and sleep loop
so that you burst to no more than 50KB/s average.

Completely untestest and for illustration only :)

file_obj = open('downloade d.ext', 'wb')
interval = 1.0 # seconds eg. 0.5 or 2.0
# smaller the interval, the less bursty and smoother the throughput
max_speed = 51200 # 50k * 1024 = bytes
data_count = 0 # keep track of the amount of data transferred
time_next = time.time() + interval
while 1:
buf = sock.recv(512) # smaller chunks = smoother, more accurate
if len(buf) == 0:
break
data_count += len(buf)
if data_count >= max_speed * interval:
data_count = 0
sleep_for = time_next - time.time()
if sleep_for 0:
time.sleep(slee p_for)
time_next = time.time() + interval
file_obj.write( buf)
file_obj.close( )
sock.close()

[Grant Edwards]

Re: Limitate speed of a socket-based data transferring

On 2006-09-15, Dennis Lee Bieber <wlfraed@ix.net com.comwrote:

On 14 Sep 2006 04:54:48 -0700, "billie" <gnewsg@gmail.c omdeclaimed the
following in comp.lang.pytho n:
>

>Hi all. I'm writing a TCP-based application that I will use to trasfer
>binary files through the network. This piece of code represents how do
>I get a file from a remote peer and save it on my local hard drive:
>>

<code snipped>

>>
>I would like to know how could be possible to limit the file transfer
>speed (for example: don't write more than 50 Kb/sec).
>Some ideas?
>>

I'm not sure you /can/ limit the /receive/ speed.

Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.

The sender will send at whatever rate they are capable of, so
packets may just become backlogged on your receiving socket

When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.

waiting for you to read them if you add some sort of delay to
your reading loop.

--
Grant Edwards grante Yow! .. the HIGHWAY is
at made out of LIME JELLO and
visi.com my HONDA is a barbequed
OYSTER! Yum!

[Steve Holden]

Re: Limitate speed of a socket-based data transferring

Grant Edwards wrote:

On 2006-09-15, Dennis Lee Bieber <wlfraed@ix.net com.comwrote:
>

>>On 14 Sep 2006 04:54:48 -0700, "billie" <gnewsg@gmail.c omdeclaimed the
>>following in comp.lang.pytho n:
>>
>>

>>>Hi all. I'm writing a TCP-based application that I will use to trasfer
>>>binary files through the network. This piece of code represents how do
>>>I get a file from a remote peer and save it on my local hard drive:
>>>

>>
>> <code snipped>
>>

>>>I would like to know how could be possible to limit the file transfer
>>>speed (for example: don't write more than 50 Kb/sec).
>>>Some ideas?
>>>

>>
>> I'm not sure you /can/ limit the /receive/ speed.

>
>
Sure you can. Just read data from the socket at the max speed
you want to receive. The receive buffer for that socket will
fill up and the TCP window will start close up and throttle the
sender.
>
>

>>The sender will send at whatever rate they are capable of, so
>>packets may just become backlogged on your receiving socket

>
>
When that happens, the sending end of the socket will throttle
down to match the rate at which data is being read from the
socket.
>
>

>>waiting for you to read them if you add some sort of delay to
>>your reading loop.

>
>

Of course this depends crucially on the window size. Since the addition
of the window scaling TCP option it's been possible to specify very
large windows, which are useful over high-bandwidth high-delay links.

The remote (send) throttling will only start to cut in when the window
is full (since the whole point of the sliding window mechanism is to
allow continuous transmission in the face of acknowledgment delay).

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[Grant Edwards]

Re: Limitate speed of a socket-based data transferring

On 2006-09-15, Steve Holden <steve@holdenwe b.comwrote:

>>>The sender will send at whatever rate they are capable of, so
>>>packets may just become backlogged on your receiving socket

>>
>>
>When that happens, the sending end of the socket will throttle
>down to match the rate at which data is being read from the
>socket.

>
Of course this depends crucially on the window size. Since the
addition of the window scaling TCP option it's been possible
to specify very large windows, which are useful over
high-bandwidth high-delay links.

True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.

The remote (send) throttling will only start to cut in when
the window is full (since the whole point of the sliding
window mechanism is to allow continuous transmission in the
face of acknowledgment delay).

Yup.

--
Grant Edwards grante Yow! -- I have seen the
at FUN --
visi.com

[Tim Williams]

Re: Limitate speed of a socket-based data transferring

On 15/09/06, Grant Edwards <grante@visi.co mwrote:

On 2006-09-15, Steve Holden <steve@holdenwe b.comwrote:

Of course this depends crucially on the window size. Since the
addition of the window scaling TCP option it's been possible
to specify very large windows, which are useful over
high-bandwidth high-delay links.

>
True. If the window size is large compared to the amount of
data being transferred, then the throttling won't happen.
>

The remote (send) throttling will only start to cut in when
the window is full (since the whole point of the sliding
window mechanism is to allow continuous transmission in the
face of acknowledgment delay).

>
Yup.

If the OP is also writing the client, the throttling can be at the
client side to control the speed of writing to the socket. But if
multiple clients connect to the server concurrently then this may not
help either!

:)