xmlprclib/server not reusing connections

**Skip Montanaro** · Jul 18 '05, 08:42 AM

Re: xmlprclib/server not reusing connections

Roger> It appears that xmlrpclib and/or SimpleXMLRPCSer ver always use
Roger> new connections for each request. I have been trying to make
Roger> them reuse the existing connection but can't find where. Is
Roger> there any particular reason they go to such great lengths to call
Roger> shutdown/close/finish as well as making new connection objects in
Roger> xmlrpclib?

Connection reuse would be done at a lower level I think. Take a look at the
httplib module. Take a look at xmlrpclib's MultiCall class for another way
to improve performance in some situations.

Skip

**Roger Binns** · Jul 18 '05, 08:43 AM

Re: xmlprclib/server not reusing connections

> Connection reuse would be done at a lower level I think. Take a look at the[color=blue]
> httplib module. Take a look at xmlrpclib's MultiCall class for another way
> to improve performance in some situations.[/color]

multicall is of no use to me because my following requests depend on the results
of the preceding ones. SSL is really important since the traffic is going
over the open Internet. I needed to do some certificate validation which
the Python 2.3 library has no facilities for.

On the server side, things were hopeless as well. Since it would be
listening on the open Internet, and had to work on Windows as well as
Linux and Mac, the threading server initially seemed appealing, but
sadly it has no facilities for limiting the number of threads spawned.
It also makes a new thread per request, rather than the more normal
design of having a thread pool to deal with requests/connections.

xmlrpclib is coded to use the Python 1.5 compatible names (and lack of
HTTP/1.1 and persistent connections). SimpleXMLRPCReq uestHandler is
hard-coded to close connections after one request (see last line of
do_POST). The M2Crypto code that added SSL to both inherited those
implementations and design.

Basically the existing xmlrpclib and SimpleXMLRPCSer ver are hard coded
(and IMHO go out of their way) to do one request per connection, and
then shut things down. Throw in that I needed to do SSL on both ends, with
HTTP authentication, and some "light" firewalling in order to prevent
DOS attacks, I had to spend several days mashing the various classes
together with M2Crypto together.

You can see the results here:

http://cvs.sf.net/viewcvs.py/bitpim/bitpim/bitfling/xmlrpcstuff.py?view=markup

Roger

**Andrew Bennetts** · Jul 18 '05, 08:43 AM

Re: xmlprclib/server not reusing connections

On Mon, Feb 23, 2004 at 03:55:08PM -0800, Roger Binns wrote:
[...][color=blue]
>
> Basically the existing xmlrpclib and SimpleXMLRPCSer ver are hard coded
> (and IMHO go out of their way) to do one request per connection, and
> then shut things down. Throw in that I needed to do SSL on both ends, with
> HTTP authentication, and some "light" firewalling in order to prevent
> DOS attacks, I had to spend several days mashing the various classes
> together with M2Crypto together.[/color]

You might have better luck with Twisted <http://twistedmatrix.c om/>.

The XML-RPC server support in Twisted does support SSL, and HTTP/1.1
persistent connections, and it's pretty easy to use:

Page not found · Twisted Legacy Documentation Pages

http://twistedmatrix.com/documents/examples/xmlrpc.py

I don't have any experience with Twisted's XML-RPC client (although you can
see an example at
http://twistedmatrix.com/documents/e...lrpcclient.py), but glancing
at the code it doesn't support HTTP/1.1 persistent connections out of the
box... you could probably add that in relatively easily, though.
[color=blue]
> You can see the results here:
>
> http://cvs.sf.net/viewcvs.py/bitpim/...py?view=markup[/color]

Glancing at that, it looks like using Twisted to implement that would be a
lot easier, and shorter.

-Andrew.

**Chris Foote** · Jul 18 '05, 08:43 AM

Re: xmlprclib/server not reusing connections

On Mon, 23 Feb 2004, Roger Binns wrote:
[color=blue]
> multicall is of no use to me because my following requests depend on the results
> of the preceding ones. SSL is really important since the traffic is going
> over the open Internet. I needed to do some certificate validation which
> the Python 2.3 library has no facilities for.
>
> On the server side, things were hopeless as well. Since it would be
> listening on the open Internet, and had to work on Windows as well as
> Linux and Mac, the threading server initially seemed appealing, but
> sadly it has no facilities for limiting the number of threads spawned.
> It also makes a new thread per request, rather than the more normal
> design of having a thread pool to deal with requests/connections.
>
> xmlrpclib is coded to use the Python 1.5 compatible names (and lack of
> HTTP/1.1 and persistent connections). SimpleXMLRPCReq uestHandler is
> hard-coded to close connections after one request (see last line of
> do_POST). The M2Crypto code that added SSL to both inherited those
> implementations and design.
>
> Basically the existing xmlrpclib and SimpleXMLRPCSer ver are hard coded
> (and IMHO go out of their way) to do one request per connection, and
> then shut things down. Throw in that I needed to do SSL on both ends, with
> HTTP authentication, and some "light" firewalling in order to prevent
> DOS attacks, I had to spend several days mashing the various classes
> together with M2Crypto together.
>
> You can see the results here:
>
> http://cvs.sf.net/viewcvs.py/bitpim/...py?view=markup[/color]

Excellent - I've been wanting these features for some time ;-)

Chris Foote <chris@inetd.co m.au>
_ _ _
(_) | | | | Director - INETD PTY LTD
_ _ __ ___ | |_ __| | Level 2, 132 Franklin St
| | | '_ \ / _ \ | __| / _` | Adelaide SA 5000
| | | | | | | __/ | |_ | (_| | Web: http://www.inetd.com.au
|_| |_| |_| \___| \__| \__,_| Phone: (08) 8410 4566

**Roger Binns** · Jul 18 '05, 08:44 AM

Re: xmlprclib/server not reusing connections

> You might have better luck with Twisted http://twistedmatrix.com/.

I did look at twisted but really don't like it. In particular I really
don't like the event driven code, that has to add callbacks and
deal with partial state when called.

That is certainly the right way to do things if you want to avoid
using threads. It is however complicated and convoluted. See this
example:

Page not found · Twisted Legacy Documentation Pages

http://twistedmatrix.com/documents/howto/tutorial#auto20

The model I far prefer is to use multiple threads (available on all
major Python platforms) and use worker threads with work queues.
They turn out to be simpler since they don't have queue callbacks
or effectively be a glorified state machine scattered across
several functions.
[color=blue]
> Glancing at that, it looks like using Twisted to implement that would be a
> lot easier, and shorter.[/color]

Except it wouldn't unless twisted already had the necessary functionality
which it doesn't. I would have to go through a similar exercise with
twisted, which is far more complicated.

Roger

**Dave Brueck** · Jul 18 '05, 08:44 AM

Re: xmlprclib/server not reusing connections

Roger wrote:[color=blue][color=green]
> > You might have better luck with Twisted http://twistedmatrix.com/.[/color]
>
> I did look at twisted but really don't like it. In particular I really
> don't like the event driven code, that has to add callbacks and
> deal with partial state when called.
>
> That is certainly the right way to do things if you want to avoid
> using threads. It is however complicated and convoluted. See this
> example:
>
> http://twistedmatrix.com/documents/h...utorial#auto20
>
> The model I far prefer is to use multiple threads (available on all
> major Python platforms) and use worker threads with work queues.
> They turn out to be simpler since they don't have queue callbacks
> or effectively be a glorified state machine scattered across
> several functions.[/color]

Yes, but the main drawback is of course scalability. But, you can have the best
of both worlds with Stackless Python's tasklets. I few years ago I wrote an
event-based I/O framework for my company using normal Python and it's nice and
fast but programs built on it are ugly (logic split across many callback
functions, state is maintained in an awkward way as a result, etc). Over the
past week or so I've started writing a replacement framework using Stackless
and it's been *so* nice... the applications that use it _look_ like they are
threaded, but you get the performance and scalability of "traditiona l" asynch
I/O frameworks (the core of the framework still uses a loop to call
select.poll).

Here's one of my test cases, a server that sends back a dummy HTTP response:

def ConnHandler(soc k):
header = sock.Read(4096)
resp = 'HTTP/1.0 200 Ok\r\nContent-length: 5\r\nContent-type:
text/html\r\n\r\n123 45'
sock.Write(resp )
sock.Close()

engine = Engine()
s = ListenSocket('1 27.0.0.1', 7777, ConnHandler)
s.AttachToEngin e(engine)
engine.Run()

On a 900MHz P3 Linux box the above server easily handles 2000 requests per
second, which is nice. :)

Although obviously a very trivial example, it's a good enough proof-of-concept
to encourage me to continue. I haven't tested high levels of concurrent
connections yet - but a similar test that sent back larger responses didn't
have any problems serving well over 1000 simultaneous connections, so in the
worse case using Stackless won't have any performance drawbacks compared to
other approaches.

-Dave

**Fredrik Lundh** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

Roger Binns wrote:
[color=blue]
> Basically the existing xmlrpclib and SimpleXMLRPCSer ver are hard coded
> (and IMHO go out of their way) to do one request per connection, and
> then shut things down.[/color]

if you don't like the xmlrpclib default Transport, use your own. you probably
only have to override 'make_connectio n' and 'parse_response ' to get the be-
haviour you're looking for.

</F>

**Christian Tismer** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

Roger Binns wrote:
[color=blue][color=green]
>>You might have better luck with Twisted http://twistedmatrix.com/.[/color]
>
>
> I did look at twisted but really don't like it. In particular I really
> don't like the event driven code, that has to add callbacks and
> deal with partial state when called.
>
> That is certainly the right way to do things if you want to avoid
> using threads. It is however complicated and convoluted.[/color]

True and false at the same time.
While it is internally the right thing to do (better to say, it
is what actually happens, internally), it is not the best way
to write it.

This is why I developed Stackless Python.
It behaves in a similar way, acting like many callbacks,
but you +write* your code in the most natural way possible.
In a way, Stackless takes all the clumsy state keeping stuff
away from the programmer and frees his mind to write simple
top-down programs with no callbacks.
This approach has even been tested with Zope. You can write
a simple loop that sends Web pages to the user, without leaving
the loop. Internally, of course it is left, the whole program is
pickled, and restored on the next request. But you write a single
script, no methods which have to seek for state, do something, and
encode state, again.

I'm thinking of a talk about Stackless on EuroPython 2004:
"Stackless Python and the Death of the Reactive Pattern". :-)

Note that Christopher Armstrong is working on
a Stackless Reactor for Twisted!

I think, when that thing is ready, it will give both Twisted
and Stackless a boost of popularity and usability.

ciao - chris
--
Christian Tismer :^) <mailto:tismer@ stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

**Andrew Bennetts** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

On Mon, Feb 23, 2004 at 09:57:55PM -0800, Roger Binns wrote:[color=blue][color=green]
> > You might have better luck with Twisted http://twistedmatrix.com/.[/color]
>
> I did look at twisted but really don't like it. In particular I really
> don't like the event driven code, that has to add callbacks and
> deal with partial state when called.[/color]

I was initially skeptical too, but I found that the small cost of using
callbacks to be well worth the benefit of avoid threads most of the time.
Threads invite all sorts of difficult to reproduce and debug problems like
race conditions and deadlocks.

That said, event-driven programming doesn't suit every problem. Twisted
does try fairly hard to reduce the burden of asynchronous code, though:
Deferreds make chains of callbacks easy to handle, and the twisted.flow
package uses python 2.2's generators to allow synchronous-looking code to
yield to the event loop:

Page not found · Twisted Legacy Documentation Pages

http://twistedmatrix.com/documents/howto/flow

And of course Twisted's splitting of network code into reactor, factory,
protocol and transport objects is a really nice abstraction, but that would
probably translate just fine into blocking code too (except perhaps the
reactor would be largely irrelevant?).
[color=blue]
> That is certainly the right way to do things if you want to avoid
> using threads. It is however complicated and convoluted. See this
> example:
>
> http://twistedmatrix.com/documents/h...utorial#auto20[/color]

That example looks fine to me, but perhaps I've been using Twisted too long!
:)

I certainly have no problems reading that code and immediately understanding
what every part of it does.
[color=blue]
> The model I far prefer is to use multiple threads (available on all
> major Python platforms) and use worker threads with work queues.
> They turn out to be simpler since they don't have queue callbacks
> or effectively be a glorified state machine scattered across
> several functions.[/color]

That's fine. Twisted doesn't prevent you from using threads!
reactor.callInT hread(func, args...) will run a function inside a thread pool
(or you can just start and manage threads yourself), and
reactor.callFro mThread provides a safe way for a thread to run some code in
the main event loop. Twisted itself does this in e.g. the
twisted.enterpr ise.adbapi module, to provide an asynchronous interface
around a synchronous DB-API modules.
[color=blue][color=green]
> > Glancing at that, it looks like using Twisted to implement that would be a
> > lot easier, and shorter.[/color]
>
> Except it wouldn't unless twisted already had the necessary functionality
> which it doesn't. I would have to go through a similar exercise with
> twisted, which is far more complicated.[/color]

Well, Twisted already has persistent HTTP connections and SSL. So e.g. the
vast bulk of your XMLRPCRequestHa ndler.do_POST would disappear, because
twisted.protoco ls.http already does that.

Perhaps my point wasn't clear: I wasn't saying that Twisted will inherently
make any network code much much shorter[1], I was saying that Twisted
already has sufficient HTTP and SSL support to meet your needs, and that you
could re-use that. Of course, now that you've written what you need without
Twisted, you probably don't care very much :)

-Andrew.

[1] Although I think on average it probably does, and makes it easier to
write, too. The wide variety of protocols available in twisted.protoco ls
suggests to me that I'm at least part right about this...

**Roger Binns** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

> This is why I developed Stackless Python.[color=blue]
> It behaves in a similar way, acting like many callbacks,
> but you +write* your code in the most natural way possible.
> In a way, Stackless takes all the clumsy state keeping stuff
> away from the programmer and frees his mind to write simple
> top-down programs with no callbacks.[/color]

I certainly agree with that and really like your examples.

However my code also has to use wxPython, win32all, pySerial, M2Crypto
and libusb. I have no idea if all those are integrated correctly with
Stackless (and work correctly on Linux, Windows and Mac), but I was
certainly not going to be the first person to find out. And I would
still have to have fixed xmlrpc client/server portions to reuse
connections properly anyway.

Roger

**Roger Binns** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

> > Basically the existing xmlrpclib and SimpleXMLRPCSer ver are hard coded[color=blue][color=green]
> > (and IMHO go out of their way) to do one request per connection, and
> > then shut things down.[/color]
>
> if you don't like the xmlrpclib default Transport, use your own. you probably
> only have to override 'make_connectio n' and 'parse_response ' to get the be-
> haviour you're looking for.[/color]

Yes, I pointed at the code I had to write. In particular I had to do the following:

- My own Transport
- reimplementatio n of parse_response method (existing one shuts down connection)
- reimplementatio n of request method (existing one uses Python 1.5 httplib
function names and semantics)
- reimplementatio n of making a connection to reuse existing one if appropriate
- My own handler
- reimplementatio n of do_POST to get authentication headers (and not close
connections)
- reimplementatio n of finish to do proper SSL shutdown sequence
- work around bug in M2Crypto makefile method that doesn't actually do a dup
and breaks semantics if called twice

It isn't as simple as it would first appear, and the many layers of indirection
and sub-classing made it even harder to figure out what was going on, and to
override behaviour.

Roger

**Christian Tismer** · Jul 18 '05, 08:45 AM

Re: xmlprclib/server not reusing connections

Roger Binns wrote:
[color=blue][color=green]
>>This is why I developed Stackless Python.
>>It behaves in a similar way, acting like many callbacks,
>>but you *write* your code in the most natural way possible.
>>In a way, Stackless takes all the clumsy state keeping stuff
>>away from the programmer and frees his mind to write simple
>>top-down programs with no callbacks.[/color]
>
> I certainly agree with that and really like your examples.
>
> However my code also has to use wxPython, win32all, pySerial, M2Crypto
> and libusb. I have no idea if all those are integrated correctly with
> Stackless (and work correctly on Linux, Windows and Mac), but I was
> certainly not going to be the first person to find out. And I would
> still have to have fixed xmlrpc client/server portions to reuse
> connections properly anyway.[/color]

Sure. Some of my experience:
(yes, I'm using Stackless since a year now :-)

It works just *great* with wxPython. There are a few objects
which need a little care since they only live on the C stack
(mouse events for instance), but all in all it is wonderful
to use wxPython + Stackless (+ Boa Constructor + PIL + ...).
I have multiple dynamic widgets with animated graphical
content in my GUI, it is all running in a single thread,
and all my tasklets can update the GUI at any time, since
it is a single thread...

Let me know if you need sample code. Like a mouse handler,
written like a single, main program. No callbacks...

ciao - chris

--
Christian Tismer :^) <mailto:tismer@ stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

**Roger Binns** · Jul 18 '05, 08:48 AM

Re: xmlprclib/server not reusing connections

> It works just *great* with wxPython. There are a few objects[color=blue]
> which need a little care since they only live on the C stack
> (mouse events for instance),[/color]

I don't understand how that is relevant. Do I have to recompile
wxPython to use stackless? You also didn't mention anything
about the other libraries (win32all, pySerial, m2crypto, libusb)

Also, does stackless work with threads? I use a seperate
thread to do serial port stuff, and need to have the user
interface running at the same time. The serial port stuff
consists of reads with 5 second timeouts (the GIL is released
during the read).

Roger

**Christian Tismer** · Jul 18 '05, 08:49 AM

Re: xmlprclib/server not reusing connections

Roger Binns wrote:
[color=blue][color=green]
>>It works just *great* with wxPython. There are a few objects
>>which need a little care since they only live on the C stack
>>(mouse events for instance),[/color]
>
>
> I don't understand how that is relevant. Do I have to recompile
> wxPython to use stackless? You also didn't mention anything
> about the other libraries (win32all, pySerial, m2crypto, libusb)[/color]

I answered about the stuff I tried. Ok, I forgot win32all,
it works fine, too.

No, wxPython doesn't need to get recompiled, but mouse events
are dead when the stack is moved away, so you have to create
extra objects before handling the event in a different tasklet.
[color=blue]
> Also, does stackless work with threads? I use a seperate
> thread to do serial port stuff, and need to have the user
> interface running at the same time. The serial port stuff
> consists of reads with 5 second timeouts (the GIL is released
> during the read).[/color]

Yes it does. At the moment, threads are just ignored, and each
has its independent list of tasklets. This si going to change, since
I will write channels for inter-thread communication, and tasklets
which are nomading across threads.

ciao - chris
--
Christian Tismer :^) <mailto:tismer@ stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

xmlprclib/server not reusing connections

xmlprclib/server not reusing connections

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment