Non blocking socket server and storage engine

[kdeveloper]

Hello Pythonists,

I am building a non blocking socket server for incomming UDP packets.
The server needs to run at least three threads:
1. getting data and pushing to "some" storage (at the moment I use
queue),
2. acknowledge the package received
3. retrieve the information from the storage and insert it in DB.

The problem I have is that when I use queue it stores more packets in
the queue than it actually receives. An example: sent 99 UDP packets
and queue stored 600-750 entries (?) Why? I have no idea. I have
impression that I still do not understand completely how does the
queue work in python.

Another issue is that I want the server to run very rupidly. The
server needs to get at least 100 packets per second, and the same
amount of acknowledges has to be sent back.

The questions are:
1. What is the best storage engine for Python and multithreading
server?
2. Can I use queue for such problem?
3. How does actually queue work in Python? (I know how it should work
generally, but somehow it doesn't work as I expect)

Any hints & helps? Would be very grateful

Cheers
K

[Lawrence D'Oliveiro]

Re: Non blocking socket server and storage engine

kdeveloper wrote:

The server needs to run at least three threads:

Get it working without threading first.

[kdeveloper]

Re: Non blocking socket server and storage engine

On Nov 19, 10:51 am, Lawrence D'Oliveiro <l...@geek-
central.gen.new _zealandwrote:

kdeveloper wrote:

The server needs to run at least three threads:

>
Get it working without threading first.

Lawrence, I was following article [1] when building my code.
So I guess it should be ok. At the moment
I have only two threads running:
a. receiving packages and storing to queue,
b. getting packet from the queue and storing in DB.

Regards,
Krzysztof


[1] http://www.ibm.com/developerworks/ai...hon/index.html