PS: Help pickling across sockets

[Jonathan Hayward]

Just after posting the previous request, I realised that my test case
worked after I added flushes. The next problem I have is that, in the
real program, the server gets an EOFError when it first tries to read
from the socket. Relevant portions follow the error message below:

Traceback (most recent call last):
File "/var/www/cgi-bin/datamine", line 1633, in ?
find_and_output _appropriate_pa ge()
File "/var/www/cgi-bin/datamine", line 1461, in
find_and_output _appropriate_pa ge
sys.stdout.writ e(get_output())
File "/var/www/cgi-bin/datamine", line 1497, in get_output
return multitasking.ge t_page_from_ora cle()
File "/var/www/cgi-bin/datamine", line 976, in get_page_from_o racle
self.check_and_ start_oracle()
File "/var/www/cgi-bin/datamine", line 972, in check_and_start _oracle
self.start_orac le()
File "/var/www/cgi-bin/datamine", line 1083, in start_oracle
self.run_oracle ()
File "/var/www/cgi-bin/datamine", line 1056, in run_oracle
self.handle_ora cle_query(newso cket, address)
File "/var/www/cgi-bin/datamine", line 1015, in handle_oracle_q uery
self.get_thread _specific_stora ge()["environment_va riables"] = \
EOFError
Traceback (most recent call last):
File "/var/www/cgi-bin/datamine", line 1633, in ?
find_and_output _appropriate_pa ge()
File "/var/www/cgi-bin/datamine", line 1461, in
find_and_output _appropriate_pa ge
sys.stdout.writ e(get_output())
File "/var/www/cgi-bin/datamine", line 1497, in get_output
return multitasking.ge t_page_from_ora cle()
File "/var/www/cgi-bin/datamine", line 988, in get_page_from_o racle
result = cPickle.load(so ckIn)
IOError: [Errno 104] Connection reset by peer

From the server side, here are run_oracle() and handle_oracle_q uery():

def run_oracle(self ):
#thread.start_n ew_thread(\
#self.initializ e, ())
sock = socket.socket(s ocket.AF_INET, socket.SOCK_STR EAM)
sock.setsockopt (socket.SOL_SOC KET, socket.SO_REUSE ADDR, 1)
sock.bind(("", configuration.g et_search_serve r_port()))
sock.listen(5)
while 1:
try:
newsocket, address = sock.accept()
self.handle_ora cle_query(newso cket, address)
#thread.start_n ew_thread(self. handle_oracle_q uery,
(newsocket, \ #address))
except socket.error:
sock.close()
sock = socket.socket(s ocket.AF_INET,
socket.SOCK_STR EAM)
sock.setsockopt (socket.SOL_SOC KET, socket.SO_REUSE ADDR,
1)
sock.bind(("", configuration.g et_search_serve r_port()))
sock.listen(5)

def handle_oracle_q uery(self, sock, address):
sockIn = sock.makefile(" rb")
sockOut = sock.makefile(" wb")
sockIn.flush()
### Line 1015, immediately below, is where the EOF error is reported.
self.get_thread _specific_stora ge()["environment_va riables"] = \
cPickle.load(so ckIn)
sockIn.flush()
self.get_thread _specific_stora ge()["cgi"] =
cPickle.load(so ckIn)
generate_output ()
print_output(so ckOut)
sockOut.flush()
sock.close()
sockIn.close()
sockOut.close()

From the client side, here is get_page_from_o racle():

def get_page_from_o racle(self):
self.check_and_ start_oracle()
sock = socket.socket(s ocket.AF_INET, socket.SOCK_STR EAM)
try:
sock.connect((c onfiguration.ge t_search_server _ip(), \
configuration.g et_search_serve r_port()))
sockIn = sock.makefile(" rb")
sockOut = sock.makefile(" wb")
cPickle.dump(os .environ, sockOut)
sockOut.flush()
cPickle.dump(cg i.FieldStorage( ), sockOut)
sockOut.flush()
sockIn.flush()
### Line 988, immediately below, also gets an error, but I believe
this is secondary damage.
result = cPickle.load(so ckIn)
sock.close()
sockIn.close()
sockOut.close()
return result

TIA,
++ Jonathan Hayward, jonathan.haywar d@pobox.com
** To see an award-winning website with stories, essays, artwork,
** games, and a four-dimensional maze, why not visit my home page?
** All of this is waiting for you at http://JonathansCorner.com

[Irmen de Jong]

Re: PS: Help pickling across sockets

Jonathan Hayward wrote:[color=blue]
> Just after posting the previous request, I realised that my test case
> worked after I added flushes. The next problem I have is that, in the
> real program, the server gets an EOFError when it first tries to read
> from the socket.[/color]

I'm not sure, but perhaps pickle doesn't work on sockets directly?
Perhaps it needs a random-access file object.

So, read all the data from the socket yourself, into a string
buffer, and run pickle on that.

That's what I do in Pyro :-)

HTH,
--Irmen de Jong