passing a socket to a spawned process.

[Mike M]

Is it possible? In the parent process, I have a socket that binds,
listens and then accepts new connections (which creates new sockets in
the process). I want to be able to pass some of these new sockets to
a spawned process. Is it possible, and if so how?

Any help is much appreciated!!

Mike

[Marcos Dione]

Re: passing a socket to a spawned process.

On Mon, Oct 11, 2004 at 10:23:27AM -0700, Mike M wrote:[color=blue]
> Is it possible? In the parent process, I have a socket that binds,
> listens and then accepts new connections (which creates new sockets in
> the process). I want to be able to pass some of these new sockets to
> a spawned process. Is it possible, and if so how?[/color]

just accept() before fork()'ing. the socket will be cloned. then close the
socket on the parent, unless needed.

--
(Not so) Random fortune:
(Really) Random fortune:
mdione@tempest: ~$ cat /dev/random | uuencode random | head -n 3
begin 644 random
M=?:,8:@5MVVIM6 >NAYQ6O@I&H86>. 3(L"F7S>U'.9`2; %LEAR/E8M[*F=(MP
M5C>B3Y2F(X,`SW 1UPAL*+OIV'EM/Z$_-15':B#<6I99_P,H %=$S6$H<&W-!;

[Josiah Carlson]

Re: passing a socket to a spawned process.

[color=blue]
> Is it possible? In the parent process, I have a socket that binds,
> listens and then accepts new connections (which creates new sockets in
> the process). I want to be able to pass some of these new sockets to
> a spawned process. Is it possible, and if so how?
>
> Any help is much appreciated!![/color]

As someone has already mentioned, if you have the socket before the fork,
you will have the socket after the fork.


I had been working on a general file descriptor passing mechanism for a
few weeks in May, until I gave up.

From what I was able to work out, it is not possible in Windows.
SunOS 5.8 has no issues, and will work almost out of the box (pure
Python) with the right incantation (I have it somewhere if you really
want it).
A few constants are missing from the Linux fcntl.py, so this is not
really possible unless you are willing to patch your kernel and Python.

Googling for the terms: passing file descriptors Python
will get you the list of articles/research that I looked at.

One of those links has source code for the C struct usable with a BSD
for making this happen. Generally though, you can only pass file
descriptors (sockets) across a UNIX domain socket, or sometimes a pipe
(make it before you fork).


Good luck.
- Josiah

[Alex Martelli]

Re: passing a socket to a spawned process.

Marcos Dione <mdione@grulic. org.ar> wrote:
[color=blue]
> On Mon, Oct 11, 2004 at 10:23:27AM -0700, Mike M wrote:[color=green]
> > Is it possible? In the parent process, I have a socket that binds,
> > listens and then accepts new connections (which creates new sockets in
> > the process). I want to be able to pass some of these new sockets to
> > a spawned process. Is it possible, and if so how?[/color]
>
> just accept() before fork()'ing. the socket will be cloned. then close the
> socket on the parent, unless needed.[/color]

Unfortunately I believe that's platform-dependent. Specifically: your
suggestion should work on any sensible operating system... BUT a tad
over 50% of Python programs are estimated to run on Windows, and, on
THAT platform, I know your idea can't work (no fork!) and I don't know
how to answer the OP's question (except by suggesting he move, if
feasible, to any sensible platform -- any BUT Wind0ws...!-).


Alex

[Andrew MacIntyre]

Re: passing a socket to a spawned process.

On Wed, 13 Oct 2004, Alex Martelli wrote:
[color=blue]
> Marcos Dione <mdione@grulic. org.ar> wrote:
>[color=green]
> > On Mon, Oct 11, 2004 at 10:23:27AM -0700, Mike M wrote:[color=darkred]
> > > Is it possible? In the parent process, I have a socket that binds,
> > > listens and then accepts new connections (which creates new sockets in
> > > the process). I want to be able to pass some of these new sockets to
> > > a spawned process. Is it possible, and if so how?[/color]
> >
> > just accept() before fork()'ing. the socket will be cloned. then close the
> > socket on the parent, unless needed.[/color]
>
> Unfortunately I believe that's platform-dependent. Specifically: your
> suggestion should work on any sensible operating system... BUT a tad
> over 50% of Python programs are estimated to run on Windows, and, on
> THAT platform, I know your idea can't work (no fork!) and I don't know
> how to answer the OP's question (except by suggesting he move, if
> feasible, to any sensible platform -- any BUT Wind0ws...!-).[/color]

I've not tried this on Windows, but on OS/2 using the native sockets
(which is close to the Windows socket model as far as I can tell) you have
to pass the socket handle (an integer) to the child (usually via the
command line), which then attaches to the socket, and signals the parent
that it can close its reference to the socket.

The EMX (OS/2, Win32) and Cygwin (Win32) runtime environments synthesize
fork() and socket inheritance over the native environment, though at a
significant cost in run time and resources.

-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andymac@bullsey e.apana.org.au (pref) | Snail: PO Box 370
andymac@pcug.or g.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia

[Michael Loritsch]

Re: passing a socket to a spawned process.

Josiah Carlson <jcarlson@uci.e du> wrote in message news:<mailman.4 761.1097598633. 5135.python-list@python.org >...[color=blue][color=green]
> > Is it possible? In the parent process, I have a socket that binds,
> > listens and then accepts new connections (which creates new sockets in
> > the process). I want to be able to pass some of these new sockets to
> > a spawned process. Is it possible, and if so how?
> >
> > Any help is much appreciated!![/color]
>
> As someone has already mentioned, if you have the socket before the fork,
> you will have the socket after the fork.
>
>
> I had been working on a general file descriptor passing mechanism for a
> few weeks in May, until I gave up.
>
> From what I was able to work out, it is not possible in Windows.
> SunOS 5.8 has no issues, and will work almost out of the box (pure
> Python) with the right incantation (I have it somewhere if you really
> want it).
> A few constants are missing from the Linux fcntl.py, so this is not
> really possible unless you are willing to patch your kernel and Python.
>
> Googling for the terms: passing file descriptors Python
> will get you the list of articles/research that I looked at.
>
> One of those links has source code for the C struct usable with a BSD
> for making this happen. Generally though, you can only pass file
> descriptors (sockets) across a UNIX domain socket, or sometimes a pipe
> (make it before you fork).
>
>
> Good luck.
> - Josiah[/color]


Passing sockets to spawned processes is also possible in Windows, but
tricky.

While I have never passed a socket between python processes, I have
passed a socket handle from a python process to a Windows binary.
Using the win32api extensions packaged with the ActiveState
distribution, one should be able to pass the socket between python
processes just as well.

The basic pseudo-code for doing so is as follows:

Parent process:
1) Obtain socket connection in parent server process from a client
process.
2) Obtain Windows process id -> use win32api.GetCur rentProcessId()
- This process id will later be sent to the child process.
3) Spawn child process, obtaining a pipe to child_stdin -> use one of
the os.popen functions.
4) Using child_stdin pass parent process id (obtained in 2) to the
child process.
5) Next, using child_stdin pass over the socket file id, using
socket.fileno() .

Now the child process should have everything it needs to listen on the
socket.

Child process:
1) Read parent process id from stdin.
2) Read socket file id from stdin.
3) Obtain a handle to the parent process using the process id -> use
win32api.OpenPr ocess(). This handle will be needed in the next
call...
4) Turn your socket file id into a socket -> use
win32api.Duplic ateHandle().
- hSourceProcess is the handle obtained in 3
- hSource is the file id

You are now done, and should be able to read data on the socket in the
child process!

Regards,

Michael Loritsch

[Josiah Carlson]

Re: passing a socket to a spawned process.


On 16 Oct 2004 00:32:23 -0700
loritsch@gmail. com (Michael Loritsch) wrote:
[color=blue]
> Josiah Carlson <jcarlson@uci.e du> wrote in message news:<mailman.4 761.1097598633. 5135.python-list@python.org >...[color=green][color=darkred]
> > > Is it possible? In the parent process, I have a socket that binds,
> > > listens and then accepts new connections (which creates new sockets in
> > > the process). I want to be able to pass some of these new sockets to
> > > a spawned process. Is it possible, and if so how?
> > >
> > > Any help is much appreciated!![/color]
> >
> > As someone has already mentioned, if you have the socket before the fork,
> > you will have the socket after the fork.
> >
> >
> > I had been working on a general file descriptor passing mechanism for a
> > few weeks in May, until I gave up.
> >
> > From what I was able to work out, it is not possible in Windows.
> > SunOS 5.8 has no issues, and will work almost out of the box (pure
> > Python) with the right incantation (I have it somewhere if you really
> > want it).
> > A few constants are missing from the Linux fcntl.py, so this is not
> > really possible unless you are willing to patch your kernel and Python.
> >
> > Googling for the terms: passing file descriptors Python
> > will get you the list of articles/research that I looked at.
> >
> > One of those links has source code for the C struct usable with a BSD
> > for making this happen. Generally though, you can only pass file
> > descriptors (sockets) across a UNIX domain socket, or sometimes a pipe
> > (make it before you fork).
> >
> >
> > Good luck.
> > - Josiah[/color]
>
>
> Passing sockets to spawned processes is also possible in Windows, but
> tricky.
>
> While I have never passed a socket between python processes, I have
> passed a socket handle from a python process to a Windows binary.
> Using the win32api extensions packaged with the ActiveState
> distribution, one should be able to pass the socket between python
> processes just as well.
>
> The basic pseudo-code for doing so is as follows:
>
> Parent process:
> 1) Obtain socket connection in parent server process from a client
> process.
> 2) Obtain Windows process id -> use win32api.GetCur rentProcessId()
> - This process id will later be sent to the child process.
> 3) Spawn child process, obtaining a pipe to child_stdin -> use one of
> the os.popen functions.
> 4) Using child_stdin pass parent process id (obtained in 2) to the
> child process.
> 5) Next, using child_stdin pass over the socket file id, using
> socket.fileno() .
>
> Now the child process should have everything it needs to listen on the
> socket.
>
> Child process:
> 1) Read parent process id from stdin.
> 2) Read socket file id from stdin.
> 3) Obtain a handle to the parent process using the process id -> use
> win32api.OpenPr ocess(). This handle will be needed in the next
> call...
> 4) Turn your socket file id into a socket -> use
> win32api.Duplic ateHandle().
> - hSourceProcess is the handle obtained in 3
> - hSource is the file id
>
> You are now done, and should be able to read data on the socket in the
> child process![/color]


Goodness, I guess it is possible in Windows. That kind of thing is
really useful for a "pre-forked" server with multiple processes, or even
a super-daemon that listens on ports specified by other daemon processes
in order to handle automatic failover.

Looks like I have a reason to download pywin32 after all.

Now if only it could be done in linux out of the box.

- Josiah

[Martin Atkins]

Re: passing a socket to a spawned process.

I recently wrote a c module for Python, for passing file descriptors and process
credentials down unix domain sockets under Linux. (using sendmsg(2))

It was pretty easy, and I could send it to anyone interested. Contact me
off-list, since I don't normally read this group.

Perhaps there is a need for a OS-independant wrapper around versions
specifically for Linux, Windows, Solaris, etc?

Martin

[Josiah Carlson]

Re: passing a socket to a spawned process.


martin_ml@parva t.com (Martin Atkins) wrote:[color=blue]
>
> I recently wrote a c module for Python, for passing file descriptors and process
> credentials down unix domain sockets under Linux. (using sendmsg(2))
>
> It was pretty easy, and I could send it to anyone interested. Contact me
> off-list, since I don't normally read this group.
>
> Perhaps there is a need for a OS-independant wrapper around versions
> specifically for Linux, Windows, Solaris, etc?
>
> Martin[/color]

I'll take the source for the C version. And in general, yes, Python
could use such a thing for all platforms, though I'm not sure all
platforms support it.

- Josiah

[Donn Cave]

Re: passing a socket to a spawned process.

In article <mailman.5620.1 098982967.5135. python-list@python.org >,
Josiah Carlson <jcarlson@uci.e du> wrote:
[color=blue]
> martin_ml@parva t.com (Martin Atkins) wrote:[color=green]
> >
> > I recently wrote a c module for Python, for passing file descriptors and
> > process
> > credentials down unix domain sockets under Linux. (using sendmsg(2))
> >
> > It was pretty easy, and I could send it to anyone interested. Contact me
> > off-list, since I don't normally read this group.
> >
> > Perhaps there is a need for a OS-independant wrapper around versions
> > specifically for Linux, Windows, Solaris, etc?
> >
> > Martin[/color]
>
> I'll take the source for the C version. And in general, yes, Python
> could use such a thing for all platforms, though I'm not sure all
> platforms support it.[/color]

Rest assured, not all platforms support it.

Donn Cave, donn@u.washingt on.edu

[Martin C.Atkins]

Re: passing a socket to a spawned process.

I have (finally!) put a package up on our website that provides a
Python module for sending file descriptors down Unix-domain sockets, on Linux.

See the first item under:

http://www.mca-ltd.com/index.php?PAGE=resources/home.php

Please let me know of any problems, obvious omissions, etc...

BTW: I just found out about bindd (see the README), and could probably
clean the code up from the examples in its sourcecode, but the current
version of fdcred works for me!

Sorry it has taken so long!

Regards,
Martin
--
Martin C. Atkins martin_ml@parva t.com
Parvat Infotech Private Limited http://www.parvat.com{/,/martin}

[Dave Cole]

Re: passing a socket to a spawned process.

Martin C.Atkins wrote:[color=blue]
> I have (finally!) put a package up on our website that provides a
> Python module for sending file descriptors down Unix-domain sockets, on Linux.
>
> See the first item under:
> http://www.mca-ltd.com/index.php?PAG...urces/home.php
>
> Please let me know of any problems, obvious omissions, etc...
>
> BTW: I just found out about bindd (see the README), and could probably
> clean the code up from the examples in its sourcecode, but the current
> version of fdcred works for me!
>
> Sorry it has taken so long![/color]

I notice that you have used the LGPL for your code. Could you be
convinced to use a Python style license?

This would mean that some time in the (hopefully not too distant) future
the code could be added to the standard socket module.

- Dave

--

http://www.object-craft.com.au