Architecture design

[Richard Heathfield]

Re: Architecture design

s0suk3@gmail.co m said:

On Oct 22, 2:36 am, Richard Heathfield <rjh@see.sig.in validwrote:

<snip>

>Threads were invented for people who can't
>write state machines (which, incidentally, are far easier to debug).

>
Do state machines really have anything to do with parallel processing?

Most people who use threads do so not because they need parallel
processing, but because they think they do. For example, it's commonplace
to see server code with one thread per client, which is a ludicrous misuse
of resources. In a great many cases, threads are simply not a good fit to
the problem. (This says nothing about their portability, of course.)

In C/C++, you can use frameworks offering a single interface to the
underlying threading environment.

>>

>If you really need to use threads, that's obviously a good way to go.
>But of course most programs would be better off without them.

>>

Either you need them or you don't, of course. I'm just saying that
they have significant advantages over processes in general.

>>
>That sounds like a factoid to me (like the factoid that "integer
>operations are faster than floating-point operations"). And threads have
>significant *dis*advantages that should not be glossed over.

>
That would sound much more convincing if you stated what disadvantages
those are, and how you can avoid them using processes or state
machines.

If you don't know already, I see little value in continuing this
discussion. But that doesn't mean I'm ducking the question.

The most obvious disadvantage is that threads can compete over object
values. Thread A might try to read the value whilst Thread B is in the
middle of updating that value, for example. So you have to do the whole
semaphore/mutex thing. And you have to be careful to avoid a deadlock,
where Thread A has locked resource C and is waiting for resource D,
whereas Thread B has locked resource D and is waiting for resource C.

Writing thread-safe code is a far from trivial exercise. Writing a state
machine solution is much simpler and generally less bug-prone (at least,
in my experience).

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Nick Keighley]

Re: Architecture design

On 22 Oct, 01:01, s0s...@gmail.co m wrote:

On Oct 21, 11:30 am, Lew Pitcher <lpitc...@teksa vvy.comwrote:

On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
(alesanRemoveMe Ple...@manoweb. com) wrote:

Ben Bacarisse wrote:

>...and if you can't be talked out of using threads,

>

Why does everybody hates threads so much :) I often fail to see the
advantages of processes for 'small' things.

>

<off-topic>
My guess is that threads are often more bother than they are worth.

>

Given the possibility of programmed creation of processes (i.e. an api to
launch processes), threads are redundant.

>
fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems),

you are comparing oranges with orchards.

threads and processes are the things to compare. fork()
is just one OS specific way of creating processes.
Compare like with like if you are going to talk portability.

so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

why? Why are fork-less system less flexible. What is the connection
between process creation and IPC?

<snip>

Programs that use threads often suffer from poor design/implementation,

>
That has nothing to do with threads themselves. Maybe you've seen a
lot of such applications, but they're poor design is purely their
implementors' fault.

fair point. People can write crap in any language.
Tho' threads are pretty easy to make mistakes with...

<snip>

--
Nick Keighley

[Antoninus Twink]

Re: Architecture design

On 22 Oct 2008 at 0:01, s0suk3@gmail.co m wrote:

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems), so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

I don't really agree with this assessment.

I'd say that for communication purposes, fork()ed processes are much
less flexible than threads, and an external process provides most of the
flexibility of a fork()ed process.

OK, so you have some nice conveniences with fork() like being able to
open a pipe, have the file descriptors copied to the child, then use it
for parent-to-child or child-to-parent communication, but IPC between a
parent and child process isn't really much easier or more convenient
than between two arbitrary processes.

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 3:26 am, Richard Heathfield <rjh@see.sig.in validwrote:

s0suk3@gmail.co m said:
>

On Oct 22, 2:36 am, Richard Heathfield <rjh@see.sig.in validwrote:

>
<snip>
>

Threads were invented for people who can't
write state machines (which, incidentally, are far easier to debug).

>

Do state machines really have anything to do with parallel processing?

>
Most people who use threads do so not because they need parallel
processing, but because they think they do. For example, it's commonplace
to see server code with one thread per client, which is a ludicrous misuse
of resources. In a great many cases, threads are simply not a good fit to
the problem. (This says nothing about their portability, of course.)

How do you propose writing a server that needs to handle multiple
clients synchronously without parallel processing? The only
alternative I've heard so far is select(), but it's only optimal if
the server isn't receiving too many requests.

In C/C++, you can use frameworks offering a single interface to the
underlying threading environment.

>

If you really need to use threads, that's obviously a good way to go.
But of course most programs would be better off without them.

>

Either you need them or you don't, of course. I'm just saying that
they have significant advantages over processes in general.

>

That sounds like a factoid to me (like the factoid that "integer
operations are faster than floating-point operations"). And threads have
significant *dis*advantages that should not be glossed over.

>

That would sound much more convincing if you stated what disadvantages
those are, and how you can avoid them using processes or state
machines.

>
If you don't know already, I see little value in continuing this
discussion. But that doesn't mean I'm ducking the question.
>
The most obvious disadvantage is that threads can compete over object
values. Thread A might try to read the value whilst Thread B is in the
middle of updating that value, for example. So you have to do the whole
semaphore/mutex thing. And you have to be careful to avoid a deadlock,
where Thread A has locked resource C and is waiting for resource D,
whereas Thread B has locked resource D and is waiting for resource C.
>
Writing thread-safe code is a far from trivial exercise. Writing a state
machine solution is much simpler and generally less bug-prone (at least,
in my experience).

I think writing thread-safe code is trivial once you adopt the set of
coding practices that lead to thread-safe code. Dealing with the
threads themselves may be less trivial, but it's the cost of having
the added functionality. Anyway, remember the old saying

"Keep it simple: as simple as possible, but no simpler."

I think you're trying to keep it simpler :-)

Sebastian

[Richard Heathfield]

Re: Architecture design

s0suk3@gmail.co m said:

On Oct 22, 3:26 am, Richard Heathfield <rjh@see.sig.in validwrote:

>s0suk3@gmail.co m said:
>>

On Oct 22, 2:36 am, Richard Heathfield <rjh@see.sig.in validwrote:

>>
><snip>
>>

>Threads were invented for people who can't
>write state machines (which, incidentally, are far easier to debug).

>>

Do state machines really have anything to do with parallel processing?

>>
>Most people who use threads do so not because they need parallel
>processing, but because they think they do. For example, it's
>commonplace to see server code with one thread per client, which is a
>ludicrous misuse of resources. In a great many cases, threads are simply
>not a good fit to the problem. (This says nothing about their
>portability, of course.)

>
How do you propose writing a server that needs to handle multiple
clients synchronously without parallel processing?

One problem with off-topic discussions is that it's difficult to know where
to draw a line.

The only
alternative I've heard so far is select(), but it's only optimal if
the server isn't receiving too many requests.

Yes, you use select(). If you have too many requests, you add servers.
Servers are cheap.

<snip>

>>
>Writing thread-safe code is a far from trivial exercise. Writing a state
>machine solution is much simpler and generally less bug-prone (at least,
>in my experience).

>
I think writing thread-safe code is trivial once you adopt the set of
coding practices that lead to thread-safe code.

I have quite a few programs installed on this machine that were written by
people who think as you do. These programs crash rather often.

Dealing with the
threads themselves may be less trivial, but it's the cost of having
the added functionality. Anyway, remember the old saying
>
"Keep it simple: as simple as possible, but no simpler."
>
I think you're trying to keep it simpler :-)

Think as you will, but I disagree.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 3:41 am, Antoninus Twink <nospam@nospam. invalidwrote:

On 22 Oct 2008 at  0:01, s0suk3@gmail.co m wrote:
>

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems), so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

>
I don't really agree with this assessment.
>
I'd say that for communication purposes, fork()ed processes are much
less flexible than threads,

Yes, that's what I said.

and an external process provides most of the
flexibility of a fork()ed process.
>
OK, so you have some nice conveniences with fork() like being able to
open a pipe, have the file descriptors copied to the child, then use it
for parent-to-child or child-to-parent communication, but IPC between a
parent and child process isn't really much easier or more convenient
than between two arbitrary processes.

How isn't it more easier or convenient if you have those mechanisms
available? With external processes, all you have are the standard I/O
streams connected as pipes between the processes. The OS may provide
more specialized IPC mechanisms (such as UNIX domain sockets, under
UNIX, or just Internet domain sockets connected to the loopback
interface), but these are very non-portable and usually lead to a
great increase in complexity.

Sebastian

[Ian Collins]

Re: Architecture design

Richard Heathfield wrote:

s0suk3@gmail.co m said:

>I think writing thread-safe code is trivial once you adopt the set of
>coding practices that lead to thread-safe code.

>
I have quite a few programs installed on this machine that were written by
people who think as you do. These programs crash rather often.
>

It's seldom trivial, but thread-safe programming is a skill well worth
mastering in these days of multi-core CPUs. I can't remember the last
time I used a single core desktop.

Would you be happy if the developers of your OS shared your Luddite
attitude and refused to build a threaded kernel?

--
Ian Collins

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 3:26 am, Nick Keighley <nick_keighley_ nospam@hotmail. com>
wrote:

On 22 Oct, 01:01, s0s...@gmail.co m wrote:

>On Oct 21, 11:30 am, Lew Pitcher <lpitc...@teksa vvy.comwrote:

>>On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
>>(alesanRemove MePle...@manowe b.com) wrote:
>>>Ben Bacarisse wrote:
>>>>...and if you can't be talked out of using threads,

>

>>>Why does everybody hates threads so much :) I often fail to see the
>>>advantages of processes for 'small' things.

>

>><off-topic>
>>My guess is that threads are often more bother than they are worth.

>

>>Given the possibility of programmed creation of processes (i.e. an api to
>>launch processes), threads are redundant.

>

>fork() is known to be the process creation equivalent to threads
>creation, and it provides most of the flexibility that threading does.
>However, fork() is less efficient that threads, and much less portable
>than threads (since it's only supported on POSIX systems),

>
you are comparing oranges with orchards.
>
threads and processes are the things to compare. fork()
is just one OS specific way of creating processes.
Compare like with like if you are going to talk portability.

I was comparing threads with fork() because, as far as I can see,
fork() is the way to create a process that provides the most
flexibility (e.g., you can set up things like pipes right before the
fork for IPC).

so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

>
why? Why are fork-less system less flexible. What is the connection
between process creation and IPC?

Well, when your process divides into two threads or two processes, you
sometimes need those two threads or processes to continue
communicating. For example, sometimes programs spawn a producer thread
that does some computation and then places the gathered information in
a global data structure, which a consumer thread will check
periodically. (This is often the case in GUI programs, where the
"consumer" thread is the main thread which handles the windows, and
the "producer" thread is the one that does the long-running task so
that the windows can remain active.) IPC is one of the main things
that makes parallel processing useful; if processes or threads just
went on their own after creation, their use would be very limited.

<snip>

Sebastian

[Antoninus Twink]

Re: Architecture design

On 22 Oct 2008 at 9:22, s0suk3@gmail.co m wrote:

On Oct 22, 3:41 am, Antoninus Twink <nospam@nospam. invalidwrote:

>On 22 Oct 2008 at  0:01, s0suk3@gmail.co m wrote:

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.

>>
>I don't really agree with this assessment.
>>
>I'd say that for communication purposes, fork()ed processes are much
>less flexible than threads,

>
Yes, that's what I said.

Well, you actually said that fork() provides most of the flexibility
that threading does... but let's not get into silly word games like the
"regulars" play.

>OK, so you have some nice conveniences with fork() like being able to
>open a pipe, have the file descriptors copied to the child, then use it
>for parent-to-child or child-to-parent communication, but IPC between a
>parent and child process isn't really much easier or more convenient
>than between two arbitrary processes.

>
How isn't it more easier or convenient if you have those mechanisms
available?

It can be easier and more convenient in some circumstances, as I said.
But the extra options available to you for IPC between parent and child
processes versus two arbitrary processes are extremely limited.

With external processes, all you have are the standard I/O streams
connected as pipes between the processes. The OS may provide more
specialized IPC mechanisms (such as UNIX domain sockets, under UNIX,
or just Internet domain sockets connected to the loopback interface),
but these are very non-portable

Sockets and pipes are both specified by POSIX, and both are widely
portable.

and usually lead to a great increase in complexity.

IPC is inherently complex.

[Antoninus Twink]

Re: Architecture design

On 22 Oct 2008 at 7:32, Richard Heathfield wrote:

In my own personal experience, neither fork() nor threads are
especially portable, and to argue about which is *more* portable seems
to me to be an angels-on-pinheads discussion.

I wondered what it was that drew you to this discussion that normally
you'd exclude as "off topic" - the lure of another angels-on-pinheads
discussion was too much.

[Antoninus Twink]

Re: Architecture design

On 22 Oct 2008 at 9:23, Richard Heathfield wrote:

s0suk3@gmail.co m said:

>The only alternative I've heard so far is select(), but it's only
>optimal if the server isn't receiving too many requests.

>
Yes, you use select(). If you have too many requests, you add servers.
Servers are cheap.

Presumably you are trying to be sarcastic, but it's obviously true that
*whatever* programming methodology you use, threads or not, resources
will still be finite, so at the end of the day if you get more requests
than you can satisfy then adding resources is the only solution.

[Nick Keighley]

Re: Architecture design

On 22 Oct, 11:09, s0s...@gmail.co m wrote:

On Oct 22, 3:26 am, Nick Keighley <nick_keighley_ nos...@hotmail. com>
wrote:

On 22 Oct, 01:01, s0s...@gmail.co m wrote:

On Oct 21, 11:30 am, Lew Pitcher <lpitc...@teksa vvy.comwrote:
>On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
>(alesanRemoveM ePle...@manoweb .com) wrote:
>>Ben Bacarisse wrote:

>>>...and if you can't be talked out of using threads,

>

>>Why does everybody hates threads so much :) I often fail to see the
>>advantages of processes for 'small' things.

>

><off-topic>
>My guess is that threads are often more bother than they are worth.

>

>Given the possibility of programmed creation of processes (i.e. an api to
>launch processes), threads are redundant.

>

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems),

>

you are comparing oranges with orchards.

>

threads and processes are the things to compare. fork()
is just one OS specific way of creating processes.
Compare like with like if you are going to talk portability.

>
I was comparing threads with fork() because, as far as I can see,
fork() is the way to create a process that provides the most
flexibility (e.g., you can set up things like pipes right before the
fork for IPC).

but if we are going to discuss portability you need to compare
comparable things (actually I think that applies much more generally!
:-) )

so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

>

why? Why are fork-less system less flexible. What is the connection
between process creation and IPC?

>
Well, when your process divides into two threads or two processes, you
sometimes need those two threads or processes to continue
communicating. For example, sometimes programs spawn a producer thread
that does some computation and then places the gathered information in
a global data structure, which a consumer thread will check
periodically. (This is often the case in GUI programs, where the
"consumer" thread is the main thread which handles the windows, and
the "producer" thread is the one that does the long-running task so
that the windows can remain active.) IPC is one of the main things
that makes parallel processing useful; if processes or threads just
went on their own after creation, their use would be very limited.

you appear to have answered a different a question from the one I
asked.
I said " What is the connection between process creation and IPC?".
You seem to be explaining why processes (or threads need to
communicate.
I agree if P/Ts were unable to communicate there would be little
point
in having them.

What does this have to do with the method of process creation?

[basically I don't accept your point that not using fork
automatically precludes using various forms of inter-process
communication]



--
Nick Keighley

[Nick Keighley]

Re: Architecture design

On 22 Oct, 09:20, s0s...@gmail.co m wrote:

On Oct 22, 3:08 am, Richard Heathfield <r...@see.sig.i nvalidwrote:

s0s...@gmail.co m said:

Like I said earlier, there are
languages such as Python and Java that try to provide portable
interfaces to operating system services.

>

Yes, they try. And surprisingly often they succeed. But if you think that
Python and Java are more portable than C, you have an astoundingly limited
view of computing.

>
Those languages are portable in a different sense than C is. C tries
to be portable across a wide range of platforms at the cost of failing
to provide many system services that may not be available under those
platforms, which greatly reduces its usability. Java and Python try to
be portable across

[a much narrower]

range of platforms while still providing
interfaces to as many system services as possible, at the cost of
being unable to provide those services on systems where they're not
available, and thus creating ambiguities in the interfaces on those
systems.
>
Unfortunately, we don't have both of those meanings of portability in
a single language yet.

I don't think it is, even in principle, possible. We will always
have both mainframes and toasters.

--
Nick Keighley

With Y2K behind us,
there's nearly 8000 years until we need COBOL programmers again!

[Richard Heathfield]

Re: Architecture design

Ian Collins said:

Richard Heathfield wrote:

>s0suk3@gmail.co m said:

>

>>I think writing thread-safe code is trivial once you adopt the set of
>>coding practices that lead to thread-safe code.

>>
>I have quite a few programs installed on this machine that were written
>by people who think as you do. These programs crash rather often.
>>

It's seldom trivial, but thread-safe programming is a skill well worth
mastering in these days of multi-core CPUs.

I agree entirely. I am not arguing that threads should not be used. I am
arguing that they are over-used.

I can't remember the last
time I used a single core desktop.
>
Would you be happy if the developers of your OS shared your Luddite
attitude and refused to build a threaded kernel?

You seem to be attempting to extend my position. I have not said that
nobody should use threads. Nor have I said that I never use threads. (In
fact, I rarely use them, but rarely != never.) What I have said is that
people often use threads when it is inappropriate, and I don't agree that
such a position is Luddite. Nor do I think it is Luddite to point out that
many people who use threads do so in sub-optimal ways (to put it too
mildly).

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Kenny McCormack]

Re: Architecture design

In article <87y70h178x.fsf @kvetch.smov.or g>,
Keith Thompson <kst@sdsc.eduwr ote:

>s0suk3@gmail.c om writes:
>[...]

>fork() is known to be the process creation equivalent to threads
>creation, and it provides most of the flexibility that threading does.
>However, fork() is less efficient that threads, and much less portable
>than threads (since it's only supported on POSIX systems), so even
>when fork() is available, threads are not redundant.

>[...]
>
>fork() is less portable than threads? Huh?

Windows has threads but not fork. That's a big chunk of the world.