Architecture design

[Alessio Sangalli]

Hi all. I am writing the documentation for the program I'm going to
develop (yes, documentation before the implementation, it's awesome!).

In short, it will run on an embedded platform wint Linux as OS. I have
some hardware devices that have to send data to and receive data from a
server on the network. I will use one TCP stream for all the communication.

Basically what I plan to do is the following:
- for the program to be as modular as possible, it will look in a
directory for "plugins" implemented with shared objects. Every plugin
will declare what "type" or data it will send or receive.
- A "main" thread will start each "plugin" as a separate thread.
- Every "plugin-thread" will manage the hardware; it will eventually
have something to send to the server and viceversa

What is the best way to implement this data exchange? I'd prefer not to
execute code found in the plugin from the "main-thread" because a poorly
written plugin may block for too much time (or indefinitely) the
operations of the rest of the plugins and the data transfer.

An idea I had was to have the main-thread or the plugin-threads insert
data in a thread-safe queue, and then send a signal to the recipient
(main or plugin depending on the data flow) to inform it has to process
the data.

What do you think?
Alessio

[Ben Bacarisse]

Re: Architecture design

Alessio Sangalli <alesanRemoveMe Please@manoweb. comwrites:
<snip>

In short, it will run on an embedded platform wint Linux as OS.

comp.unix.progr ammer is probably a reasonable place to get the overall
design discussed.

<snip>

An idea I had was to have the main-thread or the plugin-threads insert
data in a thread-safe queue, and then send a signal to the recipient
(main or plugin depending on the data flow) to inform it has to process
the data.

....and if you can't be talked out of using threads,
comp.programmin g.threads has some very good people on it who can help
with any of these specifics.

--
Ben.

[Alessio Sangalli]

Re: Architecture design

Ben Bacarisse wrote:

comp.unix.progr ammer is probably a reasonable place to get the overall
design discussed.

Thanks for the direction

...and if you can't be talked out of using threads,

Why does everybody hates threads so much :) I often fail to see the
advantages of processes for 'small' things.

bye
Alessio

[Lew Pitcher]

Re: Architecture design

On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
(alesanRemoveMe Please@manoweb. com) wrote:

Ben Bacarisse wrote:

[snip]

>...and if you can't be talked out of using threads,

>
Why does everybody hates threads so much :) I often fail to see the
advantages of processes for 'small' things.

<off-topic>
My guess is that threads are often more bother than they are worth.

Given the possibility of programmed creation of processes (i.e. an api to
launch processes), threads are redundant.

In many environments, threads do not provide any execution-speed advantage
over processes (the usual "big" argument for threads is that "the
context-switch overhead for a thread is 'less' than the context-switch
overhead for a process", but in many systems, threads and processes have
the same context-switch overhead).

Threads usually require a greater level of programming support than
processes.

Programs that use threads often suffer from poor design/implementation,
causing any advantage that threads offer to be lost (I'm thinking of
deadlock and race conditions, and of failure to disentangle logic).

There are as effective ways of data sharing for processes as there are for
threads, and inter-process data sharing usually enforces a level of data
protection that inter-thread data sharing avoids.

HTH
</off-topic>

--
Lew Pitcher

Master Codewright & JOAT-in-training | Registered Linux User #112576
http://pitcher.digitalfreehold.ca/ | GPG public key available by request
---------- Slackware - Because I know what I'm doing. ------

[s0suk3@gmail.com]

Re: Architecture design

On Oct 21, 11:30 am, Lew Pitcher <lpitcher@teksa vvy.comwrote:

On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
(alesanRemoveMe Please@manoweb. com) wrote:

Ben Bacarisse wrote:

[snip]

...and if you can't be talked out of using threads,

>

Why does everybody hates threads so much :) I often fail to see the
advantages of processes for 'small' things.

>
<off-topic>
My guess is that threads are often more bother than they are worth.
>
Given the possibility of programmed creation of processes (i.e. an api to
launch processes), threads are redundant.

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems), so even
when fork() is available, threads are not redundant. When fork() is
not available, you can only rely on creation of external processes,
which are much less flexible (e.g., there are less mechanisms for IPC)
and less efficient than threads.

Threads usually require a greater level of programming support than
processes.

Maybe, but in a well-designed application (as opposed to in a quick-
and-dirty script), that's the least of the things to worry about. You
just apply an optimal level of abstraction. Furthermore, "greater
level of programming support" often means more functionality, which
results in an increase of simplicity and flexibility in some
application areas such as IPC modeling.

Programs that use threads often suffer from poor design/implementation,

That has nothing to do with threads themselves. Maybe you've seen a
lot of such applications, but they're poor design is purely their
implementors' fault.

causing any advantage that threads offer to be lost (I'm thinking of
deadlock and race conditions, and of failure to disentangle logic).
>
There are as effective ways of data sharing for processes as there are for
threads,

When you use fork(), you do have some, but not as many as with
threads. For example, how about a synchronized, shared, global,
producer/consumer queue of data?

and inter-process data sharing usually enforces a level of data
protection that inter-thread data sharing avoids.

Sebastian

[Keith Thompson]

Re: Architecture design

s0suk3@gmail.co m writes:
[...]

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems), so even
when fork() is available, threads are not redundant.

[...]

fork() is less portable than threads? Huh?

--
Keith Thompson (The_Other_Keit h) kst-u@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Richard Heathfield]

Re: Architecture design

Keith Thompson said:

s0suk3@gmail.co m writes:
[...]

>fork() is known to be the process creation equivalent to threads
>creation, and it provides most of the flexibility that threading does.
>However, fork() is less efficient that threads, and much less portable
>than threads (since it's only supported on POSIX systems), so even
>when fork() is available, threads are not redundant.

[...]
>
fork() is less portable than threads? Huh?

Bearing in mind that Sebastian is on record as having encountered only
three (I think) installations of C ever, it is possible that *within his
own experience* his observation is accurate. All it would take would be
for fork() only to be available on one of his systems, whilst he happened
to have two systems that both supported threads in the same way.

Small sample sizes can be very misleading.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 2:07 am, Richard Heathfield <rjh@see.sig.in validwrote:

Keith Thompson said:
>

s0suk3@gmail.co m writes:
[...]

fork() is known to be the process creation equivalent to threads
creation, and it provides most of the flexibility that threading does.
However, fork() is less efficient that threads, and much less portable
than threads (since it's only supported on POSIX systems), so even
when fork() is available, threads are not redundant.

[...]

>

fork() is less portable than threads?  Huh?

>
Bearing in mind that Sebastian is on record as having encountered only
three (I think) installations of C ever, it is possible that *within his
own experience* his observation is accurate.

But I was trying to account for all platforms in general. If you think
that what I said isn't accurate outside personal experiences (which
have no practical value), stop with the word games and just say so.

Sebastian

[Richard Heathfield]

Re: Architecture design

s0suk3@gmail.co m said:

On Oct 22, 2:07 am, Richard Heathfield <rjh@see.sig.in validwrote:

>Keith Thompson said:
>>

s0suk3@gmail.co m writes:
[...]
>fork() is known to be the process creation equivalent to threads
>creation, and it provides most of the flexibility that threading
>does. However, fork() is less efficient that threads, and much less
>portable than threads (since it's only supported on POSIX systems),
>so even when fork() is available, threads are not redundant.
[...]

>>

fork() is less portable than threads? Huh?

>>
>Bearing in mind that Sebastian is on record as having encountered only
>three (I think) installations of C ever, it is possible that *within his
>own experience* his observation is accurate.

>
But I was trying to account for all platforms in general.

Then I think you failed.

If you think
that what I said isn't accurate outside personal experiences (which
have no practical value), stop with the word games and just say so.

I did say so, pretty much, and I wasn't playing word games. But I disagree
that personal experiences have no practical value. On the contrary, they
have enormous practical value. But they are not always as mappable onto
other people's experiences as we might think.

In my own personal experience, neither fork() nor threads are especially
portable, and to argue about which is *more* portable seems to me to be an
angels-on-pinheads discussion.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Richard Heathfield]

Re: Architecture design

s0suk3@gmail.co m said:

On Oct 22, 2:15 am, Richard Heathfield <rjh@see.sig.in validwrote:

>s0suk3@gmail.co m said:
>>

<snip>

But what I mean is that threads are supported on a lot of platforms
(in contrast to fork(), which is only available on POSIX systems).

>>
>The fact that threads are supported on Platform A and Platform B doesn't
>make threaded code portable between A and B *unless* the same interface
>and semantics are available on both platforms. This is by no means
>always the case.

>
And what on earth is stopping you from using a portable interface?

Partly non-availability. There isn't an interface that's portable *enough*.
And partly common sense. Threads were invented for people who can't write
state machines (which, incidentally, are far easier to debug).

In C/C++, you can use frameworks offering a single interface to the
underlying threading environment.

>>
>If you really need to use threads, that's obviously a good way to go.
>But of course most programs would be better off without them.

>
Either you need them or you don't, of course. I'm just saying that
they have significant advantages over processes in general.

That sounds like a factoid to me (like the factoid that "integer operations
are faster than floating-point operations"). And threads have significant
*dis*advantages that should not be glossed over.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 2:32 am, Richard Heathfield <rjh@see.sig.in validwrote:

s0suk3@gmail.co m said:

If you think
that what I said isn't accurate outside personal experiences (which
have no practical value), stop with the word games and just say so.

>
I did say so, pretty much, and I wasn't playing word games. But I disagree
that personal experiences have no practical value. On the contrary, they
have enormous practical value. But they are not always as mappable onto
other people's experiences as we might think.
>
In my own personal experience, neither fork() nor threads are especially
portable, and to argue about which is *more* portable seems to me to be an
angels-on-pinheads discussion.

Then you haven't used anything but C. Like I said earlier, there are
languages such as Python and Java that try to provide portable
interfaces to operating system services. On such languages, you'll be
able to write a portable threading application (without having to rely
on third-party libraries) that will work on any system where there is
an implementation of the language and that supports threads. The same
is not possible with fork(), since it's only available on POSIX
systems.

Sebastian

[Richard Heathfield]

Re: Architecture design

s0suk3@gmail.co m said:

On Oct 22, 2:32 am, Richard Heathfield <rjh@see.sig.in validwrote:

>s0suk3@gmail.co m said:

If you think
that what I said isn't accurate outside personal experiences (which
have no practical value), stop with the word games and just say so.

>>
>I did say so, pretty much, and I wasn't playing word games. But I
>disagree that personal experiences have no practical value. On the
>contrary, they have enormous practical value. But they are not always as
>mappable onto other people's experiences as we might think.
>>
>In my own personal experience, neither fork() nor threads are especially
>portable, and to argue about which is *more* portable seems to me to be
>an angels-on-pinheads discussion.

>
Then you haven't used anything but C.

Thanks for letting me know. I was under the impression that I have used and
do use a number of other languages, but clearly I was wrong about that.

Like I said earlier, there are
languages such as Python and Java that try to provide portable
interfaces to operating system services.

Yes, they try. And surprisingly often they succeed. But if you think that
Python and Java are more portable than C, you have an astoundingly limited
view of computing.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 2:36 am, Richard Heathfield <rjh@see.sig.in validwrote:

s0suk3@gmail.co m said:
>

On Oct 22, 2:15 am, Richard Heathfield <rjh@see.sig.in validwrote:

s0suk3@gmail.co m said:

>
<snip>
>

But what I mean is that threads are supported on a lot of platforms
(in contrast to fork(), which is only available on POSIX systems).

>

The fact that threads are supported on Platform A and Platform B doesn't
make threaded code portable between A and B *unless* the same interface
and semantics are available on both platforms. This is by no means
always the case.

>

And what on earth is stopping you from using a portable interface?

>
Partly non-availability. There isn't an interface that's portable *enough*.

Perhaps one of the best I've seen is Mozilla's NSPR.

And partly common sense. Threads were invented for people who can't write
state machines (which, incidentally, are far easier to debug).

Do state machines really have anything to do with parallel processing?

In C/C++, you can use frameworks offering a single interface to the
underlying threading environment.

>

If you really need to use threads, that's obviously a good way to go.
But of course most programs would be better off without them.

>

Either you need them or you don't, of course. I'm just saying that
they have significant advantages over processes in general.

>
That sounds like a factoid to me (like the factoid that "integer operations
are faster than floating-point operations"). And threads have significant
*dis*advantages that should not be glossed over.

That would sound much more convincing if you stated what disadvantages
those are, and how you can avoid them using processes or state
machines.

Sebastian

[s0suk3@gmail.com]

Re: Architecture design

On Oct 22, 3:08 am, Richard Heathfield <rjh@see.sig.in validwrote:

s0suk3@gmail.co m said:

Like I said earlier, there are
languages such as Python and Java that try to provide portable
interfaces to operating system services.

>
Yes, they try. And surprisingly often they succeed. But if you think that
Python and Java are more portable than C, you have an astoundingly limited
view of computing.

Those languages are portable in a different sense than C is. C tries
to be portable across a wide range of platforms at the cost of failing
to provide many system services that may not be available under those
platforms, which greatly reduces its usability. Java and Python try to
be portable across a wide range of platforms while still providing
interfaces to as many system services as possible, at the cost of
being unable to provide those services on systems where they're not
available, and thus creating ambiguities in the interfaces on those
systems.

Unfortunately, we don't have both of those meanings of portability in
a single language yet.

Sebastian

[Nick Keighley]

Re: Architecture design

On 21 Oct, 17:30, Lew Pitcher <lpitc...@teksa vvy.comwrote:

On October 21, 2008 11:34, in comp.lang.c, Alessio Sangalli
(alesanRemoveMe Ple...@manoweb. com) wrote:

Ben Bacarisse wrote:

...and if you can't be talked out of using threads,

>

Why does everybody hates threads so much :) I often fail to see the
advantages of processes for 'small' things.

no one said "if you don't use threads then use processes". Why not use
neither of them?

<off-topic>
My guess is that threads are often more bother than they are worth.
>
Given the possibility of programmed creation of processes (i.e. an api to
launch processes), threads are redundant.

well, no. If you *have to have parralleism" (and I'd argue the average
application has a lot less of it than some people think) then
processes
can be very heavy weight. I worked on a system that scaled badly.
It started so many processes it took *hours* to start.

In many environments, threads do not provide any execution-speed advantage
over processes (the usual "big" argument for threads is that "the
context-switch overhead for a thread is 'less' than the context-switch
overhead for a process", but in many systems, threads and processes have
the same context-switch overhead).
>
Threads usually require a greater level of programming support than
processes.
>
Programs that use threads often suffer from poor design/implementation,
causing any advantage that threads offer to be lost (I'm thinking of
deadlock and race conditions, and of failure to disentangle logic).
>
There are as effective ways of data sharing for processes as there are for
threads, and inter-process data sharing usually enforces a level of data
protection that inter-thread data sharing avoids.

and this is *all* avoided if you use neitehr threads no processes.


--
Nick Keighley