software design question

**Josiah Carlson** · Jul 18 '05, 08:15 AM

Re: software design question

> I have the following inter-class relationships:[color=blue]
>
> __main__: (in file LMCMain.py)
> imports module FileIO
> defines class LMCMain
> instanciats main = LMCMain(...)
>
> FileIO.py:
> defines class FileIO
> class FileIO needs to access "main" from LMCMain.py[/color]

I don't know what one /should/ call it, but what you have is what I
would call a circular import. You are attempting to have two modules
use the internals of each other. That is poor design.

You can use a kludge to get past poor design by using the following:

main = LMCMain()
import sys
sys.modules['FileIO'].main = main

In the future, better design is suggested.

- Josiah

**Uwe Mayer** · Jul 18 '05, 08:15 AM

Re: software design question

Josiah Carlson wrote:
[color=blue]
> I don't know what one /should/ call it, but what you have is what I
> would call a circular import. You are attempting to have two modules
> use the internals of each other. That is poor design.[/color]

[...kludge...][color=blue]
> In the future, better design is suggested.[/color]

Ok, the following is the case:
I got a GUI. This is the base class (automatically generated) from which i
derive a class that does the implementation. However, since that may become
rather big I devided the functionality into different modules, i.e. one for
presentation, one for file-IO, one for help-topic related stuff, etc.
So this main file imports the modules that implement the functionality.
Obviously these modules need to access things like "enable action"s, install
hooks, etc. from the "main part".
And thats why the circular dependency.

I know that circular dependencies are rather bad than helpful, but the only
way to circumvent this is to build everything into one big block - which I
find is rather worse: the source file grows much too large to maintain a
good overview.

So the question is rather: how to make a better design?

Ciao
Uwe

**Josiah Carlson** · Jul 18 '05, 08:15 AM

Re: software design question

> So the question is rather: how to make a better design?

import module1
....

class main:
def __init__(self, args...):
self.c1 = module1.class1( args...)
#where args... is the standard initialization for your class,
# and any additional objects/methods that c1 needs
# access to.

Pass what is needed. If you can't pass what is needed when external
module classes are initialized, then set the attribute later.

c1instance.attr ibute = value

- Josiah

**Pierre Rouleau** · Jul 18 '05, 08:15 AM

Re: software design question

>[color=blue]
> Ok, the following is the case:
> I got a GUI. This is the base class (automatically generated) from which i
> derive a class that does the implementation. However, since that may become
> rather big I devided the functionality into different modules, i.e. one for
> presentation, one for file-IO, one for help-topic related stuff, etc.
> So this main file imports the modules that implement the functionality.
> Obviously these modules need to access things like "enable action"s, install
> hooks, etc. from the "main part".
> And thats why the circular dependency.
>
> I know that circular dependencies are rather bad than helpful, but the only
> way to circumvent this is to build everything into one big block - which I
> find is rather worse: the source file grows much too large to maintain a
> good overview.
>
> So the question is rather: how to make a better design?
>[/color]

Why not look at the GUI as a "presentati on layer" that provides access
to the program functionality, like file I/O, help access, etc. Then the
file I/O, help, (etc...) can be written without any knowledge to the GUI
(and therefore testable from a python shell or a testing script) and the
Gui code just call the File I/O, help functions or members. If you need
callbacks, then provide callbacks. This way you would decouple the
GUI from the system functionality. Only GUI code would import the other
modules. The File I/O, help (etc...) would not know anything about the
GUI implementation requirements.

Hope that helps,

Pierre

**Uwe Mayer** · Jul 18 '05, 08:15 AM

Re: software design question

Pierre Rouleau wrote:[color=blue][color=green]
>> So the question is rather: how to make a better design?[/color]
>
> Why not look at the GUI as a "presentati on layer" that provides access
> to the program functionality, like file I/O, help access, etc. Then the
> file I/O, help, (etc...) can be written without any knowledge to the GUI
> (and therefore testable from a python shell or a testing script) and the
> Gui code just call the File I/O, help functions or members. If you need
> callbacks, then provide callbacks. This way you would decouple the
> GUI from the system functionality. Only GUI code would import the other
> modules. The File I/O, help (etc...) would not know anything about the
> GUI implementation requirements.[/color]

I've already got separate modules for non-GUI related functionalities , i.e.
accessing record based file formats, etc. These are imported by the module
that currently covers "Open", "Save", "Save As", "Properties ", "New" (in
the File menu).
This alone spans 442 lines (including comments and empty lines), containing
only dialog interaction and calls to the previously mentioned file-module.

Is 500 lines a good count to start splitting up the code?

Currently the "main" part is just a main widget with all its toolbars, menu-
and status bar. This part imports all other modules that then cover a small
portion of the programs functionality, i.e. one module covers all file
open, save, close related stuff, another may cover all help related
actions, another covers the presentation, modification and propagation of
the rather large collection of preferences, etc.

My current solution is thinking along the line that all these components
usually exist only once in the system, i.e. there's no reason why there
should be two objects covering the same preferences dialog.
So each component implements the list or dictionary interface by which its
data can can be accessed from other components. For example after opening a
file an access to the fileIO component like: fileIO[0:10]
would return the first 10 records from the file.
Similarly something like preferences['fileIO'] might return all preferences
for the fileIO component, etc.
Of course the component that displays the record (presentation layer) will
need to be notified of a new file that has been loaded. To realise this I
have implemented a "callback" mechanisme similar to the Qt's SIGNAL-SLOT
mechanisme: the fileIO component registers signals "fileNew", "fileOpen",
etc. under the name 'fileIO'. And when a component wants to be notified
when a new file is opened it sort of 'fileIO.registe rCallback("file New()",
self.fileNew_ca llback)'.
And then the fileIO module calls all callbacks in the list of "fileNew()"
when a new file has been opened.

Is this approach getting to complicated?

One problem I'm still having with this is: for component B to register a
callback at component A, B still got to have a reference to component A in
order to make a call like "A.registerCall back(...)"
You might not know what B needs to access, so you cannot pass "what is
needed" in the constructor, nor let the code that instanciates B set
attributes.

Ciao
Uwe

**Uwe Mayer** · Jul 18 '05, 08:15 AM

Re: software design question

Josiah Carlson wrote:
[color=blue][color=green]
>> So the question is rather: how to make a better design?[/color][/color]
[...][color=blue]
> Pass what is needed. If you can't pass what is needed when external
> module classes are initialized, then set the attribute later.
>
> c1instance.attr ibute = value[/color]

Ok, that is a way of enabling the circular dependency, but that doesn't
really get rid of it. The main program still needs the module and the
module may now access parts from the main program using the passed
parameters or the later set attributes.

That was my first approach, passing what's needed. But if module1 delegates
some task to another module, then again all args must be passed to that
module and so forth.
Now currently I've got about 7 such modules that may need to interact then I
end up in passing to each level of delegation an increasingly _huge_
parameter list and that's where I stopped thinking in that direction.

My current solution to this is to pass the instances that may need to be
publicly known as value in a key:value pair, stored in a dictionary. Thus
something like:
[color=blue][color=green][color=darkred]
>>> import singleton
>>> single = singleton.Singl eton()
>>> single['main'][/color][/color][/color]

returns the appropriate instance.

Ciao
Uwe

**John Roth** · Jul 18 '05, 08:15 AM

Re: software design question

"Uwe Mayer" <merkosh@hadiko .de> wrote in message
news:c03rii$c2t $1@news.rz.uni-karlsruhe.de...[color=blue]
>
> One problem I'm still having with this is: for component B to register a
> callback at component A, B still got to have a reference to component A in
> order to make a call like "A.registerCall back(...)"
> You might not know what B needs to access, so you cannot pass "what is
> needed" in the constructor, nor let the code that instanciates B set
> attributes.[/color]

I suspect you've got the callback logic backwards. The basic
layering pattern is "call down, notify up," and it's the upper layer's
responsibility to request the notification.

To explicate: the GUI module calls the domain
module for service. It also calls the domain module
to register callbacks, passing the function that it
want's called.

When the domain layer has something it wants to
tell the GUI layer, it simply runs a (possibly empty)
list of callback functions.

John Roth
[color=blue]
>
> Ciao
> Uwe[/color]

**Josiah Carlson** · Jul 18 '05, 08:15 AM

Re: software design question

From the sounds of your posts, you want your software to be "modular",
in that you want have one file for each of the different components, but
you don't want to spend the time to pass down all the proper references.
As a result, you want to have a single global namespace for all of
your modules, so that you don't have to pass anything down.

There is another kludge:
#main.py
import x
import y

#make a copy of the dictionaries so that
xold = dict(x.__dict__ )
yold = dict(y.__dict__ )
x.__dict__.upda te(yold)
y.__dict__.upda te(xold)

x.xrunme()
y.yrunme()

#x.py
def xrunme():
print "calling yrunyou"
yrunyou()

def xrunyou():
print "called xrunyou"

#y.py
def yrunme():
print "calling xrunyou"
xrunyou()

def yrunyou():
print "called yrunyou"

In the above, you don't even need to import the sibling modules.
Believe it or not, it works.

I would suggest you just put everything into a single file. 500 lines
is not that large. Heck, a few thousand isn't even that bad, as long as
you pay attention to what you are doing and use an editor with a
browsable source tree/class heirarchy, and comment liberally.

- Josiah

P.S. For more stuff on this topic, check the thread with subject "Basic
'import' problem", which has the same subjects pop up.

**Uwe Mayer** · Jul 18 '05, 08:16 AM

Re: software design question

Josiah Carlson wrote:[color=blue]
> From the sounds of your posts, you want your software to be "modular",
> in that you want have one file for each of the different components, but
> you don't want to spend the time to pass down all the proper references.
> As a result, you want to have a single global namespace for all of
> your modules, so that you don't have to pass anything down.[/color]

Almost. I want to write software from components. I want a simple, yet clear
and effective interface for the components to communicate with eachother.
The components will be dynamically loaded. If available they have to fit
like a piece in a puzzle, if ther're unavailable all related functionality
will be unavailable - *not* raise exceptions or cause program abortion,
etc.

I simply cannot pass all components to all others, because I don't yet know
which components there might be.

Having a single global name space is poor software design. It makes you not
having to think about "inter module communication", because there is none
to take care of, because you can call virtually everything from everywhere.
[color=blue]
> There is another kludge:[/color]
[...][color=blue]
> I would suggest you just put everything into a single file. 500 lines
> is not that large. Heck, a few thousand isn't even that bad, as long as
> you pay attention to what you are doing and use an editor with a
> browsable source tree/class heirarchy, and comment liberally.[/color]

Wow! You can't seriously suggest this as a software design recommendation!

Importing the complete namespace from one module into another is exactly
what will cause you fourty hours of debugging code afterwards.

Do you think there is a reason why in Java every class got to be put into
one source file - or is this just a bad language restriction?
Don't get me wrong, I'm not critisizing Python here.

Searching the litherature I found the following on how to model interaction
between modules:

- relations between modules should be hierachial, i.e. non-cyclic

problem with circular dependencies: "nothing works until everything works"

benefits of non circular design:
1. development and testing step by step is possible,
2. you can build subsets for
- use in other systems
- own re-use

*sandwiching*:

cyclic dependencies may be resolved by refining the modular structure:

A <==> B

Suggestion 1:

A1 ==> B ==> A2
A1 ========> A2

Suggestion 2:

A2 ==> B2 ==> A1 ==> B1
A2 =========> A1 B1
B2 =========> B1

Ciao
Uwe

**John Roth** · Jul 18 '05, 08:16 AM

Re: software design question

"Uwe Mayer" <merkosh@hadiko .de> wrote in message
news:c057uq$4ph $1@news.rz.uni-karlsruhe.de...[color=blue]
> Josiah Carlson wrote:[color=green]
> > From the sounds of your posts, you want your software to be "modular",
> > in that you want have one file for each of the different components, but
> > you don't want to spend the time to pass down all the proper references.
> > As a result, you want to have a single global namespace for all of
> > your modules, so that you don't have to pass anything down.[/color]
>
> Almost. I want to write software from components. I want a simple, yet[/color]
clear[color=blue]
> and effective interface for the components to communicate with eachother.
> The components will be dynamically loaded. If available they have to fit
> like a piece in a puzzle, if ther're unavailable all related functionality
> will be unavailable - *not* raise exceptions or cause program abortion,
> etc.
>
> I simply cannot pass all components to all others, because I don't yet[/color]
know[color=blue]
> which components there might be.
>
> Having a single global name space is poor software design. It makes you[/color]
not[color=blue]
> having to think about "inter module communication", because there is none
> to take care of, because you can call virtually everything from[/color]
everywhere.[color=blue]
>[color=green]
> > There is another kludge:[/color]
> [...][color=green]
> > I would suggest you just put everything into a single file. 500 lines
> > is not that large. Heck, a few thousand isn't even that bad, as long as
> > you pay attention to what you are doing and use an editor with a
> > browsable source tree/class heirarchy, and comment liberally.[/color]
>
> Wow! You can't seriously suggest this as a software design recommendation!
>
> Importing the complete namespace from one module into another is exactly
> what will cause you fourty hours of debugging code afterwards.
>
> Do you think there is a reason why in Java every class got to be put into
> one source file - or is this just a bad language restriction?
> Don't get me wrong, I'm not critisizing Python here.
>
> Searching the litherature I found the following on how to model[/color]
interaction[color=blue]
> between modules:
>
> - relations between modules should be hierachial, i.e. non-cyclic
>
> problem with circular dependencies: "nothing works until everything works"
>
> benefits of non circular design:
> 1. development and testing step by step is possible,
> 2. you can build subsets for
> - use in other systems
> - own re-use
>
> *sandwiching*:
>
> cyclic dependencies may be resolved by refining the modular structure:
>
> A <==> B
>
> Suggestion 1:
>
> A1 ==> B ==> A2
> A1 ========> A2
>
> Suggestion 2:
>
> A2 ==> B2 ==> A1 ==> B1
> A2 =========> A1 B1
> B2 =========> B1[/color]

As I suggested in another post, this doesn't always work in practice.
Another term for hierarchical design is layered architecture. The
rule in a layered architecture is "call down, notify up." It's the higher
layer's responsibility to call down to install callbacks so that the
lower layer can notify up at run time.

What makes this work is that the notifier on the lower layer
can't depend on there being any callbacks installed. In other
words, it should be able to do whatever it does if nobody
is listening.

John Roth[color=blue]
>
> Ciao
> Uwe[/color]

**Lothar Scholz** · Jul 18 '05, 08:16 AM

Re: software design question

Uwe Mayer <merkosh@hadiko .de> wrote in message news:<c057uq$4p h$1@news.rz.uni-karlsruhe.de>.. .
[color=blue]
> Almost. I want to write software from components.[/color]

Components must try to reduce their dependencies. So your design may
not be the best one.
[color=blue]
> Having a single global name space is poor software design.[/color]

No absolutely not. There are Programs with millions of lines written
in C that are good designed and reliable software.
Having a common prefix is nothing else then using namespaces.
[color=blue]
> Do you think there is a reason why in Java every class got to be put into
> one source file - or is this just a bad language restriction?[/color]

This has to do with dynamic loading. Nothing else.

But maybe more constructive:

In my software (Eiffel) i have a CORE_MANAGER class that has objects
like EDITOR, COMPILER etc. This are facade patterns to the
subsystems. I also have EDITOR_BRIDGE, COMPILER_BRIDGE classes. The
editor provides its functionality to the outside world only with this
class and can use only the services found in the EDITOR_BRIDGE object.

The CORE_MANAGER class now sets up the editor or some editor mock
object if the editor is not used.

This requires quite a lot of glue code but i'm now able to switch
independent subsystems on and off. So i could split my 250.000 lines
program into 8 optional subsystems.

**Lothar Scholz** · Jul 18 '05, 08:16 AM

Re: software design question

Uwe Mayer <merkosh@hadiko .de> wrote in message news:<c03rii$c2 t$1@news.rz.uni-karlsruhe.de>.. .[color=blue]
> Pierre Rouleau wrote:[color=green][color=darkred]
> >> So the question is rather: how to make a better design?[/color]
> >
> > Why not look at the GUI as a "presentati on layer" that provides access
> > to the program functionality, like file I/O, help access, etc. Then the
> > file I/O, help, (etc...) can be written without any knowledge to the GUI
> > (and therefore testable from a python shell or a testing script) and the
> > Gui code just call the File I/O, help functions or members. If you need
> > callbacks, then provide callbacks. This way you would decouple the
> > GUI from the system functionality. Only GUI code would import the other
> > modules. The File I/O, help (etc...) would not know anything about the
> > GUI implementation requirements.[/color]
>
> I've already got separate modules for non-GUI related functionalities , i.e.
> accessing record based file formats, etc. These are imported by the module
> that currently covers "Open", "Save", "Save As", "Properties ", "New" (in
> the File menu).
> This alone spans 442 lines (including comments and empty lines), containing
> only dialog interaction and calls to the previously mentioned file-module.
>
> Is 500 lines a good count to start splitting up the code?[/color]

No !
My modules are normally between 1000-3000 lines. Having to much files
(worst case scenario is PHP) is just another form of spaghetti code.
With so small files you should focus on the task that needs to be done
and only if you really do two different things in one module you
should decide to split it. Maybe a merge would help you more then
splitting.

**Josiah Carlson** · Jul 18 '05, 08:17 AM

Re: software design question

> Almost. I want to write software from components. I want a simple, yet clear[color=blue]
> and effective interface for the components to communicate with eachother.
> The components will be dynamically loaded. If available they have to fit
> like a piece in a puzzle, if ther're unavailable all related functionality
> will be unavailable - *not* raise exceptions or cause program abortion,
> etc.[/color]

Design your interface, and write your program around your interface.
Here's one:

#main
import submodule

class data_storehouse :
pass

passer = data_storehouse ()
passer.error = None
passer.main = main
#etc.

def caller(*args, **kwargs):
passer.args = args
passer.kwargs = kwargs

submodule(passe r)
if passer.error:
#handle the error

This is essentially another user was doing with their "Singleton( )"
interface, though they created a bottom-level module to hold all
application-wide information.

[color=blue]
> I simply cannot pass all components to all others, because I don't yet know
> which components there might be.[/color]

So you only pass the ones you know about when writing your application.
When functionality A gets created, before component B, likely
functionality A won't require component B. If you discover later that
it does, then you modify functionality A to use component B. You can't
magically get away from this.

[color=blue]
> Having a single global name space is poor software design. It makes you not
> having to think about "inter module communication", because there is none
> to take care of, because you can call virtually everything from everywhere.[/color]

Of course it is. It also doesn't scale very well, though Python
namespaces (dictionaries) don't seem to have issues with pollution, it
can get unweidly to handle.

[color=blue][color=green]
>>There is another kludge:[/color]
>
> [...]
>[color=green]
>>I would suggest you just put everything into a single file. 500 lines
>>is not that large. Heck, a few thousand isn't even that bad, as long as
>>you pay attention to what you are doing and use an editor with a
>>browsable source tree/class heirarchy, and comment liberally.[/color]
>
> Wow! You can't seriously suggest this as a software design recommendation![/color]

The kludge, no. That's why I called it a kludge. You can propagate
namespaces without a problem, and you don't even have to worry about the
Python import mechanisms. It ends up with the same functionality as
having a single module, without having a single module.

[color=blue]
> Importing the complete namespace from one module into another is exactly
> what will cause you fourty hours of debugging code afterwards.[/color]

I never said it was a good idea.

[color=blue]
> Do you think there is a reason why in Java every class got to be put into
> one source file - or is this just a bad language restriction?
> Don't get me wrong, I'm not critisizing Python here.[/color]

I don't know, I never thought about it, because I never learned Java. I
was in the last year they taught C/C++.

Thinking about it now, I think it is a poor language restriction. Being
able to have a half-dozen classes with related functionality in a single
module is /very/ convenient. Check out the threading or itertools
modules. Imagine having to use threading.Threa d.Thread or
itertools.imap. imap, those would be annoying and useless bubbles.
[color=blue]
> A <==> B
>
> Suggestion 1:
>
> A1 ==> B ==> A2
> A1 ========> A2
>
> Suggestion 2:
>
> A2 ==> B2 ==> A1 ==> B1
> A2 =========> A1 B1
> B2 =========> B1[/color]

You can still end up SOL if A1 requires functionality from A2. You
still have a circular dependency. As the sibling reply by John Roth says,
"this doesn't always work in practice. Another term for hierarchical
design is layered architecture. The rule in a layered architecture is
'call down, notify up.' It's the higherlayer's esponsibility to call
down to install callbacks so that the lower layer can notify up at
run time."

John says the truth.
- Josiah

**Jorge Godoy** · Jul 18 '05, 08:17 AM

Re: software design question

On Sunday 08 February 2004 01:24 John Roth wrote in
<102bb30d5422pd 4@news.supernew s.com>:
[color=blue]
> I suspect you've got the callback logic backwards. The basic
> layering pattern is "call down, notify up," and it's the upper
> layer's responsibility to request the notification.
>
> To explicate: the GUI module calls the domain
> module for service. It also calls the domain module
> to register callbacks, passing the function that it
> want's called.
>
> When the domain layer has something it wants to
> tell the GUI layer, it simply runs a (possibly empty)
> list of callback functions.[/color]

Can you exemplify it?

I'm very interested on such approach and I'm going to read a little
more about it, but examples in Python would help a lot.

TIA,
--
Godoy. <godoy@ieee.org >

software design question

software design question

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment