We are currently investigating whether to move the data files from our
application into python for ease of maintenance. Each data item turns
into a class definition with some class data. The python approach
looks great, but there is one feature that we'd like to have.
Currently the data files can include other data files. Typically
thats used to import standard definitions from somewhere else (rather
like #include in C - conceptually its a completely textual import).
We use them something like this
include("../../standard_defini tions")
include("../shared_definiti ons")
class A(base_from_sta ndard_definitio ns): pass
class B(A): pass
include("more_d efinitions")
The includes act much more like #include than import - all the symbols
from the file before the include() must be available to the included
file and the included file must export all its symbols back to the
parent.
These can of course be re-arranged to work in a pythonic way using
'from x import *' and putting "../.." on sys.path instead of the
includes. However there are over 500 of these files so I'd prefer a
more automatic solution which doesn't require re-arrangement in the
interim. (The re-arrangement is needed because "more_definitio ns"
above might refer to A, B or anything defined in
standard/shared_definiti ons leading to mutually recursive imports and
all the pain they cause)
I have implemented a working prototype, but it seems such a horrendous
bodge that there must surely be a better way! I'd really like to be
able to run an __import__ in the context of the file thats running the
include() but I haven't figured that out.
Here is the code (avert your eyes if you are of a sensitive nature ;-)
Any suggestions for improvement would be greatly appreciated!
def include(path):
# Add the include directory onto sys.path
native_path = path.replace("/", os.path.sep)
directory, module_name = os.path.split(n ative_path)
if module_name.end swith(".py"):
module_name = module_name[:-3]
old_sys_path = sys.path
if directory != "":
sys.path.insert (0, directory)
# Introspect to find the parent
# Each record contains a frame object, filename, line number, function
# name, a list of lines of context, and index within the context.
up = inspect.stack()[1]
frame = up[0]
parent_name = frame.f_globals['__name__']
parent = sys.modules[parent_name]
# Poke all the current definitions into __builtin__ so the module
# uses them without having to import them
old_builtin = __builtin__.__d ict__.copy()
overridden = {}
poked = []
for name in dir(parent):
if not (name.startswit h("__") and name.endswith(" __")):
if hasattr(__built in__, name):
overridden[name] = getattr(__built in__, name)
else:
poked.append(na me)
setattr(__built in__, name, getattr(parent, name))
# import the code
module = __import__(modu le_name, parent.__dict__ , locals(), [])
# Undo the modifications to __builtin__
for name in poked:
delattr(__built in__, name)
for name, value in overridden.item s():
setattr(__built in__, name, value)
# check we did it right! Note __builtin__.__d ict__ is read only so
# can't be over-written
if old_builtin != __builtin__.__d ict__:
raise AssertionError( "Failed to restore __builtin__ properly")
# Poke the symbols from the import back in
for name in dir(module):
if not (name.startswit h("__") and name.endswith(" __")):
setattr(parent, name, getattr(module, name))
# Restore sys.path
sys.path = old_sys_path
--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick
application into python for ease of maintenance. Each data item turns
into a class definition with some class data. The python approach
looks great, but there is one feature that we'd like to have.
Currently the data files can include other data files. Typically
thats used to import standard definitions from somewhere else (rather
like #include in C - conceptually its a completely textual import).
We use them something like this
include("../../standard_defini tions")
include("../shared_definiti ons")
class A(base_from_sta ndard_definitio ns): pass
class B(A): pass
include("more_d efinitions")
The includes act much more like #include than import - all the symbols
from the file before the include() must be available to the included
file and the included file must export all its symbols back to the
parent.
These can of course be re-arranged to work in a pythonic way using
'from x import *' and putting "../.." on sys.path instead of the
includes. However there are over 500 of these files so I'd prefer a
more automatic solution which doesn't require re-arrangement in the
interim. (The re-arrangement is needed because "more_definitio ns"
above might refer to A, B or anything defined in
standard/shared_definiti ons leading to mutually recursive imports and
all the pain they cause)
I have implemented a working prototype, but it seems such a horrendous
bodge that there must surely be a better way! I'd really like to be
able to run an __import__ in the context of the file thats running the
include() but I haven't figured that out.
Here is the code (avert your eyes if you are of a sensitive nature ;-)
Any suggestions for improvement would be greatly appreciated!
def include(path):
# Add the include directory onto sys.path
native_path = path.replace("/", os.path.sep)
directory, module_name = os.path.split(n ative_path)
if module_name.end swith(".py"):
module_name = module_name[:-3]
old_sys_path = sys.path
if directory != "":
sys.path.insert (0, directory)
# Introspect to find the parent
# Each record contains a frame object, filename, line number, function
# name, a list of lines of context, and index within the context.
up = inspect.stack()[1]
frame = up[0]
parent_name = frame.f_globals['__name__']
parent = sys.modules[parent_name]
# Poke all the current definitions into __builtin__ so the module
# uses them without having to import them
old_builtin = __builtin__.__d ict__.copy()
overridden = {}
poked = []
for name in dir(parent):
if not (name.startswit h("__") and name.endswith(" __")):
if hasattr(__built in__, name):
overridden[name] = getattr(__built in__, name)
else:
poked.append(na me)
setattr(__built in__, name, getattr(parent, name))
# import the code
module = __import__(modu le_name, parent.__dict__ , locals(), [])
# Undo the modifications to __builtin__
for name in poked:
delattr(__built in__, name)
for name, value in overridden.item s():
setattr(__built in__, name, value)
# check we did it right! Note __builtin__.__d ict__ is read only so
# can't be over-written
if old_builtin != __builtin__.__d ict__:
raise AssertionError( "Failed to restore __builtin__ properly")
# Poke the symbols from the import back in
for name in dir(module):
if not (name.startswit h("__") and name.endswith(" __")):
setattr(parent, name, getattr(module, name))
# Restore sys.path
sys.path = old_sys_path
--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick
Comment