how to overload sqrt in a module?

[Michael McNeil Forbes]

I would like to write a module that provides some mathematical functions
on top of those defined in math, cmath etc. but I would like to make it
work with "any" type that overloads the math functions.

Thus, I would like to write:

module_f.py
----
def f(x):
""" Compute sqrt of x """
return sqrt(x)
[color=blue][color=green][color=darkred]
>>> from math import *
>>> import module_f
>>> module_f.f(3)[/color][/color][/color]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "module_f.p y", line 2, in f
return sqrt(x)
NameError: global name 'sqrt' is not defined

I understand why sqrt is not in scope, but my question is: what is the
best way to do this?

Here is one fairly ugly solution:

module_g.py
-----------
def g(x,math):
return math.sqrt(x)
[color=blue][color=green][color=darkred]
>>> import math, cmath, module_g
>>> module_g.g(2,ma th)[/color][/color][/color]
1.4142135623730 951[color=blue][color=green][color=darkred]
>>> module_g.g(-2,cmath)[/color][/color][/color]
1.4142135623730 951j

I am sure there is a better way of doing this that makes use of the
type of the argument (Dynamical scoping would solve the
problem but we don't want to go there...). Note that the following
function would work fine

def f(x):
return abs(x)

because of the special member __abs__ attached to the type. There is no
corresponding member for sqrt, sin etc.

What is the "proper" pythonic way to do this?

Michael.

[Kent Johnson]

Re: how to overload sqrt in a module?

Michael McNeil Forbes wrote:[color=blue]
> I would like to write a module that provides some mathematical functions
> on top of those defined in math, cmath etc. but I would like to make it
> work with "any" type that overloads the math functions.
>
> Thus, I would like to write:
>
> module_f.py
> ----
> def f(x):
> """ Compute sqrt of x """
> return sqrt(x)
>[color=green][color=darkred]
>>>> from math import *
>>>> import module_f
>>>> module_f.f(3)[/color][/color]
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> File "module_f.p y", line 2, in f
> return sqrt(x)
> NameError: global name 'sqrt' is not defined
>
> I understand why sqrt is not in scope, but my question is: what is the
> best way to do this?[/color]

You need to import math in the module that uses it. Imports in one
module don't (in general) affect the variables available in another module.

module_f.py
----
from math import sqrt # more explicit than from math import *
def f(x):
""" Compute sqrt of x """
return sqrt(x)

Kent

[mforbes@lns.mit.edu]

Re: how to overload sqrt in a module?

>> I would like to write a module that provides some mathematical functions on[color=blue][color=green]
>> top of those defined in math, cmath etc. but I would like to make it work
>> with "any" type that overloads the math functions.
>>
>> Thus, I would like to write:
>>
>> module_f.py
>> ----
>> def f(x):
>> """ Compute sqrt of x """
>> return sqrt(x)
>>[color=darkred]
>>>>> from math import *
>>>>> import module_f
>>>>> module_f.f(3)[/color]
>>
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in ?
>> File "module_f.p y", line 2, in f
>> return sqrt(x)
>> NameError: global name 'sqrt' is not defined
>>
>> I understand why sqrt is not in scope, but my question is: what is the best
>> way to do this?[/color]
>
> You need to import math in the module that uses it. Imports in one module
> don't (in general) affect the variables available in another module.
>
> module_f.py
> ----
> from math import sqrt # more explicit than from math import *
> def f(x):
> """ Compute sqrt of x """
> return sqrt(x)
>
> Kent[/color]

I know, but the point is that I want to allow the user of the module to be
able to specify which sqrt function should be used depending on the type
of x. Importing math in the module would prevent the user from using f()
with complex types (or dimensioned types for another example).

That is why I suggested the ugly, but functional solution

module_g.py
-----------
def g(x,a_math):
"""Compute sqrt of x using a_math.sqrt)
return a_math.sqrt(x)

The user can at least specify to use cmath instead of math:
[color=blue][color=green][color=darkred]
>>> import cmath, module_g
>>> module_g.g(8j,c math)[/color][/color][/color]
(2+2j)

Michael.

[Robert Kern]

Re: how to overload sqrt in a module?

Michael McNeil Forbes wrote:[color=blue]
> Here is one fairly ugly solution:
>
> module_g.py
> -----------
> def g(x,math):
> return math.sqrt(x)
>
>[color=green][color=darkred]
>>>>import math, cmath, module_g
>>>>module_g.g( 2,math)[/color][/color]
>
> 1.4142135623730 951
>[color=green][color=darkred]
>>>>module_g. g(-2,cmath)[/color][/color]
>
> 1.4142135623730 951j
>
> I am sure there is a better way of doing this that makes use of the
> type of the argument (Dynamical scoping would solve the
> problem but we don't want to go there...). Note that the following
> function would work fine
>
> def f(x):
> return abs(x)
>
> because of the special member __abs__ attached to the type. There is no
> corresponding member for sqrt, sin etc.
>
> What is the "proper" pythonic way to do this?[/color]

Use the appropriate library which already implements the features you want.

In [8]: class A(object):
...: def __init__(self, x):
...: self.x = x
...: def sqrt(self):
...: return 2*self.x
...:
...:

In [9]: a = A(10)

In [10]: import numpy

In [11]: numpy.sqrt(a)
Out[11]: 20

In [12]: numpy.sqrt(10)
Out[12]: 3.1622776601683 795

In [13]: numpy.sqrt(10j)
Out[13]: (2.236067977499 7898+2.23606797 74997898j)

--
Robert Kern
robert.kern@gma il.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Michael McNeil Forbes]

Re: how to overload sqrt in a module?

> Michael McNeil Forbes wrote:[color=blue][color=green]
>> Here is one fairly ugly solution:
>>
>> module_g.py
>> -----------
>> def g(x,math):
>> return math.sqrt(x)
>>
>>[color=darkred]
>>>>> import math, cmath, module_g
>>>>> module_g.g(2,ma th)[/color]
>>
>> 1.4142135623730 951
>>[color=darkred]
>>>>> module_g.g(-2,cmath)[/color]
>>
>> 1.4142135623730 951j
>>
>> What is the "proper" pythonic way to do this?[/color]
>[/color]
On Thu, 2 Mar 2006, Robert Kern wrote:[color=blue]
>
> Use the appropriate library which already implements the features you want.
>
> In [8]: class A(object):
> ...: def __init__(self, x):
> ...: self.x = x
> ...: def sqrt(self):
> ...: return 2*self.x
> ...:
> ...:
>
> In [9]: a = A(10)
>
> In [10]: import numpy
>
> In [11]: numpy.sqrt(a)
> Out[11]: 20
>
> In [12]: numpy.sqrt(10)
> Out[12]: 3.1622776601683 795
>
> In [13]: numpy.sqrt(10j)
> Out[13]: (2.236067977499 7898+2.23606797 74997898j)
>
> --
> Robert Kern[/color]

Thanks. This solution is essentially to make sure that all "numeric"
objects have .sqrt etc. members, and then use a library which check
against builtin types and custom types with these extensions.

That works, as long as the appropriate library exists (or I write it
myself) (and this works in my case: I did not realize that the numpy/scipy
functions worked this way.)

Are there any other more generic solutions (without having to write
"custom" functions like numpy.sqrt)?

Michael

[Robert Kern]

Re: how to overload sqrt in a module?

Michael McNeil Forbes wrote:
[color=blue]
> Are there any other more generic solutions (without having to write
> "custom" functions like numpy.sqrt)?[/color]

No. *Someone* always has to write those "custom" functions. In the case of len()
and abs(), that someone was Guido. For numpy.sqrt(), that someone was Travis
Oliphant. For anything else you want, that someone is probably you.

--
Robert Kern
robert.kern@gma il.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Michael McNeil Forbes]

Re: how to overload sqrt in a module?

>> Are there any other more generic solutions (without having to write[color=blue][color=green]
>> "custom" functions like numpy.sqrt)?[/color]
>
> No. *Someone* always has to write those "custom" functions. In the case of len()
> and abs(), that someone was Guido. For numpy.sqrt(), that someone was Travis
> Oliphant. For anything else you want, that someone is probably you.[/color]

Thanks Guido! Thanks Travis:-)

What I was thinking was: is there some way to pass a parameter to the
module on import so that the module can then use the correct environment.

If I could pass a dictionary to the module with all of the overloaded
functions, then they could become part of the module's environment and
used appropriately. I could not find a way of passing such a dictionary.

Michael.

[Michael McNeil Forbes]

Re: how to overload sqrt in a module?

> On Thu, 2 Mar 2006 16:12:53 -0800, Michael McNeil Forbes wrote:[color=blue]
>[color=green]
>> I would like to write a module that provides some mathematical functions
>> on top of those defined in math, cmath etc. but I would like to make it
>> work with "any" type that overloads the math functions.
>>[/color][/color]
On Fri, 3 Mar 2006, Dennis Lee Bieber wrote:[color=blue]
> Since Python doesn't dispatch based on parameter types, there are
> only two choices that I can see...
>
> Either the CALLER already knows what the arguments are, and has
> coded an explicit call to the version of the function desired...[/color]

The caller does know in my case, but wants to reuse old code that performs
a complicated computation using the standard math functions. I was
thinking that maybe there was some way for the caller to pass the
appropriate functions to the module on import, but could not find a way of
doing this.
[color=blue]
> OR, the caller imports and calls your special module, and your
> special module has to import ALL possible override modules and supply a
> function that determines which of the modules is to be used...[/color]

This is essentially what numpy.sqrt() etc. does as pointed out by Robert
Kern. The numpy.sqrt() seems to check if the argument is real, complex,
or some other type and calls either math.sqrt, cmath.sqrt or x.sqrt()
appropriately. This works for my problem: I just wondered if there was a
more generic solution without having to do this work.

Thanks,
Michael.

[david mugnai]

Re: how to overload sqrt in a module?

On Thu, 02 Mar 2006 19:24:48 -0800, mforbes wrote:

[snip]
[color=blue]
> I know, but the point is that I want to allow the user of the module to be
> able to specify which sqrt function should be used depending on the type
> of x. Importing math in the module would prevent the user from using f()
> with complex types (or dimensioned types for another example).[/color]

If I don't misunderstood the problem, you can define an "init" method for
your module_g

(code not tested)

module_g.py
-----------

_functions = {}
def init(mathmodule ):
_function['sqrt'] = getattr(mathmod ule, 'sqrt', None)

def _get(name):
try:
return _functions[name]
except KeyError:
raise TypeError("you have to initialize module_g")

def sqrt(x):
return _get('sqrt')(x)

main.py
-------

import math
import module_g

module_g.init(m ath)
print module_g.sqrt(2 )


HTH

[Steven D'Aprano]

Re: how to overload sqrt in a module?

On Thu, 02 Mar 2006 22:35:51 -0800, Michael McNeil Forbes wrote:
[color=blue]
> What I was thinking was: is there some way to pass a parameter to the
> module on import so that the module can then use the correct environment.
>
> If I could pass a dictionary to the module with all of the overloaded
> functions, then they could become part of the module's environment and
> used appropriately. I could not find a way of passing such a dictionary.[/color]

Not on import, but you can do this:


import my_module
my_module.set_e nvironment("mat h") # or cmath, or numeric, or whatever

Your my_module will be like this:

# Warning: untested code.
ENVIRON = None # global variables sometimes have their uses

def f(x):
if ENVIRON is None:
raise ValueError("Uni nitialised module!")
# or use a custom exception
return ENVIRON.sqrt(x)

def set_environment (name):
global ENVIRON
ENVIRON = __import__(name )


Does this help?


--
Steven.

[Michael McNeil Forbes]

Re: how to overload sqrt in a module?

On Fri, 3 Mar 2006, david mugnai wrote:

[snip][color=blue]
>
> If I don't misunderstood the problem, you can define an "init" method for
> your module_g
>
> (code not tested)
>
> module_g.py
> -----------
>
> _functions = {}
> def init(mathmodule ):
> _function['sqrt'] = getattr(mathmod ule, 'sqrt', None)
>
> def _get(name):
> try:
> return _functions[name]
> except KeyError:
> raise TypeError("you have to initialize module_g")
>
> def sqrt(x):
> return _get('sqrt')(x)
>
> main.py
> -------
>
> import math
> import module_g
>
> module_g.init(m ath)
> print module_g.sqrt(2 )[/color]

Thanks, this gets me close. Is there anything really bad about the
following? It works exactly as I would like, but I don't want to get in
trouble down the road:

module_f
--------
import math as my_math

def set_math(custom _math):
globals()['my_math'] = custom_math

def f(x):
return my_math.sqrt(x)
[color=blue][color=green][color=darkred]
>>> import module_f
>>> module_f.f(2)[/color][/color][/color]
1.4142135623730 951[color=blue][color=green][color=darkred]
>>> import cmath
>>> module_f.set_ma th(cmath)
>>> module_f.f(2j)[/color][/color][/color]
(1+1j)

Or, if one wants to use the "from __ import *" form:

from math import *

def set_math(custom _math):
globals().updat e(custom_math._ _dict__)

def f(x):
return sqrt(x)

Michael

[Michael McNeil Forbes]

Re: how to overload sqrt in a module?

On Fri, 3 Mar 2006, Steven D'Aprano wrote:
[color=blue]
> ... you can do this:
>
> import my_module
> my_module.set_e nvironment("mat h") # or cmath, or numeric, or whatever
>
> Your my_module will be like this:
>
> # Warning: untested code.
> ENVIRON = None # global variables sometimes have their uses
>
> def f(x):
> if ENVIRON is None:
> raise ValueError("Uni nitialised module!")
> # or use a custom exception
> return ENVIRON.sqrt(x)
>
> def set_environment (name):
> global ENVIRON
> ENVIRON = __import__(name )
>
>
> Does this help?[/color]

Yes. However, this raises a question: Is this any different than
directly modifying the globals, or is it just syntactic sugar.

import math as my_math

def set_environment (name):
globals()['ENVIRON'] = __import__(name )

Thanks,
Michael.

[Kent Johnson]

Re: how to overload sqrt in a module?

Michael McNeil Forbes wrote:[color=blue]
> Is there anything really bad about the
> following? It works exactly as I would like, but I don't want to get in
> trouble down the road:
>
> module_f
> --------
> import math as my_math
>
> def set_math(custom _math):
> globals()['my_math'] = custom_math[/color]

This seems clearer to me:
def set_math(custom _math):
global my_math
my_math = custom_math
[color=blue]
> Or, if one wants to use the "from __ import *" form:
>
> from math import *
>
> def set_math(custom _math):
> globals().updat e(custom_math._ _dict__)[/color]

This will cause interesting trouble if custom_math doesn't implement all
the functions you use from math.

Kent

[Steven D'Aprano]

Re: how to overload sqrt in a module?

On Sat, 04 Mar 2006 23:07:12 -0800, Michael McNeil Forbes wrote:
[color=blue]
> On Fri, 3 Mar 2006, Steven D'Aprano wrote:
>[color=green]
>> ... you can do this:
>>
>> import my_module
>> my_module.set_e nvironment("mat h") # or cmath, or numeric, or whatever
>>
>> Your my_module will be like this:
>>
>> # Warning: untested code.
>> ENVIRON = None # global variables sometimes have their uses
>>
>> def f(x):
>> if ENVIRON is None:
>> raise ValueError("Uni nitialised module!")
>> # or use a custom exception
>> return ENVIRON.sqrt(x)
>>
>> def set_environment (name):
>> global ENVIRON
>> ENVIRON = __import__(name )
>>
>>
>> Does this help?[/color]
>
> Yes. However, this raises a question: Is this any different than
> directly modifying the globals, or is it just syntactic sugar.[/color]

The keyword "global" instructs Python to make all references to the
following name come from the global namespace. It is the correct way to
do it (but not necessarily the *only* correct way, or *always* the
correct way). In something as tiny as your example:
[color=blue]
> def set_environment (name):
> globals()['ENVIRON'] = __import__(name )[/color]

it may not make any practical difference which method you use. But in
larger, more complex code, you are creating a rod for your own back. Try
running these three functions, and explain the difference in their
behaviour.

def f1():
global parrot
parrot = 3
parrot += 1
print "parrot is %d" % parrot

def f2():
globals()["parrot"] = 3
globals()["parrot"] += 1
print "parrot is %d" % parrot

def f3():
globals()["parrot"] = 3
parrot += 1
print "parrot is %d" % parrot


Directly modifying the globals is playing around with Python's internals.
You are allowed to do that, and sometimes it is the right thing to do,
otherwise Guido wouldn't have made globals() writable.

E.g. you have code where you want -- heaven knows why -- a name to refer
to both a local and a global variable. This will work:

def func():
x = 5 # x is local
globals()['x'] = 3 # but this x is global


I don't know whether writing to globals() is guaranteed to work for all
variants of Python (CPython, Jython, PyPy, IronPython ... ) or if it is
likely to change in Python 3. But even if modifying globals() is
officially allowed, it still has a whiff of the sort of code-smell that
Joel Spolsky talks about:

http://www.joelonsoftware.com/articles/Wrong.html

In my opinion, manipulating globals() is one of those things that ring
warning bells in my head. It might not be *wrong*, exactly, but I'll want
to pay extra attention to any function that does that.


--
Steven.