Still the __new__ hell ...

[Paulo da Silva]

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.

After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.

Here is the new situation.

As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.

from datetime import date
import cPickle,string

class MyDate(date):
def __new__(cls,yea r,month=None,da y=None):
if type(year) is str:
year,month,day= map(int,string. split(year,'-'))
if year<100:
year+=2000
return date.__new__(cl s,year,month,da y)

class C1(object):
def __init__(self):
self.x=MyDate(" 2007-3-15")

def f(self):
print self.x

c1=C1()

d=cPickle.dumps (c1)
c2=cPickle.load s(d)
c2.f()

1. year is passed to __new__ as a string but with invalid
contents!

2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.

Thanks for your patience.
Paulo

[Arnaud Delobelle]

Re: Still the __new__ hell ...

On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.
>
After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.
>
Here is the new situation.
>
As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.
>
from datetime import date
import cPickle,string
>
class MyDate(date):
def __new__(cls,yea r,month=None,da y=None):
if type(year) is str:
year,month,day= map(int,string. split(year,'-'))
if year<100:
year+=2000
return date.__new__(cl s,year,month,da y)
>
class C1(object):
def __init__(self):
self.x=MyDate(" 2007-3-15")
>
def f(self):
print self.x
>
c1=C1()
>
d=cPickle.dumps (c1)
c2=cPickle.load s(d)
c2.f()

I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:

http://docs.python.org/lib/node321.html

Something like:

class MyDate(date):
...
def __reduce__(self ):
return type(self), (self.year, self.month, self.day)

might solve your problem.

HTH

--
Arnaud

[Paul McGuire]

Re: Still the __new__ hell ...

On Mar 17, 4:31 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

Sorry to put here too many questions about __init__ __new__
stuff but I always found a new problem when using them.
I have searched for simple __new__ docs on how to do the
basic things but find none.
>
After trying the solutions people gently posted here
(thanks) I run into new trouble when I go with further
development.
>
Here is the new situation.
>
As suggested in a previous post, I used __new__ to
subclass date class but now cPickle/pickle loads
does not work.
>
from datetime import date
import cPickle,string
>
class MyDate(date):
def __new__(cls,yea r,month=None,da y=None):
if type(year) is str:
year,month,day= map(int,string. split(year,'-'))
if year<100:
year+=2000
return date.__new__(cl s,year,month,da y)
>
class C1(object):
def __init__(self):
self.x=MyDate(" 2007-3-15")
>
def f(self):
print self.x
>
c1=C1()
>
d=cPickle.dumps (c1)
c2=cPickle.load s(d)
c2.f()
>
1. year is passed to __new__ as a string but with invalid
contents!
>
2. If I had a __new__ in class C1, cPickle.loads invokes it
with only one parameter. This forces to allow __new__ accept
only 1 parameter (cls) which I don't want for the normal
situation.
>
Thanks for your patience.
Paulo

Any special reason you have to use __new__ for this factory method?
This version works, without the problems with __new__:

def getDate(*args):
if isinstance(args[0],basestring):
year,month,day = map(int,string. split(args[0],'-'))
else:
year,month,day = args
if year < 100:
year += 2000
return date(year,month ,day)

class C1(object):
def __init__(self):
#self.x=MyDate( "2007-3-15")
self.x = getDate("2007-3-15")

def f(self):
print self.x


-- Paul

[Paulo da Silva]

Re: Still the __new__ hell ...

Arnaud Delobelle escreveu:

On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

....

>I used __new__ to
>subclass date class but now cPickle/pickle loads
>does not work.
>>
>from datetime import date
>import cPickle,string
>>
>class MyDate(date):
> def __new__(cls,yea r,month=None,da y=None):
> if type(year) is str:
> year,month,day= map(int,string. split(year,'-'))
> if year<100:
> year+=2000
> return date.__new__(cl s,year,month,da y)
>>
>class C1(object):
> def __init__(self):
> self.x=MyDate(" 2007-3-15")
>>
> def f(self):
> print self.x
>>
>c1=C1()
>>
>d=cPickle.dump s(c1)
>c2=cPickle.loa ds(d)
>c2.f()

>
I haven't tried your code but I think that you may need to define a
__reduce__ method in your MyDate class in order to give a clue to the
python as to how to pickle its instances. For more details see:
>

http://docs.python.org/lib/node321.html

>
Something like:
>
class MyDate(date):
...
def __reduce__(self ):
return type(self), (self.year, self.month, self.day)
>
might solve your problem.
>
HTH
>
--
Arnaud
>

Thanks. This works exactly the way you wrote.
Yet I am misunderstandin g something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Thank you
Paulo

[Arnaud Delobelle]

Re: Still the __new__ hell ...

On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

[snip]

Thanks. This works exactly the way you wrote.
Yet I am misunderstandin g something. Can't pickle "see" that being
MyDate derived from date it also has to look at variables from date?
When do I need to do this? I am using pickle with a lot more complex
classes without this problem.

Without the MyDate.__reduce __ method, Python uses the
datetime.date._ _reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.

--
Arnaud

[Paulo da Silva]

Re: Still the __new__ hell ...

Arnaud Delobelle escreveu:

On Mar 17, 11:48 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

>Arnaud Delobelle escreveu:On Mar 17, 9:31 pm, Paulo da Silva <psdasil...@eso tericaX.ptXwrot e:

>
[snip]
>

>Thanks. This works exactly the way you wrote.
>Yet I am misunderstandin g something. Can't pickle "see" that being
>MyDate derived from date it also has to look at variables from date?
>When do I need to do this? I am using pickle with a lot more complex
>classes without this problem.

>
Without the MyDate.__reduce __ method, Python uses the
datetime.date._ _reduce__ method to pickle your MyDate instances, then
it uses MyDate.__new__ to unpickle them. But your MyDate.__new__
method does not understand the format of a pickled date. You could
also change the MyDate.__new__ method so that it does understand it,
but it wouldn't be that easy as you want it to accept a string, and
this is the format that dates are pickled in.
>
--
Arnaud
>
>
>

I see now. I need to read a little further about this stuff as soon as
I get some time to do it.

Thanks Arnaud.

[Bruno Desthuilliers]

Re: Still the __new__ hell ...

Paulo da Silva a écrit :
(snip)

Not an answer to your question, just a couple advices:

from datetime import date
import cPickle,string

The string module is mostly deprecated. You should use str type methods
whenever possible (cf below)

class MyDate(date):
def __new__(cls,yea r,month=None,da y=None):
if type(year) is str:

And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year , basestring):

year,month,day= map(int,string. split(year,'-'))

year, month, day = map(int, year.split('-'))

if year < 100:
year += 2000
return date.__new__(cl s,year,month,da y)
>

(snip)

[Paulo da Silva]

Re: Still the __new__ hell ...

Bruno Desthuilliers escreveu:

Paulo da Silva a écrit :

....

>class MyDate(date):
> def __new__(cls,yea r,month=None,da y=None):
> if type(year) is str:

>
And what if it's a unicode string ?
The correct idiom here is:
if isinstance(year , basestring):
>

Thanks.
If I do type(year) I get either int or str (may be unicode for unicode
strings) but never anything like basestring. As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring? x=u"xxxxxx"; help(x) says this is unicode based on
basestring but help does not "work" for x="xxxxxxxx".
May be the python tutorial should be upgraded to include these new
concepts. Also covering the basics of __init__/__new__ (why have both?)
would be nice.

Paulo

[greg]

Re: Still the __new__ hell ...

Paulo da Silva wrote:

As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.

--
Greg

[Steve Holden]

Re: Still the __new__ hell ...

greg wrote:

Paulo da Silva wrote:

>As a relatively inexperient
>in python, how could I know that a 'string' is an instance of
>basestring?

>
isinstance(x, basestring)
>
This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.
>

The idea is right, but the detail is completely wrong.

basestring is a *type*.

>>basestring

<type 'basestring'>

It's the base class of which both str and unicode are subclasses.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

[Alex Martelli]

Re: Still the __new__ hell ...

Steve Holden <steve@holdenwe b.comwrote:

greg wrote:

Paulo da Silva wrote:

As a relatively inexperient
in python, how could I know that a 'string' is an instance of
basestring?

isinstance(x, basestring)

This works because basestring is defined as the
tuple (str, unicode) and isinstance accepts a
tuple of types as well as just a single type.

The idea is right, but the detail is completely wrong.
>
basestring is a *type*.
>

>>basestring

<type 'basestring'>
>
It's the base class of which both str and unicode are subclasses.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).


Alex

[greg]

Re: Still the __new__ hell ...

Steve Holden wrote:

>>basestring

<type 'basestring'>

You're right, I'm not sure what made me think it
was a tuple. Maybe because people used to write
isinstance(x, (str, unicode)) before basestring
existed.

--
Greg

[Aahz]

Re: Still the __new__ hell ...

In article <1hv8x6v.1x0ceu r1nnwi46N%aleax @mac.com>,
Alex Martelli <aleax@mac.comw rote:

>Steve Holden <steve@holdenwe b.comwrote:

>>
>basestring is a *type*.
>>

> >>basestring

><type 'basestring'>
>>
>It's the base class of which both str and unicode are subclasses.

>
>I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
>Python 2.2 installation to check this right now).

Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

>>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined
--
Aahz (aahz@pythoncra ft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith

[Gabriel Genellina]

Re: Still the __new__ hell ...

En Tue, 20 Mar 2007 10:16:30 -0300, Aahz <aahz@pythoncra ft.comescribió:

In article <1hv8x6v.1x0ceu r1nnwi46N%aleax @mac.com>,
Alex Martelli <aleax@mac.comw rote:

>Steve Holden <steve@holdenwe b.comwrote:

>>>
>>basestring is a *type*.

>>
>I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
>Python 2.2 installation to check this right now).

>
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

>>>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined

On 2.2 it appeared types.StringTyp es == (str,unicode)

--
Gabriel Genellina

[Alex Martelli]

Re: Still the __new__ hell ...

Aahz <aahz@pythoncra ft.comwrote:

In article <1hv8x6v.1x0ceu r1nnwi46N%aleax @mac.com>,
Alex Martelli <aleax@mac.comw rote:

Steve Holden <steve@holdenwe b.comwrote:

>
basestring is a *type*.
>
>>basestring
<type 'basestring'>
>
It's the base class of which both str and unicode are subclasses.

I believe it used to be a tuple back in Python 2.2 (sorry, don't have a
Python 2.2 installation to check this right now).

>
Python 2.2.2 (#1, Feb 24 2003, 19:13:11)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

>basestring

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'basestring' is not defined

Thanks for double checking on my vague and apparently incorrect
historical memory! Obviously I must have been thinking of some _trick_
whereby one bound basestring to the pair to make isinstance on
basestring work in 2.2 much as it did later, rather than an intrinsic
2.2 feature.


Alex