error converting list to tuple

[Stefan Seefeld]

hi there,

I'm trying to convert a tuple to a list,
and get a 'TypeError: list objects are unhashable'.

Can anybody enlighten me as to the possible causes
for this ? Where does hashing come into play during
this conversion ?

Could it be that my runtime is corrupted ?
The code is executed from within a C++ extension
module I'm developing...

Thanks for any help,
Stefan

PS: I'm using python 2.3.3 in case that matters

[john fabiani]

Re: error converting list to tuple

Stefan Seefeld wrote:[color=blue]
> hi there,
>
> I'm trying to convert a tuple to a list,
> and get a 'TypeError: list objects are unhashable'.
>
> Can anybody enlighten me as to the possible causes
> for this ? Where does hashing come into play during
> this conversion ?
>
> Could it be that my runtime is corrupted ?
> The code is executed from within a C++ extension
> module I'm developing...
>
> Thanks for any help,
> Stefan
>
> PS: I'm using python 2.3.3 in case that matters
>
>[/color]
God I can't believe I think I can answer this question:
mylist=[]
t=('123','hello ','bye')
for count in range(0,len(t)) :
mylist.append(t[count])

It works for me.
John

[john fabiani]

Re: error converting list to tuple

john fabiani wrote:
[color=blue]
> Stefan Seefeld wrote:
>[color=green]
>> hi there,
>>
>> I'm trying to convert a tuple to a list,
>> and get a 'TypeError: list objects are unhashable'.
>>
>> Can anybody enlighten me as to the possible causes
>> for this ? Where does hashing come into play during
>> this conversion ?
>>
>> Could it be that my runtime is corrupted ?
>> The code is executed from within a C++ extension
>> module I'm developing...
>>
>> Thanks for any help,
>> Stefan
>>
>> PS: I'm using python 2.3.3 in case that matters
>>
>>[/color]
> God I can't believe I think I can answer this question:
> mylist=[]
> t=('123','hello ','bye')
> for count in range(0,len(t)) :
> mylist.append(t[count])
>
> It works for me.
> John[/color]
even better than above:

mylist=list(t)


See everyone I'm learning but it is not easy for an old man to learn new
tricks.

John

[Stefan Seefeld]

Re: error converting list to tuple

john fabiani wrote:[color=blue]
> Stefan Seefeld wrote:
>[color=green]
>> hi there,
>>
>> I'm trying to convert a tuple to a list,
>> and get a 'TypeError: list objects are unhashable'.
>>
>> Can anybody enlighten me as to the possible causes
>> for this ? Where does hashing come into play during
>> this conversion ?
>>
>> Could it be that my runtime is corrupted ?
>> The code is executed from within a C++ extension
>> module I'm developing...
>>
>> Thanks for any help,
>> Stefan
>>
>> PS: I'm using python 2.3.3 in case that matters
>>
>>[/color]
> God I can't believe I think I can answer this question:[/color]

well, unfortunately that wasn't my question <wink/>

I do:

t = ('hello',)
l = list(t)

which is the standard tuple->list conversion. And yes,
it does work when I run that code in isolation.

The point is that the above, as simple as it looks,
fails with said exception in a very specific context
(I call this code from within a C++ extension module),
so I'm wondering whether I could have messed up the
python runtime somehow or whether there are other
possible explanations for the exception.

Regards,
Stefan

[Terry Reedy]

Re: error converting list to tuple


"Stefan Seefeld" <seefeld@sympat ico.ca> wrote in message
news:a3qqc.3439 7$325.832599@ne ws20.bellglobal .com...[color=blue]
> (I call this code from within a C++ extension module),[/color]

You should have said that in the beginning since that is a whole different
ball game: extension modules in C are not Python.
[color=blue]
> so I'm wondering whether I could have messed up the
> python runtime somehow or whether there are other
> possible explanations for the exception.[/color]

It is easy to mess up in C ;-). You are most likely to get an answer (from
another C extension writer) if you post a short as possible code snippet
that exhibits the error, and the actual and complete error message.

tjr

[Stefan Seefeld]

Re: error converting list to tuple

Terry Reedy wrote:[color=blue]
> "Stefan Seefeld" <seefeld@sympat ico.ca> wrote in message[/color]
[color=blue][color=green]
>>(I call this code from within a C++ extension module),[/color]
>
>
> You should have said that in the beginning since that is a whole different
> ball game: extension modules in C are not Python.[/color]

ok, let me be more specific: From my C++ code I call a method
of a python object which happens to do the tuple->list conversion.

And as Python is implemented in C, I thought it didn't matter,
it's the same underlying API / object model, just different syntax ;-)
[color=blue]
> It is easy to mess up in C ;-). You are most likely to get an answer (from
> another C extension writer) if you post a short as possible code snippet
> that exhibits the error, and the actual and complete error message.[/color]

That's (part of) the problem: I can't reproduce this in isolation,
so I wanted to know how I could debug this.

What I do is calling 'PyObject_Str' on the python object. That object
has its '__str__' method implemented in a way that involves said
tuple->list conversion, so the call raises an exception and
the call returns 0.

Thanks,
Stefan

[Stefan Seefeld]

using the C API to access objects derived from builtin classes [Was:

Terry Reedy wrote:
[color=blue]
> It is easy to mess up in C ;-). You are most likely to get an answer (from
> another C extension writer) if you post a short as possible code snippet
> that exhibits the error, and the actual and complete error message.[/color]

I found the problem, and I'd like to share my new insight, as I don't
think it is obvious:

I hold an object reference that is a dictionary, and I modify it with
the PyDict_* family of functions.

However, the real object is not a builtin 'dict', but instead it is
derived from it:

class Dictionary(dict ):
...

I just realized that PyDict_SetItem (et al.) don't lookup the
methods in the object's __dict__, so the method that gets really
executed is not the overloaded I wrote, but the method from the
base class.

Replacing 'PyDict_SetItem ' by 'PyObject_SetIt em' did the trick.
I guess 'PyDict_SetItem ' is a shortcut to circumvent the lengthy
method lookup. Fair enough, but the python C API documentation could
be a bit more clear about that...

Kind regards,
Stefan

PS: for the curious: the missing link between the dict and
the list / tuple issue is that I derived my own dictionary
precisely to be able to use a list as key.

[Thomas Heller]

Re: using the C API to access objects derived from builtin classes

Stefan Seefeld <seefeld@sympat ico.ca> writes:
[color=blue]
> Terry Reedy wrote:
>[color=green]
>> It is easy to mess up in C ;-). You are most likely to get an answer (from
>> another C extension writer) if you post a short as possible code snippet
>> that exhibits the error, and the actual and complete error message.[/color]
>
> I found the problem, and I'd like to share my new insight, as I don't
> think it is obvious:
>
> I hold an object reference that is a dictionary, and I modify it with
> the PyDict_* family of functions.
>
> However, the real object is not a builtin 'dict', but instead it is
> derived from it:
>
> class Dictionary(dict ):
> ...
>
> I just realized that PyDict_SetItem (et al.) don't lookup the
> methods in the object's __dict__, so the method that gets really
> executed is not the overloaded I wrote, but the method from the
> base class.
>
> Replacing 'PyDict_SetItem ' by 'PyObject_SetIt em' did the trick.
> I guess 'PyDict_SetItem ' is a shortcut to circumvent the lengthy
> method lookup. Fair enough, but the python C API documentation could
> be a bit more clear about that...[/color]

I would guess the *real* problem is that you are ignoring the return
value of PyDict_SetItem (or whatever api you use). In this case you get
weird errors afterwards.

Thomas

[Stefan Seefeld]

Re: using the C API to access objects derived from builtin classes

Thomas Heller wrote:
[color=blue][color=green]
>>Replacing 'PyDict_SetItem ' by 'PyObject_SetIt em' did the trick.
>>I guess 'PyDict_SetItem ' is a shortcut to circumvent the lengthy
>>method lookup. Fair enough, but the python C API documentation could
>>be a bit more clear about that...[/color]
>
>
> I would guess the *real* problem is that you are ignoring the return
> value of PyDict_SetItem (or whatever api you use). In this case you get
> weird errors afterwards.[/color]

good shot ! Well, what the 'real' problem is is probably a matter
of perspective, but you are quite right in that I failed to check
a return value (and associated exception) so python's own code
eventually bumped into it in an unrelated context.

But back to the other issue: isn't there any text that explains
when I should be using PyDict_GetItem and when PyObject_GetIte m
and what they are doing byhind the scene ?

Regards,
Stefan

[Thomas Heller]

Re: using the C API to access objects derived from builtin classes

Stefan Seefeld <seefeld@sympat ico.ca> writes:
[color=blue]
> Thomas Heller wrote:
>[color=green][color=darkred]
>>>Replacing 'PyDict_SetItem ' by 'PyObject_SetIt em' did the trick.
>>>I guess 'PyDict_SetItem ' is a shortcut to circumvent the lengthy
>>>method lookup. Fair enough, but the python C API documentation could
>>>be a bit more clear about that...[/color]
>> I would guess the *real* problem is that you are ignoring the return
>> value of PyDict_SetItem (or whatever api you use). In this case you get
>> weird errors afterwards.[/color]
>
> good shot ! Well, what the 'real' problem is is probably a matter
> of perspective, but you are quite right in that I failed to check
> a return value (and associated exception) so python's own code
> eventually bumped into it in an unrelated context.[/color]

And that will always catch you!
[color=blue]
> But back to the other issue: isn't there any text that explains
> when I should be using PyDict_GetItem and when PyObject_GetIte m
> and what they are doing byhind the scene ?[/color]

The PyObject_GetIte m() function belongs to the Abstract Objects Layer.
<http://docs.python.org/api/abstract.html>
The docs say:

6. Abstract Objects Layer

The functions in this chapter interact with Python objects regardless
of their type, or with wide classes of object types (e.g. all
numerical types, or all sequence types). When used on object types for
which they do not apply, they will raise a Python exception.

while PyDict_GetItem( ) belongs to the Concrete Objects Layer for
PyDictObject <types http://docs.python.org/api/concrete.html>:

7. Concrete Objects Layer

The functions in this chapter are specific to certain Python object
types. Passing them an object of the wrong type is not a good idea; if
you receive an object from a Python program and you are not sure that
it has the right type, you must perform a type check first; for
example, to check that an object is a dictionary, use
PyDict_Check(). The chapter is structured like the ``family tree'' of
Python object types.

Reading the include files may also help - once you get the idea it's simple.

Thomas

[Stefan Seefeld]

Re: using the C API to access objects derived from builtin classes

Thomas Heller wrote:
[color=blue][color=green]
>>But back to the other issue: isn't there any text that explains
>>when I should be using PyDict_GetItem and when PyObject_GetIte m
>>and what they are doing byhind the scene ?[/color][/color]
[color=blue]
> 7. Concrete Objects Layer
>
> The functions in this chapter are specific to certain Python object
> types. Passing them an object of the wrong type is not a good idea; if
> you receive an object from a Python program and you are not sure that
> it has the right type, you must perform a type check first; for
> example, to check that an object is a dictionary, use
> PyDict_Check(). The chapter is structured like the ``family tree'' of
> Python object types.[/color]

yeah, I did read that. The inconsistency (or even error, I would argue) is
in the interpretation of 'is a'. PyDict_Check reports 'true' when called
on my 'Dictionary' object, yet the PyDict_* API only applies to the
slice representing the base class. I don't think that is correct, it is
at least very surprising.

The API only supports two extremes:

* the object's type is an *exact* match for a built-in type, thus a highly
optimized API can be used to access its methods.

* the object's type is unknown, it may or may not support a certain 'protocol',
i.e. for all method invocations a full method lookup has to be done.

That sounds good for the pre 2.2 world, where built-in types and user-defined
types were distinct sets.

Regards,
Stefan