How to pickle dictionaries?

[brainstaurm]

I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

# Example dictionary
patterns = {pat1: {t1:None},
                 pat2: {t1:None, t2:None},
                 pat3: {t2:None}}
# Each key is an instance of the class T
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here

# Sample instance data
t1 = T('John','age','35')
t2 = T('John','created','www.blogger.com')
pat1 = T('John','age',None)
pat2 = T('John',None,None)
pat3 = T('John','created',None)

class T(tuple):
   def __new__(cls,*args):
      a,b,c= tuple(args)
      return tuple.__new__(cls,(a,b,c))

I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.

[bartonc]

I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

# Example dictionary
patterns = {pat1: {t1:None},
                 pat2: {t1:None, t2:None},
                 pat3: {t2:None}}
# Each key is an instance of the class T
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here

# Sample instance data
t1 = T('John','age','35')
t2 = T('John','created','www.blogger.com')
pat1 = T('John','age',None)
pat2 = T('John',None,None)
pat3 = T('John','created',None)

class T(tuple):
   def __new__(cls,*args):
      a,b,c= tuple(args)
      return tuple.__new__(cls,(a,b,c))

I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.

I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.

[bvdet]

I am trying to make some of the in-memory dictionaries in my code persistent using bsddb 4.3. Here is an example dictionary in my code for which I would like to create a persistent version:

# Example dictionary
patterns = {pat1: {t1:None},
                 pat2: {t1:None, t2:None},
                 pat3: {t2:None}}
# Each key is an instance of the class T
# Each value is a dictionary, where keys are instances of the class T, and values are None. I could have used set instead of dictionary here

# Sample instance data
t1 = T('John','age','35')
t2 = T('John','created','www.blogger.com')
pat1 = T('John','age',None)
pat2 = T('John',None,None)
pat3 = T('John','created',None)

class T(tuple):
   def __new__(cls,*args):
      a,b,c= tuple(args)
      return tuple.__new__(cls,(a,b,c))

I plan to convert each key in the dictionary into a string using cPickle before I can store the key in the persistent hash.

I am looking for a way to store the values in the dictionary. Since each value is a dictionary, I am not sure how exactly to pickle it. Each value in the hash table is likely to get updated quite often; hence I need an efficient way to pickle and unpickle the values. Any ideas? Please help.

Thanks,
D.

Have you tried this:

import cPickle
fn = r'your_file'
f = open(fn, "w")
cPickle.Pickler(f).dump(patterns)
f.close()

Unpickle test:

import cPickle

class T(tuple):
   def __new__(cls,*args):
      a,b,c= tuple(args)
      return tuple.__new__(cls,(a,b,c))

if __name__ == '__main__':    

    fn = r'H:\TEMP\temsys\pickle_dict.txt'
    f = open(fn, "r")
    dd = cPickle.Unpickler(f).load()
    f.close()

    for key, value in dd.items():
        print '%s = %s' % (key, value)

>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.co m'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.co m'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values( )[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.co m'): None}
>>>

[brainstaurm]

I'd say go straight to the DB. I've posted dictionary to sql query converters in the Articles section.

Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.

[brainstaurm]

Have you tried this:

import cPickle
fn = r'your_file'
f = open(fn, "w")
cPickle.Pickler(f).dump(patterns)
f.close()

Unpickle test:

import cPickle

class T(tuple):
   def __new__(cls,*args):
      a,b,c= tuple(args)
      return tuple.__new__(cls,(a,b,c))

if __name__ == '__main__':    

    fn = r'H:\TEMP\temsys\pickle_dict.txt'
    f = open(fn, "r")
    dd = cPickle.Unpickler(f).load()
    f.close()

    for key, value in dd.items():
        print '%s = %s' % (key, value)

>>> ('John', None, None) = {('John', 'age', '35'): None, ('John', 'created', 'www.blogger.co m'): None}
('John', 'created', None) = {('John', 'created', 'www.blogger.co m'): None}
('John', 'age', None) = {('John', 'age', '35'): None}
>>> type(dd.keys()[0])
<class '__main__.T'>
>>> type(dd.values( )[0])
<type 'dict'>
>>> print dd.values()[0]
{('John', 'age', '35'): None, ('John', 'created', 'www.blogger.co m'): None}
>>>

Thanks for your response! I did try this out.

[bartonc]

Thanks, your response was useful!

Could you please point me to the article you wrote, I could not find it.

They are helper functions in this post.