A lazy-committing database object with curry?

[Tim Lesher]

I'm writing a database helper class that represents records in a SQL
database as objects. I want to be able to instantiate the objects
(from the database), play with their values locally, then lazily
commit the changes all at once (via an explicit commit call). Other
than the call to Commit(), I don't want clients of this class to have
to think about the fact that there's a database behind it all--there are
some interesting and non-obvious dependencies on the particular values
that are committed together, and I want to hide that in the Commit() call.

I'm going to have a number of these, so I wanted to come up with a
solution that's short on glue code and can easily be aggregated into
all the classes that represent record types. This is what I've come
up with:

# Simple currying class, no kwargs
class curry:
def __init__(self, fun, *args):
self.fun = fun
self.pending = args[:]
def __call__(self, *args):
return self.fun(*(self .pending + args))


# Simple table: two columns, "title" and "descriptio n"
class SomeRecordType( object):
def __init__(self, title, description):
self.__modifica tions = {}

# Avoid triggering propset on init
self.__dict__['title'] = title
self.__dict__['description'] = description

def __getValue(name , self):
try:
return self.__modifica tions[name]
except KeyError:
return self.__dict__[name]

def __setValue(name , self, newValue):
self.modificati ons[name] = newValue

name = property(curry( __getValue, 'title'),
curry(__setValu e, 'title'))
description = property(curry( __getValue, 'description'),
curry(__setValu e, 'description'))

def Commit(self):
if self.modificati ons != {}:
# - snip - Do database commit and clear modifications
self.__dict__.u pdate(self.modi fications)
self.modificati ons = {}

So I can do this:

foo = myDb.LookupTitl eRecord('some title')
print foo.description # 'foo'
foo.description = 'bar' # not updated in the DB yet
print foo.description # 'bar'
foo.Commit() # now updated in the DB


Are there any pitfalls to doing this? Am I being dazzled by the shiny
new toy that is currying? Is there another simple solution, or a
refinement of this one, that I'm not seeing?

Thanks.

--
Tim Lesher
<tim@lesher.w s>

[Jp Calderone]

Re: A lazy-committing database object with curry?

On Mon, Jan 19, 2004 at 08:45:45AM -0800, Tim Lesher wrote:[color=blue]
> I'm writing a database helper class that represents records in a SQL
> database as objects. I want to be able to instantiate the objects
> (from the database), play with their values locally, then lazily
> commit the changes all at once (via an explicit commit call). Other
> than the call to Commit(), I don't want clients of this class to have
> to think about the fact that there's a database behind it all--there are
> some interesting and non-obvious dependencies on the particular values
> that are committed together, and I want to hide that in the Commit() call.[/color]

Sounds familiar...
[color=blue]
>
> I'm going to have a number of these, so I wanted to come up with a
> solution that's short on glue code and can easily be aggregated into
> all the classes that represent record types. This is what I've come
> up with:
>
> # Simple currying class, no kwargs
> class curry:
> def __init__(self, fun, *args):
> self.fun = fun
> self.pending = args[:]
> def __call__(self, *args):
> return self.fun(*(self .pending + args))
>[/color]

You don't need to copy args, Python has done that for you already.
[color=blue]
>
> # Simple table: two columns, "title" and "descriptio n"
> class SomeRecordType( object):
> def __init__(self, title, description):
> self.__modifica tions = {}
>
> # Avoid triggering propset on init
> self.__dict__['title'] = title
> self.__dict__['description'] = description
>[/color]

Are you sure? How will the initial values be committed to the database,
if you don't catch them in the modifications dict? I guess you might be
handling initial values somewhere else, in code you didn't post...
[color=blue]
> def __getValue(name , self):
> try:
> return self.__modifica tions[name]
> except KeyError:
> return self.__dict__[name]
>
> def __setValue(name , self, newValue):
> self.modificati ons[name] = newValue
>
> name = property(curry( __getValue, 'title'),
> curry(__setValu e, 'title'))
> description = property(curry( __getValue, 'description'),
> curry(__setValu e, 'description'))[/color]

Nift. You surely don't use the full capabilities of currying here, but
you are incurring a double function call overhead on every setattr now.
Here's another way to do it:

def makeModificatio nFunctions(name ):
def modget(self, value):
try:
return self.__modifica tions[name]
except KeyError:
return self.__dict__[name]
def modset(self, value):
self.__modifica tions[name] = value
return modget, modset
title = property(*makeM odificationFunc tions('title'))
description = property(*makeM odificationFunc tions('descript ion'))

Of course, you can go one step better and use a metaclass:

class MetaRecordType( type):
def __new__(metacla ss, name, bases, ns):
for k, v in ns.items():
if v is makeModificatio nFunctions
ns[k] = property(*v(k))
return type.__new__(me taclass, name, bases, ns)

and now in the class definition simply say:

__metaclass__ = MetaRecordType

title = makeModificatio nFunctions
description = makeModificatio nFunctions
[color=blue]
>
> def Commit(self):
> if self.modificati ons != {}:[/color]

This can be just "if self.__modifica tions:"

[color=blue]
> # - snip - Do database commit and clear modifications
> self.__dict__.u pdate(self.modi fications)
> self.modificati ons = {}
>
> So I can do this:
>
> foo = myDb.LookupTitl eRecord('some title')
> print foo.description # 'foo'
> foo.description = 'bar' # not updated in the DB yet
> print foo.description # 'bar'
> foo.Commit() # now updated in the DB
>
>
> Are there any pitfalls to doing this? Am I being dazzled by the shiny
> new toy that is currying? Is there another simple solution, or a
> refinement of this one, that I'm not seeing?
>[/color]

One possible problem is failed commits. If you're doing commits
transactionally , and the transaction fails, your in memory objects will
retain their now-inconsistent values while the database remains unupdated.
If you're not using transactions.. well that's a whole other problem :)

You may want to look at xsdb and atop. Neither uses SQL, but both have
developed pretty heavily on some of the ideas you're tinkering around with.

Jp

[Tim Lesher]

Re: A lazy-committing database object with curry?

Jp Calderone <exarkun@intarw eb.us> wrote in message news:<mailman.5 14.1074539232.1 2720.python-list@python.org >...[color=blue]
> On Mon, Jan 19, 2004 at 08:45:45AM -0800, Tim Lesher wrote:
> Are you sure? How will the initial values be committed to the database,
> if you don't catch them in the modifications dict? I guess you might be
> handling initial values somewhere else, in code you didn't post...[/color]

Yes... this class will only be working on rows that already exist in
the database--in other words, it only has to cope with updates and
deletes, not inserts.
[color=blue]
> Nift. You surely don't use the full capabilities of currying here, but
> you are incurring a double function call overhead on every setattr now.
> Here's another way to do it:
>
> def makeModificatio nFunctions(name ):
> def modget(self, value):
> try:
> return self.__modifica tions[name]
> except KeyError:
> return self.__dict__[name]
> def modset(self, value):
> self.__modifica tions[name] = value
> return modget, modset
> title = property(*makeM odificationFunc tions('title'))
> description = property(*makeM odificationFunc tions('descript ion'))[/color]

Aha. This is what I was really looking for. I tried it with lambdas
at first and got bitten by the expressions-only limitation. Coming
from a C++ background, I keep forgetting about local functions...
[color=blue]
> Of course, you can go one step better and use a metaclass:[/color]

Wow. That's even shinier than the currying.
[color=blue]
> This can be just "if self.__modifica tions:"[/color]

I know... explicit better than implicit and all.
[color=blue]
> One possible problem is failed commits. If you're doing commits
> transactionally , and the transaction fails, your in memory objects will
> retain their now-inconsistent values while the database remains unupdated.
> If you're not using transactions.. well that's a whole other problem :)[/color]

Yep, failed commits are handled in the 'snipped' piece of code when
they're handled at all--initially this is running against a
transactionless database, but the place is there to handle
transactions--that's why I don't clear the modifications until after
the update is made.
[color=blue]
> You may want to look at xsdb and atop. Neither uses SQL, but both have
> developed pretty heavily on some of the ideas you're tinkering around with.[/color]

Thanks; I will.

--
Tim Lesher
<tim@lesher.w s>