big objects and avoiding deepcopy?

[Reckoner]

I am writing an algorithm that takes objects (i.e. graphs with
thousands of nodes) into a "hypothetic al" state. I need to keep a
history of these hypothetical objects depending on what happens to
them later. Note that these hypothetical objects are intimately
operated on, changed, and made otherwise significantly different from
the objects they were copied from.

I've been using deepcopy to push the objects into the hypothetical
state where I operate on them heavily. This is pretty slow since the
objects are very large.

Is there another way to do this without resorting to deepcopy?

by the way, the algorithm works fine. It's just this part of it that I
am trying to change.

Thanks in advance.

[Robert Kern]

Re: big objects and avoiding deepcopy?

Reckoner wrote:

I am writing an algorithm that takes objects (i.e. graphs with
thousands of nodes) into a "hypothetic al" state. I need to keep a
history of these hypothetical objects depending on what happens to
them later. Note that these hypothetical objects are intimately
operated on, changed, and made otherwise significantly different from
the objects they were copied from.
>
I've been using deepcopy to push the objects into the hypothetical
state where I operate on them heavily. This is pretty slow since the
objects are very large.
>
Is there another way to do this without resorting to deepcopy?
>
by the way, the algorithm works fine. It's just this part of it that I
am trying to change.

This is similar to implementing "Undo" functionality in applications. One
solution is to define every operation you can do on the data structure as a pair
of functions, one which does the "forward" operation on the data structure and
one which does the "backward" operation which will return the modified data
structure back to its original state. Each time you do a forward operation,
append the pair of functions to a list (along with any auxiliary data that you
need). Once you have finished with the hypothetical operations, you can work
your way backwards through the list and using the "backwards" operations.

This works fairly well if you have a single data structure that you are managing
this way and a limited set of operations to track. If you have multiple
interacting objects and a large set of operations, things can become cumbersome.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Aaron Brady]

Re: big objects and avoiding deepcopy?

On Oct 24, 1:11 pm, Reckoner <recko...@gmail .comwrote:

I am writing an algorithm that takes objects (i.e. graphs with
thousands of nodes) into a "hypothetic al" state. I need to keep a
history of these  hypothetical objects depending on what happens to
them later. Note that these hypothetical objects are intimately
operated on, changed, and made otherwise significantly different from
the objects they were copied from.
>
I've been using deepcopy to push the objects into the hypothetical
state where I operate on them heavily. This is pretty slow since the
objects are very large.
>
Is there another way to do this without resorting to deepcopy?
>
by the way, the algorithm works fine. It's just this part of it that I
am trying to change.
>
Thanks in advance.

This solution takes a level of indirection.

Each graph has a stack of namespaces mapping names to nodes.

G:
{ 0: nodeA, 1: nodeB, 2: nodeC }
G-copy:
G, { 0: nodeD }
G-copy2:
G, { 1: nodeE }
G-copy-copy:
G-copy, { 3: nodeF }

If a key isn't found in the dictionary of a graph, its parent graph is
searched, and so on. Then G-copy[ 0 ] is nodeD, G-copy[ 1 ] is nodeB,
G-copy2[ 2 ] is nodeC, G-copy-copy[ 0 ] is nodeD. It might take a
significant change to your implementation, however.

[bearophileHUGS@lycos.com]

Re: big objects and avoiding deepcopy?

Robert Kern:

This is similar to implementing "Undo" functionality in applications.<

In a quite-high-level language (like Python, but not necessarily in
Python itself) it may become eventually advantageous to add some (even
limited) built-in form of undo. Both to give a simpler way to
implement a undo functionality into user-level programs (that is to
implement the Undo command in a program with GUI), but more
importantly to help the programmer too in some more general
programming tasks.

So it's probably a feature we'll see in the next generation of high-
level languages, among few other features that today are missing in
Python (only sometimes missing for performance reasons).

Bye,
bearophile

[Terry Reedy]

Re: big objects and avoiding deepcopy?

bearophileHUGS@ lycos.com wrote:

Robert Kern:

>This is similar to implementing "Undo" functionality in applications.<

>
In a quite-high-level language (like Python, but not necessarily in
Python itself) it may become eventually advantageous to add some (even
limited) built-in form of undo.

Right now, I believe, one could develop an undo module with undolist and
undodict classes that wrap instance mutations with an append to an
undoinfo list.

[Aaron Brady]

Re: big objects and avoiding deepcopy?

On Oct 27, 1:10 pm, Terry Reedy <tjre...@udel.e duwrote:

bearophileH...@ lycos.com wrote:

Robert Kern:

This is similar to implementing "Undo" functionality in applications.<

>

In a quite-high-level language (like Python, but not necessarily in
Python itself) it may become eventually advantageous to add some (even
limited) built-in form of undo.

>
Right now, I believe, one could develop an undo module with undolist and
undodict classes that wrap instance mutations with an append to an
undoinfo list.

The trick would be forking instances, so that you could operate on
separate histories/branches independently. Not necessary for most
undo purposes, of course.