Rewriting a bash script in python

[Chris Jones]

I wrote a trivial backup script that does the following:

If tonight is the first day of the month:

save last month's archive
do a full backup of the system

else:

save last night's differential backup
back up all changes relative to the current month's full backup

endif

I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".

I am thinking of rewriting it in python using OOP tactics/strategy.

Please advise.

Thanks!
CJ

[Ben Finney]

Re: Rewriting a bash script in python

Chris Jones <cjns1989@gmail .comwrites:

I am thinking of rewriting it in python using OOP tactics/strategy.
>
Please advise.

I advise you not to have the object-oriented programming hammer as
your only tool, because it's easy to then treat every problem as
though it were a nail.

What you describe seems trivially a procedural sequence of branching
steps using simple types from the standard library. Use functions to
encapsulate the conceptually-related steps, and call those functions
in a simple branch as you demonstrated. I see no need to apply
object-oriented techniques for this problem.

import datetime

def is_full_backup_ required(when):
""" Determine whether a full backup is required for datetime ‘when’ """
result = False
if determine_wheth er_full_backup_ required_for_da tetime(when):
result = True
return result

def most_recent_inc remental_backup (when):
""" Return the most recent incremental backup before datetime ‘when’ """
backup = most_recent_bac kup(when)
return backup

def save_previous_f ull_archive(whe n):
""" Save the full archive previous to datetime ‘when’ """

def save_previous_i ncremental_arch ive(when):
""" Save the incremental archive previous to datetime ‘when’ """

def perform_full_ba ckup(system):
""" Do a full backup of the system """

def perform_increme ntal_backup(sys tem):
""" Do an incremental backup of the system """

def get_this_system ():
""" Return the current system for backups """
system = determine_what_ this_system_is( )
return system

system = get_this_system ()
when = datetime.dateti me.now()
if is_full_backup_ required(when):
save_previous_m onth_archive(wh en)
perform_full_ba ckup(system)
else:
save_previous_i ncremental_arch ive(when)
perform_increme ntal_backup(sys tem)


I don't see any need for creating new classes for any of this, unless
a “system” is more complicated than just a string for the system
name, or a “backup” is more complicated than a tarball file.

Instead, the problem seems one best solved simply by abstracting the
steps involved at each point of the *procedure*, as I've done in my
example. Each one of those functions could be very simple or could be
complex enough to spawn a whole host of helper functions, possibly
even separate modules. That seems a more useful approach than trying
to force-fit object-orientation to this procedural problem.

--
\ “Some mornings, it's just not worth chewing through the leather |
`\ straps.” —Emo Philips |
_o__) |
Ben Finney

[Lawrence D'Oliveiro]

Re: Rewriting a bash script in python

In message <mailman.3490.1 225856545.3487. python-list@python.org >, Chris
Jones wrote:

I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".

Engineering rule #1: if it works, don't fix it.

[Jorgen Grahn]

Re: Rewriting a bash script in python

On Wed, 05 Nov 2008 17:42:43 +1300, Lawrence D'Oliveiro <ldo@geek-central.gen.new _zealandwrote:

In message <mailman.3490.1 225856545.3487. python-list@python.org >, Chris
Jones wrote:
>

>I wrote this in bash and although it's worked w/o a glitch for the last
>couple of months .. I'm not comfortable with .. "the look & feel of it".

>
Engineering rule #1: if it works, don't fix it.

Especially if it handles your backups ;-)

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.se R'lyeh wgah'nagl fhtagn!

[Jorgen Grahn]

Re: Rewriting a bash script in python

On Tue, 04 Nov 2008 22:11:37 -0500, Chris Jones <cjns1989@gmail .comwrote:

I wrote a trivial backup script that does the following:
>
If tonight is the first day of the month:
>
save last month's archive
do a full backup of the system
>
else:
>
save last night's differential backup
back up all changes relative to the current month's full backup
>
endif
>
I wrote this in bash and although it's worked w/o a glitch for the last
couple of months .. I'm not comfortable with .. "the look & feel of it".

This sounds like a job for cron(8) -- it's designed for exactly these
things: doing things based on calendar events. I assume you already
run your backups from cron rather than staying up every night to do
your stuff, so what about changing this in your crontab

00 1 * * * full_or_increme ntal_backup

to this?

00 1 1 * * backup --full
00 1 2-31 * * backup --incremental

(Or maybe two different scripts suit you better.)

That way you needn't implement your own logic for deciding when to do
a full backup. (You still need a way to find the latest full backup,
though.)

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.se R'lyeh wgah'nagl fhtagn!

[Marc 'BlackJack' Rintsch]

Re: Rewriting a bash script in python

On Wed, 05 Nov 2008 17:12:08 +0000, Jorgen Grahn wrote:

On Wed, 05 Nov 2008 17:42:43 +1300, Lawrence D'Oliveiro
>

>Engineering rule #1: if it works, don't fix it.

>
Especially if it handles your backups ;-)

Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)

Ciao,
Marc 'BlackJack' Rintsch

[Chris Jones]

Re: Rewriting a bash script in python

On Tue, Nov 04, 2008 at 11:11:17PM EST, Ben Finney wrote:

Chris Jones <cjns1989@gmail .comwrites:
>

I am thinking of rewriting it in python using OOP tactics/strategy.

Please advise.

>
I advise you not to have the object-oriented programming hammer as
your only tool, because it's easy to then treat every problem as
though it were a nail.

Naturally! But then.. knowing when to use a given programming technique
is part of the problem. I have a spouse who just ruined a perfectly good
chef's knife that cost me about $80.00 because she didn't bother looking
for a screwdriver .. and thank goodness I didn't have to take her the ER
to boot.

What you describe seems trivially a procedural sequence of branching
steps using simple types from the standard library.

This is for a standalone laptop.

I went to the trouble of designing, coding, and cron'ing my B/U script
for two reasons:

1. I wanted the laptop backed up regularly and -- see other posts in
this thread.. "restorable ".

2. The more I worked on this project, the more interesting it became ..
along "what happens if" lines .. and less trivial.

Now that's over and done with.

But then I started thinking .. what if for instance I had to scale my
effort from my single system to a large "data center" with hundred of
hosts .. with different backup policies .. what if I had to take into
account time slots where the backups ran .. had to consider bandwidth &
performance issues .. media costs & their rotation and such like.

In a vague way I had this feeling that if added machines one at a time,
my current script would not scale all that well.

I hope I'm making some sense but this is why, rather than take your word
for it, which I tend to intuitively agree with anyway .. I would much
prefer trying to oop this myself and try to figure why in this instance
using object-orientation would probably only add much overhead and
complexity without any real benefits.

IOW--we only learn by our own mistakes.

Use functions to encapsulate the conceptually-related steps, and call
those functions in a simple branch as you demonstrated. I see no need
to apply object-oriented techniques for this problem.

import datetime
>
def is_full_backup_ required(when):
""" Determine whether a full backup is required for datetime 'when' """
result = False
if determine_wheth er_full_backup_ required_for_da tetime(when):
result = True
return result
>
def most_recent_inc remental_backup (when):
""" Return the most recent incremental backup before datetime 'when' """
backup = most_recent_bac kup(when)
return backup
>
def save_previous_f ull_archive(whe n):
""" Save the full archive previous to datetime 'when' """
>
def save_previous_i ncremental_arch ive(when):
""" Save the incremental archive previous to datetime 'when' """
>
def perform_full_ba ckup(system):
""" Do a full backup of the system """
>
def perform_increme ntal_backup(sys tem):
""" Do an incremental backup of the system """
>
def get_this_system ():
""" Return the current system for backups """
system = determine_what_ this_system_is( )
return system
>
system = get_this_system ()
when = datetime.dateti me.now()
if is_full_backup_ required(when):
save_previous_m onth_archive(wh en)
perform_full_ba ckup(system)
else:
save_previous_i ncremental_arch ive(when)
perform_increme ntal_backup(sys tem)

Thank you very much for providing this example skeleton.

I don't see any need for creating new classes for any of this, unless
a "system" is more complicated than just a string for the system
name, or a "backup" is more complicated than a tarball file.

Well .. relative to my far-fetched example of scaling above .. I thought
that maybe a "target system" -- one to backup, that is .. would be an
obvious class .. that one could find some criteria or other that would
categorize systems into subgroups .. with related backup policies, for
instance .. and each actual system would end up being an instance of
these latter "sub-classes".

Likewise a "backup" .. full or incremental .. with a system name and a
calendar date as natural attributes ..?

No harm in trying?

:-)

Instead, the problem seems one best solved simply by abstracting the
steps involved at each point of the *procedure*, as I've done in my
example.

To loosely quote someone's post on this thread .. I have AFAIK already
"solved" my problem .. So why bother? .. As indicated above this is a
case of looking at the problem from a different angle and hopefully
learn something.

Each one of those functions could be very simple or could be
complex enough to spawn a whole host of helper functions, possibly
even separate modules. That seems a more useful approach than trying
to force-fit object-orientation to this procedural problem.

Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?

Also, what classes of problems would be naturals for OOP?

Thanks!
CJ

[Tim Rowe]

Re: Rewriting a bash script in python

2008/11/6 Chris Jones <cjns1989@gmail .com>:

Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?

Practice, largely, so you're doing the right thing (provided you don't
trust your /real/ backup data to a tutorial program). If you find that
the program is at its neatest when it's just a couple of methods in
one class, it was a procedural problem.

Actually, from your description of the problem -- that the interest is
in all the "what if" cases -- suggests that it's a specification
problem rather than a design or implementation problem, so rather than
recoding it it might be worth looking at various approaches to
requirements capture and specification.

--
Tim Rowe

[Lawrence D'Oliveiro]

Re: Rewriting a bash script in python

In message <6ne3j3Fkre17U9 @mid.uni-berlin.de>, Marc 'BlackJack' Rintsch
wrote:

Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)

That's why a backup system needs to be absolutely simple as possible. Avoid
complicated formats, a straightforward mirrored filesystem (e.g. updated
via rsync) is the easiest to verify that you can read your precious data.

[Lawrence D'Oliveiro]

Re: Rewriting a bash script in python

In message <mailman.3534.1 225932216.3487. python-list@python.org >, Chris
Jones wrote:

But then I started thinking .. what if for instance I had to scale my
effort from my single system to a large "data center" with hundred of
hosts .. with different backup policies .. what if I had to take into
account time slots where the backups ran .. had to consider bandwidth &
performance issues .. media costs & their rotation and such like.

<http://www.google.co.n z/search?q=second +system+effect>

[Chris Jones]

Re: Rewriting a bash script in python

On Wed, Nov 05, 2008 at 08:12:40PM EST, Tim Rowe wrote:

2008/11/6 Chris Jones <cjns1989@gmail .com>:
>

Begs the question .. how do I tell what is an object-oriented vs. a
procedural problem?

>
Practice, largely, so you're doing the right thing (provided you don't
trust your /real/ backup data to a tutorial program). If you find that
the program is at its neatest when it's just a couple of methods in
one class, it was a procedural problem.

but you do have to go through the motions .. to "find" it ..

Actually, from your description of the problem -- that the interest is
in all the "what if" cases -- suggests that it's a specification
problem rather than a design or implementation problem, so rather than
recoding it it might be worth looking at various approaches to
requirements capture and specification.

I'm one of them that need to get their hands dirty a bith .. lack the
power of abstraction. Need to do some building before I can specify.

[Lawrence D'Oliveiro]

Re: Rewriting a bash script in python

In message <mailman.3545.1 225941087.3487. python-list@python.org >, Chris
Jones wrote:

I'm one of them that need to get their hands dirty a bith .. lack the
power of abstraction. Need to do some building before I can specify.

Nothing wrong with rapid prototyping. :)

[Chris Jones]

Re: Rewriting a bash script in python

On Wed, Nov 05, 2008 at 09:21:38PM EST, Lawrence D'Oliveiro wrote:

In message <6ne3j3Fkre17U9 @mid.uni-berlin.de>, Marc 'BlackJack' Rintsch
wrote:
>

Well, if it handles your backups it doesn't work. It just pretends until
you really *need* the backed up data. ;-)

>
That's why a backup system needs to be absolutely simple as possible. Avoid
complicated formats, a straightforward mirrored filesystem (e.g. updated
via rsync) is the easiest to verify that you can read your precious data.
--
http://mail.python.org/mailman/listinfo/python-list

_Any_ system and backup systems in particular.

There are tradeoffs though .. the above is beyond my budget ..

Couldn't agree more with what you're saying, though.

Less is better!

[Tim Rowe]

Re: Rewriting a bash script in python

2008/11/6 Lawrence D'Oliveiro <ldo@geek-central.gen.new _zealand>:

Nothing wrong with rapid prototyping. :)

That's one requirements capture methodology, certainly. Actually,
there's quite a lot wrong with rapid prototyping, but there's quite a
lot wrong with all other requirements capture methodologies too, so
rapid prototyping is up there with the rest of them.


--
Tim Rowe

[Lawrence D'Oliveiro]

Re: Rewriting a bash script in python

In message <mailman.3558.1 225968534.3487. python-list@python.org >, Tim Rowe
wrote:

Actually, there's quite a lot wrong with rapid prototyping, but there's
quite a lot wrong with all other requirements capture methodologies too,
so rapid prototyping is up there with the rest of them.

Sounds like what Churchill said about democracy: that it's the worst system,
apart from all the others. :)