Time script help sought!

[kpp9c]

I am kind of in a bit of a jam (okay a big jam) and i was hoping that
someone here could give me a quick hand. I had a few pages of time
calculations to do. So, i just started in on them typing them in my
time calculator and writing them in by hand. Now i realize, that i
really need a script to do this because:

1. It turns out there are hundreds of pages of this stuff.
2. I have to do something similar in again soon.
3. By doing it by hand i am introducing wonderful new errors!
4. It all has to be typed up anyway (which means weeks of work and even
more typos!)

The input would like so:

Item_1 TAPE_1 1 00:23 8:23

Item_2 TAPE_1 2 8:23 9:41

Item_3 TAPE_1 3 9:41 10:41
Item_3 TAPE_1 4 10:47 11:19
Item_3 TAPE_1 5 11:21 11:55
Item_3 TAPE_1 6 11:58 12:10
Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape sound.
Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape sound.

Item_4 TAPE_1 9 24:33
Item_4 TAPE_1 10 25:48
Item_4 TAPE_1 11 29:48
Item_4 TAPE_1 12 31:46
Item_4 TAPE_1 13 34:17 Electronic sounds.
Item_4 TAPE_1 14 35:21
Item_4 TAPE_1 15 36:06
Item_4 TAPE_1 16 37:01 37:38

These are analog tapes that were digitized (on to CD or a digital tape)
that have now been exported as individual files that are meant to be
part of an on-line audio archive. The timings refer to the time display
on the CD or digital tape. The now all have to adjusted so that each
item starts at 0.00 since they have all been edited out of their
context and are now all individual items that start at 00:00. So Item_1
which was started at 00:23 on the tape and ended at 8:23 needs to have
23 seconds subtracted to it so that it says:

Item_1 TAPE_1 1 00:00 08:00

Item_2 TAPE_1 2 08:23 09:41

would change to:

Item_2 TAPE_1 2 00:00 01:18

etc.

but as always you may notice a wrinkle.... some items have many times
(here 6) indicated:

Item_3 TAPE_1 3 9:41 10:41
Item_3 TAPE_1 4 10:47 11:19
Item_3 TAPE_1 5 11:21 11:55
Item_3 TAPE_1 6 11:58 12:10
Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape sound.
Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape sound.

This is all a single sound file and these separate times mark where
there was a break, defect, or edit in the individual item. These have
to be adjusted as well to show where these events would appear in the
new sound file which now starts at 00:00.

Item_3 TAPE_1 3 00:00 01:00 ----
Item_3 TAPE_1 4 01:00 01:38 ----
Item_3 TAPE_1 5 01:38 02:14 ----
Item_3 TAPE_1 6 02:14 02:29 ----
Item_3 TAPE_1 7 02:29 03:04 Defect in analog tape sound.
Item_3 TAPE_1 8 03:04 14:39 Defect in analog tape sound.

Further wrinkles: Some have start and end times indicated, some only
start times. I suppose that the output would ideally have both.... some
have comments and others don't ... and I need these comments echo-ed or
since i probably need to make a database or table eventually non
comments just have some place holder.

I'd have a lot of similar type calculations to do... I was hoping and
praying that some one here was feeling generous and show me the way and
then, of course i could modify that to do other tasks... Usually i am
happy to take the long road and all but i'll be honest, i am in a big
jam here and this huge task was just dumped on me. I am frankly a
little desperate for help on this and hoping someone is feeling up to
spoon feeding me a clear modifiable example that works. Sorry.....
cheers,

kevin

[Paul Rubin]

Re: Time script help sought!

"kpp9c" <kp8@mac.com> writes:[color=blue]
> These are analog tapes that were digitized (on to CD or a digital tape)
> that have now been exported as individual files that are meant to be
> part of an on-line audio archive. ...
> I was hoping and
> praying that some one here was feeling generous and show me the way...[/color]

Is this online archive going to be accessible by the public for free?
What's in the archive? If you're asking for volunteer labor it's
generally appropriate to say precisely what that the labor is for.

[kpp9c]

Re: Time script help sought!

Yes, Ultimately it will be part of a large digital archive available
for researchers on site and eventually probably on-line for the New
York Public Library. It is a huge undertaking and most of the
soundfiles have been made. I (we) are struggling with the sheer size
of the documentation.. .. Sorry about that.... i should have been more
clear, epecially since i am sort of begging for a little help. Sorry, i
am slightly overwhelmed at the moment...

[Mark McEahern]

Re: Time script help sought!

kpp9c wrote:
[color=blue]
>The input would like so:
>
>[/color]

[...]

Attached is a first cut at a parser that actually uses the raw content
of your original email. You'll notice that the net effect is that the
parser instance's items attribute contains the source ordered list of
items with attributes for each of the various parts of the line. From
this, it should be pretty easy to adjust the times and what not.

Cheers,

// m

#!/usr/bin/env python

"""usage: %prog
"""

raw = """I am kind of in a bit of a jam (okay a big jam) and i was hoping that
someone here could give me a quick hand. I had a few pages of time
calculations to do. So, i just started in on them typing them in my
time calculator and writing them in by hand. Now i realize, that i
really need a script to do this because:

1. It turns out there are hundreds of pages of this stuff.
2. I have to do something similar in again soon.
3. By doing it by hand i am introducing wonderful new errors!
4. It all has to be typed up anyway (which means weeks of work and even
more typos!)

The input would like so:

Item_1 TAPE_1 1 00:23 8:23

Item_2 TAPE_1 2 8:23 9:41

Item_3 TAPE_1 3 9:41 10:41
Item_3 TAPE_1 4 10:47 11:19
Item_3 TAPE_1 5 11:21 11:55
Item_3 TAPE_1 6 11:58 12:10
Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape sound.
Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape sound.

Item_4 TAPE_1 9 24:33
Item_4 TAPE_1 10 25:48
Item_4 TAPE_1 11 29:48
Item_4 TAPE_1 12 31:46
Item_4 TAPE_1 13 34:17 Electronic sounds.
Item_4 TAPE_1 14 35:21
Item_4 TAPE_1 15 36:06
Item_4 TAPE_1 16 37:01 37:38

These are analog tapes that were digitized (on to CD or a digital tape)
that have now been exported as individual files that are meant to be
part of an on-line audio archive. The timings refer to the time display
on the CD or digital tape. The now all have to adjusted so that each
item starts at 0.00 since they have all been edited out of their
context and are now all individual items that start at 00:00. So Item_1
which was started at 00:23 on the tape and ended at 8:23 needs to have
23 seconds subtracted to it so that it says:

Item_1 TAPE_1 1 00:00 08:00

Item_2 TAPE_1 2 08:23 09:41

would change to:

Item_2 TAPE_1 2 00:00 01:18

etc.

but as always you may notice a wrinkle.... some items have many times
(here 6) indicated:

Item_3 TAPE_1 3 9:41 10:41
Item_3 TAPE_1 4 10:47 11:19
Item_3 TAPE_1 5 11:21 11:55
Item_3 TAPE_1 6 11:58 12:10
Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape sound.
Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape sound.

This is all a single sound file and these separate times mark where
there was a break, defect, or edit in the individual item. These have
to be adjusted as well to show where these events would appear in the
new sound file which now starts at 00:00.

Item_3 TAPE_1 3 00:00 01:00 ----
Item_3 TAPE_1 4 01:00 01:38 ----
Item_3 TAPE_1 5 01:38 02:14 ----
Item_3 TAPE_1 6 02:14 02:29 ----
Item_3 TAPE_1 7 02:29 03:04 Defect in analog tape sound.
Item_3 TAPE_1 8 03:04 14:39 Defect in analog tape sound.

Further wrinkles: Some have start and end times indicated, some only
start times. I suppose that the output would ideally have both.... some
have comments and others don't ... and I need these comments echo-ed or
since i probably need to make a database or table eventually non
comments just have some place holder.

I'd have a lot of similar type calculations to do... I was hoping and
praying that some one here was feeling generous and show me the way and
then, of course i could modify that to do other tasks... Usually i am
happy to take the long road and all but i'll be honest, i am in a big
jam here and this huge task was just dumped on me. I am frankly a
little desperate for help on this and hoping someone is feeling up to
spoon feeding me a clear modifiable example that works. Sorry.....
cheers,

kevin

-- http://mail.python.org/mailman/listinfo/python-list """

import optparse
import re

pat = re.compile('\s+ ')

class Item:

def __init__(self, line):
parts = pat.split(line)
self.name, self.tape, self.number, self.start = parts[:4]
if len(parts) == 5:
self.end = parts[4]
else:
self.end = None
if len(parts) > 5:
self.comment = ' '.join(parts[5:])
else:
self.comment = None

class Parser:

def __init__(self):
self.items = []

def feed(self, line):
item = Item(line)
self.items.appe nd(item)

def parseCommandLin e(usage, requiredArgCoun t, argv=None):
"""Parse the command line and return (options, args).

Raise an error if there are insufficient positional arguments as
specified by requiredArgCoun t.
"""
parser = optparse.Option Parser(usage)
## parser.add_opti on('-x',
## '--xxx',
## action='',
## default='',
## help='')
options, args = parser.parse_ar gs(argv)
if len(args) < requiredArgCoun t:
parser.error('M issing parameters.')
return options, args

def main(argv=None) :
usage = __doc__
requiredArgCoun t = 0
options, args = parseCommandLin e(usage, requiredArgCoun t, argv)
filename = args[0]
parser = Parser()
for line in raw.split('\n') :
if not line.startswith ('Item_'):
continue
parser.feed(lin e)

if __name__ == '__main__':
main()

[sp1d3rx@gmail.com]

Re: Time script help sought!

read below for my sample script....

kpp9c wrote:[color=blue]
> I am kind of in a bit of a jam (okay a big jam) and i was hoping[/color]
that[color=blue]
> someone here could give me a quick hand. I had a few pages of time
> calculations to do. So, i just started in on them typing them in my
> time calculator and writing them in by hand. Now i realize, that i
> really need a script to do this because:
>
> 1. It turns out there are hundreds of pages of this stuff.
> 2. I have to do something similar in again soon.
> 3. By doing it by hand i am introducing wonderful new errors!
> 4. It all has to be typed up anyway (which means weeks of work and[/color]
even[color=blue]
> more typos!)
>
> The input would like so:
>
> Item_1 TAPE_1 1 00:23 8:23
>
> Item_2 TAPE_1 2 8:23 9:41
>
> Item_3 TAPE_1 3 9:41 10:41
> Item_3 TAPE_1 4 10:47 11:19
> Item_3 TAPE_1 5 11:21 11:55
> Item_3 TAPE_1 6 11:58 12:10
> Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape[/color]
sound.[color=blue]
> Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape[/color]
sound.[color=blue]
>
> Item_4 TAPE_1 9 24:33
> Item_4 TAPE_1 10 25:48
> Item_4 TAPE_1 11 29:48
> Item_4 TAPE_1 12 31:46
> Item_4 TAPE_1 13 34:17 Electronic sounds.
> Item_4 TAPE_1 14 35:21
> Item_4 TAPE_1 15 36:06
> Item_4 TAPE_1 16 37:01 37:38
>
> These are analog tapes that were digitized (on to CD or a digital[/color]
tape)[color=blue]
> that have now been exported as individual files that are meant to be
> part of an on-line audio archive. The timings refer to the time[/color]
display[color=blue]
> on the CD or digital tape. The now all have to adjusted so that each
> item starts at 0.00 since they have all been edited out of their
> context and are now all individual items that start at 00:00. So[/color]
Item_1[color=blue]
> which was started at 00:23 on the tape and ended at 8:23 needs to[/color]
have[color=blue]
> 23 seconds subtracted to it so that it says:
>
> Item_1 TAPE_1 1 00:00 08:00
>
> Item_2 TAPE_1 2 08:23 09:41
>
> would change to:
>
> Item_2 TAPE_1 2 00:00 01:18
>
> etc.
>
> but as always you may notice a wrinkle.... some items have many times
> (here 6) indicated:
>
> Item_3 TAPE_1 3 9:41 10:41
> Item_3 TAPE_1 4 10:47 11:19
> Item_3 TAPE_1 5 11:21 11:55
> Item_3 TAPE_1 6 11:58 12:10
> Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape[/color]
sound.[color=blue]
> Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape[/color]
sound.[color=blue]
>
> This is all a single sound file and these separate times mark where
> there was a break, defect, or edit in the individual item. These have
> to be adjusted as well to show where these events would appear in the
> new sound file which now starts at 00:00.
>
> Item_3 TAPE_1 3 00:00 01:00 ----
> Item_3 TAPE_1 4 01:00 01:38 ----
> Item_3 TAPE_1 5 01:38 02:14 ----
> Item_3 TAPE_1 6 02:14 02:29 ----
> Item_3 TAPE_1 7 02:29 03:04 Defect in analog tape[/color]
sound.[color=blue]
> Item_3 TAPE_1 8 03:04 14:39 Defect in analog tape[/color]
sound.[color=blue]
>
> Further wrinkles: Some have start and end times indicated, some only
> start times. I suppose that the output would ideally have both....[/color]
some[color=blue]
> have comments and others don't ... and I need these comments echo-ed[/color]
or[color=blue]
> since i probably need to make a database or table eventually non
> comments just have some place holder.
>
> I'd have a lot of similar type calculations to do... I was hoping and
> praying that some one here was feeling generous and show me the way[/color]
and[color=blue]
> then, of course i could modify that to do other tasks... Usually i am
> happy to take the long road and all but i'll be honest, i am in a big
> jam here and this huge task was just dumped on me. I am frankly a
> little desperate for help on this and hoping someone is feeling up to
> spoon feeding me a clear modifiable example that works. Sorry.....
> cheers,
>
> kevin[/color]
--------START--------
inp = file("input1.tx t",'r') # input file is opened readonly
x = inp.readline() #read in the first line of the file
x = x.upper().split (None,5) #convert it to uppercase and split into 5
segments
print x #show the line as splitted and converted
print x[1] #show the second element
start = x[3].split(":") #split the minutes from the seconds
end = x[4].split(":") #split the minutes from the seconds
print "Start at:", start[0], "minutes and ", start[1], "seconds."
start_in_second s = int(start[0])*60 + int(start[1]) #converts
minutes/seconds to seconds
print start_in_second s , "seconds offset."
print "End at:", end[0],"minutes and",end[1], "seconds."
end_in_seconds = int(end[0])*60 + int(end[1])#converts minutes/seconds
to seconds
print end_in_seconds , "seconds offset."
totaltime = end_in_seconds - start_in_second s #calculate the length of
the segment
print "Total time of segment in seconds:", totaltime
print "Total time of segment in minutes/seconds:", totaltime/60,
"minutes and", totaltime % 60, "seconds."
# ^^^ converts seconds back to minutes and seconds.
--------END--------

This should give you an excellent starting point.

[sp1d3rx@gmail.com]

Re: Time script help sought!

Gosh Mark you must be a friggin genius or something. I can't even begin
to read your code. Anyways, I think mine is easier to understand. My
program has all the functions (save a couple) that you would need for
your project. It's an exercise for you to copy and paste what you want
where you want it and to form a basic program structure out of it. some
of my comments spilled on to the next line (thanks google!) so you will
have to clean it up a bit.

[kpp9c]

Re: Time script help sought!

Thanks for this Everyone!

Trying to work with all the stuff folks are giving me on this i a have
come across a problem... down
the line i notice that some of the times will also have an hour as well
as in H:M:S (e.g. 1:22:40)

so in some cases i would need to convert H:M:S to sec and some just M:S


or should there just be a fun that passes all the times and converts
them to H:M:S first and
just appends a 00: ((e.g. 00:02:07) if there is no hour value?
cheers,

-kp

[sp1d3rx@gmail.com]

Re: Time script help sought!

using my code above...
start = x[3].split(":") #split the minutes from the seconds
this splits something like 1:22:40 into three parts
1(hours), 22(mintes), 40(seconds)
so, you must add a special case handler...
if len(start) == 3:
{
start[1] = int(start[0]) * 60 + int(start[1])
start = start[1], start[2]
}
and there you go....
what this does is take the hours , multiply them by 60 and add it to
the minutes.
You will also have to change your output function to convert from
"seconds" to "hours, minutes, seconds".

[kpp9c]

Re: Time script help sought!

I also notice that there is the is the 'datetime' module, which is new
to version 2.3, which i now have access to. My feeling is that this
will do much of what i want, but i can't get my head round the standard
library reference stuff

http://www.python.org/doc/lib/module-datetime.html

I don't have any texts with me either and it probably is too new to be
in the Python Standard Library book by Fredrik Lundh or the Python
Essential Reference by David Beazley


-kevin-

[sp1d3rx@gmail.com]

Re: Time script help sought!

post your script here so we can go over it with you.

[kpp9c]

Re: Time script help sought!

still working on it and also fixing the input data. I think for
simplicity and consistency's sake i will have *all* time values input
and output as hh:mm:ss maybe that would be easier.... but i have a few
thousand find and replaceeseseses to do now (yes i am doing them by
hand)

grr... this is hard!

[kpp9c]

Re: Time script help sought!

so all the imput will look more like this now... ( no comments either)

tem_133, DAT_20, 7, 00:58:25, 01:15:50

Item_134, DAT_20, 8, 01:15:50, 01:32:15

Item_135, DAT_21, 1, 00:01:00, 00:36:15

Item_136, DAT_60, 3, 00:18:30
Item_136, DAT_60, 4, 00:19:30
Item_136, DAT_60, 5, 00:23:00, 00:28:00

Item_137, DAT_21, 4, 00:37:00, 00:47:00

Item_139, DAT_21, 2, 00:36:15
Item_139, DAT_21, 3, 00:42:15, 01:00:50

Item_140, DAT_21, 4, 01:00:50
Item_140, DAT_21, 5, 01:25:10, 01:26:35

...... snip...

[Paul McGuire]

Re: Time script help sought!

"kpp9c" <kp8@mac.com> wrote in message
news:1105480151 .431987.327080@ c13g2000cwb.goo glegroups.com.. .[color=blue]
> still working on it and also fixing the input data. I think for
> simplicity and consistency's sake i will have *all* time values input
> and output as hh:mm:ss maybe that would be easier.... but i have a few
> thousand find and replaceeseseses to do now (yes i am doing them by
> hand)
>
> grr... this is hard!
>[/color]

Oh, I wasn't going to chime in on this thread, your data looked so
well-formed that I wouldn't recommend pyparsing, but there is enough
variability going on here, I thought I'd give it a try. Here's a pyparsing
treatment of your problem. It will accommodate trailing comments or none,
leading hours or none on timestamps, and missing end times, and normalizes
all times back to the item start time.

Most of your processing logic will end up going into the processVals()
routine. I've put various examples of how to access the parsed tokens by
field name, and some helper methods for converting to and from seconds and
hh:mm:ss or mm:ss times.

-- Paul


from pyparsing import *

data = """
Item_1 TAPE_1 1 00:23 8:23

Item_2 TAPE_1 2 8:23 9:41

Item_3 TAPE_1 3 9:41 10:41
Item_3 TAPE_1 4 10:47 11:19
Item_3 TAPE_1 5 11:21 11:55
Item_3 TAPE_1 6 11:58 12:10
Item_3 TAPE_1 7 12:15 12:45 Defect in analog tape sound.
Item_3 TAPE_1 8 12:58 24:20 Defect in analog tape sound.

Item_4 TAPE_1 9 24:33
Item_4 TAPE_1 10 25:48
Item_4 TAPE_1 11 29:48
Item_4 TAPE_1 12 31:46
Item_4 TAPE_1 13 34:17 Electronic sounds.
Item_4 TAPE_1 14 35:21
Item_4 TAPE_1 15 36:06
Item_4 TAPE_1 16 37:01 01:37:38
"""

def toSecs(tstr):
fields = tstr.split(":")
secs = int(fields[-1])
secs += int(fields[-2])*60
if len(fields)>2: secs += int(fields[-3])*60*60
return secs

def secsToTime(secs ):
s = secs % 60
m = ((secs - s) / 60 ) % 60
h = (secs >= 3600 and (secs - s - m*60 ) / 3600 or 0)
return "%02d:%02d:%02d " % (h,m,s)

# globals for normalizing timestamps
lastItem = ""
itemStart = 0

# put logic here for processing various parse fields
def processVals(s,l ,t):
global lastItem,itemSt art
print t.item,t.tape,t .recnum
if not t.item == lastItem :
lastItem = t.item
itemStart = toSecs(t.start)

startSecs = toSecs(t.start)
print secsToTime(star tSecs),"(%s)" % secsToTime(star tSecs-itemStart)

if t.end:
endSecs = toSecs(t.end)
print secsToTime(endS ecs),"(%s)" % secsToTime(endS ecs-itemStart)
print endSecs-startSecs,"elap sed seconds"
print secsToTime(endS ecs-startSecs),"ela psed time"
else:
print "<no end time>"
print t.comment
print

# define structure of a line of data - sorry about the clunkiness of the
optional trailing fields
integer = Word(nums)
timestr = Combine(integer + ":" + integer + Optional(":" + integer))
dataline = ( Combine("Item_" +integer).setRe sultsName("item ") +
Combine("TAPE_" +integer).setRe sultsName("tape ") +
integer.setResu ltsName("recnum ") +
timestr.setResu ltsName("start" ) +
Optional(~LineE nd() + timestr, default="").set ResultsName("en d")
+
Optional(~LineE nd() + empty +
restOfLine,defa ult="-").setResultsNa me("comment") )

# set up parse handler that will process the actual fields
dataline.setPar seAction(proces sVals)

# now parse the little buggers
OneOrMore(datal ine).parseStrin g(data)

will print out:

Item_1 TAPE_1 1
00:00:23 (00:00:00)
00:08:23 (00:08:00)
480 elapsed seconds
00:08:00 elapsed time
-

Item_2 TAPE_1 2
00:08:23 (00:00:00)
00:09:41 (00:01:18)
78 elapsed seconds
00:01:18 elapsed time
-

Item_3 TAPE_1 3
00:09:41 (00:00:00)
00:10:41 (00:01:00)
60 elapsed seconds
00:01:00 elapsed time
-

Item_3 TAPE_1 4
00:10:47 (00:01:06)
00:11:19 (00:01:38)
32 elapsed seconds
00:00:32 elapsed time
-

Item_3 TAPE_1 5
00:11:21 (00:01:40)
00:11:55 (00:02:14)
34 elapsed seconds
00:00:34 elapsed time
-
....

[sp1d3rx@gmail.com]

Re: Time script help sought!

man, now that is beautifully done. Paul, I wish I knew about pyparsing
a while ago. I could have used it in a few projects. :)

[Paul McGuire]

Re: Time script help sought!

<sp1d3rx@gmail. com> wrote in message
news:1105485509 .346577.292130@ f14g2000cwb.goo glegroups.com.. .[color=blue]
> man, now that is beautifully done. Paul, I wish I knew about pyparsing
> a while ago. I could have used it in a few projects. :)
>[/color]
Thanks for the compliment! I'll be the first to admit that pyparsing can be
a bit persnickety in some applications, especially when you have to trap on
end-of-line, or have some columnar-dependent syntax. pyparsing really works
best with data that has some syntactic structure to it, but is not
whitespace sensitive (pyparsing's default behavior is to skip over
whitespace). I don't recommend it when the data is well-formed - regexp's
or even just string.split() will blow it away from a performance (and
simplicity) standpoint. For instance, if the OP knew that all his data was
itemId, tapeId, recordNum, startTime, endTime, comment, he could have most
easily parsed it with

item, tape, recordNum, startTime, endTime, comment = dataline.split( None,5)

It's just those optional and variable-formatted fields that getcha. :)

-- Paul

pyparsing can be downloaded at http://pyparsing.sourceforge.net.