python skipping lines?

[lisa.engblom@gmail.com]

Hi,

I've just started programming in python, and have run into an
unexpected problem. I am using python to pull text data from some csv
files. I have one file that has the important identifiers (name, etc)
and other files with lots of other data. I wrote a function that takes
the file name and identifiers as inputs, selects the data I need, and
outputs it to another file. This function (PullData) seems to work
fine.

In the same .py file, but separate from the function, I entered used
"for line in file" to step through each line in my identifier file and
feed the identifiers into my PullData function. It works fine for the
first entry in my identifier file, but it will not call the function
again. It will execute lines before or after the appropriate number of
times given the number of lines in the identifier file. i just put in
a dummy counter variable to try to figure out why it wasn't working)
.... but it always skips the function call after the first line.

Any ideas of what could be the problem?

[Jordan Greenberg]

Re: python skipping lines?

lisa.engblom@gm ail.com wrote:

Hi,

<SNIP>

Any ideas of what could be the problem?
>

Hard to say without seeing your code.

Jordan Greenberg

--
Posted via a free Usenet account from http://www.teranews.com

[lisa.engblom@gmail.com]

Re: python skipping lines?

thats easy enough to solve

"""test text.py
for playing around with my text editing task
"""

UnitList = open('/Python25/working/FacList.txt', 'r')
RawData = open('/Python25/working/data.txt', 'r')
Output = open('/Python25/working/output.txt', 'a')

def PullHourlyData( filename, facility, unit):
for line in filename:
data = line.split('"," ')
CurrentFacility = data[1]
CurrentUnit = data[3]
CurrentSO2Mass = data[9] #in lb/hour
#print CurrentFacility
#print CurrentUnit

if facility == CurrentFacility and unit == CurrentUnit:

#print >Output, '"%s", "%s", "%s"' %
(CurrentFacilit y, CurrentUnit, CurrentSO2Mass)
print '"%s", "%s", "%s"' % (CurrentFacilit y,
CurrentUnit, CurrentSO2Mass)
else:
print facility
print unit
print CurrentFacility
print CurrentUnit
print "\n"


counter = 0

for combos in UnitList:
print counter
FandU = combos.split('" ,"')
#print combos
FacilityName = FandU[0]
UnitName = FandU[1]
#print FacilityName
#print UnitName
FacilityName = FacilityName.st rip('"')
UnitName = UnitName.strip( '",\n')
print FacilityName
print UnitName

PullHourlyData( RawData, FacilityName, UnitName)
counter += 1


UnitList.close( )
RawData.close()
Output.close()
print "Done!"

Jordan Greenberg wrote:

lisa.engblom@gm ail.com wrote:

Hi,

<SNIP>

Any ideas of what could be the problem?

>
Hard to say without seeing your code.
>
Jordan Greenberg
>
--
Posted via a free Usenet account from http://www.teranews.com

[Larry Bates]

Re: python skipping lines?

lisa.engblom@gm ail.com wrote:

thats easy enough to solve
>
"""test text.py
for playing around with my text editing task
"""
>
UnitList = open('/Python25/working/FacList.txt', 'r')
RawData = open('/Python25/working/data.txt', 'r')
Output = open('/Python25/working/output.txt', 'a')
>
def PullHourlyData( filename, facility, unit):
for line in filename:
data = line.split('"," ')
CurrentFacility = data[1]
CurrentUnit = data[3]
CurrentSO2Mass = data[9] #in lb/hour
#print CurrentFacility
#print CurrentUnit
>
if facility == CurrentFacility and unit == CurrentUnit:
>
#print >Output, '"%s", "%s", "%s"' %
(CurrentFacilit y, CurrentUnit, CurrentSO2Mass)
print '"%s", "%s", "%s"' % (CurrentFacilit y,
CurrentUnit, CurrentSO2Mass)
else:
print facility
print unit
print CurrentFacility
print CurrentUnit
print "\n"
>
>
counter = 0
>
for combos in UnitList:
print counter
FandU = combos.split('" ,"')
#print combos
FacilityName = FandU[0]
UnitName = FandU[1]
#print FacilityName
#print UnitName
FacilityName = FacilityName.st rip('"')
UnitName = UnitName.strip( '",\n')
print FacilityName
print UnitName
>
PullHourlyData( RawData, FacilityName, UnitName)
counter += 1
>
>
UnitList.close( )
RawData.close()
Output.close()
print "Done!"
>
Jordan Greenberg wrote:

>lisa.engblom@gm ail.com wrote:

>>Hi,

><SNIP>

>>Any ideas of what could be the problem?
>>>

>Hard to say without seeing your code.
>>
>Jordan Greenberg
>>
>--
>Posted via a free Usenet account from http://www.teranews.com

>

The first time through the loop you read through RawData and are at
the bottom of the file. You either need to seek back to the beginning
or you need to close and reopen the file each time through the loop.

Suggestions:

1) In PullHourData the first argument is filename. In fact
it is not a filename but rather a file pointer. Just a little
confusing for anyone coming along behind you.

2) If the data is well-formed CSV you should probably take a look
at the csv module. It handles CSV data better than splitting on
commas (which can be dangerous as there can be commas inside of
literal data).

-Larry

[Jussi Salmela]

Re: python skipping lines?

lisa.engblom@gm ail.com wrote:

RawData = open('/Python25/working/data.txt', 'r')

You open this file only once. The first time in here:

def PullHourlyData( filename, facility, unit):
for line in filename:

reads all of the file - nothing left for the other function calls!

A better way would be to read the file in once to a list and give that
lkist to the function to process.

HTH,
Jussi

[Tim Chase]

Re: python skipping lines?

I'm not sure if this will /solve/ your problem, but it's
something I noticed...

UnitList = open('/Python25/working/FacList.txt', 'r')
RawData = open('/Python25/working/data.txt', 'r')

Here, you open RawData once...

Output = open('/Python25/working/output.txt', 'a')
>
def PullHourlyData( filename, facility, unit):
for line in filename:

[cut]

counter = 0
>
for combos in UnitList:

[cut]

PullHourlyData( RawData, FacilityName, UnitName)
counter += 1

and your first pass through this loop, you exhaust RawData by
reading to the end. The second pass through the UnitList loop,
you pass the exhausted RawData to PullHourlyData( ). I'm thinking
you'd need to RawData.seek(0) or some such "rewind" ability. A
slightly ugly alternative would be just opening/closing the
RawData file for each pass through the loop. Another, possibly
cleaner option (depending on the size of RawData's contents)
would be to just read the whole thing into an in-memory list with
something like

data = RawData.readlin es()

and then just pass "data" to PullHourlyData( ) each time...no need
to rewind (like VHS vs. DVD :)

Part of the confusion stems from the fact that what you refer to
as "filename" (sounds like it should be a string containing the
path to the file) is actually a file object that contains state.

-tkc

[lisa.engblom@gmail.com]

Re: python skipping lines?

Thanks, everyone for replying so promptly. I got it to work the way I
intended, and have some ideas for how to make it much cleaner.

- Lisa