py-serial + CSV

[McBooCzech]

Hi
I am just trying to analyze (parse) data from the serial port (I have
connected GPS receiver to the ttyS0, so I can read ASCII characters in
the CSV form on the serial port 1).
I am doing this just to understand how Python works (yes, you can call
me Python/Linux newbie :)
My environment is Fedora Core 4, Python 2.4.1

CSV alone (to read CSV data from the file) and py-serial alone (to
read data from the serial port) are working flawlessly.

Even I was trying to google through this group and through the
Internet, I am not able to read (and parse) CSV data directly from the
serial port.

data from my serial port (using py-serial) I am getting this way:
[color=blue][color=green][color=darkred]
>>> import serial
>>> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
>>> s.readline()[/color][/color][/color]
'$GPRMC,101236. 331,A,5026.1018 ,N,01521.6653,E ,0.0,328.1,2308 05,,*09\r\n'

my next intention was to do something like this:

import csv
r = csv.reader(s.re adline())
for currentline in r:
if currentline[0] == '$GPRMC':
print currentline[2]
print currentline[4]

but it does not work

Thanks for your comments

Petr Jakes

[Michael Hoffman]

Re: py-serial + CSV

McBooCzech wrote:
[color=blue]
> r = csv.reader(s.re adline())[/color]

csv.reader expects an iterable, not a str.
--
Michael Hoffman

[McBooCzech]

Re: py-serial + CSV

So do I have to save to the file first and analyze later on?

Or there is an other way how to process it (read from the serial and
analyze data) on the fly?

Petr Jakes

[Michael Hoffman]

Re: py-serial + CSV

McBooCzech wrote:[color=blue]
> So do I have to save to the file first and analyze later on?[/color]

Certainly not.
[color=blue]
> Or there is an other way how to process it (read from the serial and
> analyze data) on the fly?[/color]

I've never used py-serial, so I was hoping someone else would jump in
with the best way to do this (or perhaps you can find it in the
documentation). One way would be to use the iter() built-in to make it
into an iterable, like this:

iterable = iter(s.readline , "")

Note that you do not call s.readline() here; you are only supplying the
bound method itself as an argument. iter() will call it later. I'm
assuming that s.readline returns an empty string when there is no more
input. Please replace that with whatever end sentinel it actually uses.
(None?)

Another way would be to use s.readlines(), which returns a list, which
is iterable, but you would have to wait until the entire list were
collected before beginning processing.
--
Michael Hoffman

[Grant Edwards]

Re: py-serial + CSV

On 2005-08-23, McBooCzech <petr@tpc.cz> wrote:
[color=blue][color=green][color=darkred]
>>>> import serial
>>>> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
>>>> s.readline()[/color][/color]
> '$GPRMC,101236. 331,A,5026.1018 ,N,01521.6653,E ,0.0,328.1,2308 05,,*09\r\n'
>
> my next intention was to do something like this:
>
> import csv
> r = csv.reader(s.re adline())
> for currentline in r:
> if currentline[0] == '$GPRMC':
> print currentline[2]
> print currentline[4]
>
> but it does not work[/color]

For something that simple (the data itself doesn't contain any
commas), it's probably easier to use the string's split method
rahter than CSV. Try something like this:

line = s.readline()
words = line.split(',')

--
Grant Edwards grante Yow! Now KEN and BARBIE
at are PERMANENTLY ADDICTED to
visi.com MIND-ALTERING DRUGS...

[Dennis Lee Bieber]

Re: py-serial + CSV

On Tue, 23 Aug 2005 14:11:59 +0100, Michael Hoffman
<cam.ac.uk@mh39 1.invalid> declaimed the following in comp.lang.pytho n:
[color=blue]
> Note that you do not call s.readline() here; you are only supplying the
> bound method itself as an argument. iter() will call it later. I'm
> assuming that s.readline returns an empty string when there is no more
> input. Please replace that with whatever end sentinel it actually uses.
> (None?)
>
> Another way would be to use s.readlines(), which returns a list, which
> is iterable, but you would have to wait until the entire list were
> collected before beginning processing.[/color]

The latter wouldn't work in this situation -- as the only way to end
the list is to disconnect the serial port <G>

Most GPS receivers, when set to send the messages, will do so at a
rate of one message every 1 or 2 seconds, as I recall.
--[color=blue]
> =============== =============== =============== =============== == <
> wlfraed@ix.netc om.com | Wulfraed Dennis Lee Bieber KD6MOG <
> wulfraed@dm.net | Bestiaria Support Staff <
> =============== =============== =============== =============== == <
> Home Page: <http://www.dm.net/~wulfraed/> <
> Overflow Page: <http://wlfraed.home.ne tcom.com/> <[/color]

[McBooCzech]

Re: py-serial + CSV

Sorry, I did not mentioned the data flow from the serial port is
permanent/continuous (as long as the GPS receiver is connected to the
serial port). The input data are commning every second, they are comma
separated and they are looking like:

$GPGGA,174525.6 17,5026.1080,N, 01521.6724,E,1, 05,1.8,306.5,M, ,,,0000*0B
$GPGSA,A,3,02,0 9,05,06,14,,,,, ,,,3.6,1.8,3.1* 31
$GPGSV,3,1,09,3 0,74,294,,05,65 ,093,49,06,40,2 23,32,02,39,089 ,49*78
$GPRMC,174525.6 17,A,5026.1080, N,01521.6724,E, 0.0,005.8,23080 5,,*0A
etc....
[color=blue]
>From the rows they are begining with $GPRMC I want to get following[/color]
data only (positions 2,4,6)
174525.617
5026.1080
01521.672

This (according to your suggestions) is my code which works for me

import serial
s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
while 1:
line = s.readline()
words = line.split(',')
if words[0]=="$GPRMC":
print words[1], words[3], words[5]

I just wonder if there is some beter (or as you are saying "more
pythonic":) aproach how to write such a piece of code.
--
Petr Jakes

[Steve Holden]

Re: py-serial + CSV

McBooCzech wrote:[color=blue]
> Sorry, I did not mentioned the data flow from the serial port is
> permanent/continuous (as long as the GPS receiver is connected to the
> serial port). The input data are commning every second, they are comma
> separated and they are looking like:
>
> $GPGGA,174525.6 17,5026.1080,N, 01521.6724,E,1, 05,1.8,306.5,M, ,,,0000*0B
> $GPGSA,A,3,02,0 9,05,06,14,,,,, ,,,3.6,1.8,3.1* 31
> $GPGSV,3,1,09,3 0,74,294,,05,65 ,093,49,06,40,2 23,32,02,39,089 ,49*78
> $GPRMC,174525.6 17,A,5026.1080, N,01521.6724,E, 0.0,005.8,23080 5,,*0A
> etc....
>[color=green]
>>From the rows they are begining with $GPRMC I want to get following[/color]
> data only (positions 2,4,6)
> 174525.617
> 5026.1080
> 01521.672
>
> This (according to your suggestions) is my code which works for me
>
> import serial
> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
> while 1:
> line = s.readline()
> words = line.split(',')
> if words[0]=="$GPRMC":
> print words[1], words[3], words[5]
>
> I just wonder if there is some beter (or as you are saying "more
> pythonic":) aproach how to write such a piece of code.[/color]

That code is quite tidy. You could save yourself the split on lines that
weren't of interest, though frankly this isn't essential - this task
won't use 1% of CPU on almost any computer built in the last five years.
But, if you are interested in seeing other solutions you might consider
it, and it does avoid the split when it's not necessary.

while 1:
line = s.readline()
if line.startswith ("$GPRMC"):
words = line.split(",")
print words[1], words[3], words[5]

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

[Sergei Organov]

Re: py-serial + CSV

Steve Holden <steve@holdenwe b.com> writes:[color=blue]
> McBooCzech wrote:[/color]
[...][color=blue][color=green]
> > $GPRMC,174525.6 17,A,5026.1080, N,01521.6724,E, 0.0,005.8,23080 5,,*0A
> > etc....
> >[/color]
>[/color]
[...][color=blue][color=green]
> > s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
> > while 1:
> > line = s.readline()
> > words = line.split(',')
> > if words[0]=="$GPRMC":
> > print words[1], words[3], words[5]
> > I just wonder if there is some beter (or as you are saying "more[/color]
>[color=green]
> > pythonic":) aproach how to write such a piece of code.[/color]
>
> That code is quite tidy. You could save yourself the split on lines
> that weren't of interest, though frankly this isn't essential - this
> task won't use 1% of CPU on almost any computer built in the last five
> years. But, if you are interested in seeing other solutions you might
> consider it, and it does avoid the split when it's not necessary.
>
>
> while 1:
> line = s.readline()
> if line.startswith ("$GPRMC"):[/color]

"$GPRMC," would be better, -- not to match something like "$GPRMC174. ..".

--
Sergei.

[McBooCzech]

Re: py-serial + CSV

Sergei, I do not realy understand your comment??? Am I missing
something?

BTW, is there some possibility to address lists like:

print words [1; 3; 5]
instead of
print words[1], words[3], words[5]

Petr Jakes

[Dennis Lee Bieber]

Re: py-serial + CSV

On 23 Aug 2005 12:59:03 -0700, "McBooCzech " <petr@tpc.cz> declaimed the
following in comp.lang.pytho n:
[color=blue]
> Sergei, I do not realy understand your comment??? Am I missing
> something?
>
> BTW, is there some possibility to address lists like:
>
> print words [1; 3; 5]
> instead of
> print words[1], words[3], words[5]
>[/color]

Mix & Matching from two messages <G>

2>$GPRMC,174525 .617,A,5026.108 0,N,01521.6724, E,0.0,005.8,230 805,,*0A
2>etc....
2>
2>>From the rows they are begining with $GPRMC I want to get following
2>data only (positions 2,4,6)
2>174525.617
2>5026.1080
2>01521.672
2>
Interesting that you don't want to handle conditions crossing
Greenwich/dateline, or equator (since you ignore the N/S, E/W tags.

2>This (according to your suggestions) is my code which works for me
2>
2>import serial
2>s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
2>while 1:
2> line = s.readline()
2> words = line.split(',')
2> if words[0]=="$GPRMC":
2> print words[1], words[3], words[5]

Well... Let's see

#open port, etc...
while True:
line = s.readline()
if line.startswith ("$GPRMC"):
(msg, timestamp, junk,
raw_latitude, NS_indicator,
raw_longitude) = line.split(",")[:6]
print timestamp, raw_latitude, raw_longitude

[color=blue][color=green][color=darkred]
>>> line = "$GPRMC,174525. 617,A,5026.1080 ,N,01521.6724,E ,0.0,005.8,2308 05,,*0A"
>>> (msg, timestamp, junk,[/color][/color][/color]
.... raw_latitude, NS_indicator,
.... raw_longitude) = line.split(",")[:6][color=blue][color=green][color=darkred]
>>> print timestamp, raw_latitude, raw_longitude[/color][/color][/color]
174525.617 5026.1080 01521.6724[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]
--[color=blue]
> =============== =============== =============== =============== == <
> wlfraed@ix.netc om.com | Wulfraed Dennis Lee Bieber KD6MOG <
> wulfraed@dm.net | Bestiaria Support Staff <
> =============== =============== =============== =============== == <
> Home Page: <http://www.dm.net/~wulfraed/> <
> Overflow Page: <http://wlfraed.home.ne tcom.com/> <[/color]

[Sybren Stuvel]

Re: py-serial + CSV

McBooCzech enlightened us with:[color=blue]
> This (according to your suggestions) is my code which works for me
>
> import serial
> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
> while 1:
> line = s.readline()
> words = line.split(',')
> if words[0]=="$GPRMC":
> print words[1], words[3], words[5]
>
> I just wonder if there is some beter (or as you are saying "more
> pythonic":) aproach how to write such a piece of code.[/color]

You could use regular expressions instead. And to make it even more
pythonic, replace the "while" and the "line = s.readline()" with
"for line in s:"

Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa

[Grant Edwards]

Re: py-serial + CSV

On 2005-08-24, Sybren Stuvel <sybrenUSE@YOUR thirdtower.com. imagination> wrote:
[color=blue][color=green]
>> import serial
>> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
>> while 1:
>> line = s.readline()
>> words = line.split(',')
>> if words[0]=="$GPRMC":
>> print words[1], words[3], words[5]
>>
>> I just wonder if there is some beter (or as you are saying "more
>> pythonic":) aproach how to write such a piece of code.[/color]
>
> You could use regular expressions instead.[/color]

"There is a programmer who has a problem to solve. He decides
to use regular expressions. Now he has two problems."
[color=blue]
> And to make it even more pythonic, replace the "while" and the
> "line = s.readline()" with "for line in s:"[/color]

Serial port objects aren't iterable.

--
Grant Edwards grante Yow! I was making donuts
at and now I'm on a bus!
visi.com

[Michael Hoffman]

Re: py-serial + CSV

McBooCzech wrote:[color=blue]
> This (according to your suggestions) is my code which works for me
>
> import serial
> s = serial.Serial(p ort=0,baudrate= 4800, timeout=20)
> while 1:
> line = s.readline()
> words = line.split(',')
> if words[0]=="$GPRMC":
> print words[1], words[3], words[5]
>
> I just wonder if there is some beter (or as you are saying "more
> pythonic":) aproach how to write such a piece of code.[/color]

import csv

from serial import Serial

port = Serial(port=0, baudrate=4800, timeout=20)

for row in csv.reader(iter (port.readline, None)):
if row[0] == "$GPMRC":
print row[1], row[3], row[5]
--
Michael Hoffman

[McBooCzech]

Re: py-serial + CSV

Thanks you all, guys, for your suggestions and help. Everything now
works great :)
Regards
Petr Jakes