playing with pyGoogle - strange codec error

[Brian Blazer]

Hello,

I am playing around with pyGoogle and encountered an error that I have
never seen, and I am unsure how to correct for it. Here is a code
snippet:

for r in data.results:
print 'Title: ',r.title
print 'URL: ',r.URL
print 'Summary: ',r.snippet
print

Everything works fine until I get to r.snippet. Here is the error:

UnicodeEncodeEr ror: 'ascii' codec can't encode character '\ua9' in
position 119: ordinal not in range(128)

Any help is appreciated.

Thanks,
Brian
--
Nail a post to the Spalted Board. Free WW'ing software and forums.
Regular freebies! http://www.spaltedboard.com

[Richard Brodie]

Re: playing with pyGoogle - strange codec error


"Brian Blazer" <brian@brianand kate.com> wrote in message
news:2005040410 062316807%brian @brianandkateco m...
[color=blue]
> Everything works fine until I get to r.snippet. Here is the error:
>
> UnicodeEncodeEr ror: 'ascii' codec can't encode character '\ua9' in
> position 119: ordinal not in range(128)[/color]

You have a character there (the copyright sign) that isn't in the ASCII
set. If you have anything other than plain ASCII, you need to consider
encoding. This reference might help:

http://diveintopython.org/xml_processing/unicode.html

[Brian Blazer]

Re: playing with pyGoogle - strange codec error

On 2005-04-05 07:32:12 -0500, "Richard Brodie" <R.Brodie@rl.ac .uk> said:
[color=blue]
>
> "Brian Blazer" <brian@brianand kate.com> wrote in message
> news:2005040410 062316807%brian @brianandkateco m...
>[color=green]
>> Everything works fine until I get to r.snippet. Here is the error:
>>
>> UnicodeEncodeEr ror: 'ascii' codec can't encode character '\ua9' in
>> position 119: ordinal not in range(128)[/color]
>
> You have a character there (the copyright sign) that isn't in the ASCII
> set. If you have anything other than plain ASCII, you need to consider
> encoding. This reference might help:
>
> http://diveintopython.org/xml_processing/unicode.html[/color]

Thank you. That is a great reference.

Brian

--
Nail a post to the Spalted Board. Free WW'ing software and forums.
Regular freebies! http://www.spaltedboard.com

[Brian Blazer]

Re: playing with pyGoogle - strange codec error

On 2005-04-04 10:06:23 -0500, Brian Blazer <brian@brianand kate.com> said:

<snip>

You know, I am beginning to think that I MAY have stumbled on a bug
here. At first I was thinking that this issue was related to the
offending character being out of range for the Mac. Then I tried it on
A MS machine and a linux box; all with the same error.

This does not happen when I wrote the same script in java. This is
making me wonder if there is an issue with the wrapper for the google
api that was originally done in java.

For the sake of it, here is the full code (minus my google key). It is
going to look wierd, but those print statements are there so that I
dont have to open the file it is writing to every time I want to see
stuff. it has my name hard coded into the search query. The commented
r.snippet.encod e(mac_roman) was there to see if by changing the
encoding, I could make it work (no luck). I also tried putting

#-*- coding: utf-8 -*-

right after the shebang (as listed here:
http://www.python.org/peps/pep-0263.html). Again, no help.

Anyway, here is the code ------------------------>

import google

google.LICENSE_ KEY = 'insertKeyHere'
#print google.doSpelli ngSuggestion('h elllo')
data = google.doGoogle Search('Brian Blazer')
print 'Found %d results' % len(data.result s)

searchData = open('searchDat a.txt','w')

for r in data.results:
# r.snippet.encod e('mac_roman')
searchData.writ e ('Title: ' + r.title + '\n' + '\n')
searchData.writ e ('URL: ' + r.URL + '\n' + '\n')
searchData.writ e ('Snippet: ' + r.snippet + '\n' + '\n'+'\n')
print r.URL
print r.title
print r.snippet



--
Nail a post to the Spalted Board. Free WW'ing software and forums.
Regular freebies! http://www.spaltedboard.com

[Erik Max Francis]

Re: playing with pyGoogle - strange codec error

Brian Blazer wrote:
[color=blue]
> You know, I am beginning to think that I MAY have stumbled on a bug
> here. At first I was thinking that this issue was related to the
> offending character being out of range for the Mac. Then I tried it on
> A MS machine and a linux box; all with the same error.[/color]

The problem, common to all three, is that you're using a terminal whose
default encoding doesn't specify a valid encoding for the copyright
character (in the first case, the default encoding is 'ascii'; it is
likely the case for the others, as well).

When you print a Unicode string, by default it is encoded to your
default encoding. The problem is this cannot be done faithfully with a
string containing a non-ASCII symbol (like the copyright character which
is actually triggering it for you). So, consequently, the encoding is
failing with an error.

What you probably want here is either to use another encoding, or to
specify what to do in the case that the encoding is not possible.
Either encode to a different encoding (one which you know your terminal
supports even though it is not detected, e.g., 'latin-1'), or specify
what to do with errors in the encoding (e.g., 'ignore', which removes
the offending characters, or 'replace', which replaces them with
question marks):

aUnicodeString. decode('latin-1')
aUnicodeString. decode('ascii', 'replace')
[color=blue]
> This does not happen when I wrote the same script in java. This is
> making me wonder if there is an issue with the wrapper for the google
> api that was originally done in java.[/color]

Java does not handle Unicode the same way.

--
Erik Max Francis && max@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Drifting from woman-who-tries misconstrued / Shifting to woman-wise
-- Lamya

[Brian Blazer]

Re: playing with pyGoogle - strange codec error

On 2005-04-05 13:55:48 -0500, Erik Max Francis <max@alcyone.co m> said:
<snip>

Thank you, that worked.

Brian

--
Nail a post to the Spalted Board. Free WW'ing software and forums.
Regular freebies! http://www.spaltedboard.com