Hi,
I'm writting a application using python standard logging system. I
encounter some problem with unicode message passed to logging library.
I found that unicode message will be messed up by logging handler.
piese of StreamHandler:
try:
self.stream.wri te(fs % msg)
except UnicodeError:
self.stream.wri te(fs % msg.encode("UTF-8"))
It just write the message to stream. If there is some unicode error,
it would rewrite msg with utf8 encoding.
I write some code to try:
import sys
print u'¤¤¤å¦r´ú¸Õ'
print sys.stdout.enco ding
sys.stdout.writ e(u'¤¤¤å')
result of that program:
¤¤¤å¦r´ú¸Õ
cp950
Traceback (most recent call last):
File "update_stockpr ice.py", line 92, in <module>
sys.stdout.writ e(u'ä¸æ?')
UnicodeEncodeEr ror: 'ascii' codec can't encode characters in position
0-1: ordin
al not in range(128)
It show that....
1. print statement encode what it get with stream.encoding ?
2. stream.write don't do anything like encoding, just write it
(because it might be binary data?)
So the problem is : the StreamHandler of standard logging library use
stream.write to log message, if there is unicode error, unicode string
will be encode to utf8. This behavior mess my unicode up.
Here I modify the code of StreamHandler:
try:
print >self.stream, msg
#self.stream.wr ite(fs % msg)
except UnicodeError:
self.stream.wri te(fs % msg.encode("UTF-8"))
I replace stream.write with print statement, so that it will try to
use stream.encoding to encode msg. Now everything works fine.
My question is :
Could the behavior of StreamHandler be considered as a bug?
If it is, how to report this bug?
Is my solution correct?
Are there any side effect will caused by doing so?
If the code I write is fine, and solve that problem, how to report it
to Python's project?
I think this could be helpful for people who also encountered this
problem.
Thanks.
Victor Lin.
I'm writting a application using python standard logging system. I
encounter some problem with unicode message passed to logging library.
I found that unicode message will be messed up by logging handler.
piese of StreamHandler:
try:
self.stream.wri te(fs % msg)
except UnicodeError:
self.stream.wri te(fs % msg.encode("UTF-8"))
It just write the message to stream. If there is some unicode error,
it would rewrite msg with utf8 encoding.
I write some code to try:
import sys
print u'¤¤¤å¦r´ú¸Õ'
print sys.stdout.enco ding
sys.stdout.writ e(u'¤¤¤å')
result of that program:
¤¤¤å¦r´ú¸Õ
cp950
Traceback (most recent call last):
File "update_stockpr ice.py", line 92, in <module>
sys.stdout.writ e(u'ä¸æ?')
UnicodeEncodeEr ror: 'ascii' codec can't encode characters in position
0-1: ordin
al not in range(128)
It show that....
1. print statement encode what it get with stream.encoding ?
2. stream.write don't do anything like encoding, just write it
(because it might be binary data?)
So the problem is : the StreamHandler of standard logging library use
stream.write to log message, if there is unicode error, unicode string
will be encode to utf8. This behavior mess my unicode up.
Here I modify the code of StreamHandler:
try:
print >self.stream, msg
#self.stream.wr ite(fs % msg)
except UnicodeError:
self.stream.wri te(fs % msg.encode("UTF-8"))
I replace stream.write with print statement, so that it will try to
use stream.encoding to encode msg. Now everything works fine.
My question is :
Could the behavior of StreamHandler be considered as a bug?
If it is, how to report this bug?
Is my solution correct?
Are there any side effect will caused by doing so?
If the code I write is fine, and solve that problem, how to report it
to Python's project?
I think this could be helpful for people who also encountered this
problem.
Thanks.
Victor Lin.
Comment