Python 2.4.1 hang

[Mahesh]

Hi,

This is on WinXP SP1.

I needed to get to the POST body and while I was trying out various
regular expressions, one of them caused Python to hang. The Python
process was taking up 100% of the CPU. I couldn't even see the "Max
recursion depth exceeded message". Is this a bug? Code below:

import re

s = \
"""POST /TradeManagement-RT3/ReportControlle r.Servlet HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/vnd.ms-excel, application/vnd.ms-powerpoint,
application/msword, */*
Referer: https://dummy.com/TradeManagement-RT...portSearch.jsp
Accept-Language: en-us
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR
1.0.3705)
Host: dummy.com
Content-Length: 267
Connection: Keep-Alive
Cache-Control: no-cache
Cookie:
SMSESSION=RNwIN GbjtRijIplKDE8E Z79NtSJREhKu4Oo gQqQD1PTukTIE3p clwfkkj2b5YSFsc bW97A8QQxk1066r mtqwatlBQsnxr2h 6fvAPiazYWex297 WmDDjPd05RjXNhV qiPmhxSN9nOVP6P q1igcFC5b3R4AFH WFcz+lW1QyUz+1y eaLfupKDkwaV7jP 0qgPbccioWUpEmn 762OyqCnehjuVBJ 9hDBGO07Bx8Pv/tHd0l6xjFOt6Ybt HG9IfMaKhrnPwmd tyo8c/4trmRNO84Boqwht OzhrJTVuPjzYN2u xg04ZgAt+j75gSA 9OPYYymirfwx5zB nhHvQz7ezGQqUPe 45l3CvnRhkFVM/kOAYdY2Cdlv+15E MCVqJLT+2cMRCPP Y+vlqlgsY30h5V9 NWiG+AdXKQ8LEUn PEhnSYhyIo1a3Fz B1yr+E/CZfXkNi1lMrG0Hi wU+NJVK7rY0deee 7gFeiq8T0660eq2 WOVF7USMESTOAbS DsR6Ejo+rRscvHf X7uzvu1pRw0Phw7 ffF0pr/nBhunq4v7/dW6WXOzvWAEocBX K9/Hl5Ua63X/UxXVs8g6psI0mqo RziFWw+O4t4jjn1 fS2e1YvvtAGRPIc NeEEPSCgqEhSUKo Gz1qysPoK87Mgfl IaHt/PsOeRCYSS/53B87RH9RrcaJEg rHyIBZNuzEjD1AG 4Uud5oKi88902RW 3IATHnH1E4UntvE do/NbCcNbgN/dGWEvBnBzLn6KYx d4PxG0pQ3vr3qDD a7v0i9eXq9t6++t lM1tIS/XIHZc4bfGKPdZC3 0Dtw7HwUc7bl74/SHVEEcgzgXJPkCH 2zSHaxyot3sqGHC wDa3AmuUkaPSC+i viVHlTe3Uk4KsOn nG94UIwB4yv+mlk Xqnw0JwausWVuet CIm+cDIuvZXgRYg hjZnNcNsji0k15d dr8j4=;
CCTMRT3=O0PVLBM WBVD2115CLLS4RE I

EntrySourceDesc ription=All&SEA RCHReportType=A ll&SEARCHReport Status=All&avai lable=IC&SEARCH ReportCommodity =All&SortContra ct_Year=1&SortT rade_Price=&Sor tOrder_Type=&So rtBuy_Sell_Ind= &SortAccount_Nu mber=&SortExter nal_TradeId=&Gr oupBy=contract& command=PrelimR eportCommand"""

#pattern_str = "^POST.*\\r\\n\ \r((\\n)|(\\n[^\r]*))"
#pattern_str = "^POST.*\\n((\\ n)|(\\n[^\r]*&))"
pattern_str = "^POST(.*\\n*)+ \\n\\n" # <--- Offending pattern

pattern = re.compile(patt ern_str)

match = pattern.match(s );

if match:
print match.groups()

[Fredrik Lundh]

Re: Python 2.4.1 hang

Mahesh wrote:
[color=blue]
> I needed to get to the POST body and while I was trying out various
> regular expressions, one of them caused Python to hang. The Python
> process was taking up 100% of the CPU. I couldn't even see the "Max
> recursion depth exceeded message". Is this a bug?[/color]

no, it's just a very stupid way to implement a trivial operation.
[color=blue]
> import re
>
> s = \
> """POST /TradeManagement-RT3/ReportControlle r.Servlet HTTP/1.1
> /snip>
>
> #pattern_str = "^POST.*\\r\\n\ \r((\\n)|(\\n[^\r]*))"
> #pattern_str = "^POST.*\\n((\\ n)|(\\n[^\r]*&))"
> pattern_str = "^POST(.*\\n*)+ \\n\\n" # <--- Offending pattern[/color]

the first .* is a variable-length match. so is the second .*. and then you're putting it
inside a repeated capturing group. and then you're applying it to a moderately large
string. the poor engine has to check zillions of combinations before finding something
that works.

if you want to split on "\r\n\r\n", use split:

header, body = message.split(" \r\n\r\n")

for more robust code, consider using the rfc822 module:

f = StringIO.String (message)
request = f.readline()
header = rfc822.Message( f)
body = f.read()

</F>

[Mahesh]

Re: Python 2.4.1 hang

Yes, it is stupid but I am debugging some poorly written C++ code so I
cannot change it. It was easier for me to use python to try various
combinations (since the C++ code uses a non-standard re engine). I just
chanced upon the problem and was curious as to what Python was up to.

Thanks for clearing that up.