HTML parsing bug?

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • g_no_mail_please@yahoo.com
    #1

    HTML parsing bug?

    Python 2.3.5 seems to choke when trying to parse html files, because it
    doesn't realize that what's inside <!-- --> is a comment in HTML,
    even if this comment is inside <script> </script>, especially if it's a
    comment inside that script code too.

    The html file:

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
    <html><head><ti tle>Choke on this</title>
    <script language="JavaS cript">
    <!--
    // </ht ml> - this is a comment in JavaScript, which is itself inside
    an HTML comment
    -->
    </script>
    </head>
    <body>
    Hey there
    </body>
    </html>


    The Python program:

    from urllib2 import urlopen
    from HTMLParser import HTMLParser
    f = urlopen("file:///PATH_TO_THE_ABO VE/index.html")
    p = HTMLParser()
    p.feed(f.read() )

    Tags: None
    • G.
      #2
      Re: HTML parsing bug?

      > // </ht ml> - this is a comment in JavaScript, which is itself inside[color=blue]
      > an HTML comment[/color]

      This is supposed to be one line. Got wrapped during posting.

        Comment

        • Richard Brodie
          #3
          Re: HTML parsing bug?


          <g_no_mail_plea se@yahoo.com> wrote in message
          news:1138632328 .306349.241430@ g44g2000cwa.goo glegroups.com.. .
          [color=blue]
          > Python 2.3.5 seems to choke when trying to parse html files, because it
          > doesn't realize that what's inside <!-- --> is a comment in HTML,
          > even if this comment is inside <script> </script>, especially if it's a
          > comment inside that script code too.[/color]

          Actually, you are technically incorrect; try validating the code you posted.
          Google found this explanation: http://lachy.id.au/log/2005/05/script-comments
          Feeding even slightly invalid HTML to the standard library parser will often
          choke it. If you can't guarantee clean sources, best use Tidy first or another
          parser entirely.



            Comment

            • Istvan Albert
              #4
              Re: HTML parsing bug?

              > this is a comment in JavaScript, which is itself inside an HTML comment

              Don't nest HTML comments. Occasionaly it may break the browsers as
              well.

              (I remember this from one of the weirdest of bughunts : whenever the
              number of characters between nested HTML comments was divisible by four
              the page would render incorrectly ... or something of that sorts)

              i.

                Comment

                • Tim Roberts
                  #5
                  Re: HTML parsing bug?

                  "Istvan Albert" <istvan.albert@ gmail.com> wrote:[color=blue]
                  >[color=green]
                  >> this is a comment in JavaScript, which is itself inside an HTML comment[/color]
                  >
                  >Don't nest HTML comments. Occasionaly it may break the browsers as
                  >well.[/color]

                  Did you read the post? He didn't nest HTML comments. He put a Javascript
                  comment inside an HTML comment, inside a <script></script> pair. Virtually
                  every page with Javascript does exactly the same thing.
                  --
                  - Tim Roberts, timr@probo.com
                  Providenza & Boekelheide, Inc.

                    Comment

                    • Fredrik Lundh
                      #6
                      Re: HTML parsing bug?

                      g_no_mail_pleas e@yahoo.com wrote:
                      [color=blue]
                      > Python 2.3.5 seems to choke when trying to parse html files, because it
                      > doesn't realize that what's inside <!-- --> is a comment in HTML,
                      > even if this comment is inside <script> </script>, especially if it's a
                      > comment inside that script code too.[/color]

                      nope. what's inside <!-- --> is not a comment if it's inside a <script>
                      or <style> tag. read the spec:

                      Basic HTML data types
                      http://www.w3.org/TR/REC-html40/types.html#type-cdata


                      "Although the STYLE and SCRIPT elements use CDATA for their data
                      model, for these elements, CDATA must be handled differently by
                      user agents. Markup and entities must be treated as raw text and
                      passed to the application as is. The first occurrence of the
                      character sequence "</" (end-tag open delimiter) is treated as
                      terminating the end of the element's content. In valid documents,
                      this would be the end tag for the element."

                      in your case, the first occurrence of "</" is not the end tag.

                      you can disable proper parsing by setting the CDATA_CONTENT_E LEMENTS
                      attribute on the parser instance, before you start parsing. by default, it is
                      set to

                      CDATA_CONTENT_E LEMENTS = ("script", "style")

                      setting it to an empty tuple disables HTML-compliant handling for these
                      elements:

                      p = HTMLParser()
                      p.CDATA_CONTENT _ELEMENTS = ()
                      p.feed(f.read() )

                      </F>



                        Comment

                        • Istvan Albert
                          #7
                          Re: HTML parsing bug?

                          >> this is a comment in JavaScript, which is itself inside an HTML comment
                          [color=blue]
                          > Did you read the post?[/color]

                          misread it rather ...

                            Comment

                            Previous template Next
                            Working...