Assembler Parser/Lexer in Python

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Simon Foster
    #1

    Assembler Parser/Lexer in Python


    Anyone have any experience or pointers to how to go about creating
    a parser lexer for assemble in Python. I was thinking of using PLY
    but wonder whether it's too heavyweight for what I want. Anyone have
    any thoughts?
    --
    Simon Foster
    Somewhere in the West of England
    Tags: None
    • djw
      #2
      Re: Assembler Parser/Lexer in Python

      Simon Foster wrote:
      [color=blue]
      >
      > Anyone have any experience or pointers to how to go about creating
      > a parser lexer for assemble in Python. I was thinking of using PLY
      > but wonder whether it's too heavyweight for what I want. Anyone have
      > any thoughts?
      > --
      > Simon Foster
      > Somewhere in the West of England[/color]

      Maybe kwParser?

      http://olympus.het.brown.edu/doc/gadfly/


      -Don

        Comment

        • Patrick Maupin
          #3
          Re: Assembler Parser/Lexer in Python

          Simon Foster wrote:
          [color=blue]
          > Anyone have any experience or pointers to how to go about creating
          > a parser lexer for assemble in Python. I was thinking of using PLY
          > but wonder whether it's too heavyweight for what I want. Anyone have
          > any thoughts?[/color]

          There are, of course, lots of tools available to help you out with
          this, but as you probably realize, most of them have heavyweight
          features which help out for higher-level languages, but not really
          so much for assembler. Also, most of them will probably not give you
          great help for assembly language macros, which are typically more
          full-featured than C macros, in that they know something about
          the actual program being built. (Please note that I am _not_
          cross-posting this to comp.lang.lisp, and also that if you want
          to parse pre-existing assembly language, it will look _nothing_
          like lisp, so the built-in parser wouldn't help you out in any
          case. Also note that YMMV, but while I use macros _extensively_
          in assembly language, I have personally never felt the necessity
          of having any sort of macro processor in Python :)

          I had a similar problem, in maintaining a system with over 10MB of
          crufty ancient assembly language. I had conflicting goals of wanting
          to use Python so I could easily and correctly do different things with
          the source code (code rewriting, automatic HTML generation, some
          lint-like operations, etc.) and wanting operations to complete rapidly
          so that I could do some of it in the typical Python experimental mode.

          I wrote a lexer using a tiny bit of C and a Pyrex wrapper. The
          partitioning was such that the C code knows nothing about Python,
          and the Pyrex interface handles the higher layers of the tokenization.

          The lexer performs a single tokenization pass over an entire file,
          (with the Pyrex calling the C code once per line) and returns a list
          of token tuples (one tuple per line). Macro lines which invoke text-
          pasting operations are flagged, and the lexer is re-run on these
          lines when they are encountered at parse time.

          A separate (and very simple!) Python script generates a .h file
          which contains the lowest-level lexer tables.

          I did an earlier version of this in mxTextTools, which is not too
          bad for such a thing if a) for whatever reason, you don't want to
          write your own C extensions, and b) you're not doing too much
          maintenance on the actual lexer.

          I also played around with re, but if you do that, you will quickly
          come to realize why lexer generators are popular :)

          More recently, I played a little bit with psyco. If I didn't care
          quite as much about speed and didn't already have the C code, I
          might consider one of the existing parser/lexer generators in
          conjunction with psyco. Unfortunately, I've only dabbled in
          a very minor way with some of these packages, so I couldn't begin
          to compare their strengths and weaknesses.

          Hope this helps.

          Pat

            Comment

            • phil hunt
              #4
              Re: Assembler Parser/Lexer in Python

              On Thu, 06 Nov 2003 22:06:45 GMT, Simon Foster <simon@uggs.dem on.co.uk> wrote:[color=blue]
              >
              >Anyone have any experience or pointers to how to go about creating
              >a parser lexer for assemble in Python. I was thinking of using PLY
              >but wonder whether it's too heavyweight for what I want. Anyone have
              >any thoughts?[/color]

              I've found SPARK useful:

              <http://pages.cpsc.ucal gary.ca/~aycock/spark/>

              --
              "It's easier to find people online who openly support the KKK than
              people who openly support the RIAA" -- comment on Wikipedia
              (Email: <zen20000@zen.c o.ku>, but first subtract 275 and reverse
              the last two letters).


                Comment

                Previous template Next
                Working...