Trivial string substitution/parser

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Samuel
    #1

    Trivial string substitution/parser

    Hi,

    How would you implement a simple parser for the following string:

    ---
    In this string $variable1 is substituted, while \$variable2 is not.
    ---

    I know how to write a parser, but I am looking for an elegant (and lazy)
    way. Any idea?

    -Samuel
    Tags: None
    • Duncan Booth
      #2
      Re: Trivial string substitution/parser

      Samuel <newsgroups@deb ain.orgwrote:
      Hi,
      >
      How would you implement a simple parser for the following string:
      >
      ---
      In this string $variable1 is substituted, while \$variable2 is not.
      ---
      >
      I know how to write a parser, but I am looking for an elegant (and lazy)
      way. Any idea?
      >
      The elegant and lazy way would be to change your specification so that $
      characters are escaped by $$ not by backslashes. Then you can write:
      >>from string import Template
      >>t = Template("In this string $variable1 is substituted, while
      $$variable2 is not.")
      >>t.substitute( variable1="hell o", variable2="worl d")
      'In this string hello is substituted, while $variable2 is not.'

      If you must insist on using backslash escapes (which introduces the
      question of how you get backslashes into the output: do they have to be
      escaped as well?) then use string.Template with a custom pattern.

        Comment

        • Samuel
          #3
          Re: Trivial string substitution/parser

          On Sun, 17 Jun 2007 11:00:58 +0000, Duncan Booth wrote:
          The elegant and lazy way would be to change your specification so that $
          characters are escaped by $$ not by backslashes. Then you can write:
          >
          >>>from string import Template
          >>>...
          Thanks, however, turns out my specification of the problem was
          incomplete: In addition, the variable names are not known at compilation
          time.
          I just did it that way, this looks fairly easy already:

          -------------------
          import re

          def variable_sub_cb (match):
          prepend = match.group(1)
          varname = match.group(2)
          value = get_variable(va rname)
          return prepend + value

          string_re = re.compile(r'(^ |[^\\])\$([a-z][\w_]+\b)', re.I)

          input = r'In this string $variable1 is substituted,'
          input += 'while \$variable2 is not.'

          print string_re.sub(v ariable_sub_cb, input)
          -------------------

          -Samuel

            Comment

            • Josiah Carlson
              #4
              Re: Trivial string substitution/parser

              Samuel wrote:
              On Sun, 17 Jun 2007 11:00:58 +0000, Duncan Booth wrote:
              >
              >The elegant and lazy way would be to change your specification so that $
              >characters are escaped by $$ not by backslashes. Then you can write:
              >>
              >>>>from string import Template
              >>>>...
              >
              Thanks, however, turns out my specification of the problem was
              incomplete: In addition, the variable names are not known at compilation
              time.
              You mean at edit-time.
              >>t.substitute( variable1="hell o", variable2="worl d")
              Can be replaced by...
              >>t.substitute( **vars)
              ....as per the standard **kwargs passing semantics.


              - Josiah

                Comment

                • Graham Breed
                  #5
                  Re: Trivial string substitution/parser

                  Samuel wote:
                  Thanks, however, turns out my specification of the problem was
                  incomplete: In addition, the variable names are not known at compilation
                  time.
                  I just did it that way, this looks fairly easy already:
                  >
                  -------------------
                  import re
                  >
                  def variable_sub_cb (match):
                  prepend = match.group(1)
                  varname = match.group(2)
                  value = get_variable(va rname)
                  return prepend + value
                  >
                  string_re = re.compile(r'(^ |[^\\])\$([a-z][\w_]+\b)', re.I)
                  >
                  input = r'In this string $variable1 is substituted,'
                  input += 'while \$variable2 is not.'
                  >
                  print string_re.sub(v ariable_sub_cb, input)
                  -------------------
                  It gets easier:

                  import re

                  def variable_sub_cb (match):
                  return get_variable(ma tch.group(1))

                  string_re = re.compile(r'(? <!\\)\$([A-Za-z]\w+)')

                  def get_variable(va rname):
                  return globals()[varname]

                  variable1 = 'variable 1'

                  input = r'In this string $variable1 is substituted,'
                  input += 'while \$variable2 is not.'

                  print string_re.sub(v ariable_sub_cb, input)

                  or even

                  import re

                  def variable_sub_cb (match):
                  return globals()[match.group(1)]

                  variable1 = 'variable 1'
                  input = (r'In this string $variable1 is substituted,'
                  'while \$variable2 is not.')

                  print re.sub(r'(?<!\\ )\$([A-Za-z]\w+)', variable_sub_cb , input)


                  Graham

                    Comment

                    • Duncan Booth
                      #6
                      Re: Trivial string substitution/parser

                      Josiah Carlson <josiah.carlson @sbcglobal.netw rote:
                      Samuel wrote:
                      >On Sun, 17 Jun 2007 11:00:58 +0000, Duncan Booth wrote:
                      >>
                      >>The elegant and lazy way would be to change your specification so
                      >>that $ characters are escaped by $$ not by backslashes. Then you can
                      >>write:
                      >>>
                      >>>>>from string import Template
                      >>>>>...
                      >>
                      >Thanks, however, turns out my specification of the problem was
                      >incomplete: In addition, the variable names are not known at
                      >compilation time.
                      >
                      You mean at edit-time.
                      >
                      >t.substitute(v ariable1="hello ", variable2="worl d")
                      >
                      Can be replaced by...
                      >
                      >t.substitute(* *vars)
                      >
                      ...as per the standard **kwargs passing semantics.
                      You don't even need to do that. substitute will accept a dictionary as a
                      positional argument:

                      t.substitute(va rs)

                      If you use both forms then the keyword arguments take priority.

                      Also, of course, vars just needs to be something which quacks like a dict:
                      it can do whatever it needs to do such as looking up a database or querying
                      a server to generate the value only when it needs it, or even evaluating
                      the name as an expression; in the OP's case it could call get_variable.

                      Anyway, the question seems to be moot since the OP's definition of 'elegant
                      and lazy' includes regular expressions and reinvented wheels.

                      .... and in another message Graham Breed wrote:
                      def get_variable(va rname):
                      return globals()[varname]
                      Doesn't the mere thought of creating global variables with unknown names
                      make you shudder?

                        Comment

                        • Graham Breed
                          #7
                          Re: Trivial string substitution/parser

                          Duncan Booth wote:
                          Also, of course, vars just needs to be something which quacks like a dict:
                          it can do whatever it needs to do such as looking up a database or querying
                          a server to generate the value only when it needs it, or even evaluating
                          the name as an expression; in the OP's case it could call get_variable.
                          And in case that sounds difficult, the code is

                          class VariableGetter:
                          def __getitem__(sel f, key):
                          return get_variable(ke y)
                          Anyway, the question seems to be moot since the OP's definition of 'elegant
                          and lazy' includes regular expressions and reinvented wheels.
                          Your suggestion of subclassing string.Template will also require a
                          regular expression -- and a fairly hairy one as far as I can work out
                          from the documentation. There isn't an example and I don't think it's
                          the easiest way of solving this problem. But if Samuel really wants
                          backslash escaping it'd be easier to do a replace('$$','$ $$$') and
                          replace('\\$', '$$') (or replace('\\$',' \\$$') if he really wants the
                          backslash to persist) before using the template.

                          Then, if he really does want to reject single letter variable names,
                          or names beginning with a backslash, he'll still need to subclass
                          Template and supply a regular expression, but a simpler one.
                          ... and in another message Graham Breed wrote:
                          def get_variable(va rname):
                          return globals()[varname]
                          >
                          Doesn't the mere thought of creating global variables with unknown names
                          make you shudder?
                          Not at all. It works, it's what the shell does, and it's easy to test
                          interactively. Obviously the application code wouldn't look like
                          that.


                          Graham

                            Comment

                            • Graham Breed
                              #8
                              Re: Trivial string substitution/parser

                              Duncan Booth wote:
                              If you must insist on using backslash escapes (which introduces the
                              question of how you get backslashes into the output: do they have to be
                              escaped as well?) then use string.Template with a custom pattern.
                              If anybody wants this, I worked out the following regular expression
                              which seems to work:

                              (?P<escaped>\\) \$ | # backslash escape pattern
                              \$(?:
                              (?P<named>[_a-z][_a-z0-9]*) | # delimiter and Python identifier
                              {(?P<braced>[_a-z][_a-z0-9]*)} | # delimiter and braced identifier
                              (?P<invalid>) # Other ill-formed delimiter exprs
                              )

                              The clue is string.Template .pattern.patter n

                              So you compile that with verbose and case-insensitive flags and set it
                              to "pattern" in a string.Template subclass. (In fact you don't have
                              to compile it, but that behaviour's undocumented.) Something like
                              >>regexp = """
                              .... (?P<escaped>\\\ \)\\$ | # backslash escape pattern
                              .... \$(?:
                              .... (?P<named>[_a-z][_a-z0-9]*) | # delimiter and identifier
                              .... {(?P<braced>[_a-z][_a-z0-9]*)} | # ... and braced identifier
                              .... (?P<invalid>) # Other ill-formed delimiter exprs
                              .... )
                              .... """
                              >>class BackslashEscape (Template):
                              .... pattern = re.compile(rege xp, re.I | re.X)
                              ....


                              Graham

                                Comment

                                Previous template Next
                                Working...