pyparsing Combine without merging sub-expressions

[Steven Bethard]

Within a larger pyparsing grammar, I have something that looks like::

wsj/00/wsj_0003.mrg

When parsing this, I'd like to keep around both the full string, and the
AAA_NNNN substring of it, so I'd like something like::

>>foo.parseStri ng('wsj/00/wsj_0003.mrg')

(['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})

How do I go about this? I was using something like::

>>digits = pp.Word(pp.nums )
>>alphas = pp.Word(pp.alph as)
>>wsj_name = pp.Combine(alph as + '_' + digits)
>>wsj_path = pp.Combine(alph as + '/' + digits + '/' + wsj_name +

... '.mrg')

But of course then all I get back is the full path::

>>wsj_path.pars eString('wsj/00/wsj_0003.mrg')

(['wsj/00/wsj_0003.mrg'], {})

I could leave off the final Combine and add a parse action::

>>wsj_path = alphas + '/' + digits + '/' + wsj_name + '.mrg'
>>def parse_wsj_path( string, index, tokens):

... wsj_name = tokens[4]
... return ''.join(tokens) , wsj_name
...

>>wsj_path.setP arseAction(pars e_wsj_path)
>>wsj_path.pars eString('wsj/00/wsj_0003.mrg')

([('wsj/00/wsj_0003.mrg', 'wsj_0003')], {})

But that then allows whitespace between the pieces of the path, which
there shouldn't be::

>>wsj_path.pars eString('wsj / 00 / wsj_0003.mrg')

([('wsj/00/wsj_0003.mrg', 'wsj_0003')], {})

How do I make sure no whitespace intervenes, and still have access to
the sub-expression?

Thanks,

STeVe

[Paul McGuire]

Re: pyparsing Combine without merging sub-expressions


Steven Bethard wrote:

Within a larger pyparsing grammar, I have something that looks like::
>
wsj/00/wsj_0003.mrg
>
When parsing this, I'd like to keep around both the full string, and the
AAA_NNNN substring of it, so I'd like something like::
>

>>foo.parseStri ng('wsj/00/wsj_0003.mrg')

(['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
>
How do I go about this? I was using something like::
>

>>digits = pp.Word(pp.nums )
>>alphas = pp.Word(pp.alph as)
>>wsj_name = pp.Combine(alph as + '_' + digits)
>>wsj_path = pp.Combine(alph as + '/' + digits + '/' + wsj_name +

... '.mrg')
>
But of course then all I get back is the full path::
>

>>wsj_path.pars eString('wsj/00/wsj_0003.mrg')

(['wsj/00/wsj_0003.mrg'], {})
>

The tokens are what the tokens are, so if you want to replicate a
sub-field, then you'll need a parse action to insert it into the
returned tokens. BUT, if all you want is to be able to easily *access*
that sub-field, then why not give it a results name? Like this:

wsj_name = pp.Combine(alph as + '_' + digits).setResu ltsName("name")

Leave everything else the same, but now you can access the name field
independently from the rest of the combined tokens.

result = wsj_path.parseS tring('wsj/00/wsj_0003.mrg')
print result.dump()
print result.name
print result.asList()

-- Paul

[Steven Bethard]

Re: pyparsing Combine without merging sub-expressions

Dennis Lee Bieber wrote:

On Sat, 20 Jan 2007 13:49:52 -0700, Steven Bethard
<steven.bethard @gmail.comdecla imed the following in comp.lang.pytho n:
>

>Within a larger pyparsing grammar, I have something that looks like::
>>
> wsj/00/wsj_0003.mrg
>>
>When parsing this, I'd like to keep around both the full string, and the
>AAA_NNNN substring of it, so I'd like something like::
>>

> >>foo.parseStri ng('wsj/00/wsj_0003.mrg')

> (['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
>>

If working file name/paths, why not use the functions in os.path?

Two reasons. First, as I mentioned, this is within a larger pyparsing
grammar so it's not as easy to switch back and forth between the two.
Second, I do want to do some data validation (e.g. the name of the file
needs to be in a particular format) so I either need to post-process the
os.path approach or just do it in pyparsing.

>But that then allows whitespace between the pieces of the path, which
>there shouldn't be::
>>

If you didn't have whitespace coming in, there shouldn't be any
going out. If you do, you likely have malformed data and probably should
detect it earlier...

Well that's the intention of using pyparsing here. With a proper
grammar, pyparsing can detect the malformed data for me and throw an error.

STeVe

[Steven Bethard]

Re: pyparsing Combine without merging sub-expressions

Paul McGuire wrote:

Steven Bethard wrote:

>Within a larger pyparsing grammar, I have something that looks like::
>>
> wsj/00/wsj_0003.mrg
>>
>When parsing this, I'd like to keep around both the full string, and the
>AAA_NNNN substring of it, so I'd like something like::
>>

> >>foo.parseStri ng('wsj/00/wsj_0003.mrg')

> (['wsj/00/wsj_0003.mrg', 'wsj_0003'], {})
>>
>How do I go about this? I was using something like::
>>

> >>digits = pp.Word(pp.nums )
> >>alphas = pp.Word(pp.alph as)
> >>wsj_name = pp.Combine(alph as + '_' + digits)
> >>wsj_path = pp.Combine(alph as + '/' + digits + '/' + wsj_name +

> ... '.mrg')

[snip]

BUT, if all you want is to be able to easily *access*
that sub-field, then why not give it a results name? Like this:
>
wsj_name = pp.Combine(alph as + '_' + digits).setResu ltsName("name")
>
Leave everything else the same, but now you can access the name field
independently from the rest of the combined tokens.

Works great. Thanks!

STeVe