Python's regular expression?

[Davy]

Hi all,

I am a C/C++/Perl user and want to switch to Python (I found Python is
more similar to C).

Does Python support robust regular expression like Perl?

And Python and Perl's File content manipulation, which is better?

Any suggestions will be appreciated!
Best regards,
Davy

[Lawrence Oluyede]

Re: Python's regular expression?

"Davy" <zhushenli@gmai l.com> writes:[color=blue]
> Does Python support robust regular expression like Perl?[/color]

Yep, Python regular expression is robust. Have a look at the Regex Howto:
http://www.amk.ca/python/howto/regex/ and the re module:

http://docs.python.org/lib/module-re.html

--
Lawrence - http://www.oluyede.org/blog
"Nothing is more dangerous than an idea
if it's the only one you have" - E. A. Chartier

[Mirco Wahab]

Re: Python's regular expression?

Hi Davy wrote:
[color=blue]
> I am a C/C++/Perl user and want to switch to Python[/color]

OK
[color=blue]
> (I found Python is more similar to C).[/color]

;-) More similar than what?
[color=blue]
> Does Python support robust regular expression like Perl?[/color]

It supports them fairly good, but it's
not 'integrated' - at least it feels not
integrated for me ;-) If you did a lot of
Perl, you know what 'integrated' means ...
[color=blue]
> And Python and Perl's File content manipulation, which is better?[/color]

What is a 'file content manipulation'?
Did you mean 'good xxx level file IO',
where xxx means either 'low' or 'high'?
[color=blue]
> Any suggestions will be appreciated![/color]

Just try to start a small project in Python -
from source that you already have in C or Perl
or something.


Regards

Mirco

[Davy]

Re: Python's regular expression?

Hi Mirco,

Thank you!

More similar than Perl ;-)

And what's 'integrated' mean (must include some library)?

I like C++ file I/O, is it 'low' or 'high'?

Regards,
Davy

[Davy]

Re: Python's regular expression?

By the way, is there any tutorial talk about how to use the Python
Shell (IDE). I wish it simple like VC++ :)

Regards,
Davy

[Mirco Wahab]

Re: Python's regular expression?

Hi Davy
[color=blue][color=green]
> > More similar than Perl ;-)[/color][/color]

But C has { }'s everywhere, so has Perl ;-)
[color=blue][color=green]
> > And what's 'integrated' mean (must include some library)?[/color][/color]

Yes. In Python, regular expressions are just
another function library - you use them like
in Java or C.

In Perl, it's part of the core language, you
use the awk-style (eg: /.../) regular expressions
everywhere you want.

If you used regexp in C/C++ before, you can use them
in almost the same way in Python - which may give you
an easy start.

BTW. Python has some fine extensions to the
perl(5)-Regexes, e.g. 'named backreferences' .

But you won't see much regular expressions
in Python code posted to this group, maybe
because it looks clunky - which is unpythonic ;-)

Lets see - a really simple find/match
would look like this in Python:

import re

t = 'blue socks and red shoes'
p = re.compile('(bl ue|white|red)')
if p.match(t):
print t

which prints the text 't' because of
the positive pattern match.

In Perl, you write:

use Acme::Pythonic;

$t = 'blue socks and red shoes'
if ($t =~ /(blue|white|red )/):
print $t

which is one line shorter (no need
to compile the regular expression
in advance).
[color=blue][color=green]
> > I like C++ file I/O, is it 'low' or 'high'?[/color][/color]

C++ has afaik actually three levels of I/O:

(1) - (from C, very low) operating system level, included
by <io.h> which provides direct access to operating system
services (read(), write(), lseek() etc.)

(2) - C-Standard-Library buffered IO, included by <stdio.h>,
provides structured 'mid-level' access like (block-) fread()/
fwrite(), line read (fgets()) and formatted I/O (fprintf()/
fscanf())

(3) - C++/streams library (high level, <fstream>, <iostream>, <sstream>),
which abstracts out the i/o devices, provides the same set of
functionality for any abstract input or output.

Perl provides all three levels of I/O, the 'abstracting' is introduced
by modules which tie 'handle variables' to anything that may receive
or send data.

Python also does a good job on all three levels, but provides
the (low level) operating system I/O by external modules (afaik).
I didn't do much I/O in Python, so I can't say much here.

Regards

Mirco

[John Machin]

Re: Python's regular expression?

On 8/05/2006 10:31 PM, Mirco Wahab wrote:
[snip][color=blue]
>
> Lets see - a really simple find/match
> would look like this in Python:
>
> import re
>
> t = 'blue socks and red shoes'
> p = re.compile('(bl ue|white|red)')
> if p.match(t):[/color]

What do you expect when t == "green socks and red shoes"? Is it possible
that you mean to use search() rather than match()?
[color=blue]
> print t
>
> which prints the text 't' because of
> the positive pattern match.
>
> In Perl, you write:
>
> use Acme::Pythonic;
>
> $t = 'blue socks and red shoes'
> if ($t =~ /(blue|white|red )/):
> print $t
>
> which is one line shorter (no need
> to compile the regular expression
> in advance).[/color]

There is no need to compile the regex in advance in Python, either.
Please consider the module-level function search() ...
if re.search(r"blu e|white|red", t):
# also, no need for () in the regex.

[Duncan Booth]

Re: Python's regular expression?

Mirco Wahab wrote:
[color=blue]
> Lets see - a really simple find/match
> would look like this in Python:
>
> import re
>
> t = 'blue socks and red shoes'
> p = re.compile('(bl ue|white|red)')
> if p.match(t):
> print t
>
> which prints the text 't' because of
> the positive pattern match.
>
> In Perl, you write:
>
> use Acme::Pythonic;
>
> $t = 'blue socks and red shoes'
> if ($t =~ /(blue|white|red )/):
> print $t
>
> which is one line shorter (no need
> to compile the regular expression
> in advance).
>[/color]

There is no need to compile the regular expression in advance in Python
either:

t = 'blue socks and red shoes'
if re.match('(blue |white|red)', t):
print t

The only advantage to compiling in advance is a small speed up, and most of
the time that won't be significant.

[Mirco Wahab]

Re: Python's regular expression?

Hi John
[color=blue][color=green]
>> import re
>>
>> t = 'blue socks and red shoes'
>> p = re.compile('(bl ue|white|red)')
>> if p.match(t):[/color]
>
> What do you expect when t == "green socks and red shoes"? Is it possible
> that you mean to use search() rather than match()?[/color]

This is interesting.
What's in this example the difference then between:

import re

t = 'blue socks and red shoes'
if re.compile('blu e|white|red').m atch(t):
print t

and

t = 'blue socks and red shoes'
if re.search('blue |white|red', t):
print t
[color=blue]
> There is no need to compile the regex in advance in Python, either.
> Please consider the module-level function search() ...
> if re.search(r"blu e|white|red", t):
> # also, no need for () in the regex.[/color]

Thats true. Thank you for pointing this out.
But what would be an appropriate use
of search() vs. match()? When to use what?

I answered the posting in the first place
because also I'm coming from a C/C++/Perl
background and trying to get along in Python.

Thanks,

Mirco

[Mirco Wahab]

Re: Python's regular expression?

Hi Duncan
[color=blue]
> There is no need to compile the regular expression in advance in Python
> either:
> ...
> The only advantage to compiling in advance is a small speed up, and most of
> the time that won't be significant.[/color]

I read 'some' introductions into Python Regexes
and got confused in the first place when to use
what and why.

After some minutes in this NG I start to get
the picture. So I narrowed the above regex-question
down to a nice equivalence between Perl and Python:

Python:

import re

t = 'blue socks and red shoes'
if re.match('blue| white|red', t):
print t

t = 'blue socks and red shoes'
if re.search('blue |white|red', t):
print t

Perl:

use Acme::Pythonic;

$t = 'blue socks and red shoes'
if $t =~ /blue|white|red/:
print $t


And Python Regexes eventually lost (for me) some of
their (what I believed) 'clunky appearance' ;-)

Thanks

Mirco

[John Machin]

Re: Python's regular expression?

On 8/05/2006 11:13 PM, Mirco Wahab wrote:[color=blue]
> Hi John
>[color=green][color=darkred]
>>> import re
>>>
>>> t = 'blue socks and red shoes'
>>> p = re.compile('(bl ue|white|red)')
>>> if p.match(t):[/color]
>> What do you expect when t == "green socks and red shoes"? Is it possible
>> that you mean to use search() rather than match()?[/color]
>
> This is interesting.
> What's in this example the difference then between:[/color]

I suggest that you (a) read the description on the difference between
search and match in the manual (b) try out search and match on both
your original string and the one I proposed.
[color=blue]
>
> import re
>
> t = 'blue socks and red shoes'
> if re.compile('blu e|white|red').m atch(t):
> print t
>
> and
>
> t = 'blue socks and red shoes'
> if re.search('blue |white|red', t):
> print t[/color]
[snip][color=blue]
>
> But what would be an appropriate use
> of search() vs. match()? When to use what?[/color]

ReadTheFantasti cManual :-)

[Mirco Wahab]

Re: Python's regular expression?

Hi John
[color=blue][color=green]
>> But what would be an appropriate use
>> of search() vs. match()? When to use what?[/color]
>
> ReadTheFantasti cManual :-)[/color]

From the manual you mentioned, i don't get
the point of 'match'. So why should you use
an extra function entry match(),

re.match('whate ver', t):

which is, according to the FM,
equivalent to (a special case of?)

re.search('^wha tever', t):

For me, it looks like match() should
be used on simple string comparisons
like a 'ramped up C-strcmp()'.

Or isn't ist? Maybe I dont get it ;-)

Thanks

Mirco

[Nick Craig-Wood]

Re: Python's regular expression?

Mirco Wahab <peace.is.our.p rofession@gmx.d e> wrote:[color=blue]
> After some minutes in this NG I start to get
> the picture. So I narrowed the above regex-question
> down to a nice equivalence between Perl and Python:
>
> Python:
>
> import re
>
> t = 'blue socks and red shoes'
> if re.match('blue| white|red', t):
> print t
>
> t = 'blue socks and red shoes'
> if re.search('blue |white|red', t):
> print t
>
> Perl:
>
> use Acme::Pythonic;
>
> $t = 'blue socks and red shoes'
> if $t =~ /blue|white|red/:
> print $t
>
> And Python Regexes eventually lost (for me) some of
> their (what I believed) 'clunky appearance' ;-)[/color]

If you are used to perl regexes there is one clunkiness of python
regexpes which you'll notice eventually...

Let's make the above example a bit more "real world", ie use the
matched item in some way...

Perl:

$t = 'blue socks and red shoes';
if ( $t =~ /(blue|white|red )/ )
{
print "Colour: $1\n";
}

Which prints

Colour: blue

In python you have to express this like

import re

t = 'blue socks and red shoes'
match = re.search('(blu e|white|red)', t)
if match:
print "Colour:", match.group(1)

Note the extra variable "match". You can't do assignment in an
expression in python which makes for the extra verbiosity, and you
need a variable to store the result of the match in (since python
doesn't have the magic $1..$9 variables).

This becomes particularly frustrating when you have to do a series of
regexp matches, eg

if ( $t =~ /(blue|white|red )/ )
{
print "Colour: $1\n";
}
elsif ( $t =~ /(socks|tights)/)
{
print "Garment: $1\n";
}
elsif ( $t =~ /(boot|shoe|trai ner)/)
{
print "Footwear: $1\n";
}

Which translates to

match = re.search('(blu e|white|red)', t)
if match:
print "Colour:", match.group(1)
else:
match = re.search('(soc ks|tights)', t)
if match:
print "Garment:", match.group(1)
else:
match = re.search('(boo t|shoe|trainer) ', t)
if match:
print "Footwear:" , match.group(1)
# indented ad infinitum!

You can use a helper class to get over this frustration like this

import re

class Matcher:
def search(self, r,s):
self.value = re.search(r,s)
return self.value
def __getitem__(sel f, i):
return self.value.grou p(i)

m = Matcher()
t = 'blue socks and red shoes'

if m.search(r'(blu e|white|red)', t):
print "Colour:", m[1]
elif m.search(r'(soc ks|tights)', t):
print "Garment:", m[1]
elif m.search(r'(boo t|shoe|trainer) ', t):
print "Footwear:" , m[1]

Having made the transition from perl to python a couple of years ago,
I find myself using regexpes much less. In perl everything looks like
it needs a regexp, but python has a much richer set of string methods,
eg .startswith, .endswith, good subscripting and the nice "in"
operator for strings.

--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

[Duncan Booth]

Re: Python's regular expression?

Nick Craig-Wood wrote:
[color=blue]
> Which translates to
>
> match = re.search('(blu e|white|red)', t)
> if match:
> print "Colour:", match.group(1)
> else:
> match = re.search('(soc ks|tights)', t)
> if match:
> print "Garment:", match.group(1)
> else:
> match = re.search('(boo t|shoe|trainer) ', t)
> if match:
> print "Footwear:" , match.group(1)
> # indented ad infinitum![/color]

This of course gives priority to colours and only looks for garments or
footwear if the it hasn't matched on a prior pattern. If you actually
wanted to match the first occurrence of any of these (or if the condition
was re.match instead of re.search) then named groups can be a nice way of
simplifying the code:

PATTERN = '''
(?P<c>blue|whit e|red)
| (?P<g>socks|tig hts)
| (?P<f>boot|shoe |trainer)
'''
PATTERN = re.compile(PATT ERN, re.VERBOSE)
TITLES = { 'c': 'Colour', 'g': 'Garment', 'f': 'Footwear' }

match = PATTERN.search( t)
if match:
grp = match.lastgroup
print "%s: %s" % (TITLES[grp], match.group(grp ))

For something this simple the titles and group names could be the same, but
I'm assuming real code might need a bit more.

[Mirco Wahab]

Re: Python's regular expression?

Hi Duncan
[color=blue]
> Nick Craig-Wood wrote:[color=green]
>> Which translates to
>> match = re.search('(blu e|white|red)', t)
>> if match:
>> else:
>> if match:
>> else:
>> if match:[/color]
>
> This of course gives priority to colours and only looks for garments or
> footwear if the it hasn't matched on a prior pattern. If you actually
> wanted to match the first occurrence of any of these (or if the condition
> was re.match instead of re.search) then named groups can be a nice way of
> simplifying the code:[/color]

A good point. And a good example when to use named
capture group references. This is easily extended
for 'spitting out' all other occuring categories
(see below).
[color=blue]
> PATTERN = '''
> (?P<c>blue|whit e|red)
> ...[/color]

This is one nice thing in Pythons Regex Syntax,
you have to emulate the ?P-thing in other
Regex-Systems more or less 'awk'-wardly ;-)
[color=blue]
> For something this simple the titles and group names could be the
> same, but I'm assuming real code might need a bit more.[/color]
Non no, this is quite good because it involves
some math-generated table-code lookup.

I managed somehow to extend your example in order
to spit out all matches and their corresponding
category:

import re

PATTERN = '''
(?P<c>blue |white |red )
| (?P<g>socks|tig hts )
| (?P<f>boot |shoe |trainer)
'''

PATTERN = re.compile(PATT ERN , re.VERBOSE)
TITLES = { 'c': 'Colour', 'g': 'Garment', 'f': 'Footwear' }

t = 'blue socks and red shoes'
for match in PATTERN.findite r(t):
grp = match.lastgroup
print "%s: %s" %( TITLES[grp], match.group(grp ) )

which writes out the expected:
Colour: blue
Garment: socks
Colour: red
Footwear: shoe

The corresponding Perl-program would look like this:

$PATTERN = qr/
(blue |white |red )(?{'c'})
| (socks|tights )(?{'g'})
| (boot |shoe |trainer)(?{'f' })
/x;

%TITLES = (c =>'Colour', g =>'Garment', f =>'Footwear');

$t = 'blue socks and red shoes';
print "$TITLES{$^ R}: $^N\n" while( $t=~/$PATTERN/g );

and prints the same:
Colour: blue
Garment: socks
Colour: red
Footwear: shoe

You don't have nice named match references (?P<..>)
in Perl-5, so you have to emulate this by an ordinary
code assertion (?{..}) an set some value ($^R) on
the fly - which is not that bad in the end (imho).

(?{..}) means "zero with code assertion",
this sets Perl-predefined $^R to its evaluated
value from the {...}

As you can see, the pattern matching related part
reduces from 4 lines to one line.

If you wouldn't need dictionary lookup and
get away with associated categories, all
you'd have to do would be this:

$PATTERN = qr/
(blue |white |red )(?{'Colour'})
| (socks|tights )(?{'Garment'})
| (boot |shoe |trainer)(?{'Fo otwear'})
/x;

$t = 'blue socks and red shoes';
print "$^R: $^N\n" while( $t=~/$PATTERN/g );

What's the point of all that? IMHO, Python's
Regex support is quite good and useful, but
won't give you an edge over Perl's in the end.

Thanks & Regards

Mirco