unicode "em space" in regex

[Xah Lee]

how to represent the unicode "em space" in regex?

e.g. i want do something like this:

fracture=re.spl it(r'\342371*\| \342371*',mylin e,re.U)

Xah
xah@xahlee.org
∑ http://xahlee.org/

[Klaus Alexander Seistrup]

Re: unicode "em space" in regex

Xah Lee :
[color=blue]
> how to represent the unicode "em space" in regex?
>
> e.g. i want do something like this:
>
> fracture=re.spl it(r'\342371*\| \342371*',mylin e,re.U)[/color]

I'm not sure what you're trying to do, but would it help you to use
it's name:
[color=blue][color=green][color=darkred]
>>> EM_SPACE = u'\N{EM SPACE}'
>>> fracture = myline.split(EM _SPACE)[/color][/color][/color]

?

Cheers,

--
Klaus Alexander Seistrup
Magnetic Ink, Copenhagen, Denmark

http://magnetic-ink.dk/

[Martin v. Löwis]

Re: unicode "em space" in regex

Xah Lee wrote:[color=blue]
> how to represent the unicode "em space" in regex?[/color]

You will have to pass a Unicode literal as the regular expression,
e.g.

fracture=re.spl it(u'\u2003*\\| \u2003*',myline ,re.U)

Notice that, in raw Unicode literals, you can still use \u to
escape characters, e.g.

fracture=re.spl it(ur'\u2003*\| \u2003*',myline ,re.U)

Regards,
Martin

[Xah Lee]

Re: unicode "em space" in regex

Thanks. Is it true that any unicode chars can also be used inside regex
literally?

e.g.
re.search(ur'†ƒ+',mystring,re .U)

I tested this case and apparently i can. But is it true that any
unicode char can be embedded in regex literally. (does this apply to
the esoteric ones such as other non-printing chars and combining
forms...)

----
Related...:

The official python doc:

http://python.org/doc/2.4.1/lib/module-re.html

says:

"Regular expression pattern strings may not contain null bytes, but can
specify the null byte using the \number notation."

What is meant by null bytes here? Unprintable chars?? and the "\number"
is meant to be decimal? and in what encoding?

Xah
xah@xahlee.org
∑ http://xahlee.org/

[Reinhold Birkenfeld]

Re: unicode "em space" in regex

Xah Lee wrote:
[color=blue]
> "Regular expression pattern strings may not contain null bytes, but can
> specify the null byte using the \number notation."
>
> What is meant by null bytes here? Unprintable chars?? and the "\number"
> is meant to be decimal? and in what encoding?[/color]

The null byte is a byte with the integer value 0. Difficult, isn't it.

The \number notation is, as you could read in http://docs.python.org/ref/strings.html,
octal.

Reinhold

[Fredrik Lundh]

Re: unicode "em space" in regex

Xah Lee wrote:
[color=blue]
> "Regular expression pattern strings may not contain null bytes, but can
> specify the null byte using the \number notation."
>
> What is meant by null bytes here? Unprintable chars??[/color]

no, null bytes. "\0". "\x00". ord(byte) == 0. chr(0).
[color=blue]
> and the "\number" is meant to be decimal?[/color]

octal. this is explained on the "Regular Expression Syntax" page.
[color=blue]
> and in what encoding?[/color]

null byte encoding? you're confused.

</F>

[Martin v. Löwis]

Re: unicode &quot;em space&quot; in regex

Xah Lee wrote:[color=blue]
> Thanks. Is it true that any unicode chars can also be used inside regex
> literally?
>
> e.g.
> re.search(ur'†ƒ+',mystring,re .U)
>
> I tested this case and apparently i can.[/color]

Yes. In fact, when you write u"\u2003" or u" " doesn't matter
to re.search. Either way you get a Unicode object with U+2003
in it, which is processed by SRE.
[color=blue]
> But is it true that any
> unicode char can be embedded in regex literally. (does this apply to
> the esoteric ones such as other non-printing chars and combining
> forms...)[/color]

Yes. To SRE, only the Unicode ordinal values matter. To determine
whether something matches, it needs to have the same ordinal value
in the string that you have in the expression. No interpretation
of the character is performed, except for the few characters that
have markup meaning in regular expressions (e.g. $, \, [, etc)

Regards,
Martin