Help with cleaning input text - removing control characters

[Peter O'Reilly]

I have an HTML form with a textarea input box. When the user conducts a
post request (e.g. clicks the submit button), an HTML preview page is
presented to them with the information they have filled out in the prior
page's form elements.

Naturally some users like to copy and paste text into the textarea box and
presumably do so from say a word processor program. Some Macintosh based
users I know of experience problems with foreign looking characters
appearing in the HTML output, i.e tiny square boxes. The server processing
their requests is PC/Microsoft Windows (2000) based.

To fix the problem, I know this is a matter of removing certain control
characters. I would like to write some client side Javascript validation
code to handle this.

The problem for me is two-fold. I do not have a Mac/PowerPC to use for
testing. I am not all that familiar with Macs or know what control
characters to screen for. (About the only thing I know is Mac and Windows
use different control character representations for line feeds or carriage
returns or both).

Can someone shed some light on this for me? For example, which characters
to look for in parsing strings, i.e. \n, \t, etc. Thanks.

--
Peter O'Reilly

[Evertjan.]

Re: Help with cleaning input text - removing control characters

Peter O'Reilly wrote on 05 aug 2004 in comp.lang.javas cript:
[color=blue]
> To fix the problem, I know this is a matter of removing certain control
> characters. I would like to write some client side Javascript validation
> code to handle this.
>[/color]

<input
onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,'')"[color=blue]
>[/color]

removes anything that is not alphanumeric or space after loss of focus

--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[Mick White]

Re: Help with cleaning input text - removing control characters

Evertjan. wrote:
[color=blue]
> Peter O'Reilly wrote on 05 aug 2004 in comp.lang.javas cript:
>
>[color=green]
>>To fix the problem, I know this is a matter of removing certain control
>>characters. I would like to write some client side Javascript validation
>>code to handle this.
>>[/color]
>
>
> <input
> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,'')"[/color]

onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,' ')"

Replace any character that is not a-z or a number with a space.
Better, no?
Mick[color=blue]
>
>
> removes anything that is not alphanumeric or space after loss of focus
>[/color]

[Mick White]

Re: Help with cleaning input text - removing control characters

Mick White wrote:
[color=blue]
> Evertjan. wrote:
>[color=green]
>> Peter O'Reilly wrote on 05 aug 2004 in comp.lang.javas cript:
>>
>>[color=darkred]
>>> To fix the problem, I know this is a matter of removing certain control
>>> characters. I would like to write some client side Javascript
>>> validation
>>> code to handle this.
>>>[/color]
>>
>>
>> <input
>> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,'')"[/color]
>
>
> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,' ')"
>
> Replace any character that is not a-z or a number with a space.
> Better, no?
> Mick[/color]

Oops, you're right, I didn't notice the space in your "not" character set.
Mick[color=blue]
>[color=green]
>>
>>
>> removes anything that is not alphanumeric or space after loss of focus
>>[/color][/color]

[Evertjan.]

Re: Help with cleaning input text - removing control characters

Mick White wrote on 05 aug 2004 in comp.lang.javas cript:[color=blue][color=green]
>> <input
>> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,'')"[/color]
>
> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,' ')"
>
> Replace any character that is not a-z or a number with a space.
> Better, no?
>[/color]

Better, yes.
But not quite complete:

==============

Replace any group of characters that are
not a-z
or A-Z
or a number
or a space
with a space:

onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,' ')"

[this will leave multiple spaces as they are,
but replace multiple repaceants with one space]

==============

Replace any character that is
not a-z
or A-Z
or a number
or a space
with a space:

onchange="this. value=this.valu e.replace(/[^a-z\d ]/ig,' ')"

[this will leave multiple spaces as they are,
and replace multiple repaceants with multiple spaces]


==============

Replace any group of characters that are
not a-z
or A-Z
or a number
with a space:

onchange="this. value=this.valu e.replace(/[^a-z\d]+/ig,' ')"

[this will replace multiple white space with one space,
and replace multiple repaceants with multiple spaces]

===============

not tested, beware of any silly mistake.

--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[Peter O'Reilly]

Re: Help with cleaning input text - removing control characters

Evertjan & Mick,

Thank you both for the very helpful replies and code samples.

To be honest though, I am a little bit uncomfortable with the "what to
allow" approach. Don't get me wrong, your regular expressions are great, but
I'm afraid it may be a bit too aggressive in replacing text. For example,
consideration must be given for characters like !, @, #, $ ~ , etc. Of
course, those characters can always be added to the regular expression. I'm
afraid I will not think of all possible allowable characters.

Instead, a "what not to allow" approach would be most ideal,
e.g.specificall y targeting those few characters to screen out. What those
characters are is a mystery to me.
Perhaps String.charCode At() approach is needed?

Thanks again/dank u wel.

--
Peter O'Reilly

[Evertjan.]

Re: Help with cleaning input text - removing control characters

Peter O'Reilly wrote on 05 aug 2004 in comp.lang.javas cript:
[color=blue]
> Instead, a "what not to allow" approach would be most ideal,
> e.g.specificall y targeting those few characters to screen out. What
> those characters are is a mystery to me.[/color]

If you do not know the character or it's ascii value or it's unicode value,
it will be very difficult to specify a positive exclusion, Peter.

onchange="this. value=this.valu e.replace(/[@\\\n\x08\x1b\u 00A9]+/ig,' ')"

This will exclude:
The @
the \ itself (\\)
the linfeed char (\n)
the backspace (\x08 = hex 8)
the escape (\x21 = hex 1b = decimal 27)
the unicode copyright symbol (\u00A9 = ©)


--
Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

[Peter O'Reilly]

Re: Help with cleaning input text - removing control characters

> If you do not know the character or it's ascii value or it's unicode
value,[color=blue]
> it will be very difficult to specify a positive exclusion, Peter.[/color]

Evertjan, it's good to see that you are finally catching on. If someone
could shed some more insight into the original query, that would be great.
I'm sure someone else here must have experienced such problem and found a
solution for it.

In particular information on the character encoding issues or type(s) used
by English Macintosh users
(versus the IBM-PC/OEM ASCII character set I am accustomed to) would be
helpful.


--
Peter "UTF-8" O'Reilly

[Lāʻie Techie]

Re: Help with cleaning input text - removing control characters

On Thu, 05 Aug 2004 16:33:49 +0000, Evertjan. wrote:
[color=blue]
> <input
> onchange="this. value=this.valu e.replace(/[^a-z\d ]+/ig,'')"[/color]

Don't forget to perform this validation on the server side, too, for those
with JavaScript disabled in their browser.

La'ie Techie