url encoding string (Code Worth Recommending Project)

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Peter Michaux

    #1

    url encoding string (Code Worth Recommending Project)

    encodeURICompon ent is commonly used to serialize forms for use with
    XMLHttpRequest requests. For a perfect simulation of browser form
    requests, the goal is to serialize form data in the <URL:
    http://www.w3.org/TR/1999/REC-html401-19991224/interact/forms.html#h-17.13.4.1>
    application/x-www-form-urlencoded</astandardized format.

    The handling of whitespace by encodeURICompon ent() is different from
    the x-www-form-urlencoded standard. According to the x-www-form-
    urlencoded standard, a space should be encoded as a "+" and a newline
    should be encoded as a "%0D%0A".

    The JavaScript encodeURICompon ent function encodes a space as "%20".
    On Windows operating system a new line is encoded as "%0D%0A" but on
    Mac OS X a new line is encoded as just "%0A". I don't know about
    Linux. Can someone check using a textarea in a form?

    Some developers may need to have the application/x-www-form-urlencoded
    format. We can wrap encodeURICompon ent.

    if (typeof encodeURICompon ent != 'undefined' &&
    String.prototyp e.replace) {

    var urlEncode = function(s) {
    return encodeURICompon ent(s).replace(/%20/, '+').replace(/(.
    {0,3})(%0A)/g,
    function(m, p1, p2) {return p1+(p1=='%0D'?' ':'%0D')+p2;});
    };
    })();

    }

    The feature detection for String.prototyp e.replace is not sufficient
    because we really need to know if the second argument to replace can
    be a function. I imagine we can test this by using try-catch and
    actually using replace with a function as the second argument. Does
    anyone know a way to check without using try-catch?


    For developers with server-side toolkits capable of dealing with the
    output of encodeURICompon ent the regexp work is not necessary and can
    use just this simple little function.

    if (typeof encodeURICompon ent != 'undefined') {

    var urlEncode = function(s) {
    return encodeURICompon ent(s);
    };

    }


    There could even be a version for forms without textareas so only the
    spaces need replacement.

    if (typeof encodeURICompon ent != 'undefined' &&
    String.prototyp e.replace) {

    var urlEncode = function(s) {
    return encodeURICompon ent(s).replace(/%20/, '+');
    };
    })();

    }



    --------------

    BTW, I set up svn/trac for the code and posted the getOptionValue code
    that resulted from the previous thread on form serialization.

    <URL: http://cljs.michaux.ca/trac/browser/trunk/src/getOptionValue>

    Please email me if you would like wiki editing and ticket creation/
    commenting permissions. Let me know if you have a username/password
    preference.

    Peter
  • pr

    #2
    Re: url encoding string (Code Worth Recommending Project)

    Peter Michaux wrote:
    On Windows operating system a new line is encoded as "%0D%0A" but on
    Mac OS X a new line is encoded as just "%0A". I don't know about
    Linux. Can someone check using a textarea in a form?
    >
    Same as Windows: before%0D%0Aaft er

    Linux xxxx 2.6.17-12-generic #2 SMP Sun Sep 23 22:56:28 UTC 2007 i686
    GNU/Linux

    Comment

    • Bart Van der Donck

      #3
      Re: url encoding string (Code Worth Recommending Project)

      Peter Michaux wrote:
      The JavaScript encodeURICompon ent function encodes a space as "%20".
      On Windows operating system a new line is encoded as "%0D%0A" but on
      Mac OS X a new line is encoded as just "%0A". I don't know about
      Linux. Can someone check using a textarea in a form?
      Yes but %0A is only because Mac OSX is a UNIX derivate (which
      historically always used %0A).

      Mac until OS9 should return %0D.



      %0A: Multics, Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac
      OS X, etc.), BeOS, Amiga, RISC OS, and others
      %0D%0A: DEC RT-11 and most other early non-Unix, non-IBM OSes, CP/M,
      MP/M, MS-DOS, OS/2, Microsoft Windows
      %0D: Commodore machines, Apple II family and Mac OS up to version 9

      --
      Bart

      Comment

      • Peter Michaux

        #4
        Re: url encoding string (Code Worth Recommending Project)

        On Nov 21, 1:44 am, Bart Van der Donck <b...@nijlen.co mwrote:
        Peter Michaux wrote:
        The JavaScript encodeURICompon ent function encodes a space as "%20".
        On Windows operating system a new line is encoded as "%0D%0A" but on
        Mac OS X a new line is encoded as just "%0A". I don't know about
        Linux. Can someone check using a textarea in a form?
        >
        Yes but %0A is only because Mac OSX is a UNIX derivate (which
        historically always used %0A).
        >
        Mac until OS9 should return %0D.
        >

        >
        %0A: Multics, Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac
        OS X, etc.), BeOS, Amiga, RISC OS, and others
        %0D%0A: DEC RT-11 and most other early non-Unix, non-IBM OSes, CP/M,
        MP/M, MS-DOS, OS/2, Microsoft Windows
        %0D: Commodore machines, Apple II family and Mac OS up to version 9
        Thanks pr and Bart. It seems the two options ('%0D' and '%0A') need to
        be corrected. How about this?

        var urlencode = (function() {

        var f = function(s) {
        return encodeURICompon ent(s).replace(/%20/,
        '+').replace(/(.{0,3})(%0A)/g,
        function(m, p1, p2) {return p1+(p1=='%0D'?' ':'%0D')+p2;}
        ).replace(/(%0D)(.{0,3})/g,
        function(m, p1, p2) {return p1+(p2=='%0A'?' ':'%0A')+p2;});
        };

        try {
        if (f('\n \r') == '%0D%0A+%0D%0A' ) {
        return f;
        }
        }
        catch(e) {}

        })();


        Using try-catch is a nice way to avoid host object feature testing and
        ensure that all the necessary parts of the regexp are working as
        expected. This makes a full test of the new feature in a very small
        space.

        However, try-catch doesn't appeal to some programmers. Any other
        option to test that replace can take a function for the second
        argument?

        Peter

        Comment

        • pr

          #5
          Re: url encoding string (Code Worth Recommending Project)

          Peter Michaux wrote:
          However, try-catch doesn't appeal to some programmers. Any other
          option to test that replace can take a function for the second
          argument?
          If it's backwards-compatibility you mainly want to test for, then AFAIK
          try...catch arrived in the language about the same time as the ability
          to use a function in String.replace( ) (JS 1.3 & 1.4, IE 5 & 5.5). So if
          the feature isn't present then a try...catch feature test is almost
          equally likely to fall over.

          Unless you expect somebody with a seven-year-old browser to show up or
          you know of a recent implementation that can't successfully use
          functions in String.replace( ), it would seem reasonable not to test at
          all. I couldn't think of a test for this that would work pre-Netscape
          6.0 without browser sniffing.

          Comment

          • Peter Michaux

            #6
            Re: url encoding string (Code Worth Recommending Project)

            On Nov 22, 5:28 am, pr <p...@porl.glob alnet.co.ukwrot e:
            Peter Michaux wrote:
            However, try-catch doesn't appeal to some programmers. Any other
            option to test that replace can take a function for the second
            argument?
            >
            If it's backwards-compatibility you mainly want to test for, then AFAIK
            try...catch arrived in the language about the same time as the ability
            to use a function in String.replace( ) (JS 1.3 & 1.4, IE 5 & 5.5). So if
            the feature isn't present then a try...catch feature test is almost
            equally likely to fall over.
            >
            Unless you expect somebody with a seven-year-old browser to show up or
            you know of a recent implementation that can't successfully use
            functions in String.replace( ), it would seem reasonable not to test at
            all. I couldn't think of a test for this that would work pre-Netscape
            6.0 without browser sniffing.
            Randy Webb frequently mentions his dislike for try-catch and says his
            cell phone doesn't support it. I really don't have a problem with try-
            catch other than I don't like using features when they aren't
            necessary.

            It seems that if a browser doesn't support a function second argument
            then this would be sufficient.

            var urlencode = (function() {

            var f = function(s) {
            return encodeURICompon ent(s).replace(/%20/,
            '+').replace(/(.{0,3})(%0A)/g,
            function(m, p1, p2) {return p1+(p1=='%0D'?' ':'%0D')+p2;}
            ).replace(/(%0D)(.{0,3})/g,
            function(m, p1, p2) {return p1+(p2=='%0A'?' ':'%0A')+p2;});
            };

            if (f('\n \r') == '%0D%0A+%0D%0A' ) {
            return f;
            }

            })();

            The reason this is ok is because the if conditional will be false (not
            throw an error). JavaScript will automatically type convert the
            function second argument of replace to a string.

            "asdf".repl ace(/s/, function(){});

            outputs something like this

            afunction(){}df

            The Function.protot ype.toString function is standard but it's behavior
            is implementation dependent.

            So I think try-catch may not be necessary after all.

            Peter

            Comment

            • Randy Webb

              #7
              Re: url encoding string (Code Worth Recommending Project)

              Peter Michaux said the following on 11/22/2007 10:06 AM:
              On Nov 22, 5:28 am, pr <p...@porl.glob alnet.co.ukwrot e:
              >Peter Michaux wrote:
              >>However, try-catch doesn't appeal to some programmers. Any other
              >>option to test that replace can take a function for the second
              >>argument?
              >If it's backwards-compatibility you mainly want to test for, then AFAIK
              >try...catch arrived in the language about the same time as the ability
              >to use a function in String.replace( ) (JS 1.3 & 1.4, IE 5 & 5.5). So if
              >the feature isn't present then a try...catch feature test is almost
              >equally likely to fall over.
              >>
              >Unless you expect somebody with a seven-year-old browser to show up or
              >you know of a recent implementation that can't successfully use
              >functions in String.replace( ), it would seem reasonable not to test at
              >all. I couldn't think of a test for this that would work pre-Netscape
              >6.0 without browser sniffing.
              >
              Randy Webb frequently mentions his dislike for try-catch and says his
              cell phone doesn't support it. I really don't have a problem with try-
              catch other than I don't like using features when they aren't
              necessary.
              It auto-updated itself about a week ago and according to Motorola any
              cell phone that has that browser was also auto-updated. I don't think I
              have to say I am glad it happened.

              It supports about 90% of what IE7 supports, including try/catch.

              --
              Randy
              Chance Favors The Prepared Mind
              comp.lang.javas cript FAQ - http://jibbering.com/faq/index.html
              Javascript Best Practices - http://www.JavascriptToolbox.com/bestpractices/

              Comment

              • Thomas 'PointedEars' Lahn

                #8
                Re: url encoding string (Code Worth Recommending Project)

                pr wrote:
                Peter Michaux wrote:
                >However, try-catch doesn't appeal to some programmers. Any other
                >option to test that replace can take a function for the second
                >argument?
                >
                If it's backwards-compatibility you mainly want to test for, then AFAIK
                try...catch arrived in the language about the same time as the ability
                to use a function in String.replace( )
                No, it did not. How did you get that strange idea?
                (JS 1.3 & 1.4, IE 5 & 5.5).
                Your figures are way off.




                PointedEars
                --
                var bugRiddenCrashP ronePieceOfJunk = (
                navigator.userA gent.indexOf('M SIE 5') != -1
                && navigator.userA gent.indexOf('M ac') != -1
                ) // Plone, register_functi on.js:16

                Comment

                • Peter Michaux

                  #9
                  Re: url encoding string (Code Worth Recommending Project)

                  On Nov 23, 10:08 am, Thomas 'PointedEars' Lahn <PointedE...@we b.de>
                  wrote:
                  pr wrote:
                  Peter Michaux wrote:
                  However, try-catch doesn't appeal to some programmers. Any other
                  option to test that replace can take a function for the second
                  argument?
                  >
                  If it's backwards-compatibility you mainly want to test for, then AFAIK
                  try...catch arrived in the language about the same time as the ability
                  to use a function in String.replace( )
                  >
                  No, it did not. How did you get that strange idea?
                  >
                  (JS 1.3 & 1.4, IE 5 & 5.5).
                  >
                  Your figures are way off.
                  >
                  http://PointedEars.de/es-matrix#try
                  Nice page but I don't see anything about when String.prototyp e.replace
                  could take a function as the second argument.

                  Peter

                  Comment

                  • Thomas 'PointedEars' Lahn

                    #10
                    Re: url encoding string (Code Worth Recommending Project)

                    Thomas 'PointedEars' Lahn wrote:
                    Peter Michaux wrote:
                    >On Nov 23, 10:08 am, Thomas 'PointedEars' Lahn <PointedE...@we b.de>>[...] I don't see anything about when String.prototyp e.replace
                    >could take a function as the second argument.
                    >
                    Implicitly, it does. That feature could not have been available before
                    function expressions were introduced. (I might emphasize that by stating
                    it in a row below explicitly, but then I would have to do it for several
                    methods.)
                    I have spotted a bug in the Matrix ;-)

                    String.prototyp e.replace() was only described with a string as second
                    argument, which probably has contributed not only to your confusion.
                    Sorry about that.

                    I have done some tests now and updated the ES Matrix accordingly. I have
                    also fixed an error (of omission) at MDC: the method was not part of
                    ECMAScript specifications before Edition *3*.

                    The replace() method of String values returns a new string with one, some, or all matches of a pattern replaced by a replacement. The pattern can be a string or a RegExp, and the replacement can be a string or a function called for each match. If pattern is a string, only the first occurrence will be replaced. The original string is left unchanged.



                    PointedEars
                    --
                    Prototype.js was written by people who don't know javascript for people
                    who don't know javascript. People who don't know javascript are not
                    the best source of advice on designing systems that use javascript.
                    -- Richard Cornford, cljs, <f806at$ail$1$8 300dec7@news.de mon.co.uk>

                    Comment

                    • Richard Cornford

                      #11
                      Re: url encoding string (Code Worth Recommending Project)

                      Thomas 'PointedEars' Lahn wrote:
                      Peter Michaux wrote:
                      <snip>
                      >but I don't see anything about when String.prototyp e.replace
                      >could take a function as the second argument.
                      >
                      Implicitly, it does. That feature could not have been
                      available before function expressions were introduced.
                      <snip>

                      The second argument to - String.prototyp e.replace - does not have to be
                      a function expression. It can be (and very often should be) a reference
                      to a function via a named property of a Variable object (an Identifier
                      in the actual code). You can do that as soon as you have first class
                      functions (as function objects) that can be passed by reference, which
                      is pretty much from day one.

                      Richard.

                      Comment

                      • Richard Cornford

                        #12
                        Re: url encoding string (Code Worth Recommending Project)

                        Peter Michaux wrote:
                        <snip>
                        The reason this is ok is because the if conditional will be
                        false (not throw an error). JavaScript will automatically
                        type convert the function second argument of replace to a
                        string.
                        >
                        "asdf".repl ace(/s/, function(){});
                        >
                        outputs something like this
                        >
                        afunction(){}df
                        >
                        The Function.protot ype.toString function is standard but
                        it's behavior is implementation dependent.
                        >
                        So I think try-catch may not be necessary after all.
                        Given the number of times I have seen you referring people to my page in
                        the FAQ notes about feature detecting it is disappointing that you have
                        forgotten that the example of pure javascript feature detection that I
                        used on that page was a test for - String.prototyp e.replace - accepting
                        a function reference as its second argument or not (and it did not use
                        try-catch).

                        Richard.

                        Comment

                        • Thomas 'PointedEars' Lahn

                          #13
                          Re: url encoding string (Code Worth Recommending Project)

                          Richard Cornford wrote:
                          Thomas 'PointedEars' Lahn wrote:
                          >Peter Michaux wrote:
                          <snip>
                          >>but I don't see anything about when String.prototyp e.replace
                          >>could take a function as the second argument.
                          >Implicitly, it does. That feature could not have been
                          >available before function expressions were introduced.
                          <snip>
                          >
                          The second argument to - String.prototyp e.replace - does not have to be
                          a function expression. It can be (and very often should be) a reference
                          to a function via a named property of a Variable object (an Identifier
                          in the actual code).
                          "Very often should be" -- why? That would pollute the namespace and, given
                          the current support of runtime environments, it would pollute it needlessly.
                          I would have concurred if you had said "there are occasions where it is
                          advisable" but not with this.
                          You can do that as soon as you have first class functions (as function
                          objects) that can be passed by reference, which is pretty much from day one.
                          Nevertheless, String.prototyp e.replace() was not introduced before
                          JavaScript 1.2, JScript 1.0, and ECMAScript 3, and function expressions
                          were introduced with JavaScript 1.2, JScript 3.0, and ECMAScript 3. So
                          yes, it looks like there is a correlation.


                          PointedEars
                          --
                          var bugRiddenCrashP ronePieceOfJunk = (
                          navigator.userA gent.indexOf('M SIE 5') != -1
                          && navigator.userA gent.indexOf('M ac') != -1
                          ) // Plone, register_functi on.js:16

                          Comment

                          • Peter Michaux

                            #14
                            Re: url encoding string (Code Worth Recommending Project)

                            On Nov 24, 8:50 am, "Richard Cornford" <Rich...@litote s.demon.co.uk>
                            wrote:
                            Peter Michaux wrote:
                            >
                            <snip>
                            >
                            The reason this is ok is because the if conditional will be
                            false (not throw an error). JavaScript will automatically
                            type convert the function second argument of replace to a
                            string.
                            >
                            "asdf".repl ace(/s/, function(){});
                            >
                            outputs something like this
                            >
                            afunction(){}df
                            >
                            The Function.protot ype.toString function is standard but
                            it's behavior is implementation dependent.
                            >
                            So I think try-catch may not be necessary after all.
                            >
                            Given the number of times I have seen you referring people to my page in
                            the FAQ notes about feature detecting it is disappointing that you have
                            forgotten that the example of pure javascript feature detection that I
                            used on that page was a test for - String.prototyp e.replace - accepting
                            a function reference as its second argument or not (and it did not use
                            try-catch).
                            At least I had the joy of rediscovery :-)

                            Peter

                            Comment

                            Working...