multiline string split problem and fix

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Brian

    multiline string split problem and fix

    I have been working on a data reception system.. I am still finding my
    way around Javascript, though I am accomplishing much.

    I just fixed a flaw that was really hard to find. The symptoms are
    this:

    I get a multiline string returned to Javascript from a Proxy+Google
    Maps API GDownloadUrl()
    The data, when added to a DOM table looked fine, about 20 lines in CSV
    format

    Sunrise,-119.098,35.345, 0.0<br>
    SwanDancer,-119.345,35.567, 1.0<br>
    .... etc

    (I don't know why the <br>'s are there, but that's what it looks like)
    So using a suggestion from this newsgroup, I perform two subsequent
    split()'s

    var index, index2;
    var strCSVFile = data;
    var arrayCSVFile;

    arrayCSVFile = strCSVFile.spli t( "<br>" );

    for ( index = 0; index < arrayCSVFile.le ngth; index++ )
    {
    arrayCSVFile[ index ] = arrayCSVFile[ index ].split( ',' );
    // do stuff to the elements
    }

    I use both strCSVFile *and* arrayCSVFile to be doubly sure I wasn't
    somehow clobbering something, though in theory there needs to be only
    the original string. At any rate, what I see is this (after HOURS of
    trying and finally using str.charCodeAt( ))

    10|32|32|32|32| 32|32|32|32|83| 117|110|114|105 |115|101| len=16
    10|10|32|32|32| 32|32|32|32|32| 83|119|97|110|6 8|97|110|99|101 |114|
    len=20
    .... etc

    %^!@#$^%@ <- that's cursing, people
    So I am now hand clipping some number of LF and SPACE chars using
    str.charCodeAt( ). On top of that, my furtive attempts at RegEx
    replacements along the way had been SILENTLY FAILING. Probably because
    of the leading LF(s). I had no idea, and it took valuable time..

    I looked for split() gotcha's but never found anything like this. I
    thought I tried changing the split to "<BR>\r" at one point, but I
    probably did the return instead of line feed... Also, that would NOT
    handle the first line case ?!

    This is what is happening, and I now have tedious code to handle it.
    Looking back on the original recv'd data, it does indeed have a leading
    LF|SPACE's, with two LF's on every subsequent row. I never saw them.
    How could I ? When I aded them to the HTML page to check the data, they
    didn't show
    This was awful.

    FYI

  • Brian

    #2
    Re: multiline string split problem and fix

    btw- I just added

    strCSVFile.repl ace( / /g, '');
    strCSVFile.repl ace( / \n/g, '');
    strCSVFile.repl ace( / \r/g, '');

    to clean the data (the whole block before the split()'s. Am I making a
    mistake in the RegEx? they don't work...

    Comment

    • Brian

      #3
      Re: multiline string split problem and fix

      >
      strCSVFile.repl ace( / /g, '');
      strCSVFile.repl ace( / \n/g, '');
      strCSVFile.repl ace( / \r/g, '');
      >
      >
      hmmm, very late night typing.. I meant

      strCSVFile.repl ace( / +/g, '');
      strCSVFile.repl ace( /\n+/g, '');
      strCSVFile.repl ace( /\r+/g, '');

      Comment

      • mick white

        #4
        Re: multiline string split problem and fix

        Brian wrote:
        >
        hmmm, very late night typing.. I meant
        >
        strCSVFile.repl ace( / +/g, '');
        strCSVFile.repl ace( /\n+/g, '');
        strCSVFile.repl ace( /\r+/g, '');
        >
        Perhaps you need:
        someVariable=st rCSVFile.replac e(/\s/g,'');

        No need for "+", since you are using the "g" modifier. And why not
        assign the result of the statement to a variable? (Unless you want to
        destroy the original string.)
        Mick

        Comment

        • RobG

          #5
          Re: multiline string split problem and fix


          Brian wrote:
          I have been working on a data reception system.. I am still finding my
          way around Javascript, though I am accomplishing much.
          >
          I just fixed a flaw that was really hard to find. The symptoms are
          this:
          >
          I get a multiline string returned to Javascript from a Proxy+Google
          Maps API GDownloadUrl()
          The data, when added to a DOM table looked fine, about 20 lines in CSV
          format
          >
          Sunrise,-119.098,35.345, 0.0<br>
          SwanDancer,-119.345,35.567, 1.0<br>
          ... etc
          Try something like:

          var strCSVFile = 'Sunrise,-119.098,35.345, 0.0<br>'
          + 'SwanDancer,-119.345,35.567, 1.0<br>';

          /* Remove any leading or trailing br elements */
          strCSVFile = strCSVFile.repl ace(/(^<br>)|(<br>$)/g,'');

          var arrayCSVFile = strCSVFile.spli t('<br>');
          var recordElement;

          /* arrayCSVFile is now an array with two elements:
          *
          * ['Sunrise,-119.098,35.345, 0.0',
          * 'SwanDancer,-119.345,35.567, 1.0']
          */
          for (var i=0, len=arrayCSVFil e.length; i<len; i++){
          arrayCSVFile[i] = arrayCSVFile[i].split(',');

          /* arrayCSVFile is still an array with two elements,
          * but each is now an array of 4 elements:
          *
          * [
          * ['Sunrise', '-119.098', '35.345', '0.0'],
          * ['SwanDancer', '-119.345', '35.567', '1.0']
          * ]
          */
          for (var j=0, len2=arrayCSVFi le[i].length; j<len2; j++){
          recordElement = arrayCSVFile[i][j];

          /* recordElement will be each element in turn, i.e.
          * 'Sunrise', then '-119.098', then '35.345', and so on
          */

          alert('Record: ' + (i+1) + ' of ' + len
          + '\nElement: ' + (j+1) + ' of ' + len2
          + '\nValue: ' + recordElement);
          }
          }

          If you want to remove all whitespace (all spaces, tabs, linefeeds,
          returns, the lot) than add .replace(/\s/g, '') to the end of the line
          where the leading and trailing br's are replaced.


          --
          Rob

          Comment

          • Brian

            #6
            Re: multiline string split problem and fix


            RobG wrote:
            Brian wrote:...
            Try something like:
            >
            var strCSVFile = 'Sunrise,-119.098,35.345, 0.0<br>'
            + 'SwanDancer,-119.345,35.567, 1.0<br>';
            >
            /* Remove any leading or trailing br elements */
            strCSVFile = strCSVFile.repl ace(/(^<br>)|(<br>$)/g,'');
            >
            var arrayCSVFile = strCSVFile.spli t('<br>');
            var recordElement;
            >
            /* arrayCSVFile is now an array with two elements:
            *
            * ['Sunrise,-119.098,35.345, 0.0',
            * 'SwanDancer,-119.345,35.567, 1.0']
            */
            for (var i=0, len=arrayCSVFil e.length; i<len; i++){
            arrayCSVFile[i] = arrayCSVFile[i].split(',');
            >
            /* arrayCSVFile is still an array with two elements,
            * but each is now an array of 4 elements:
            *
            * [
            * ['Sunrise', '-119.098', '35.345', '0.0'],
            * ['SwanDancer', '-119.345', '35.567', '1.0']
            * ]
            */
            for (var j=0, len2=arrayCSVFi le[i].length; j<len2; j++){
            recordElement = arrayCSVFile[i][j];
            >
            /* recordElement will be each element in turn, i.e.
            * 'Sunrise', then '-119.098', then '35.345', and so on
            */
            >
            alert('Record: ' + (i+1) + ' of ' + len
            + '\nElement: ' + (j+1) + ' of ' + len2
            + '\nValue: ' + recordElement);
            }
            }
            >

            hey, Rob, I think you have dome something like this before (!) The
            clarity of your formatting alone is helpful. I, also, split twice to
            end up with CSV[][].

            I ended up going a different direction at the end though. Rather than
            make a single, flat line of elements, why not use the JS Object/perl
            hash/ObjectiveC collection(?)/STL Map idiom. that is, and array of
            objects, a series of name/value pairs. years ago I called it a
            dictionary, or associative array (name/value pairs), but didn't use it
            much. These days, it seems to have sprung into popularity, with
            language support to just 'toss in' elements as needed (though I am not
            using it that way here)

            So the final lines of the my version turns into:
            // Obj w/named fields, the result of processing the CSV
            var csvDataObj = {};
            csvDataObj.recA rray = [];

            // process each record
            for ( index = 0; index < arrayCSVFile.le ngth; index++ )
            {
            arrayCSVFile[ index ] = arrayCSVFile[ index ].split( ',' );

            // build data container to pass out to other processes
            var tRecObj = {};
            tRecObj.recName = arrayCSVFile[index][0];
            tRecObj.recLat = arrayCSVFile[index][1];
            tRecObj.recLng = arrayCSVFile[index][2];
            tRecObj.recAlt = arrayCSVFile[index][3];

            csvDataObj.recA rray.push( tRecObj);
            }

            // Now do something with arrayCSVFile[][]
            gMyCore.process AllDevices( csvDataObj);

            On the other end, I could "discover" the elements, but since its only
            Javascript ;-) I use direct knowledge of the contents in the code.

            I wrote it, worked the first time. So some things are going ok :-)

            Comment

            • Brian

              #7
              Re: multiline string split problem and fix


              mick white wrote:
              Brian wrote:

              strCSVFile.repl ace( / +/g, '');
              strCSVFile.repl ace( /\n+/g, '');
              strCSVFile.repl ace( /\r+/g, '');
              Perhaps you need:
              someVariable=st rCSVFile.replac e(/\s/g,'');
              >
              No need for "+", since you are using the "g" modifier. And why not
              assign the result of the statement to a variable? (Unless you want to
              destroy the original string.)
              I would happily destry the original string! but, I am reticent to
              admit, those simple replace's were failing silently. Haven't revisited
              it quite yet.. on a deadline.. but will look soon

              Comment

              • Dr J R Stockton

                #8
                Re: multiline string split problem and fix

                In comp.lang.javas cript message
                <erehh.4447$D9. 4178@twister.ny roc.rr.com>, Sun, 17 Dec 2006 16:41:46,
                mick white <mick@mickweb.c omwrote:
                >Brian wrote:
                >
                > hmmm, very late night typing.. I meant
                > strCSVFile.repl ace( / +/g, '');
                > strCSVFile.repl ace( /\n+/g, '');
                > strCSVFile.repl ace( /\r+/g, '');
                >>
                Perhaps you need:
                >someVariable=s trCSVFile.repla ce(/\s/g,'');
                >
                >No need for "+", since you are using the "g" modifier. And why not
                >assign the result of the statement to a variable? (Unless you want to
                >destroy the original string.)

                Str.replace should have no effect on Str's existence. The .replace
                method generates a new string, but does not destroy the old one.

                AFAIK, the only type of Object which has Methods provided to change its
                value is the Date Object.


                I would expect, if there are instances of multiple /r, for the /r+
                version to be slightly faster, since it calls for fewer replacements.
                That could be implementation-dependent. A quick test shows a slight
                gain in speed when using +.

                It's a good idea to read the newsgroup and its FAQ. See below.

                --
                (c) John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v6.05 IE 6
                <URL:http://www.jibbering.c om/faq/ A FAQ for news:comp.lang. javascript.
                <URL:http://www.merlyn.demo n.co.uk/js-index.htmjscr maths, dates, sources.
                <URL:http://www.merlyn.demo n.co.uk/TP/BP/Delphi/jscr/&c, FAQ items, links.

                Comment

                • mick white

                  #9
                  Re: multiline string split problem and fix

                  Dr J R Stockton wrote:
                  mick white <mick@mickweb.c omwrote:
                  >Perhaps you need:
                  >someVariable=s trCSVFile.repla ce(/\s/g,'');
                  >>
                  >No need for "+", since you are using the "g" modifier. And why not
                  >assign the result of the statement to a variable? (Unless you want to
                  >destroy the original string.)
                  >
                  >
                  Str.replace should have no effect on Str's existence. The .replace
                  method generates a new string, but does not destroy the old one.
                  var StringA="A B C"
                  var StringA=StringA .replace(/\s/g,'');
                  alert(StringA);
                  But you're correct, /technically/.

                  Mick

                  Comment

                  • Dr J R Stockton

                    #10
                    Re: multiline string split problem and fix

                    In comp.lang.javas cript message
                    <jOChh.4643$D9. 1848@twister.ny roc.rr.com>, Mon, 18 Dec 2006 20:24:47,
                    mick white <mick@mickweb.c omwrote:
                    >Dr J R Stockton wrote:
                    > Str.replace should have no effect on Str's existence. The .replace
                    >>method generates a new string, but does not destroy the old one.
                    >
                    >var StringA="A B C"
                    >var StringA=StringA .replace(/\s/g,'');
                    >alert(StringA) ;
                    >But you're correct, /technically/.
                    It is not the .replace that destroys "A B C", but the subsequent
                    assignment.

                    --
                    (c) John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v6.05 IE 6
                    news:comp.lang. javascript FAQ <URL:http://www.jibbering.c om/faq/index.html>.
                    <URL:http://www.merlyn.demo n.co.uk/js-index.htmjscr maths, dates, sources.
                    <URL:http://www.merlyn.demo n.co.uk/TP/BP/Delphi/jscr/&c, FAQ items, links.

                    Comment

                    • Brian

                      #11
                      Re: multiline string split problem and fix


                      Dr J R Stockton wrote:
                      ...
                      It's a good idea to read the newsgroup and its FAQ. See below.
                      >
                      --
                      (c) John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v6.05 IE 6
                      <URL:http://www.jibbering.c om/faq/ A FAQ for news:comp.lang. javascript.
                      <URL:http://www.merlyn.demo n.co.uk/js-index.htmjscr maths, dates, sources.
                      <URL:http://www.merlyn.demo n.co.uk/TP/BP/Delphi/jscr/&c, FAQ items, links.
                      Searching the three FAQs listed in your signature for the term
                      'replace' yields nothing at all about string replaces...

                      Rather than empathy or insight, a blind "read the FAQs" with no
                      specific pointer, and in this case FAQs that have no reference to my
                      problems. is in itself less than helpful.

                      page 526 of Javascript, the Definitive Guide, 4th Ed. does say that the
                      string replace() returns a new string with the replacement, but fails
                      to emphasize that the original argument is untouched.

                      On another note - Had this been C, would have checked the binary
                      contents of the array returned remotely immediately, by rote, in any
                      decent development envirnment. Since its web, and .js, and I'm still
                      getting used to it, I got caught by an annoying situation. Now I know.
                      I trust the thread will be helpful to someone someday.

                      Comment

                      • Dr J R Stockton

                        #12
                        Re: multiline string split problem and fix

                        In comp.lang.javas cript message
                        <1166668649.755 714.156400@79g2 000cws.googlegr oups.com>, Wed, 20 Dec 2006
                        18:37:29, Brian <googleAcct@scr eenlight.comwro te:
                        >
                        >Dr J R Stockton wrote:
                        >...
                        >It's a good idea to read the newsgroup and its FAQ. See below.
                        >Searching the three FAQs listed in your signature for the term
                        >'replace' yields nothing at all about string replaces...
                        If I had wished to give you advice about your problem, I would have done
                        so in an article *directly* following-up to one of yours.
                        >Rather than empathy or insight, a blind "read the FAQs" with no
                        >specific pointer, and in this case FAQs that have no reference to my
                        >problems. is in itself less than helpful.
                        If you have indeed searched all three URLs, you should have learned
                        quite a bit that can be useful to you in the future. You should also
                        have seen no case showing that the string parameter of .replace is
                        changed, and many cases in which the value of S.replace(a, b) is
                        assigned for use. You should also have seen a couple of references to
                        ECMA-262.
                        >page 526 of Javascript, the Definitive Guide, 4th Ed. does say that the
                        >string replace() returns a new string with the replacement, but fails
                        >to emphasize that the original argument is untouched.
                        I'll take your word for that; but what do other pages say? However, if
                        it is clear that a value is RETURNed, is it not then unreasonable to
                        expect the original string (which is not an argument) to be altered?
                        That would be superfluous.


                        It's a good idea to read the newsgroup and its FAQ. See below.

                        --
                        (c) John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v6.05 IE 6
                        news:comp.lang. javascript FAQ <URL:http://www.jibbering.c om/faq/index.html>.
                        <URL:http://www.merlyn.demo n.co.uk/js-index.htmjscr maths, dates, sources.
                        <URL:http://www.merlyn.demo n.co.uk/TP/BP/Delphi/jscr/&c, FAQ items, links.

                        Comment

                        Working...