tokens to string

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • hello12
    New Member
    • Oct 2006
    • 16

    tokens to string

    I am a newbie at Java. I am doing an assignment on string manipulations. Can anyone give me tips on how to make tokens back into a sentence.
    Eg.
    Good ------>(to) Good Morning Everyone!!
    Morning
    Everyone!!
    thanks in advance
  • r035198x
    MVP
    • Sep 2006
    • 13225

    #2
    Originally posted by hello12
    I am a newbie at Java. I am doing an assignment on string manipulations. Can anyone give me tips on how to make tokens back into a sentence.
    Eg.
    Good ------>(to) Good Morning Everyone!!
    Morning
    Everyone!!
    thanks in advance
    Do you have the words/tokens in an array or in a StringTokenizer ?

    Comment

    • hello12
      New Member
      • Oct 2006
      • 16

      #3
      StringTokenizer .

      Comment

      • Ganon11
        Recognized Expert Specialist
        • Oct 2006
        • 3651

        #4
        So you have a StringTokenizer whose contents are "Good", "Morning", and "Everyone!"

        Make a new String variable set to the empty string "". While your Tokenizer has more tokens, append the nextToken to the end of your String, plus a space.

        Comment

        • r035198x
          MVP
          • Sep 2006
          • 13225

          #5
          Originally posted by Ganon11
          So you have a StringTokenizer whose contents are "Good", "Morning", and "Everyone!"

          Make a new String variable set to the empty string "". While your Tokenizer has more tokens, append the nextToken to the end of your String, plus a space.
          Yes try that and post your code (using code tags) if you get any problem.

          Comment

          • hello12
            New Member
            • Oct 2006
            • 16

            #6
            I tried the code it's working but there is a problem..
            I tokenized the sentences from a file
            Hello World
            Good Morning Everyone!!
            I was suppose to reverse each word and output the result: right now i am
            getting
            olloH dlroW dooG gninroM !!enoyrevE ( how do i make this into 2 different lines like in the file)?
            try{
            BufferedReader Buf = new BufferedReader( new FileReader(File name));
            String line;

            while((line = Buf.readLine()) != null){

            StringTokenizer token = new StringTokenizer (line);
            int number = token.countToke ns();
            while(token.has MoreTokens()){
            String word = token.nextToken ();
            String hold = (reverseFunctio n(word));
            StringTokenizer tok = new StringTokenizer (hold);
            var += (tok.nextToken( )+ " ");


            }
            System.out.prin tln(var);
            System.out.prin tln(number);


            }
            Thanks for the help

            Comment

            • hello12
              New Member
              • Oct 2006
              • 16

              #7
              sorry i forgot the documentation part
              try{
              //reads from file
              BufferedReader Buf = new BufferedReader( new FileReader(File name));
              String line;

              while((line = Buf.readLine()) != null){

              StringTokenizer token = new StringTokenizer (line);
              int number = token.countToke ns();
              while(token.has MoreTokens()){
              String word = token.nextToken ();//Tokenizes the file

              String hold = (reverseFunctio n(word));// calls a function to reverse words//
              StringTokenizer tok = new StringTokenizer (hold);
              var += (tok.nextToken( )+ " "); //combines the word to form the sentence


              }
              System.out.prin tln(var); //prints out the new sentence
              System.out.prin tln(number);


              }

              Comment

              • Ganon11
                Recognized Expert Specialist
                • Oct 2006
                • 3651

                #8
                If your input file has the words in two seperate lines, you can repeat the following process for each line:

                1) Get the next line from your input file.
                2) Make a StringTokenizer with the input string.
                3) Reverse each token and hold in a larger string variable.
                4) Output the larger string variable

                This will reverse each word in each line individually, rather than the whole file.

                Comment

                • r035198x
                  MVP
                  • Sep 2006
                  • 13225

                  #9
                  Originally posted by hello12
                  I tried the code it's working but there is a problem..
                  I tokenized the sentences from a file
                  Hello World
                  Good Morning Everyone!!
                  I was suppose to reverse each word and output the result: right now i am
                  getting
                  olloH dlroW dooG gninroM !!enoyrevE ( how do i make this into 2 different lines like in the file)?
                  try{
                  BufferedReader Buf = new BufferedReader( new FileReader(File name));
                  String line;

                  while((line = Buf.readLine()) != null){

                  StringTokenizer token = new StringTokenizer (line);
                  int number = token.countToke ns();
                  while(token.has MoreTokens()){
                  String word = token.nextToken ();
                  String hold = (reverseFunctio n(word));
                  StringTokenizer tok = new StringTokenizer (hold);
                  var += (tok.nextToken( )+ " ");


                  }
                  System.out.prin tln(var);
                  System.out.prin tln(number);


                  }
                  Thanks for the help
                  I don't see why you are going through the trouble of the tokenizer. Why not just

                  Code:
                   while((line = Buf.readLine()) != null){ 
                  	  String hold = reverseFunction(line);
                  	  System.out.println(hold);
                  }

                  Comment

                  • hello12
                    New Member
                    • Oct 2006
                    • 16

                    #10
                    this is my first asiignment i was kind of exploring different types and methods.. i guess :)

                    Comment

                    • hello12
                      New Member
                      • Oct 2006
                      • 16

                      #11
                      when i try reversing like you said by passing the string line to reverse function it is even reversing the string.
                      Eg: Hello World! ---> !dlroW olleH
                      but i want ----> olleH !dlorW

                      Comment

                      • Ganon11
                        Recognized Expert Specialist
                        • Oct 2006
                        • 3651

                        #12
                        My suggestion should still work - have you tried that yet?

                        Comment

                        • r035198x
                          MVP
                          • Sep 2006
                          • 13225

                          #13
                          Originally posted by hello12
                          when i try reversing like you said by passing the string line to reverse function it is even reversing the string.
                          Eg: Hello World! ---> !dlroW olleH
                          but i want ----> olleH !dlorW
                          I see what the fuss is about then. Generally to tokenize a string, you should use the string.split() method

                          Comment

                          • hello12
                            New Member
                            • Oct 2006
                            • 16

                            #14
                            yes i tried your method too.. it is giving me the same result. I guess i would have to make some changes in the reverseFunction for your method ro work.I'll keep trying by finding what string.split() does and by changing reverseFunction around :)
                            thanks

                            Comment

                            • r035198x
                              MVP
                              • Sep 2006
                              • 13225

                              #15
                              Originally posted by hello12
                              yes i tried your method too.. it is giving me the same result. I guess i would have to make some changes in the reverseFunction for your method ro work.I'll keep trying by finding what string.split() does and by changing reverseFunction around :)
                              thanks
                              No you don't need to change your reverse function.

                              Code:
                               String[] tokens = line.split("\\s");
                              then reverse each token and append as Ganon has shown you how to

                              Comment

                              Working...