Using StringTokenizer in Java

Package:
java.util
Class:
StringTokenizer
Methods:
nextToken()- returns string value of token
hasMoreTokens() - returns boolean value to check if there is more tokens left
nextElement()- returns string value of token
hasMoreElements() - returns boolean value to check if there is more tokens left
countTokens() - returns an int value, will count the number of tokens

When you want access the methods of the StringTokenizer class you will need to import the package that StringTokenizer belongs to. StringTokenizer forms part of the util package.

We need to make use of the reserved keyword import to import the package into our class.

The following statement will import the util package.
import java.util.*;

The following statement will import only the StringTokenizer class in the util package.
import java.util.StringTokenizer;

Once you have included this statement you will be able to make calls to the methods that belong to the StringTokenizer

According to www.dictionary.com the definition of delimiter is as follows: "
a blank space, comma, or other character or symbol that indicates the beginning or end of a character string, word, or data item."

For example:
In the following string "one,two,three,four" you will notice a commonality between the data. It can also be rewritten in the following format "data,data,data". Following this format we notice the "," is the commonality and thus is the delimter.

More examples:
With the string "13:14:15", the delimiter is the ":"
With the string "a b c", the delimiter is the " "
With the string "12a78a90", the delimiter is the "a"

Write a Java application to display the different names and numbers for the following string "Tazz:20 Daffy:40 Tweety:80"

Solution

import java.util.*;

public class MyStringTokenizer {

String data="Tazz:20 Daffy:40 Tweety:80";

public static void main(String[] args) {
new MyStringTokenizer().startApp();
}

public void startApp(){

StringTokenizer toons=new StringTokenizer(data," ");

while(toons.hasMoreTokens()){

StringTokenizer data_of_toons=new StringTokenizer(toons.nextToken(),":");

while(data_of_toons.hasMoreTokens()){
String name=data_of_toons.nextToken();
String number=data_of_toons.nextToken();

System.out.println("Name of toon "+name+" Number "+number);
}
}

}
}

Looking at the string that was provided we could make use of substring to extract all the data that we needed. The only problem is that when the string changes to "Tazz:20 Daffy:40 Tweety:80 Porky:99" the application written with substring will not work.

StringTokenizer will provide us a mechanism to extract the data as long as the string is constructed with the following format: Name:Number Name: Number ...

Looking at the format of how the string is constructed we can use StringTokenizer to extarxt the data.

import java.util.*;

This will import the util package and will allow us to make use of the methods of StringTokenizer.

String data="Tazz:20 Daffy:40 Tweety:80";

Define the string that was provided, the string is in the format as discussed above.


public static void main(String[] args) {
new MyStringTokenizer().startApp();
}


The main method is the entry point into our application. I also create an anonomous object to go from a static method to a non-static environment.


StringTokenizer toons=new StringTokenizer(data," ");


We make use of the constructor to create the tokens. We need to specify the string that contains the data and specify the delimiter as parameters for the constructor.

We will break the string up first with the space (" ") delimiter. Then we will seperate the name from the number by making use of another StringTokenizer and then make use of the ":" delimiter.

Basic Syntax for the constructor of the StringTokenizer Class

StringTokenizer(String string,String delimiter)


When no delimter is specified the default delimiter is a space (" ")

The object that we will use to access the first tokens is called toons.

Each token will now have string values as follows:
token 1: "Tazz:20"
token 2: "Daffy:40"
toekn 3: "Tweety:80"


while(toons.hasMoreTokens()){

}


The while is used to iterate thought all the tokens. The hasMoreTokens() method checks if there are more tokens left. This is done by returning a boolean value. True if there are more tokens and False should the tokens be completed. This will cause the while to exit.

StringTokenizer data_of_toons=new StringTokenizer(toons.nextToken(),":");

We need to seperate the name from the number, this data is contained in the token created by the first StringTokenizer. The nextToken() method contains the string value of the data in each token. We will use the delimiter ":" as the parameter to seperate the two pieces of data.


while(data_of_toons.hasMoreTokens()){

}

We will again make use of a while to iterate through the token to extract the data from the token.


String name=data_of_toons.nextToken();
String number=data_of_toons.nextToken();


We will now save the content of each token into a variable/member. This variable/member will then be used to display the data.

System.out.println("Name of toon "+name+" Number "+number);

This print the value of each variable/member to the console.

What is substring?

Returns a new string that is a substring of this string. The substring begins with the character at the specified index and extends to the end of this string.

The syntax for using substring:

substring(int beginIndex,int endIndex)

Example:
"hamburger".substring(4, 8) returns "urge"

Lets compare using substring and then StringTokenizer to extract data and add the data together and then display the answer.

Using substring
String marks_of_class="67@45@67@100@55@90@6@56";
int num1=Integer.parseInt(marks_of_class.substring(0,2));
int num2=Integer.parseInt(marks_of_class.substring(3,5));
int num3=Integer.parseInt(marks_of_class.substring(6,8));
int num4=Integer.parseInt(marks_of_class.substring(9,12));
int num5=Integer.parseInt(marks_of_class.substring(13,15));
int num6=Integer.parseInt(marks_of_class.substring(16,18));
int num7=Integer.parseInt(marks_of_class.substring(19,20));
int num8=Integer.parseInt(marks_of_class.substring(21,23));

int ans=num1+num2+num3+num4+num5+num6+num7+num8;

System.out.println("Sum is "+ans);

Using StringTokenizer

StringTokenizer token_num=new StringTokenizer(marks_of_class,"@");
int ans_stringtokenizer=0;

while(token_num.hasMoreTokens()){ ans_stringtokenizer=ans_stringtokenizer+Integer.parseInt(token_num.nextToken());
}

System.out.println("Sum is "+ans_stringtokenizer);

Please note two important aspects here:
1) The number of lines of code that was used.
2) What do you think will happen should the string value change to "100@12@@1@67@45@67@10@50@34@60@56"? You need to change the code for using substring, and for StringTokenizer..you dont need to change a single line of code.