Few Words About StringTokenizer Class For ISC Students

This post is mainly for ISC students as StringTokenizer class is in their syllabus. StringTokenizer is a predefined class of java.util package which can be used to break a string into tokens.

Constructors of StringTokenizer class

StringTokenizer class has three overloaded version of constructors and each version works with slight difference.

StringTokenizer(String s): This constructor of StringTokenizer class taken a string object as argument and break the string into tokens considering blank space as delimiters which means this constructor breaks the argument string into words.

Suppose the argument string is “This is a sample test”, the tokens will be “This”, “is”, “a”, “sample” and “test”.

In several programs we need to break a sentence into words and in such case we mostly use a combination of ‘indexOf ()’ and ‘substring ()’ function to perform the job. Using StringTokenizer class constructor, these type of jobs can be easily performed.

Second version of StringTokenizer class constructor is StringTokenizer( String s, String delim): Here the first argument is the string object to be broken into tokens and the second argument is the delimiter according to which the first argument will be broken into tokens. Here the default delimiter blank space, as in first version of constructor will be ignored.

Suppose the first argument is “This is a sample test, You have to wait for @another test” and the second argument is “,@” then the first argument string will be broken into tokens where the delimiter characters appears means the first token will be “This is a sample test”, the second token will be “You have to wait for” and the third token will be “another test”.

Third version of StringTokenizer class constructor is StringTokenizer( String s, String delim, boolean bool), where ‘bool’ is set to true: This version of constructor acts as the second version of constructor and associate the delimiter with the token. If we use the above sentence as first argument like StringTokenizer(This is a sample test, You have to wait for @another test”, “,@”, true); the tokens will be as follows “This is a sample test,”, “You have to wait for @” and “another test”.

StringTokenizer class has a function ‘ int countTokens()’ which the returns the number of tokens associated with a StringTokenizer class object.

Another function of StringTokenizer class is ‘boolean hasMoreTokens ()’ which retuens true if there is token with the StringTokenizer class object otherwise returns false.

The next most useful StringTokenizer class function is ‘ String nextToken()’ which returns the tokens from the StringTokenizer class object.

Here is a sample program on StringTokenizer class

import java.io.*;

import java.util.*;

class Str

{

  BufferedReader br=new BufferedReader(new InputStreamReader(System.in));

  String str,s;

  StringTokenizer stk;

  int i;

  public void show() throws Exception

 {

   System.out.println("Enter the sentence:");

   str=br.readLine();

  stk=new StringTokenizer(str);

  i=stk.countTokens();

  System.out.println("Number of tokens in the sentence="+i);

  System.out.println("The tokens are as follows\n");

  while(stk.hasMoreTokens())

  {

   s=stk.nextToken();

   System.out.println(s);

 }

 }

  public static void main(String args[])throws Exception

  {

   Stk ob=new Stk();

   ob.show();

   }

   }

Related Post: BlueJ Programs on String/Sentence