Date: 2013feb5
Update: 2025oct13
Language: Java
Keywords: byte, stream, input, token, tokenizer
Q. Java: How to use StreamTokenizer on a String
A. Use ByteArrayInputStream to make an InputStream
and then convert that into a Reader as shown in this full example:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.Reader;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StreamTokenizer;
class Demo {
public static final void main(String[] args) throws IOException {
String s = "String 123 I want to parse";
System.out.println("input=" + s);
InputStream is = new ByteArrayInputStream(s.getBytes());
Reader reader = new BufferedReader(new InputStreamReader(is));
StreamTokenizer streamTokenizer = new StreamTokenizer(reader);
// Now we can use StreamTokenizer...
int currentToken = streamTokenizer.nextToken();
while (currentToken != StreamTokenizer.TT_EOF) {
if (streamTokenizer.ttype == StreamTokenizer.TT_NUMBER) {
System.out.println("token=number " + streamTokenizer.nval);
}
else {
System.out.println("token=string " + streamTokenizer.sval);
}
currentToken = streamTokenizer.nextToken();
}
}
}
Output:
input=String 123 I want to parse
token=string String
token=number 123.0
token=string I
token=string want
token=string to
token=string parse