Dave's Brain

Browse - Programming Tips - How can I convert a string into tokens?

Date: 2012jun20
Language: Java
Keywords: token, tokenize

Q.  How can I convert a string into tokens?

A.  Java has these classes/methods builtin:

String.split() java.io.StreamTokenizer java.util.StringTokenizer java.util.Scanner
If you want something fancier look for a "lexer". Or look for a parser for the language you are dealing with. eg JSON, XML, RSS etc have parsers.

Add a comment

Sign in to add a comment
Copyright © 2008-2018, dave - Code samples on Dave's Brain is licensed under the Creative Commons Attribution 2.5 License. However other material, including English text has all rights reserved.