public class Tokenizer extends StreamTokenizer implements Serializable
Constructor and Description |
---|
Tokenizer(Reader text)
creating a tokenizer for the source stream
|
Tokenizer(String text) |
Modifier and Type | Method and Description |
---|---|
int |
lineno() |
int[] |
offsetToRowColumn(int offset) |
alice.tuprolog.Token |
readToken()
reads next available token
|
int |
tokenOffset() |
int |
tokenStart() |
commentChar, eolIsSignificant, lowerCaseMode, nextToken, ordinaryChar, ordinaryChars, parseNumbers, pushBack, quoteChar, resetSyntax, slashSlashComments, slashStarComments, toString, whitespaceChars, wordChars
public Tokenizer(String text)
public Tokenizer(Reader text)
public alice.tuprolog.Token readToken() throws InvalidTermException, IOException
InvalidTermException
IOException
public int lineno()
lineno
in class StreamTokenizer
public int tokenOffset()
public int tokenStart()
public int[] offsetToRowColumn(int offset)