Qizx fe-4.4p2 API

com.qizx.api.util.text
Class LexicalTokenizer

java.lang.Object
  extended by com.qizx.xquery.impl.Lexer
      extended by com.qizx.api.util.text.LexicalTokenizer

public class LexicalTokenizer
extends com.qizx.xquery.impl.Lexer

Iterator on tokens of a XQuery script. Used for implementing syntax coloring in Qizx Studio.

Returns the lexical group and the position and size of each token.


Field Summary
static int LC_COMMENT
          Lexical category of XQuery comments.
static int LC_FUNCTION
          Lexical category of XQuery function calls.
static int LC_KEYWORD
          Lexical category of reserved keywords.
static int LC_MISC
          Lexical category of others tokens.
static int LC_NAME
          Lexical category of identifiers.
static int LC_NUMBER
          Lexical category of numeric literals.
static int LC_PRAGMA
          Lexical category of XQuery pragmas.
static int LC_SPACE
          Lexical category of whitespace.
static int LC_STRING
          Lexical category of string literals.
static int LC_TAG
          Lexical category of element constructors.
 
Fields inherited from class com.qizx.xquery.impl.Lexer
debug, ERR_SYNTAX, T_END
 
Constructor Summary
LexicalTokenizer(String script)
          Constructs a tokenizer of a XQuery script.
 
Method Summary
 String getSpace()
          Gets the space that is before the current token.
 int getSpaceLength()
          Size of leading space before the current token.
 int getTokenLength()
          Returns the length of the current token.
 int getTokenStart()
          Returns the start position of the current token.
 String getTokenValue()
          Returns the text of the current token: null if the end is reached.
 int nextToken()
          Moves to next token and returns its category.
 
Methods inherited from class com.qizx.xquery.impl.Lexer
match, startLexer
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LC_TAG

public static final int LC_TAG
Lexical category of element constructors.

See Also:
Constant Field Values

LC_SPACE

public static final int LC_SPACE
Lexical category of whitespace.

See Also:
Constant Field Values

LC_NUMBER

public static final int LC_NUMBER
Lexical category of numeric literals.

See Also:
Constant Field Values

LC_STRING

public static final int LC_STRING
Lexical category of string literals.

See Also:
Constant Field Values

LC_MISC

public static final int LC_MISC
Lexical category of others tokens.

See Also:
Constant Field Values

LC_NAME

public static final int LC_NAME
Lexical category of identifiers.

See Also:
Constant Field Values

LC_KEYWORD

public static final int LC_KEYWORD
Lexical category of reserved keywords.

See Also:
Constant Field Values

LC_COMMENT

public static final int LC_COMMENT
Lexical category of XQuery comments.

See Also:
Constant Field Values

LC_PRAGMA

public static final int LC_PRAGMA
Lexical category of XQuery pragmas.

See Also:
Constant Field Values

LC_FUNCTION

public static final int LC_FUNCTION
Lexical category of XQuery function calls.

See Also:
Constant Field Values
Constructor Detail

LexicalTokenizer

public LexicalTokenizer(String script)
Constructs a tokenizer of a XQuery script.

Parameters:
script - source XQuery script to tokenize
Method Detail

nextToken

public int nextToken()
              throws CompilationException
Moves to next token and returns its category.

Overrides:
nextToken in class com.qizx.xquery.impl.Lexer
Returns:
token code (LC_xxx)
Throws:
CompilationException - if parsing error

getTokenValue

public String getTokenValue()
Returns the text of the current token: null if the end is reached.


getTokenStart

public int getTokenStart()
Returns the start position of the current token.


getTokenLength

public int getTokenLength()
Returns the length of the current token.


getSpaceLength

public int getSpaceLength()
Size of leading space before the current token.


getSpace

public String getSpace()
Gets the space that is before the current token. Returns null if no space.


© 2010 Axyana Software