Extension SDK 10.1.2

oracle.javatools.parser
Interface Lexer

All Known Implementing Classes:
AbstractLexer

public interface Lexer

The Lexer interface describes the set of calls that need to be implemented by a Lexer for any given language. It is intended for providing lexical information only, and does not do any syntactical or semantic translation.

The underlying document model passed into any Lexer implementation must implement the ReadTextBuffer interface to insulate the Lexer from the actual model. Please refer to the ReadTextBuffer interface for more information on why this interface is being used.

Token information is returned via the LexerToken structure. This is for convenience so that multiple pieces of (possibly language-specific) information may be returned to clients in the same call as the lex() operation. Clients need to call the createLexerToken() of a given Lexer implementation to create the LexerToken object corresponding to that given language and Lexer.

See Also:
LexerToken, ReadTextBuffer

Field Summary
static int TK_EOF
          Constant indicating that there are no more tokens remaining because the Lexer reached the end of the file.
static int TK_NOT_FOUND
          Constant indicating a bad token value.
 
Method Summary
 void backup()
          Unlexes the last found token.
 LexerToken createLexerToken()
          Creates a new empty LexerToken for a specific Lexer implementation to be passed in to the lex() call to be filled in.
 ReadTextBuffer getTextBuffer()
          Gets the text buffer that is being used by this Lexer.
 int lex(LexerToken lexedToken)
          Scans the text buffer at the current position and returns the token that was found.
 void setPosition(int offset)
          Sets the current lex (read) position to the given offset in the buffer.
 void setTextBuffer(ReadTextBuffer buffer)
          Sets the text buffer that is to be used by this Lexer.
 

Field Detail

TK_EOF

public static final int TK_EOF
Constant indicating that there are no more tokens remaining because the Lexer reached the end of the file.

See Also:
Constant Field Values

TK_NOT_FOUND

public static final int TK_NOT_FOUND
Constant indicating a bad token value. This is returned by the KeywordTable when a keyword lookup is not found.

See Also:
Constant Field Values
Method Detail

setTextBuffer

public void setTextBuffer(ReadTextBuffer buffer)
Sets the text buffer that is to be used by this Lexer.

Parameters:
buffer - the text buffer to use

getTextBuffer

public ReadTextBuffer getTextBuffer()
Gets the text buffer that is being used by this Lexer.

Returns:
the text buffer being used

setPosition

public void setPosition(int offset)
Sets the current lex (read) position to the given offset in the buffer. It is the clients responsibility to ensure that this offset corresponds to the start of a token, otherwise unexpected (and incorrect) results may occur.

Parameters:
offset - the offset for the next lex() operation.

createLexerToken

public LexerToken createLexerToken()
Creates a new empty LexerToken for a specific Lexer implementation to be passed in to the lex() call to be filled in.

Returns:
a newly created LexerToken

lex

public int lex(LexerToken lexedToken)
Scans the text buffer at the current position and returns the token that was found. The token and offset information is also stored in the lexedToken instance passed in to the call.

Parameters:
lexedToken - the instance passed in where token info is stored
Returns:
the token that was found, same as calling lexedToken.getToken() (for convenience)

backup

public void backup()
Unlexes the last found token. The next call to lex() will return the last token and offset information found.


Extension SDK

 

Copyright © 1997, 2004, Oracle. All rights reserved.