antlr
Class TokenBuffer

java.lang.Object
  |
  +--antlr.TokenBuffer

public class TokenBuffer

extends Object

Untamed: A Stream of Token objects fed to the parser from a Tokenizer that can be rewound via mark()/rewind() methods.

A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.

See Also:

antlr.Token, antlr.Tokenizer, antlr.TokenQueue

Field Summary

protected TokenStream	input
(package private) int	markerOffset
(package private) int	nMarkers
(package private) int	numToConsume
(package private) TokenQueue	queue

Constructor Summary

TokenBuffer(TokenStream input_)
Enabled: Create a token buffer

Method Summary

void	consume() Enabled: Mark another token for deferred consumption
private void	fill(int amount) Ensure that the token buffer is sufficiently full
TokenStream	getInput() Enabled: return the Tokenizer (needed by ParseView)
int	LA(int i) Enabled: Get a lookahead token value
Token	LT(int i) Enabled: Get a lookahead token
int	mark() Enabled: Return an integer marker that can be used to rewind the buffer to its current state.
void	rewind(int mark) Enabled: Rewind the token buffer to a marker.
private void	syncConsume() Sync up deferred consumption

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

input

protected TokenStream input

nMarkers

int nMarkers

markerOffset

int markerOffset

numToConsume

int numToConsume

queue

TokenQueue queue

Constructor Detail

TokenBuffer

public TokenBuffer(TokenStream input_)

Enabled: Create a token buffer

Method Detail

consume

public final void consume()

Enabled: Mark another token for deferred consumption

fill

private final void fill(int amount)
                 throws TokenStreamException

Ensure that the token buffer is sufficiently full

TokenStreamException

getInput

public TokenStream getInput()

Enabled: return the Tokenizer (needed by ParseView)

LA

public final int LA(int i)
             throws TokenStreamException

Enabled: Get a lookahead token value

TokenStreamException

LT

public final Token LT(int i)
               throws TokenStreamException

Enabled: Get a lookahead token

TokenStreamException

mark

public final int mark()

Enabled: Return an integer marker that can be used to rewind the buffer to its current state.

rewind

public final void rewind(int mark)

Enabled: Rewind the token buffer to a marker.

Parameters:

mark - Marker returned previously from mark()

syncConsume

private final void syncConsume()

Sync up deferred consumption