Using c++ During a token recognition error in the antlr4 Lexer how can I set a token type to it?
for eg. in the Lexer file I can add
@lexer::members {
enum {
UNKNOWN_TOKEN = INT_MAX
}
}
And when a token recognition error occurs, I can set it to UNKNOWN_TOKEN
.
I’d like to do this because I’m using the antlr grammar for syntax highlighting a gerebr file
The visitor is used to analyze the data in the file and the lexer is extended with a member that uses getAllTokens()
for colorizing the text.
I already highlight other errors and deprecated lines in the file by using
void syntaxError(antlr4::Recognizer* recognizer, antlr4::Token* offendingSymbol, size_t line, size_t charPositionInLine, const std::string& msg, std::exception_ptr e)
in the BaseErrorListner
During a token recognition error offendingSymbol
is a nullptr
so it isn’t really useful.
I have tried to use a catch all block in the lexer by doing
//All other stuff...
//End of default_mode.
UNKNOWN_TOKEN : .+?;
//Other modes...
but that messes with the parsing in unpredictable ways.
Even otherwise, I’d like to not use a catch-all block.
Yejneshwar Sivamoorthy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.