I’m working on creating a lexer for a custom configuration file format using JFlex. However, I am encountering an issue with my .flex file and receiving an “Unexpected character” error during compilation.

Here is the content of my .flex file:

%{
import com.intellij.psi.TokenType;
import com.intellij.psi.tree.IElementType;
import static de.flox.floxsupport.FloxTypes.*;

StringBuffer string = new StringBuffer();

private IElementType token(int type) {
    switch (type) {
        case FLOX_CONFIG: return FLOX_CONFIG;
        case FLOX_KEYWORD: return FLOX_KEYWORD;
        case FLOX_NUMBER: return FLOX_NUMBER;
        case FLOX_STRING: return FLOX_STRING;
        case FLOX_SYMBOL: return FLOX_SYMBOL;
        default: return TokenType.BAD_CHARACTER;
    }
}
%}

%
%unicode
%cup
%line
%column

LineTerminator = r|n|rn
InputCharacter = [^rn]
WhiteSpace     = {LineTerminator} | [ tf]

TraditionalComment   = "/*" [^*] ~"*/" | "/*" "*"+ "/"
EndOfLineComment     = "//" {InputCharacter}* {LineTerminator}?

Identifier = [a-zA-Z_][a-zA-Z0-9_]*
DecIntegerLiteral = 0 | [1-9][0-9]*

%state STRING

%%

/* Lexical rules */

/* keywords and symbols */
"@"[a-zA-Z_][a-zA-Z0-9_]* { return token(FLOX_CONFIG); }
[a-zA-Z_][a-zA-Z0-9_]* { return token(FLOX_KEYWORD); }
[0-9]+ { return token(FLOX_NUMBER); }
"([^"\]|\.)*" { return token(FLOX_STRING); }

/* single-line comments */
"//".* { /* Ignore single-line comments */ }

/* whitespace */
s+ { /* Ignore whitespace */ }

/* symbols and punctuation */
[{}:.,] { return token(FLOX_SYMBOL); }

/* handle errors */
. { return TokenType.BAD_CHARACTER; }

<STRING> {
    " {
        yybegin(YYINITIAL);
        return token(FLOX_STRING, string.toString());
    }
    [^nr"\]+ { string.append(yytext()); }
    \t { string.append('t'); }
    \n { string.append('n'); }
    \r { string.append('r'); }
    \" { string.append('"'); }
    \ { string.append('\'); }
}

/* catch-all for unexpected characters */
[^] { throw new Error("Illegal character <" + yytext() + ">"); }

Error Message:

`Reading ".\src\main\resources\FloxLexer.flex"

Error in file ".\src\main\resources\FloxLexer.flex" (line 33):
Unexpected character
"@"[a-zA-Z_][a-zA-Z0-9_]* { return token(FLOX_CONFIG); }
^
1 error, 0 warnings.

I attempted to use JFlex to compile a lexer definition for a custom configuration format. The .flex file includes patterns for keywords, symbols, numbers, strings, and comments, and uses JFlex-specific syntax to define how tokens are recognized and returned.

What I tried:

1. Creating the .flex File:

   • I wrote a lexer specification using JFlex syntax to match various tokens, including keywords starting with the @ symbol.

2. Running JFlex:

   • I used the JFlex tool to compile the .flex file into a Java lexer class. Specifically, I executed the command:

     java -jar /path/to/jflex.jar FloxLexer.flex
     

   • I replaced /path/to/jflex.jar with the actual path to the JFlex JAR file.

What I was expecting:

• I expected JFlex to successfully generate a Java lexer class from the .flex file without errors.
• The lexer should correctly recognize tokens such as keywords starting with the @ symbol, numbers, strings, and various punctuation marks, and generate the appropriate token types.