I’m learning about compiler construction and have a question regarding the handling of structural tokens such as parentheses (), braces {}, and semicolons ;. I understand that these tokens are crucial for parsing but are not typically included in the symbol table. How does the compiler track and manage these tokens if they are not stored in the symbol table?
Here’s what I understand so far:
- Lexical Analysis: Converts source code into a stream of tokens
including identifiers, keywords, literals, operators, and structural
tokens like (), {}, and ;. - Symbol Table: Primarily used to track entities like variables,
functions, and literals that need to be referenced later. Does not
typically include structural tokens.
What I’m Curious About:
- How does the parser use structural tokens during syntax analysis if
they are not recorded in the symbol table? - What mechanisms or data structures are used to track the positions
and usages of these tokens during parsing?
Example
Consider the following C++ code:
int main() {
int a = 10;
float b = 5.5;
a = a + b;
return 0;
}
-
How are the parentheses () in main() and the braces {} around the
function body tracked during the parsing of this code? -
What role do these tokens play in the construction of the parse tree
or abstract syntax tree (AST)?
Also, check that my Symbol table is correct (Lexical Analysis Phase)