What programming language is this? It’s a Lisp dialect used by GCC internals.
(define_insn "*call_symbol"
[(call (mem:DI (match_operand:DI 0 "" ""))
(match_operand 1 "" ""))
(use (match_operand 2 "" ""))
(clobber (reg:DI LR_REGNUM))]
"GET_CODE (operands[0]) == SYMBOL_REF
&& !aarch64_is_long_call_p (operands[0])"
"bl\t%a0"
[(set_attr "type" "call")]
)
Link to source file
It is called machine description files. These .md files are used to generate a significant part of the C (now C++) code of the back end.
BTW, you could also customize GCC using a Lisp-like language: MELT
Some documentation is linked from MELT documentation page, and to get a global picture about GCC I strongly recommend taking several hours on the (Indian) GCC resource center.
This (mine) answer contains pictures which might be helpful to you.
2