I get this error:
pyspark.sql.utils.ParseException: mismatched input 'my_schema' expecting {')', ','}(line 86, pos 33)
for this code:

SELECT
my_table.id AS id,
my_table.name AS name,
    ...
    '210624100'                                                      AS segment,
    'shard01'                                                        AS shard_name,
    'ndb'                                                     AS database_name
FROM my_schema.my_table LEFT JOIN my_schema.some_table
ON my_schema.my_table.ID = my_schema.some_table.ID
    LEFT JOIN LATERAL (SELECT * FROM my_schema.another_table a_t WHERE a_t.NAME = my_table .NAME,
---------------------------------^^^
    AND a_t.ID = my_table.ID AND a_t.IS_ACTIVE IS NOT NULL LIMIT 1) active on TRUE
WHERE
    my_schema.my_table.type_id in(2, 8)