Im running dbt core against a mysql database and I’m running into the limits of the group_concat_max_len
setting and need to increase it. Normally I would do that by running:
<code>SET SESSION group_concat_max_len=100000;
</code>
<code>SET SESSION group_concat_max_len=100000;
</code>
SET SESSION group_concat_max_len=100000;
This can’t be done directly in DBT, so instead I tried to run it as a pre_hook
:
<code>{{ config(
pre_hook = "SET SESSION group_concat_max_len=100000",
) }}
... some SQL statement involving GROUP BY
</code>
<code>{{ config(
pre_hook = "SET SESSION group_concat_max_len=100000",
) }}
... some SQL statement involving GROUP BY
</code>
{{ config(
pre_hook = "SET SESSION group_concat_max_len=100000",
) }}
... some SQL statement involving GROUP BY
But when I check the offending rows, they are still being cut by the GROUP BY
.
So what is the proper way to set a session variable in DBT?