I have a Python worksheet on Snowflake and I am using this to deploy an udf. I want to add a update SQL inside this function but I believe I am missing something to find most efficient way.

When I use session.sql(update_query).collect() I am getting memory error:

SQL compilation error: Compilation memory exhausted.

Other methods I found from documents rewrites the whole table as I see and I do not want that since target table is very large.

Any thoughts?