I am using a Databrick Shared Cluster with Runtime 13.3LTS.

When I run the following code:

spark.read.json("xxx").toJSON()

I get the following error:

Attribute toJSON is not supported in the Spark Connect as it depends
on JVM.

I know the root cause that RDDs are not allowed. But how can I mitigate the issue without moving to user cluster?