What is taking PySpark so much time in this loop?
I am requesting an Api (limited to 30calls per min so I do not parallelize) in a PySpark notebook (azure synapse analytics), and it takes ages in the synapse notebook while it’s lightning fast on my local env (PY3 kernel – jupyter notebook).