Relative Content

Tag Archive for pythonapifor-looppyspark

What is taking PySpark so much time in this loop?

I am requesting an Api (limited to 30calls per min so I do not parallelize) in a PySpark notebook (azure synapse analytics), and it takes ages in the synapse notebook while it’s lightning fast on my local env (PY3 kernel – jupyter notebook).