Relative Content

Tag Archive for unit-testingapache-sparkpysparkcontinuous-integrationtox

Tox with pyspark

We implemented unit tests for the Pyspark scenarios. But as tox creates the runtime environment and installs the required packages, every time I run tox, it has to install pyspark which takes around 3 minutes everytime. So how can I avoid this long package installation runtime so that my test stage in CI/CD pipeline will be executed faster?