We implemented unit tests
for the Pyspark
scenarios. But as tox
creates the runtime environment and installs the required packages, every time I run tox
, it has to install pyspark which takes around 3 minutes
everytime. So how can I avoid this long package installation runtime so that my test stage in CI/CD pipeline will be executed faster?
Is there any other alternative for this? I did my due diligence but couldn’t find anything relevant to my problem.