I am trying to build an opensource Docker image that would allow to configure pyspark and spark snowflake connector. I have provided the error log and contents of the Dockerfile below.

Here’s the link to Docker Image Repo: https://github.com/Snowflake-Labs/icetire

Please help resolve this issue. I am using Docker Desktop on Windwows 10 Pro and leveraging wsl. I am sitting behind Zscaler VPN.
I am very new to using these platforms/technolgies. I will appreciate any help here.

I am trying to build a docker image using the following command:

docker build –pull -t icetire .

The contents of the Dockerfile are as follows:

- Update lines 17 to 22 (beginning with ARG) with the correct levels to be deployed which executes deploy_snowflake.sh Script

- Note: For the jupyter scala kernel, the version can be set with the variable scala_kernel_version

Questions: Zohar Nissare-Houssen - [email protected]

#Start from the following core stack & driver levels versions#Picks right image to fix M1 Chip issuesFROM jupyter/all-spark-notebook@sha256:efe985059f915e7c96a2de7b4ce15fdc110582261ccecf3120fdf5d5c2acae3e

#Setting up default levels for all drivers and connectors#You can either customize levels in the section below or add as arguments while building the docker

USER rootARG almond_version=0.10.9ARG scala_kernel_version=2.12.11ARG odbc_version=2.25.5ARG jdbc_version=3.13.9ARG spark_version=2.11.0-spark_3.3ARG snowsql_version=1.2.23

#Installing base OS packages prerequisites including jdk 8 and jdk 11 versionsRUN apt-get update && apt-get install -y apt-utils && apt-get install -y libssl-dev libffi-dev && apt-get install -y vim && apt-get install -y iodbc libiodbc2-dev && apt-get install -y openjdk-8-jdk && apt-get install -y openjdk-11-jdk

#Installing Snowpark Scala Almond Jupyter KernelRUN update-alternatives --set java /usr/lib/jvm/java-11-openjdk-amd64/bin/javaRUN sudo -u jovyan /opt/conda/bin/curl --cacert /etc/ssl/certs/ca-certificates.crt -Lo coursier https://git.io/coursier-cliRUN chown -R jovyan:users /home/jovyan/coursier && chmod +x /home/jovyan/coursierRUN sudo -u jovyan /home/jovyan/coursier launch --fork almond:$almond_version --scala $scala_kernel_version -- --installCOPY ./kernel.json /home/jovyan/.local/share/jupyter/kernels/scala/RUN chown jovyan:users /home/jovyan/.local/share/jupyter/kernels/scala/kernel.json

#Creating conda Environment with Python 3.8 for Snowpark for Python called pysnowparkUSER jovyanRUN conda create -n pysnowpark -c https://repo.anaconda.com/pkgs/snowflake python=3.8

#Setting up default Python environment to support pyspark and snowflake python connectorRUN conda install -c conda-forge jupyterlab-plotly-extension --yesSHELL ["conda", "run", "/bin/bash", "-c"]RUN pip install --upgrade pipRUN pip install --upgrade pyarrowRUN pip install --upgrade snowflake-connector-python[pandas]RUN pip install --upgrade snowflake-sqlalchemyRUN pip install --upgrade plotlyRUN pip install --upgrade pyodbc

#Setting up Snowpark Python conda environmentSHELL ["conda", "run", "-n", "pysnowpark", "/bin/bash", "-c"]RUN pip install --user ipykernelRUN python -m ipykernel install --user --name=pysnowparkRUN pip install pandas nbformat plotly scikit-plot pyarrow==8.0.0 seaborn matplotlibRUN pip install snowflake-snowpark-python

#Deploying Snowflake Connectors and DriversUSER rootSHELL ["/bin/bash", "-c"]COPY ./deploy_snowflake.sh /RUN chmod +x /deploy_snowflake.shRUN /deploy_snowflake.sh

#Deploying Sample ScriptsRUN mkdir /home/jovyan/samplesCOPY ./pyodbc.ipynb /home/jovyan/samplesCOPY ./Python.ipynb /home/jovyan/samplesCOPY ./spark.ipynb /home/jovyan/samplesCOPY ./SQLAlchemy.ipynb /home/jovyan/samplesRUN chown -R jovyan:users /home/jovyan/samplesRUN sudo -u jovyan /opt/conda/bin/jupyter trust /home/jovyan/samples/pyodbc.ipynbRUN sudo -u jovyan /opt/conda/bin/jupyter trust /home/jovyan/samples/Python.ipynbRUN sudo -u jovyan /opt/conda/bin/jupyter trust /home/jovyan/samples/spark.ipynbRUN sudo -u jovyan /opt/conda/bin/jupyter trust /home/jovyan/samples/SQLAlchemy.ipynb

Here's the error Log:


% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     00     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.