Relative Content

Tag Archive for razure-synapsesparkrazure-notebooksazure-synapse-pipeline

Azure Synapse Apache Spark Pools: .gz package added but notebook run error says not found

I have notebooks that contain r code. On their own, they work fine when run manually. To schedule and automate these workbooks, we have to use pipelines to call the r notebooks. However, pipelines don’t allow inline install of the r library packages. So, we have to install those packages on the Apache spark pool. I was able to do this successfully for all of the packages and their dependencies. I’ve run into a problem with the HydroVuR.gz package (This also happens to be our 1 custom package. hmmm??). While I have been able to add all of the packages to the Apache Spark Pool, when I run the notebook, I get an error specific to only our Hydrovu package.