Our team has started migrating init scripts from DBFS. Our runtime version is 10.4, so we are considering moving the scripts to either ADLS or Workspace. The second option seems much easier than the first one. The main problem is that our init scripts interact with .whl files,
and according to Microsoft’s instructions, it’s not recommended to use Workspace for storing init scripts in this case. What do you think about this?
The script code looks something like this:
#!/bin/sh
mkdir -p ~/.config/example/
touch ~/.config/example/config.conf
cat <<EOF > ~/.config/example/config.conf
[install]
find-links = /path/to/packages/
EOF
for FILE in /path/to/packages/*.whl;
do
pip install ${FILE} --find-link /path/to/packages/
done
I have been working on migration planning for quite some time. The plan is to move the init script and .whl files from DBFS to ADLS, but this approach will take much more time than just putting the init script in the Workspace. I hope someone can advise if it’s possible to use the Workspace for migrating this type of script.
Chvyaken is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.