I have seen a lot of articles explaining how to return the output from a databricks notebook in Data Factory. Is it possible to do this from a workflow?
I am new to Databricks, but the project I am working on calls a series of Databricks notebooks from ADF, every call is done via a databricks workflow, so notebooks are never called directly.
Essentially, this is what is happening now –
- Pipeline calls workflow and waits
- Workflow triggers a notebook
- Notebook code executes (python), returns a value with this code – dbutils.notebook.exit(‘test’)
- Pipeline tries to get the output and fails with this error – “cannot be evaluated because property ‘runOutput’ doesn’t exist”