I am trying to pytest a logic that has dbutils.widget
which is a Databricks feature inside a docker container.
My file structure looks like this:
├── Dockerfile
├── main
│ ├── src
│ │ ├── schemas.py
│ │ └── main.py
│ └── test
│ └── test_main.py
# main.py
def get_dbutils(spark):
from pyspark.dbutils import DBUtils
return DBUtils(spark)
def get_date_from_widget():
dates = get_dbutils.widgets.get("dates")
return dates
Part of my pytest logic:
# test_main.py
def test_get_date_from_widget(mocker, test_data, expected_result):
mock_get = mocker.patch("src.main.dbutils.widgets.get")
# Configure the mock behavior
mock_get.return_value = test_data
result = get_date_from_widget()
dates = get_dbutils.widgets.get("dates")
The error I get is:
AttributeError: 'function' object has no attribute 'widgets'
Am I missing something or is it not possible to pytest Databricks features with Docker?