I had a Databricks workspace that was not Unity Catalog enabled. So I had to create in terraform databricks groups in workspace level.

Now my Workspace has UC, I can create account groups from the interface. But I can not manage to make it work with terraform.

The documentation says that we need to add a provider databricks with host https://accounts.azuredatabricks.net/.

Originally this is my tf code:

# Used to create clusters/sql warehouses and policies in the workspace
provider "databricks" {
  azure_workspace_resource_id = module.databricks_01.id
  azure_client_id             = local.service_principal_application_id
  azure_client_secret         = var.secrets.SERVICE_PRINCIPAL_SECRET
  azure_tenant_id             = var.secrets.TENANT_ID
}

# this is inside a module, providers are defined at root of project, not in sub modules
resource "databricks_group" "this" {
  for_each                   = data.azuread_group.this
  display_name               = each.key
  external_id                = data.azuread_group.this[each.key].object_id
  workspace_access           = var.groups[each.key].workspace_access
  databricks_sql_access      = var.groups[each.key].databricks_sql_access
  allow_cluster_create       = var.groups[each.key].allow_cluster_create
  allow_instance_pool_create = var.groups[each.key].allow_instance_pool_create
  force                      = false
}

This is what I tried:

# Used to create clusters/sql warehouses and policies in the workspace
provider "databricks" {
  azure_workspace_resource_id = module.databricks_01.id
  azure_client_id             = local.service_principal_application_id
  azure_client_secret         = var.secrets.SERVICE_PRINCIPAL_SECRET
  azure_tenant_id             = var.secrets.TENANT_ID
}
provider "databricks" {
  alias               = "accounts"
  host                = "https://accounts.azuredatabricks.net"
  account_id          = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # My personal account ID from https://accounts.azuredatabricks.net/ at top right "My Account"
  azure_client_id     = local.service_principal_application_id
  azure_client_secret = var.secrets.SERVICE_PRINCIPAL_SECRET
  azure_tenant_id     = var.secrets.TENANT_ID
}

# this is inside a module, providers are defined at root of project, not in sub modules
resource "databricks_group" "this" {
  provider                   = databricks.accounts
  for_each                   = data.azuread_group.this
  display_name               = each.key
  external_id                = data.azuread_group.this[each.key].object_id
  workspace_access           = var.groups[each.key].workspace_access
  databricks_sql_access      = var.groups[each.key].databricks_sql_access
  allow_cluster_create       = var.groups[each.key].allow_cluster_create
  allow_instance_pool_create = var.groups[each.key].allow_instance_pool_create
  force                      = false
}

I’ve first removed databricks_group.this by commenting the code and made an apply. So all workspaces groups are removed.
Then uncommented my code and added the provider option to specify databricks.accounts

But I get this error that I don’t understand because the ressource is not in the state:

> terraform apply -var-file=context_and_secrets.json -auto-approve
╷
│ Error: Provider configuration not present
│
│ To work with module.databricks_policy.databricks_group.this its original provider configuration at
│ module.databricks_policy.provider["registry.terraform.io/databricks/databricks"].accounts is required, but it has been   
│ removed. This occurs when a provider configuration is removed while objects created by that provider still exist in the  
│ state. Re-add the provider configuration to destroy module.databricks_policy.databricks_group.this, after which you can  
│ remove the provider configuration again.