I have an simple Azure CLI function that deployes the databricks workspace to a dedicated RG.
This deployment is basically done via DevOps pipelines.
My Azure CLI code looks like this:
function Create-Databricks {
param (
[string] $rgName,
[string] $dbName
)
az databricks workspace create --resource-group $rgName --name $dbName --location westeurope --output none
}
And my DevOps pipeline looks like this:
name: Deploy-Resources
trigger:
branches:
include:
- main
paths:
include:
- configuration/variables.ini
variables:
- template: common/Common-Variables.yml
pool:
vmImage: 'windows-latest'
steps:
- task: AzurePowerShell@5
inputs:
azureSubscription: '$(serviceConnectionName)'
ScriptType: 'FilePath'
ScriptPath: 'infrastructure/Main.ps1'
azurePowerShellVersion: 'LatestVersion'
I would like to add 2 more things after the worksape is deployed. Enable it for UC and create 1 external location pointing to my datalake in azure.
After some searching it appears that I should use Databricks CLI but Im confused on how can I integrate Azire CLI and Databricks CLI togather. Plus, as I understand it will requrie the installation and then authentication?
Any advice would be much appreciated, or some example codes or reading material to get started.
Reason why I want to use both languages is that I deploy some other resources for training purposes per trainee (Using for each command)
short version of the code (
foreach ($key in $iniContent["trainees"].Keys) {
# Get the trainee name
$traineeName = $iniContent["trainees"][$key]
# Set Azure context
# az account set --subscription "721fd46a-a810-4202-aa25-8b58b0410e1b"
$names = $traineeName -split ' '
$lastname = ($traineeName -split ' ')[1].Substring(0, [Math]::Min(5, $traineeName.IndexOf(" "))).ToLower()
# RG name
$resourceGroupName = "rg-academy-fundamentals-$lastname"
# SA name
$storageName = "stacademyazf$lastname"
# DBS name
$databricksName = "dbs-academy-azf-$lastname"
# AC name
Create-ResourceGroup -rgName $resourceGroupName -saName $storageName -tags $tags
Create-StorageAccount -rgName $resourceGroupName -saName $storageName -tags $tags
Create-Databricks -rgName $resourceGroupName -dbName $databricksName -tags $tags
and after the databricks deployment done for that specific “trainee”, then I would like to also prepare some UC ( with one schema in it) and external location.
1