I’m trying to install Secrets Store CSI Driver using aks add-ons in my private cluster. This cluster is deployed under my company subscription, therefore, there are some network limitations when it comes to pull images from official resources.
We use Harbor to pull images from official repos and we already configured the registry to pull from mcr.microsoft.com, the thing is when we try to install the add-on using the recommended method by Microsoft by running in the CLI
az aks enable-addons --addons azure-keyvault-secrets-provider --name myAKSCluster --resource-group myResourceGroup
the image, obviously, is trying to be pull from mcr.microsoft.com, and I want to use a custom URL from harbor harbor.mycompany.com .
How can I set the repo from which the image should be pulled?
I already tried editing the daemonset, but for some reason the image always back to mcr.microsoft.com.
1
The reason why even after editing the Daemon Set, the image reverts to mcr.microsoft.com is due to AKS’s add-on management. When AKS manages an add-on like the Secrets Store CSI Driver, it can automatically revert any manual changes made to the underlying resources, including Daemon Set, during updates or reconciliation processes.
As a workaround, disable the AKS-managed add-on entirely and install the Secrets Store CSI Driver manually using Helm, where you can control the image source.
Example-
az aks disable-addons --addons azure-keyvault-secrets-provider --name myAKSCluster --resource-group myResourceGroup
Now AKS won’t automatically revert the Daemon Set.
Now you can manually install the Secrets Store CSI Driver Using Helm
helm repo add secrets-store-csi-driver https://azure.github.io/secrets-store-csi-driver-provider-azure/charts
helm repo update
Create a values.yaml file where you specify the custom Harbor registry for both the CSI Driver and the Azure provider
You can take reference from here .
roughly something like this-
image:
registry: harbor.mycompany.com
repository: azure/secrets-store-csi
tag: latest # Specify the correct tag
providerAzure:
image:
registry: harbor.mycompany.com
repository: azure/provider-azure
tag: latest # Specify the correct tag
linux:
resources: {}
logLevel: 2
Install the Secrets Store CSI Driver manually using Helm
helm install csi-secrets-store secrets-store-csi-driver/csi-secrets-store-provider-azure
--namespace kube-system
--values values.yaml
Once done, verify that the DaemonSet is using images from harbor.mycompany.com
kubectl get daemonset csi-secrets-store -n kube-system -o yaml | grep "image"
Therefore, by disabling the AKS add-on and manually installing the Secrets Store CSI Driver using Helm, you can have full control over Daemon Set configuration thereby preventing AKS from reverting your changes.