I am using a SecretClient to retrieve a shared secret stored in Azure KeyVault in my ASP.NET Core app on startup. When hosted in Azure, the App Service is configured with a Managed Identity which has SecretUser permissions to my Azure KeyVault. Recently, I noticed that after a few days (have not pinned down timeframe, unfortunately), the app regularly goes “down”, and when I look in the Application Event Logs, I see the following error:
Azure.Identity.AuthenticationFailedException: ManagedIdentityCredential authentication failed: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No connection could be made because the target machine actively refused it. (127.0.0.1:41071)) (No connection could be made because the target machine actively refused it. (127.0.0.1:41071)) (No connection could be made because the target machine actively refused it. (127.0.0.1:41071)) (No connection could be made because the target machine actively refused it. (127.0.0.1:41071))
The inciting call in the Stack Trace for this error is SecretClient.GetSecret.
Currently, I should be creating the SecretClient once and storing it in a singleton “KeyVaultService”, which I use to retrieve secrets as necessary. This specific one should only be retrieved on startup/connection, as it is for an OIDC client.
A restart of the App Service “fixes” the error and gets the app running again, but it eventually returns to this state.
Some snippets from my Code:
Building the Azure Credential:
if (!string.IsNullOrEmpty(clientId))
{
azureCredential = new DefaultAzureCredential(new DefaultAzureCredentialOptions()
{
ManagedIdentityClientId = clientId,
});
}
Adding the Azure Secret Client:
// Add any azure clients/services. This should be before our services, as they may use/consume these azure clients/services.
services.AddAzureClients(clientBuilder =>
{
KeyVaultConfig keyVaultConfig = new KeyVaultConfig(configuration, keyVaultConfigSection);
// Add secret client.
clientBuilder.AddSecretClient(keyVaultConfig.KeyVaultUri).WithCredential(keyVaultConfig.Credential).WithName(keyVaultName);
});
Retrieving the Secret (where the error happens):
public Azure.Response<KeyVaultSecret> GetSecret(string clientName, string secretName)
{
if (!SecretClients.ContainsKey(clientName))
{
var client = SecretClientFactory.CreateClient(clientName);
SecretClients.Add(clientName, client);
}
return SecretClients[clientName].GetSecret(secretName);
}