We have few REST linkedServices in ADF which are making API calls to get some data.
The problem is, they only seems to work in GIT mode or preview but not on the Live Data Factory and the JSON looks different after deployment either via arm deployment or by usinbg ADF Studio.
When in Live mode, I get 403 error when running the pipeline which is not an issue in Git mode…
This is the only type of LS that json looks different after deployment and I’ve tested that in blank ADF and in some of our environments. Tried with HTTP and the json looks exactly the same in Git and in Live Mode.
I’ve even created dummy REST LS in Dev like below:
{
"name": "test_LS",
"properties": {
"annotations": [],
"type": "RestService",
"typeProperties": {
"url": "https://api.test.com",
"enableServerCertificateValidation": true,
"authenticationType": "Anonymous",
"authHeaders": {
"X-API-Key": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "LS_KEYVAULT",
"type": "LinkedServiceReference"
},
"secretName": "SOME-API-Key"
}
}
}
}
}
I’ve tested connection and it works fine and when I go to pipeline with copy data, I can get the data in preview.
But… this is how the LS looks on live mode after publishing it:
{
"name": "test_LS",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"annotations": [],
"type": "RestService",
"typeProperties": {
"url": "https://api.test.com",
"enableServerCertificateValidation": true,
"authenticationType": "Anonymous",
"encryptedCredential": "ew0KICAiVmVyc2lvbiI6ICIyMDE3LTExLTMwIiwNCiAgIlByb3RlY3Rpb25Nb2RlIjogIktleSIsDQogICJTZWNyZXRDb250ZW50VHlwZSI6ICJQbGFpbnRleHQiLA0KICAiQ3JlZGVudGlhbElkIjogIkRBVEFGQUNUT1JZQDI5NjIwQkFBLTc3RkItNEU1Ni04MDg3LTkwMkRFREY0MUMyNF81Zjg2NDkyNi1kNTY2LTRjNDItOTUwMC0zMzBiMGFlOTljZTAiDQp9"
}
}
}
As You can see, those looks different and autHeaders are missing for some strange reason and I’m pretty sure this is the reason why it does not work. The question is why tho.
If this is somehow encrypted by default, that is great but it does not work after it being encrypted…
Spoke to support earlier today, ADF engineering team are aware and actively working on a resolution – but no ETA. It was suggested to me that the issue only arose when a keyvault was used in the Linked Service – so a short term workaround was possible by removing the keyvault and specifying the value directly…. but thats not been my experience
I have a REST LS, which takes the value for the auth header from a keyvault secret, and the keyvault LS is, in turn, using a global parameter supplying the keyvault location. I thought maybe this level of parameterization and keyvault usage was part of the problem. However, I sat on the call and showed the support person that the issue occurs on simpler setups – even just creating a simple rest LS, with a url explicitly supplied, and a header added which is also just a string – at that point the LS is immediately published to live, and shows up on live without the header.
So – no workarounds I’m aware of, I’m currently applying the header directly in production DF, then debugging, for one or two copy Data Tasks that use an LS with an auth header. These are failing when triggered. Not ideal.
Update: We managed to figure out a workaround!
For some reason, the authHeaders only get encrypted when a single value is provided. When two auth headers are provided, the value does not get encrypted or get published correctly.
So our work around is to just add a test value to every auth header:
Messy. But it works! Most APIs only check for the auth headers they require, so it shouldn’t affect data retrieval.
Auth headers are published correctly when two (or more) of them are specified.
Microsoft has stated that this is due to a bug, and have offered a (lousy) workaround: https://learn.microsoft.com/en-us/answers/questions/2036344/auzure-synapse-rest-linked-service-doesnt-save-aut
From your screenshot, it appears that you have added the value from Azure Key Vault (AKV).
If this is correct:
The issue is due to a bug, and the product group is working on a fix. However, there is no ETA at this time.
Workaround: You need to remove the AKV until the permanent fix is deployed.
We tried hardcoding the values instead of referring to the key vault, but this did not solve our issue. The hardcoded value would be encrypted in the same fashion and the LS would not work.
Answer from Microsoft PG:
ADF Rest Linked Service Auth headers issue had already been fixed since 6th Sept, 2024.
User can save auth header either in AKV or plain text, it can work fine.
Currently if you input plain text in auth headers, it will disappear after saving and refreshing. Because Credentials are encrypted. It’s known experience and will not block functionality.