I am trying to ingest lineage from ADF to Purview and the connection appears to be set up successfully however I cannot see any lineage data in Purview.
The set up is:
ADF
Running a Copy Data activity to copy a csv from one ADLS Gen2 container to another in the same storage account using a self-hosted integration runtime.
It is showing as connected to the Purview account and when I run the pipeline from my git branch in debug mode it shows the lineage status as succeeded.
Purview
Classic Purview account set up with firewall settings ‘Disabled for ingestion only’.
Connection to the ADF is showing as connected and after I run an ADF pipeline, the name appears through navigation via the monitoring tab, however the number of assets and relationships ingested is always 0.
Connectivity
I have followed the steps from the documentation to set up the private endpoints to the ingestion storage account after adding the ADF to Purview. (https://learn.microsoft.com/en-us/purview/catalog-private-link-end-to-end#option-2—enable-account-portal-and-ingestion-private-endpoint-on-existing-microsoft-purview-accounts)
I have also checked the ADF managed identity has the Data Curator role in Purview.
I am expecting to find new entities for the ADF pipeline and activity in Purview to connect up the csv files I have already scanned in separately however nothing is showing up, even after waiting a few hours.
From what I can tell this doesn’t seem to be an issue with connectivity nor one of the limitations of Purview as it is a very basic copy activity. What could be the cause of this? Has anyone faced a similar issue?
Thanks
2