I am currently working on a data flow in Azure Data Factory where I need to check if the contents of a CSV file match with the contents of an SQL table. For this purpose, I am using the Exists activity.
While I am able to see the results of the Exists activity in the Data Preview section, I am struggling to find a way to get the total number of rows in the result set.
Here is what I have done so far:
- Created a data flow.
- Used the Exists activity to compare the CSV file (Blob Storage) with the Azure SQL table.
- Previewed the data using the Data Preview section, which shows the matching rows.
However, I need to get the count of these rows programmatically so I can use it in further activities within the pipeline.
My questions are:
- How can I get the row count of the result from the Exists activity?
- Is there a built-in function or expression in Azure Data Factory that can help me achieve this?
- Do I need to add additional activities or transformations to capture the row count, and if so, which ones?
Any guidance or examples would be greatly appreciated!
Additional Details:
- I am using Azure Data Factory version 2.
- The CSV file is stored in Azure Blob Storage.
- The SQL table is hosted in Azure SQL Database.
Thank you!