I am experiencing unexpected behavior in Azure Data Factory (ADF)’s data preview while working on a data flow. The data preview continues to show older data even after refreshing it. I have to turn off and turn on the data flow debug to view the updated data. How can I resolve this issue and ensure the data preview reflects the most recent changes immediately?
Furthermore, I am encountering issues during CI/CD pipeline deployment of ADF changes from my developer environment to production:
In some data flows, certain columns on the Sink side are either removed or unselected in the mapping after deployment.
During execution, some data flows fail without any clear reason, but when I copy the activities to a newly created pipeline, the issue is resolved and the pipeline runs successfully.
How can I troubleshoot these issues?