I am building up a new solution for the problem as below
Every Sunday I will be receiving high volume of data from a different vendors into my Azure cosmos DB. After this I want to retrieve the same set of data to apply some business logic and calculations then push that into my another Azure synapse cosmos DB with updated information.
Once the business logic and calculations are applied to the source cosmos DB data and putting that into Azure synapse cosmos DB, I wanted to push back the updates back to source cosmos DB.
For this I am reviewing below solutions
Azure Batch to read data from source cosmos and add business logic, here I am creating pools and tasks per vendor and nature of data.
With Azure batch I wrote c#.net based console app, which is also helping me to write unit test for the business logic and calculation but
Azure batch is consuming more computes and adding more nodes because of heavy data I am processing here.
Another option is to use Azure Data Factory, using data factory I can do same reading source cosmos and add business logic on data then push it to synapse cosmos and back to source cosmos.
Issue with ADF is that I can’t write test
ADF is cheaper compare to Azure batch.
So, wanted your opinion from the above list what would fit in my scenario