I am stuck in deciding how to implement the below scenario in my application (Spring boot + angular).
- User will upload a file with email ids (max records estimated is 10k) could be multiple times a month. Each file is a complete data set so we can delete the older records and insert the latest file as the fresh data set.
- For each email id we have to find the country of the user from a rest api.
I thought the below logic:
- Each time a user uploads this file store it in cloud storage.
- Run a batch to delete all existing records from table, read the file and insert all records.
In this scenario i will have to loop in and hit the service for all 10k records to get the user country which i think is not ideal.
Can someone please let me know how can i handle this use case. Can i just find the difference in data and use that? Or any other way you see this can implemented?