Sync using partial updates in a dataset?
I’m trying to find a solution to allow our users to stay in sync with an API that updates often. Right now we export our data once a month in JSON lines format. We want users to keep up with daily changes to records via the API, but it’s difficult to do because we update our records A LOT. We release a lot of new features, and when we do this we update 15M to 20M records per day (out of 250M total).