I am streaming huge amount of data from mongodb to other service.
For this i am using mongodb stream. But due to some reason if stream stops i want to rerun the job from where it stopped rather than starting it from start.
My question is if i store the last document id where it failed and rerun the stream from that document will this work ? Does streaming mongodb collection preserve same order every time or do i need to add sortBy for this ?
This is the stream i am using
db
.collection(PRODUCTS_COLLECTION)
.aggregate<MongoProduct>([
{
$lookup: {
from: 'prices',
localField: 'sku',
foreignField: 'sku',
as: 'price'
}
},
{
$project: {
sku: 1,
}
},
{
$match: {}
}
])
.stream();