export const cacheRecommendations = async () => {
    const userCursor = UserModel.find({})
        .cursor()
        .addCursorFlag("noCursorTimeout", true);
    let counter = 0;

    for await (const user of userCursor) {
        console.log("counter", counter);
        counter++;

        await calculateRecommendationsWithTensorflow(user._id);
    }
};

This code iterates over every user in the database. The calculations from calculateRecommendationsWithTensorflow take a couple of minutes per user to complete. It uses a tensorflow model to iterate over many documents to find the best rated ones.

The problem is that randomly after user number 100 I get an error:
cursor id 7158571831382380588 not found. It’s always at the same cursor position. At this point the function is running for already round about 9 hours.

When I try to skip the first 99 users, the calculations proceed normally for user 100 and the users afterwards:

export const cacheRecommendations = async () => {
    const userCursor = UserModel.find({})
        .cursor()
        .addCursorFlag("noCursorTimeout", true);
    let counter = 0;

    for await (const user of userCursor) {
        console.log("counter", counter);
        counter++;

        if (counter < 99) continue; // <--- SKIP THE FIRST USERS, THEN THE CURSOR WORKS CORRECTLY

        await calculateRecommendationsWithTensorflow(user._id);
    }
};

It is pretty hard to debug because I didn’t find a way to make it fail without waiting 9 hours yet. Another important information might be that the code is invoked in a worker thread, since Tensorflow otherwise blocks the main thread.

If anyone has an idea how to solve this weird bug, I’d be happy to hear about it.