I am using optimistic responses in Apollo to add a new object to a list instantly in the UI. I use the cache.modify() function to update the query and it works fine for the first time but only once, every next attempt to mutate does not update the UI instantly (takes about 3~5 seconds to update).

Here is the code that reads the deliveries.

const { status } = useSession();
const { data: { deliveries } = { deliveries: null } } = useQuery(
    GET_DELIVERIES,
    {
        skip: status !== "authenticated",
        fetchPolicy: "cache-and-network"
    }
);

useEffect(() => {
    console.log("Deliveries updated", deliveries, Date.now());
}, [deliveries]);

The mutation:

addDelivery({
    variables: { location: locationObj, offset },
    optimisticResponse: {
        addDelivery: {
            __typename: "Delivery",
            id: `${Math.round(Math.random() * 999999)}`,
            createdAt: Date.now(),
            expectedEnd: null,
            routeId: null,
            location: {
                id: `${Math.round(Math.random() * 999999)}`,
                __typename: "GeoJSON",
                type: "Feature",
                geometry: {
                    __typename: "Geometry",
                    ...locationObj.geometry,
                },
                properties: {
                    __typename: "Properties",
                    ...locationObj.properties,
                },
            },
        },
    },
    update(cache, { data: { addDelivery } }) {
        cache.modify({
            fields: {
                deliveries(existingDeliveries) {
                    console.log("Modifying cache.", existingDeliveries, addDelivery, Date.now());
                    const newDelivery = cache.writeFragment({
                        data: addDelivery,
                        fragment: gql(`
                            fragment NewDelivery on Delivery {
                                id
                                location {
                                    id
                                    type
                                    geometry {
                                        type
                                        coordinates
                                    }
                                    properties {
                                        gid
                                        name
                                        street
                                        housenumber
                                    }
                                }
                                createdAt
                                expectedEnd
                                routeId
                            }
                        `),
                    });
                    return [...existingDeliveries, newDelivery];
                },
            },
        });
    },
});

The Apollo Client:

const cache = new InMemoryCache({
    typePolicies: {
        Query: {
            fields: {
                deliveries: {
                    merge(existing = [], incoming) {
                        return incoming;
                    },
                },
            },
        },
        Geometry: {
            keyFields: false, 
        },
        Properties: {
            keyFields: false,
        },
        GeoJSON: {
            fields: {
                properties: {
                    merge(existing = [], incoming) {
                        return incoming;
                    },
                },
                geometry: {
                    merge(existing = [], incoming) {
                        return incoming;
                    },
                },
            },
        },
    },
});

const client = new ApolloClient({
    link: splitLink,
    cache,
    connectToDevTools: true,
});

In the console it says that the cache is indeed updated instantly, but the UI is not updated at the same time, only works for the first mutation. Is there any reason why this is happening or any way to fix?