Firebase limit issue: Error fetching records for page: 11 FirebaseError: Limit value in the structured query is over the maximum value of 10000.
When I fetch data from Firebase, I encounter the maximum value issue after 10,000 records. I am using Vue.js for the frontend and Firebase for the database. My database has more than 100,000 records, and I am fetching 1,000 records per page. When I go to the 10th page, the data is fetched correctly. However, when I go to the 11th page, I get an error from Firebase, which I have attached below, and the page gets stuck.
Error fetching records for page: 11 FirebaseError: Limit value in the structured query is over the maximum value of 10000.
at new n (index.cjs.js:160:23)
at t.ni (index.cjs.js:3312:16)
at t.pi (index.cjs.js:3471:195)
at n.onMessage (index.cjs.js:9955:33)
at eval (index.cjs.js:9908:26)
at eval (index.cjs.js:9939:37)
According to my research, I found that I should avoid using offset and manage data based on the last document ID, so I did that as well. But still, when I go to the next page, I am getting the last record’s doc ID from the previous page, but I am still facing the same issue.
Workable code with offset
rebindConv: firestoreAction(async ({ bindFirestoreRef }, { uid, page, limit }) => {
if (!uid) {
throw new Error("UID is required for rebindConvosNew");
}
// Get reference to the collection and apply the limit
const conversationsRef = db.collection('Conversations')
.where('Uid', '==', uid)
.orderBy('DateUpdated', 'desc');
// Calculate the starting point for the page
let query = conversationsRef.limit(limit);
// For pages beyond the first, fetch the starting point
if (page > 1) {
const offset = (page - 1) * limit;
// Get the last document of the previous page
const snapshot = await conversationsRef.limit(offset).get();
if (!snapshot.empty) {
const lastDoc = snapshot.docs[snapshot.docs.length - 1];
query = conversationsRef.startAfter(lastDoc).limit(limit);
} else {
throw new Error("No data found for the requested page.");
}
}
if (page > 1) {
const offset = (page - 1) * limit;
const snapshot = await conversationsRef.limit(offset).get();
const lastDocId = snapshot.docs.length > 0 ? snapshot.docs[snapshot.docs.length - 1].id : null;
// eslint-disable-next-line
console.log("lastDocId:", lastDocId);
// eslint-disable-next-line
console.log("Fetched records for page:", page - 1, snapshot.docs[snapshot.docs.length - 1].id);
}
// Bind the query to the 'convos' store state
return bindFirestoreRef('convos', query);
})
Workable code with LastDocId
rebindConv: firestoreAction(async ({ bindFirestoreRef }, { uid, page, limit }) => {
if (!uid) {
throw new Error("UID is required for rebindConvosNew");
}
// Get reference to the collection and apply the order
const conversationsRef = db.collection('Conversations')
.where('Uid', '==', uid)
.orderBy('DateUpdated', 'desc');
let query = conversationsRef.limit(limit); // Default query with limit
let lastDocId = null;
// For page > 1, we need to fetch the last document of the previous page
if (page > 1) {
// Fetch documents up to the previous page (not including current page)
const snapshot = await conversationsRef.limit((page - 1) * limit).get();
if (snapshot.empty) {
throw new Error("No data found for the requested page.");
}
// Get the last document from the previous page to use for pagination
lastDocId = snapshot.docs[snapshot.docs.length - 1].id;
// Log the last document ID for debugging
// eslint-disable-next-line
console.log("lastDocId for page " + (page - 1) + ":", lastDocId);
// Now, use startAfter to start fetching from the next page
query = conversationsRef.startAfter(snapshot.docs[snapshot.docs.length - 1]).limit(limit);
}
// Bind the query to the 'convos' store state
return bindFirestoreRef('convos', query);
})
Does anyone have a solution for this? If yes, please help
1