I have a Qt application, which sends and receives messages. Messages are stored in a local MongoDB.
The application has a message list window, which shows sent, received and all messages (depending on the view). For performance reasons the application has a QList, which holds QPairs of Mongo ID and the message itself, so more or less all messages are available in the application.
We want to limit the no. of shown messages, because the view gets slower for 1000 and more messages.
The simple approach was just to limit the no. of pairs stored in the QList.
However, I’m not quite sure, if this is true. Recently I’ve read here (EDIT: found it) a question which basically said “Your application code won’t be smarter/faster/better for stuff a DB is designed for (e.g. sorting, getting a subset, etc.)”.
In addition keeping the QList in sync with the DB is causing us headaches and we have some corner cases like an very old message can be updated and then has to be displayed again.
So the question is, would a DB-only approach (e.g. querying the DB for 1000 messages whenever the view has to be updated) be faster then storing the acutally shown messages in the QList and trying to keep them in sync.
4
This seems more like a separation of concerns between the: QList, view and MongoDB.
Focus on the purpose of the QList and then make sure the synch capabilities get the job done. Do you need all the records for some purpose outside of the view?
We want to limit the no. of shown messages, because the view gets
slower for 1000 and more messages.
If the QList for whatever purpose outside of displaying a view, needs to hold more than the 1000 view limit, you need something else to limit the records coming from the QList to the view (if that’s how you need it to work.). Otherwise, you may need some other logic pulling the MongoDB data with a 1000 record limit with the sole purpose of feeding the view.
Is the limitation of the view (not sure exactly what that is) holding enough records in memory or the control you’re using as the display for the user (Data Grid, Combo Box, repeater, etc.)?
1
It’s impossible to say whether loading from the database will be faster without running some tests or knowing how often the window gets refreshed. My guess is that the perceived performance loading or working in the messages window will become constant rather than getting worse as the number of messages increases.
A not-insignificant upside to it would be simplifying your code a bit – you’ll no longer need any code to keep the list in sync with the database. You’ll just grab the latest (perhaps configurable number of) messages when loading or refreshing the messages window, so you won’t have to worry about stale data.
As you said in a comment:
Let’s say 300 would probably be also sufficient.
Retrieving a very small amount of data from the database will very likely not be noticeably slower, and if 300 messages can cover 90% of use cases and you can provide a way for users to retrieve older messages when needed, I bet you’ll come out ahead performance-wise.
1