I have the following scenario:
I have a user that needs to see some news feeds from different sites and I want to paginate them and sort them by date. The problem is I don’t understand how to do that schemantically.
Fetch top-N from each site and then limit the results? That could result in inconsistent results if for example:
I ask the first 100 results from each site and then select in total the newest 100 from all sites. The second time I do this if site A had all of its first 100 results newer than the other sites , it is a problem. I will ask from the sites results from 100-200 where site B’s results should come from 0-100 because they were not displayed the last time.
How to do this kind of aggreggation without fetching all available (or an obsene amount of results) from each site?
4
You have to remember the start point when paginating, so if you retrieve the latest results for all sites, you have to be able to repeat the query for the same result set – that will depend on the result data you get back, for example, I will use an ID value for my SQL data results, but it could be based on date for other systems.
Once you have this, you can repeat the query asking for next (or previous) pages of data. If you ever want to start again, you’ll have to include a reload or refresh option that resets the start point. You only need the start point data for paginated queries, never the first one (though it might help if you have data that doesn’t return something you can use, you might have to store the datetime you requested the data before calling, and pass that through to prevent a new entry appearing between the time requested and results returned).
1
There can be no general rule.
If all of A’s articles are newer than B why would you expect any B articles in your list?
You need to nail down your expected behaviour and make a decision based on that. Also think about what happens if one resource is unavailable at query time etc.