I have an asp.net core razor pages application that displays data in a javascript chart. In some scenarios, this chart can have up to 1 300 000 datapoints/database rows. Depending on the selected date range in the chart, I want to make a request to the server to calculate stats like min, max, and average values for the selected time range.
The problem I’m running into is how best to fetch the data once the request reaches the server. Making a database query for 1 300 000 rows every time a new date range is selected isn’t the best strategy. Some ideas I’ve come up with are to pass the data from the front-end in the request or cache the data on the server using a MemoryCache and fetch it during the request to avoid a database query. But I am unsure if storing 1 300 000 in a MemoryCache is good practice or even doable.
In short, what is a time-efficient way to store 1 300 000 rows of data for future processing when requests are made from the front end?