I am currently querying data from a Parquet data lake stored in Azure Data Lake Gen2 via Grafana using Microsoft SQL Server as my data source. My query looks as below:
SELECT
DATEADD(SECOND, DATEDIFF(SECOND, '2020', t) / ($__interval_ms/1000) * ($__interval_ms/1000), '2020') AS time,
AVG(accelerationx) as AVG_accelerationx, AVG(accelerationy) as AVG_accelerationy, AVG(accelerationz) as AVG_accelerationz
FROM
OPENROWSET(
BULK 'https://cssdatalakestoragegen2.dfs.core.windows.net/cssdatalakestoragegen2filesystem/3BA199E2/CAN2_gnssimu/*/*/*/*',
FORMAT = 'PARQUET'
) AS r
WHERE
CONCAT(r.filepath(1), '-', r.filepath(2), '-', r.filepath(3)) BETWEEN '2020-10-28' AND '2020-10-28'
AND t BETWEEN '2020-10-28 14:44:00' AND '2020-10-28 14:48:00'
GROUP BY
DATEDIFF(SECOND, '2020', t) / ($__interval_ms/1000)
ORDER BY time
OFFSET 0 ROWS;
This works when the value of $__interval_ms
is 1000 or larger, but fails for smaller values. This is a problem as I need to be able to zoom in/out of my plot dynamically – switching between grouping data points e.g. at 5 min intervals to 1 ms intervals.
Is there a way to tweak the query so that I achieve the above? I seem to encounter errors related to overflow and other issues when doing modifications to this.