I configured elastic_exporter to get elastic metrics in prometheus.
How to calculate and display the following indicators in grafana using prometheus metrics (see photo)?
Screenshot
I have already tried many instructions and metrics, the values are far from reality (which are shown by kibana).
Can anyone suggest which metrics of prometheus i can use to calculate indexing rate, indexing latency, search rate and search latency for many indexes and nodes like in kibana? Thanks in advance
Not prometheus specific, but basically,
- a rate metric is computed as: the number of operations / time taken to do those operations
- a latency metric is computed as: the time taken to perform operations / number of operations during that time
Taking search as an example, you need to query the index stats (GET index/_stats/search) and take two subsequent values of query_time_in_millis and query_total (e.g. 10 seconds apart).
Then you subtract the T2 value from the T1 value for both metrics and you get their derivatives for that 10 seconds interval, i.e.
delta_ops = query_total(t2) - query_total(t1)delta_time = query_time_in_millis(t2) - query_time_in_millis(t1)
Then:
- the search rate is
delta_ops / delta_time - the search latency is
delta_time / delta_ops
You can look at how Stack Monitoring computes those metrics:
- Search rate
- Search latency
2