Hope everyone is doing fine.

I have been trying to find a way to monitor my webserver (nginx 1.24.0) for resource utilisation per each request such as CPU and memory usage for each HTTP request handled by my NGINX server and integrate with a monitoring solution such as Prometheus and Grafana.

What I’ve Tried

1. I have tried multiple exporters for prometheus such as:
   1.1. Node Exporter.
   1.2. Process Exporter.
   1.3. Nginx-Prometheus-Exporter.
   and tried to combine them in custom promql queries such as:

rate(node_cpu_seconds_total{mode="user"}[5m]) / nginx_connections_active

This promql query returns empty set which might be due to the there is no connection b/w currently active connections and my total cpu seconds (correct me, if I am wrong).

2. Using process exporter I can indeed find the memory and cpu utilisation of Nginx worker process but it does not give my individual details about each request.

3. I also tried configuring NGINX with custom logging and push these logs using telegraf tail input module, but I did not seem to find any variable’s in the nginx logging documentation that specifies my need:

Custom log format:

[IP] – [Method] – [Request Endpoint] – [Time taken to process the request] – [CPU seconds] – [Ram usage]

Example: 127.0.0.1 – GET /nginx_status – 0.02s – 1ms – 12.1mb

4. Then I tried finding custom solutions such as bash scripts to create these logs myself and push them to prometheus, did not have any luck with that either.

—

Any help is highly appreciated, please let me know if there’s a way in which I can track each HTTP requests Resource utilisation on my server (ubuntu 20.04).