I want to create a resource tracker for my pipelines (the majority are deployed in a Cloud Run Service). so that I can save a log every time they run with the CPU & Memory usage, something like this:
"Resources Tracker": {
"Memory Consumed (MB)": 15.01171875,
"CPU Consumed (%)": 0.0
}
- I’ve tried to use psutil but it doesn’t work when the pipeline is deployed (CPU tracks 0.0%).
- I’m currently trying to use the Google Monitoring API to get this metrics, however, it has been complicated, this is what I have
from google.cloud import monitoring_v3
from google.cloud.monitoring_v3 import types
import datetime
client = monitoring_v3.MetricServiceClient()
project_name = client.common_project_path('tracker')
def cpu_allocation_time():
metric_type = 'run.googleapis.com/container/cpu/allocated'
now = datetime.datetime.now()
interval = types.TimeInterval({
'end_time': {'seconds': int(now.timestamp())},
'start_time': {'seconds': int((now - datetime.timedelta(minutes=5)).timestamp())},
})
aggregation = types.Aggregation({
'alignment_period': {'seconds': 60},
'per_series_aligner': monitoring_v3.Aggregation.Aligner.ALIGN_MEAN,
})
request = types.ListTimeSeriesRequest({
'name': project_name,
'filter': f'metric.type="{metric_type}"',
'interval': interval,
'view': monitoring_v3.ListTimeSeriesRequest.TimeSeriesView.FULL,
'aggregation': aggregation,
})
response = client.list_time_series(request=request)
for result in response:
print(result)
- I’m trying to guide myself using the following websites:
Monitoring-Api-Metrics
Monitoring-resources-cloud-run-revisions
Monitoring-rest-resources-timeseries
New contributor
lapiceroazul4 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.