I am using pm2 and nodejs/nestjs on ec2 machine, i created a script to filter the logs between a date range, but this is not filtering properly
Here is the filter_logs.sh
#!/bin/bash
# Usage: ./filter_logs.sh "06/18/2024, 5:23:00 AM" "06/18/2024, 5:51:00 AM"
# Input Parameters
START_TIME=$1
END_TIME=$2
if [ -z "$START_TIME" ] || [ -z "$END_TIME" ]; then
echo "Usage: $0 <start_time> <end_time>"
echo "Example: $0 "06/18/2024, 5:23:00 AM" "06/18/2024, 5:51:00 AM""
exit 1
fi
# Function to convert date-time to epoch
convert_to_epoch() {
local datetime=$1
datetime=$(echo "$datetime" | sed 's/,//g') # Remove the comma
date -d "$datetime" +%s
}
# Convert input date-time to epoch
start_epoch=$(convert_to_epoch "$START_TIME")
end_epoch=$(convert_to_epoch "$END_TIME")
echo "Looking for logs between: $START_TIME and $END_TIME"
if [ -z "$start_epoch" ] || [ -z "$end_epoch" ]; then
echo "Error in date conversion. Please check the date format."
exit 1
fi
# Filter logs using awk
pm2 logs | awk -v start_epoch="$start_epoch" -v end_epoch="$end_epoch" '
{
match($0, /[Nest] [0-9]+ - ([0-9]{2}/[0-9]{2}/[0-9]{4}, [0-9]{2}:[0-9]{2}:[0-9]{2} (AM|PM))/, arr);
if (arr[1] != "") {
log_time = arr[1];
gsub(",", "", log_time); # Remove the comma
cmd = "date -d"" log_time "" +"%s"";
cmd | getline epoch;
close(cmd);
if (epoch >= start_epoch && epoch <= end_epoch) {
print $0;
}
}
}'
and then
chmod +x filter_logs.sh
my logs are like this using pm2 logs 0
0|backend | [Nest] 55110 - 06/18/2024, 7:15:03 AM LOG [Request ID: 812dca29-1501-4eaf-b340-e630d3010d4b] [payment.service.ts] Found Payments Count: 2
0|backend | [Nest] 55110 - 06/18/2024, 7:15:03 AM LOG [Request ID: 812dca29-1501-4eaf-b340-e630d3010d4b] [payment.service.ts] Added payments []
0|backend | [Nest] 55110 - 06/18/2024, 7:16:57 AM LOG [Request ID: 5a1cd0be-6225-4a70-ab16-7c87f6064698] [applicants.service.ts] Applicant id 615 updated by user id 4.
0|backend | [Nest] 55110 - 06/18/2024, 7:17:14 AM LOG [Request ID: 1f69a88a-e715-4596-8ed3-117c95f8ac97] [s3.service.ts] Uploading docs/ENQ_684/cyhOBKcL.pdf
0|backend | [Nest] 55110 - 06/18/2024, 7:17:14 AM LOG [Request ID: 1f69a88a-e715-4596-8ed3-117c95f8ac97] [s3.service.ts] Uploaded docs/ENQ_684/cyhOBKcL.pdf
0|backend | [Nest] 55110 - 06/18/2024, 7:17:51 AM LOG [Request ID: 3242a4e1-b115-4f4f-b99a-44d21f8b4381] [s3.service.ts] Uploading docs/ENQ_684/oHM1D8Mi.mp3
0|backend | [Nest] 55110 - 06/18/2024, 7:17:51 AM LOG [Request ID: 3242a4e1-b115-4f4f-b99a-44d21f8b4381] [s3.service.ts] Uploaded docs/ENQ_684/oHM1D8Mi.mp3
0|backend | [Nest] 55110 - 06/18/2024, 7:23:34 AM LOG [Request ID: 5a3e981c-c6ea-4daf-9547-18f277455fdf] [applicants.service.ts] Applicant id 615 updated by user id 4.
0|backend | [Nest] 55110 - 06/18/2024, 7:23:46 AM LOG [Request ID: 8e0f449a-8c00-425c-b4fd-06c9e84ab7f3] [s3.service.ts] Uploading docs/ENQ_684/26kHSyQY.png
0|backend | [Nest] 55110 - 06/18/2024, 7:23:46 AM LOG [Request ID: 8e0f449a-8c00-425c-b4fd-06c9e84ab7f3] [s3.service.ts] Uploaded docs/ENQ_684/26kHSyQY.png
0|backend | [Nest] 55110 - 06/18/2024, 7:24:50 AM LOG [Request ID: fa79b09b-5a7f-4964-ac9c-4b75cab2779d] [s3.service.ts] Uploading docs/ENQ_684/NkpMZdJb.pdf
0|backend | [Nest] 55110 - 06/18/2024, 7:24:50 AM LOG [Request ID: fa79b09b-5a7f-4964-ac9c-4b75cab2779d] [s3.service.ts] Uploaded docs/ENQ_684/NkpMZdJb.pdf
but if i hit
./filter_logs.sh "06/18/2024, 7:23:00 AM" "06/18/2024, 7:23:59 AM"
it is not showing correct date range instead
ubuntu@ip-111-0-0-0:~$ ./filter_logs.sh "06/18/2024, 7:23:00 AM" "06/18/2024, 7:23:59 AM"
Looking for logs between: 06/18/2024, 7:23:00 AM and 06/18/2024, 7:23:59 AM
any help on this?
and if we can filter for request id as well