I have tried the below two options in crontab
and expecting to get only stderr
in the log file. However, some non-error lines related to mongodump command in the script file appear in the log file. I have seen various posts on stderr piping here, but couldn’t get what I wanted. Where am I going wrong?
The two crontab options that are tried:
<code>0 0 * * * /root/scripts/dev-vps_backup.sh >> logfile.log 2>&1 > /dev/null
0 0 * * * /root/scripts/dev-vps_backup.sh 2>&1 >> all.log | tee -a all.log err.log >&2
<code>0 0 * * * /root/scripts/dev-vps_backup.sh >> logfile.log 2>&1 > /dev/null
0 0 * * * /root/scripts/dev-vps_backup.sh 2>&1 >> all.log | tee -a all.log err.log >&2
</code>
0 0 * * * /root/scripts/dev-vps_backup.sh >> logfile.log 2>&1 > /dev/null
0 0 * * * /root/scripts/dev-vps_backup.sh 2>&1 >> all.log | tee -a all.log err.log >&2
the bash script file:
start_time=$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)
echo "Started Script: " $(basename "$0") "@ $start_time"
#########UPDATE THIS###########
### setting up variables here
######################## END of UPDATE
mkdir -p $local_backupfolder
mkdir -p $local_dumps_backupfolder
mkdir -p $local_assets_backupfolder
remote_base_backupfolder=root@$remote_IP:/root/r-backups/$vps_name
gzfile=$local_dumps_backupfolder/$vps_name--dump--$start_time.gz
dumpMsg="$vps_name mongodump StartTime:$start_time, EndTime:"
if mongodump --gzip --archive=$gzfile ; then ### THIS generates lines that are shown in stderr file
echo "DONE: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
echo "ERR: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name mongodump" $recipient_email
zipMsg1="$vps_name - $app_name1 - ZIP file for folder: $local_assets_folder1. StartTime:$start_time, EndTime:"
if zip -r $local_assets_backupfolder/$vps_name--$app_name1--assets-$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S).zip $local_assets_folder1 ; then
echo "DONE: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
echo "ERR: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name - $app_name1 - assets local ZIP" $recipient_email
some1-some_command # to generate error
<code>#!/bin/bash
set -e
#start time var
start_time=$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)
echo "Started Script: " $(basename "$0") "@ $start_time"
#########UPDATE THIS###########
### setting up variables here
######################## END of UPDATE
#create backup folders
mkdir -p $local_backupfolder
mkdir -p $local_dumps_backupfolder
mkdir -p $local_assets_backupfolder
# remote backup storage
remote_base_backupfolder=root@$remote_IP:/root/r-backups/$vps_name
# mongodump
gzfile=$local_dumps_backupfolder/$vps_name--dump--$start_time.gz
dumpMsg="$vps_name mongodump StartTime:$start_time, EndTime:"
if mongodump --gzip --archive=$gzfile ; then ### THIS generates lines that are shown in stderr file
echo "DONE: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
else
echo "ERR: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name mongodump" $recipient_email
exit
fi
zipMsg1="$vps_name - $app_name1 - ZIP file for folder: $local_assets_folder1. StartTime:$start_time, EndTime:"
if zip -r $local_assets_backupfolder/$vps_name--$app_name1--assets-$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S).zip $local_assets_folder1 ; then
echo "DONE: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
else
echo "ERR: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name - $app_name1 - assets local ZIP" $recipient_email
exit
fi
some1-some_command # to generate error
## some more lines
</code>
#!/bin/bash
set -e
#start time var
start_time=$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)
echo "Started Script: " $(basename "$0") "@ $start_time"
#########UPDATE THIS###########
### setting up variables here
######################## END of UPDATE
#create backup folders
mkdir -p $local_backupfolder
mkdir -p $local_dumps_backupfolder
mkdir -p $local_assets_backupfolder
# remote backup storage
remote_base_backupfolder=root@$remote_IP:/root/r-backups/$vps_name
# mongodump
gzfile=$local_dumps_backupfolder/$vps_name--dump--$start_time.gz
dumpMsg="$vps_name mongodump StartTime:$start_time, EndTime:"
if mongodump --gzip --archive=$gzfile ; then ### THIS generates lines that are shown in stderr file
echo "DONE: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
else
echo "ERR: $dumpMsg $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name mongodump" $recipient_email
exit
fi
zipMsg1="$vps_name - $app_name1 - ZIP file for folder: $local_assets_folder1. StartTime:$start_time, EndTime:"
if zip -r $local_assets_backupfolder/$vps_name--$app_name1--assets-$(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S).zip $local_assets_folder1 ; then
echo "DONE: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)"
else
echo "ERR: $zipMsg1 $(TZ=Asia/Calcutta date +%d-%m-%Y_%H-%M-%S)" | mailx -s "ERR: $vps_name - $app_name1 - assets local ZIP" $recipient_email
exit
fi
some1-some_command # to generate error
## some more lines
err.log file:
<code>2024-08-02T10:33:01.116+0200 writing admin.system.version to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
2024-08-02T10:33:01.118+0200 done dumping admin.system.version (1 document)
2024-08-02T10:33:01.118+0200 writing myca.syslogs to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
.....some more lines related to mongodump command....
/root/scripts/fvc1-vps_backup.sh: line 61: some1-some_command: command not found
<code>2024-08-02T10:33:01.116+0200 writing admin.system.version to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
2024-08-02T10:33:01.118+0200 done dumping admin.system.version (1 document)
2024-08-02T10:33:01.118+0200 writing myca.syslogs to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
.....some more lines related to mongodump command....
/root/scripts/fvc1-vps_backup.sh: line 61: some1-some_command: command not found
</code>
2024-08-02T10:33:01.116+0200 writing admin.system.version to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
2024-08-02T10:33:01.118+0200 done dumping admin.system.version (1 document)
2024-08-02T10:33:01.118+0200 writing myca.syslogs to archive '/root/backups/dumps/fvc1-vps--dump--02-08-2024_14-03-01.gz'
.....some more lines related to mongodump command....
/root/scripts/fvc1-vps_backup.sh: line 61: some1-some_command: command not found
all.log file has the additional lines related to echo
command and zip
command.
EXPECTED: Only the last line in err.log file – which is the error. How do I achieve this?
UPDATE
tried this crontab:
<code>59 11 * * * /root/scripts/fvc1-vps_backup.sh >> /root/scripts/logfile.log
<code>59 11 * * * /root/scripts/fvc1-vps_backup.sh >> /root/scripts/logfile.log
</code>
59 11 * * * /root/scripts/fvc1-vps_backup.sh >> /root/scripts/logfile.log
resultant logfile:
<code>Started Script: fvc1-vps_backup.sh @ 02-08-2024_15-29-01
DONE: fvc1-vps mongodump StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-01
adding: root/jobs/assets/ (stored 0%)
adding: root/jobs/assets/tempfiles/ (stored 0%)
...... some more lines resulting from zip command.....
adding: root/jobs/assets/invoiceDocs/ (stored 0%)
DONE: fvc1-vps - jobs - ZIP file for folder: /root/jobs/assets. StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-19
<code>Started Script: fvc1-vps_backup.sh @ 02-08-2024_15-29-01
DONE: fvc1-vps mongodump StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-01
adding: root/jobs/assets/ (stored 0%)
adding: root/jobs/assets/tempfiles/ (stored 0%)
...... some more lines resulting from zip command.....
adding: root/jobs/assets/invoiceDocs/ (stored 0%)
DONE: fvc1-vps - jobs - ZIP file for folder: /root/jobs/assets. StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-19
</code>
Started Script: fvc1-vps_backup.sh @ 02-08-2024_15-29-01
DONE: fvc1-vps mongodump StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-01
adding: root/jobs/assets/ (stored 0%)
adding: root/jobs/assets/tempfiles/ (stored 0%)
...... some more lines resulting from zip command.....
adding: root/jobs/assets/invoiceDocs/ (stored 0%)
DONE: fvc1-vps - jobs - ZIP file for folder: /root/jobs/assets. StartTime:02-08-2024_15-29-01, EndTime: 02-08-2024_15-29-19
The log file doesnot include mongodump related lines. That means these lines are going to stderr (as show in the first log file result). Not sure why?