I have fail2ban setup and it’s working great for most scanning. It triggers off any 4xx in the nginx error log. However, note the following bot scan. Somehow THIS bot is triggering my server to return 301 instead of 404, like all the others. How could it be doing this? Since it’s a 301 and not a 4xx, it walked right past my fail2ban and never got banned. I’d like to detect and prevent this.
Any suggestion on how this was done and how to prevent it?
178.20.44.82 - - [30/May/2024:21:28:48 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET /.DS_Store HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET /.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "POST /.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "GET /.env.prod HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "POST /.env.prod HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "GET /.env.production HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "POST /.env.production HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "GET /redmine/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "POST /redmine/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "GET /__tests__/test-become/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "POST /__tests__/test-become/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "POST / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view?panel=config HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view.html HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /frontend/web/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /web/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /sapi/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /_profiler/phpinfo HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /app_dev.php/_profiler/phpinfo HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /phpinfo.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /owncloud/apps/graphapi/vendor/microsoft/microsoft-graph/tests/GetPhpInfo.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:56 +0000] "GET /info.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:56 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"