We have Wildfly installed as our Application Server. There are multiple Linux servers, and Wildfly is running in Domain mode. To keep simple assume that we have 2 servers: Linux1 and Linux2 where Linux1 is the master node.
On these Wildfly instances, assume that we have 3 Web instances running on different ports. So the infra is as such:
- Linux1-Web:8200
- Linux1-Web:8201
- Linux1-Web:8202
- Linux2-Web:8200
- Linux2-Web:8201
- Linux2-Web:8202
We constantly monitor these web instances and sometimes, one of these servers utilizes very high CPU. And also, the load of that Linux server increases significantly. We’re still trying to find the root cause. Meanwhile, in order to prevent the users from lags and timeouts, we want to restart that specific web instance.
Here is a preview of our situation.
As you see, the process with the ID 4303 is at the top and currently utilizing 211. CPU. However, sometimes it increases over 4000. CPU. When that happens, we want to restart that instance which is Server-12:8206 in this example. We are going to add this to cron so we can check the situation in every minute.
Is there way to check the process CPU load and restart that specific instance after that check?