I am using Express, Node 18, and JS for my back end. Also doing clustering using pm2.
Weirdly enough, I can see that some of my APIs get very slow randomly. CPU and Memory remain fine (I know because I have the NewRelic agent installed).
I suspect that some of my processes are doing synchronous operations which are then blocking my event loop and making all other APIs slow.
I’ll give an example :
An API called getUserFromId(uid) does a null check and then a Sequelize query on MySQL, A simple SELECT query. The table has an index on uid and consistently returns the response in under 10ms. Well, sometimes this simple API, will take 3s.
Upon checking the distributed traces in NewRelic I see that the DB query took let’s say 1s and some Application code (a simple IF statement + Sequelize ORM) took the rest of 2s.
At this time, DB is free. Memory is OK, and CPU usage is OK. Is my suspicion correct that it is the event loop which is being blocked by some synchronous work?
I have started using Blocked-at today. Has anyone faced some similar issues? If yes, what were the biggest levers for you and how to solve and monitor them?