I am running my web application over AWS ECS as following.

on one docker i have an API let’s call it API_A()
and this API_A90 within it’s body it calls another Api will call it API_B() and this deployed on another docker.

When running a 100 concurrent users stress test
API_A() response time reaches 5minutes.
I’ve user newrelic profiling tool and it says the 5minutes waster while calling API_A();

Same newrelic profiling tool saying API_B() response time is 5 seconds.

It means there are 4.55 minutes was waiting the server of API_B() to pickup the request.

API_A(); developed using .Net C#.
But API_B(); developed with Python and here is the running command :

DockerFile
`
FROM python:3.9

..
..
..
..

CMD [“newrelic-admin”, “run-program”, “uvicorn”, “fast_api:app”, “–proxy-headers”, “–host”, “0.0.0.0”, “–port”, “5000”, “–timeout-keep-alive”, “10000”, “–workers”, “5”]
`

the fast_api.py file comains

if __name__ == "__main__": uvicorn.run(app, host="0.0.0.0", port=5000,timeout_keep_alive=10000,workers = 5)

During the high traffic and response time, if i triggered the docker via IP address to check the HealthCheck it also takes so long to respond, so the DNS and Load Balancer are Incent so far.

Note that the CPU and RAM utlization are perfect also, not exceeding 50%

Any suggestions to find out or resolve this ?

i tried to trigger the docker via IP address to investigate if DNS or Load Balancer causing that but still that also delays the same during the traffic time.

i also tried to increase the workers, VCpu, Ram but no improvements at all.

Expecting the API_B() docker/server to pick all requests and start processing them all on concurrent basis. as it seems the requests are queued at the door of this docker/server and picked up so late.