We have a simple app using FastAPI that execute requests to another endpoint but we are seeing poor performance on percentiles P85, P90 and P99 running around 200/rps – we only see slow responses inside our code because same metrics on API Gateway is not possible to see the same latency problem. Our P85 is around 50ms, P90 jumps to 90ms and P99 is around 200ms.

Based on that, I was imagining that we are facing some problem on how we are using aiohttp and we did some refactoring in the code:

• created a singleton aiohttp to expose a single ClientSession;
• tried to remove limit of connections from TCPConnector.

Our code is very simple:

import aiohttp
import asyncio

from socket import AF_INET

class HTTPClient:
   
    @classmethod
    def get_aiohttp_client(cls) -> aiohttp.ClientSession:
        if cls.aiohttp_client is None:            
            connector = aiohttp.TCPConnector(
                family=AF_INET,
                limit=0,
                limit_per_host=0)
            cls.aiohttp_client = aiohttp.ClientSession(connector=connector)

        return cls.aiohttp_client

    
    @classmethod
    async def close_aiohttp_client(cls) -> None:
        if cls.aiohttp_client:
            await cls.aiohttp_client.close()
            cls.aiohttp_client = None

    @classmethod 
    async def query_url(cls, url: str, values: Dict[str, Any]) -> Any: 
        start = time.perf_counter() * 1000 
        client = cls.get_aiohttp_client() 
        payload = _create_value_payload(value) 
        try: 
            async with client.post(url=url, json=payload, headers=headers) as response: 
                value_response = await response.json() 
                end = time.perf_counter() * 1000 
                total_time = round(end - start) 
                logger.debug(f"Time load {value['group']} {value['name']}: {total_time}") 
                return {value['name']: value_response} 
        except Exception as e: 
            logger.error(f"An error occurred: {e}") 
            return {}

async def make_posts(values: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    async_calls = list()
    for value in values:
        async_calls.append(HTTPClient.query_url('...', value))

    responses = await asyncio.gather(*async_calls)
    responses = [response for response in responses if responses]
    return responses

The application is running on kubernetes with uvicorn single worker:

uvicorn app:server –host 0.0.0.0 –port 30000

Any idea how to debug aiohttp to identify what’s is wrong with our code? Do you see any smell in this code or any point to be optimized?