I have noticed that when HTTP requests are made from within a thread, the file descriptors associated with the HTTP requests are never released and would continue accumulating. My suspicion is that this happens because of the urllib3 connection pools, where the connections are never released.

Simple demo to reproduce:

@pytest.mark.sandbox
def test_file_descriptors_leak():
    def request_google():
        return requests.get("https://google.com")

    for i in range(300):
        with ThreadPoolExecutor(max_workers=3) as executor:
            futures = [executor.submit(request_google) for _ in range(3)]
            for future in as_completed(futures):
                print(future.result())

        fd_count = sum(1 for _ in os.listdir("/proc/self/fd"))
        print(f"FD count: {fd_count}")

Is there a way to keep the file descriptors count constant while still running in a threaded environment?