I am having a microservice hosted as an Azure Web App that sends outbound HTTP-POST calls based on various events. The calls have the characteristics of “fire-and-forget”. Thus, I start a thread for each performed HTTP-request as follows:
t = threading.Thread(target=self.perform_call, args=(target_url, notification_dto), daemon=True)
t.name = f'notification_sending_{notification_dto.id}'
t.start()
The request itself is executed via the requests
-package as follows (without any explicit session-management etc.):
response = requests.post(target_url, timeout=5, headers=headers_dict, data=data_str)
Sporadically, I see the following error in my log-file:
Traceback (most recent call last):
File “/agents/python/urllib3/connection.py”, line 203, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/agents/python/urllib3/util/connection.py”, line 60, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/python/3.11.8/lib/python3.11/socket.py”, line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not knownThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/agents/python/urllib3/connectionpool.py”, line 790, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File “/agents/python/urllib3/connectionpool.py”, line 491, in _make_request
raise new_e
File “/agents/python/urllib3/connectionpool.py”, line 467, in _make_request
self._validate_conn(conn)
File “/agents/python/urllib3/connectionpool.py”, line 1096, in _validate_conn
conn.connect()
File “/agents/python/urllib3/connection.py”, line 611, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File “/agents/python/urllib3/connection.py”, line 210, in _new_conn
raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7b7340378a90>: Failed to resolve ‘[here is the hostname: removed for the stackoverflow question]’ ([Errno -2] Name or service not known)The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/agents/python/requests/adapters.py”, line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File “/agents/python/wrapt/wrappers.py”, line 669, in call
return self._self_wrapper(self.wrapped, self._self_instance,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/agents/python/opentelemetry/instrumentation/urllib3/init.py”, line 228, in instrumented_urlopen
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/agents/python/urllib3/connectionpool.py”, line 844, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File “/agents/python/urllib3/util/retry.py”, line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='[here is the hostname: removed for the stackoverflow question]’, port=443): Max retries exceeded with url: /api/test/v1/update (Caused by NameResolutionError(“<urllib3.connection.HTTPSConnection object at 0x7b7340378a90>: Failed to resolve ‘[here is the hostname: removed for the stackoverflow question]’ ([Errno -2] Name or service not known)”))
This happens for about 1 of 50 requests. They are all handled by the same machine, but in different threads (as described). I would be very thankful for any ideas regarding root causes.
I tried to reproduce the error on my local mac but this seems to be difficult.