i’m currently trying to deploy an API with its Opensearch “database” but i encounter problems :
My python API automatically crash when it tries to request Opensearch, here’s the log :
[2024-08-05 14:14:22 +0000] [7] [INFO] Booting worker with pid: 7
No sentence-transformers model found with name ./sentence-camembert-large. Creating a new one with mean pooling.
[2024-08-05 14:17:58 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:7)
[2024-08-05 14:17:58 +0000] [7] [ERROR] Error handling request /search?query=test&index=3
Traceback (most recent call last):
File "/embedding-api/.venv/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/embedding-api/.venv/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 178, in handle_request
respiter = self.wsgi(environ, resp.start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/flask/app.py", line 1498, in __call__
return self.wsgi_app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/main.py", line 205, in search
data = es.search(index=OPENSEARCH_INDEX, body=query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/opensearchpy/client/utils.py", line 180, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/opensearchpy/client/__init__.py", line 2359, in search
return self.transport.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/opensearchpy/transport.py", line 416, in perform_request
status, headers_response, data = connection.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/opensearchpy/connection/http_requests.py", line 199, in perform_request
response = self.session.send(prepared_request, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
self._validate_conn(conn)
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
conn.connect()
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/connection.py", line 615, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/connection.py", line 196, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/embedding-api/.venv/lib64/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
File "/embedding-api/.venv/lib64/python3.11/site-packages/gunicorn/workers/base.py", line 203, in handle_abort
sys.exit(1)
SystemExit: 1
[2024-08-05 14:17:58 +0000] [7] [INFO] Worker exiting (pid: 7)
--- AI execution : 0.20816683769226074s
[2024-08-05 14:17:59 +0000] [1] [ERROR] Worker (pid:7) was sent SIGKILL! Perhaps out of memory?
Modifying the Task execution role permissions doesn’t help. My API only tries to make _search queries. I’m using opensearch-py==2.6.0
to connect to my cluster :
hosts = [
{'host': OPENSEARCH_HOST, 'port': OPENSEARCH_PORT}
]
proxies = {}
if OPENSEARCH_HTTPS_PROXY is not None:
proxies['https'] = OPENSEARCH_HTTPS_PROXY
if OPENSEARCH_HTTP_PROXY is not None:
proxies['http'] = OPENSEARCH_HTTP_PROXY
class MyConnection(RequestsHttpConnection):
def __init__(self, *args, **kwargs):
proxies = kwargs.pop('proxies', {})
super(MyConnection, self).__init__(*args, **kwargs)
self.session.proxies = proxies
es = OpenSearch(
hosts,
use_ssl=True,
connection_class=MyConnection,
http_auth=(OPENSEARCH_USERNAME, OPENSEARCH_PASSWORD),
timeout=30,
proxies = proxies
)