Thiết kế website giá rẻ

Question

Recently I started tolean python and play with fastapi to create some simple endpoints. Everything going well, until I changed my machine.
I was using a linux machine (ubuntu) as environment, but after change to MacOS with M2, I’m having problems to make the same code run.

My flow is very simple I’m doing some image processing and generating bytes as result. (Until here everything ok, I can even save the image locally).
After that I go for this function that does the upload and get the url of the image.

I have an aws s3 instance, that i tested, but locally I’m using minio, both have the same result.

After tries to do the upload the application stuck here (is hard even to kill using ctrl+c)

2024-05-11 13:53:42,078 botocore.awsrequest [DEBUG] Waiting for 100 Continue response.
2024-05-11 13:53:43,078 botocore.awsrequest [DEBUG] No response seen from server, continuing to send the response body.

The strange part I can find the image inside the bucket.

If you have some idea, about what to do here please let me know.

Thank You

That is my function and I tried some variation (upload instead put, upload,, using resource, …)

import logging
import os

import boto3
from botocore.config import Config
from fastapi import HTTPException
from botocore.exceptions import ClientError

# Configure logger to display the SDK communications
boto3.set_stream_logger('', logging.DEBUG)

async def s3_upload(contents: bytes, key: str, bucket: str):
    logging.info(f"Uploading {key} to S3")
    s3 = boto3.client(
        "s3",
        region_name=os.getenv("AWS_REGION"),
        aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
        aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
        endpoint_url=os.getenv("AWS_URL"),
        // config=Config(signature_version="s3v4"), tried this, bigger timeout
    )
    try:
        s3.put_object(Bucket=bucket, Key=key, Body=contents)
        logging.info(f"Uploaded {key} to S3")
        url = os.getenv("AWS_S3_STORAGE_URL").format(
            bucket=bucket, region=os.getenv("AWS_REGION"), key=key
        )
        return url
    except ClientError as e:
        logging.error("Failed to upload to S3:", e)
        raise HTTPException(status_code=500, detail="Failed to upload to S3")

After reach put_object, I’m stucked with this log

2024-05-11 13:53:42,075 botocore.hooks [DEBUG] Event request-created.s3.PutObject: calling handler <function add_retry_headers at 0x1100ac9a0>
2024-05-11 13:53:42,075 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://localhost:9000/instagram-posts/a18d9e39-8c10-4a5e-a9e5-ac388fe4fd25.webp, headers={'User-Agent': b'Boto3/1.34.103 md/Botocore#1.34.103 ua/2.0 os/macos#23.4.0 md/arch#arm64 lang/python#3.12.2 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.103', 'Content-MD5': b'RJlcDqt8AMZDSQBUS+St0A==', 'Expect': b'100-continue', 'X-Amz-Date': b'20240511T045342Z', 'X-Amz-Content-SHA256': b'4e0d34884d34f66b5b54833802caed835bf45347964a71a2cd0a996bf0fcdae0', 'Authorization': b'AWS4-HMAC-SHA256 Credential=Qy3ocHylzjK3MWJCJoIX/20240511/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=850913c8257be11fc433a12a7d918d8e587e323d2478c493bcf157c9478ae7ab', 'amz-sdk-invocation-id': b'37402dbc-b5e8-42dd-a1fb-b83ad2a8a6a7', 'amz-sdk-request': b'attempt=3; max=5', 'Content-Length': '42062'}>
2024-05-11 13:53:42,076 urllib3.connectionpool [DEBUG] Starting new HTTP connection (3): 127.0.0.1:50428
2024-05-11 13:53:42,078 botocore.awsrequest [DEBUG] Waiting for 100 Continue response.
2024-05-11 13:53:43,078 botocore.awsrequest [DEBUG] No response seen from server, continuing to send the response body.

After some tries, I have this error

2024-05-11 13:55:49,948 botocore.hooks [DEBUG] Event request-created.s3.PutObject: calling handler <function add_retry_headers at 0x1100ac9a0>
2024-05-11 13:55:49,949 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://localhost:9000/instagram-posts/a18d9e39-8c10-4a5e-a9e5-ac388fe4fd25.webp, headers={'User-Agent': b'Boto3/1.34.103 md/Botocore#1.34.103 ua/2.0 os/macos#23.4.0 md/arch#arm64 lang/python#3.12.2 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.34.103', 'Content-MD5': b'RJlcDqt8AMZDSQBUS+St0A==', 'Expect': b'100-continue', 'X-Amz-Date': b'20240511T045549Z', 'X-Amz-Content-SHA256': b'4e0d34884d34f66b5b54833802caed835bf45347964a71a2cd0a996bf0fcdae0', 'Authorization': b'AWS4-HMAC-SHA256 Credential=Qy3ocHylzjK3MWJCJoIX/20240511/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=ce67081d5aff9106eed87ab5f4b8afcd640e1d72439a4960882d3c3a1ece69db', 'amz-sdk-invocation-id': b'37402dbc-b5e8-42dd-a1fb-b83ad2a8a6a7', 'amz-sdk-request': b'attempt=5; max=5', 'Content-Length': '42062'}>
2024-05-11 13:55:49,949 urllib3.connectionpool [DEBUG] Starting new HTTP connection (5): 127.0.0.1:50428
2024-05-11 13:55:49,950 botocore.awsrequest [DEBUG] Waiting for 100 Continue response.
2024-05-11 13:55:50,951 botocore.awsrequest [DEBUG] No response seen from server, continuing to send the response body.
2024-05-11 13:56:50,958 botocore.hooks [DEBUG] Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x11270eff0>
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 537, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 466, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/.pyenv/versions/3.12.2/lib/python3.12/http/client.py", line 1423, in getresponse
    response.begin()
  File "/Users/lucas/.pyenv/versions/3.12.2/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/.pyenv/versions/3.12.2/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/.pyenv/versions/3.12.2/lib/python3.12/socket.py", line 707, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/httpsession.py", line 464, in send
    urllib_response = conn.urlopen(
                      ^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 445, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 539, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: AWSHTTPConnectionPool(host='127.0.0.1', port=50428): Read timed out. (read timeout=60)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/app/api/v1/endpoints/post.py", line 33, in create_upload_file
    file_names = await create_post(files, upload_data.caption)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/app/services/post.py", line 16, in create_post
    return [await handle_file_upload(files[0])]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/app/services/post.py", line 21, in handle_file_upload
    url = await s3_upload(
          ^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/app/helper/file_upload.py", line 24, in s3_upload
    s3.put_object(Bucket=bucket, Key=key, Body=contents)
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/client.py", line 1001, in _make_api_call
    http, parsed_response = self._make_request(
                            ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/client.py", line 1027, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
INFO:     127.0.0.1:49251 - "POST /api/v1/posts/upload/ HTTP/1.1" 500 Internal Server Error
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 119, in make_request
    return self._send_request(request_dict, operation_model)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 202, in _send_request
    while self._needs_retry(
          ^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 354, in _needs_retry
    responses = self._event_emitter.emit(
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 412, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 256, in emit
    return self._emit(event_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 239, in _emit
    response = handler(**kwargs)
               ^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 207, in __call__
    if self._checker(**checker_kwargs):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 284, in __call__
    should_retry = self._should_retry(
                   ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 320, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 363, in __call__
    checker_response = checker(
                       ^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 247, in __call__
    return self._check_caught_exception(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
    raise caught_exception
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 281, in _do_get_response
    http_response = self._send(request)
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 377, in _send
    return self.http_session.send(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lucas/code/instagram-clone/backend/post-service/.venv/lib/python3.12/site-packages/botocore/httpsession.py", line 501, in send
    raise ReadTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ReadTimeoutError: Read timeout on endpoint URL: "http://localhost:9000/instagram-posts/a18d9e39-8c10-4a5e-a9e5-ac388fe4fd25.webp"

As mentioned, the stange is if I go to this url (using minio in this case) I can see the image http://localhost:9000/instagram-posts/a18d9e39-8c10-4a5e-a9e5-ac388fe4fd25.webp and the image is inside the bucket

As mentioned too, this just happen when running the project inside Mac, testing in my linux or windows I don’t have the problem.

Thank You

Thiết kế website giá rẻ

Danh mục

Waiting for 100 Continue response while trying to upload an image