I’m setting up a Google Cloud Load Balancer with an external backend pointing to a Wasabi S3 bucket. The objective is to serve video files from the Wasabi bucket through the load balancer and Cloud CDN. However, I’m consistently getting a 502 Bad Gateway error. With the logs specifying: ‘backend_connection_closed_before_data_sent_to_client’.

Setup Details:

I am new to working with the cloud so following Google’s docs I made sure of completing these steps:

• SSL certificate: I set up a google managed SSL cert that includes but mydomain.com and www.mydomain.com (active)
• NEG: I set up a NEG that points to wasabi, with s3.us-west-1.wasabisys.com as the FQDN and port 443
• Load balancer: First I made sure to add a permission to allow Google’s IPs to send requests to my bucket. Then I set up a load balancer following this other guide. I configured the load balancer with those same settings except later I had to switch the backend protocols from http/2 to http since wasabi was not responding to http2 requests. In the backend configuration of the load balancer I have 2 services, one of them created when setting up the load balancer and the 2nd one as a “new” service (named user-clips) as per the first linked guide. I also made sure to add Host: s3.us-west-1.wasabisys.com in the custom request headers.
• Host and path rules: I added a new rule that looks like this Host: *, Paths: /all-users-clip-storage-wasabi/*, Backend: user-clips.
• Private origins: The bucket is not public so I had to configure private origin auth.

The problem:
When I try to make requests to the bucket through the CDN I get different errors depending on how I try to access it. When I use google’s example:

time wget "https://my_load_balancer_ip/all-users-clip-storage-wasabi/all-users-projects/exampleUser/project-name-placeholder/video_file.mp4

I get the following error:

The certificate's owner does not match hostname ‘my_load_balancer_ip’

When I replace the IP with my domain (with or w/o the www):

Resolving www.mydomain.com (www.mydomain.com)... {my_load_balancer_ip}
Connecting to www.mydomain.com (www.mydomain.com)|{my_load_balancer_ip}|:443... connected.
HTTP request sent, awaiting response... 502 Bad Gateway
2024-08-08 23:50:46 ERROR 502: Bad Gateway.

real    0m0.342s
user    0m0.017s
sys     0m0.005s

And this is the log from that failed request:

statusDetails: "backend_connection_closed_before_data_sent_to_client"

I read from other sources that the issue lies in the load balancer is sending traffic to another entity with a short timeout. But as you can see from the logs the answer to the request is almost instant (less 1 second).

I already tried:

1. Setting the TCP to 600s
2. Verified the SSL certificate is active for both domains
3. Updated the protocol from HTTP/2 to HTTP
4. Using pre signed URL’s
5. Ensured private origin authentication is configured correctly.

Questions:

1. What could be causing the 502 Bad Gateway error in this setup?
2. Are there additional configurations required to ensure the load
   balancer communicates correctly with the Wasabi S3 backend?
3. How can I verify if the load balancer is properly routing requests
   to the backend service?

Let me know if I can provide anymore context please.
Any guidance on resolving this issue would be greatly appreciated. Thank you!!