I’m trying to replace the load balancer provided by my cloud provider and trying to setup my own using NGINX on a Linux (Ubuntu 24.04) with 16 vCores and 128GB ram which I’d hope is good enough to support the load. My setup is like this, I have a bunch of “Clients” people with iPhones which I’m running thru a Proxy (Fiddler Proxy), since I have many I have 4 backend servers which I run thru a TCP load balancer, everything works mostly fine when using the cloud provider load balancer but some of my clients are savvy enough to run a simple Speed Test which gives very low speed results (6-8Mbps download/1-2Mbps upload), which in 5G days is very laughable so they’re demanding better service. If I connect directly to my backend servers I get 200+Up/100+Down easily even under heavy load so I’m pretty sure the Load Balancer is the issue. I tried NGINX and after going thru all the learning and configuration with the help of ChatGPT I got it up and running and all is great and speed is up to par but when I put load on it .. about 80 req/sec and above it starts throwing errors and timeouts and disconnecting sessions, etc .. my config is simple since I’m only using to distribute TCP load. Wondering if someone could help here to optimize and even I’m willing to compensate for expertise in private if offered.

Here’s my basic anonymized nginx.config if that helps:

user  nginx;

worker_rlimit_nofile 65536;

worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  16384;
    multi_accept on;
    use epoll;
}

stream {
        log_format main '$remote_addr [$time_local] "$protocol" $status $bytes_sent $bytes_received';

        access_log /var/log/nginx/stream_access.log main;

        upstream my_proxy {
            server xxx.xxx.xxx.xxx:8080 max_fails=5 fail_timeout=30;
            server yyy.yyy.yyyy.yyy8080 max_fails=5 fail_timeout=30;
            server zzz.zzz.zzz.zzz:8080 max_fails=5 fail_timeout=30;
        }

        server {
            listen 7777;
            proxy_pass my_proxy;
            proxy_timeout 60s;
            proxy_connect_timeout 5s;
            proxy_socket_keepalive on;
        }
}

I made other changes to some other config file in the system suggested by ChatGPT for high volume mostly about allowing more files and connections to remain open, etc but haven’t seen any improvement from that.

might be worth to mention I’m not Linux/nginx guy at all, I mostly deal with windows as a developer so please be patient with me lol.