I am trying to run nginx on docker swarm platform using default ingress network. I have created 3 replicas on 5 node cluster. I can access nginx from first 3 node where replica is running, but I can’t access nginx(curl) from other nodes where replica is not running, looks like docker iptables issue to me, but not able to figure it out what changes are needed. I can see ingress has opened the published 8080 port on all the nodes, but can’t access nginx via curl.
Further, I checked the firewalld logs and found below related log lines.

Aug 12 07:30:37 ip-165-212-106-141 firewalld[119103]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -D DOCKER-INGRESS -p tcp --sport 8080 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain?).
2024-08-12 07:38:16 WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -nL DOCKER-INGRESS' failed: iptables v1.8.8 (nf_tables): chain `DOCKER-INGRESS' in table `filter' is incompatible, use 'nft' tool.