We have a spring application which is run via tomcat 7 where we do not have the problem. The same app is deployed with an upgraded tomcat server 9.0.59 with the same configurations.
On tomcat 7 (server) and any client (curl, chrome etc) this is my observation :
Once the HTTP1.1 payload is delived to the client the client initiaties to closes the connection.
On tomcat 9 and curl :
The behaviour is same as above.
Below is a snippet captured for curl (last column being latency)
https://github.com/iovisor/bcc/blob/master/tools/tcpstates.py
[ /usr/share/bcc/tools]# ./tcpstates | grep "10.0.12.79"
ffff9882b00aaaaa 0 swapper/2 10.14.40.x 8080 10.0.12.y 42312 SYN_RECV -> ESTABLISHED 0.016
ffff9882b00aaaaa 0 swapper/2 10.14.40.x 8080 10.0.12.y 42312 ESTABLISHED -> CLOSE_WAIT 3.238
ffff9882b00aaaaa 896675 java 10.14.40.x 8080 10.0.12.y 42312 CLOSE_WAIT -> LAST_ACK 0.687
ffff9882b00aaaaa 0 swapper/2 10.14.40.x 8080 10.0.12.y 42312 LAST_ACK -> CLOSE 0.157
On tomcat 9 with google chrome :
The client does not initiate the closing the connection and after the deafult timeout configured in tomcat (20s) , the server is forced to close the connection. On digging deeper I’ve seen that tomcat 9 sends connection header in the response headers while this is not true in tomcat 7.
[ /usr/share/bcc/tools]# ./tcpstates | grep "10.0.125.97"
ffff9882b00aaaaa 1462 java 10.14.40.x 8080 10.0.125.z 4286 SYN_RECV -> ESTABLISHED 0.009
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.z 4287 SYN_RECV -> ESTABLISHED 0.012
ffff9882b00aaaaa 896675 java 10.14.40.x 8080 10.0.125.z 4287 ESTABLISHED -> FIN_WAIT1 1271.089
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.z 4287 FIN_WAIT1 -> FIN_WAIT2 0.341
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.z 4287 FIN_WAIT2 -> CLOSE 0.004
ffff9882b00aaaaa 896675 java 10.14.40.x 8080 10.0.125.z 4286 ESTABLISHED -> FIN_WAIT1 1279.145
ffff9882b00aaaaa 1462 java 10.14.40.x 8080 10.0.125.z 4286 FIN_WAIT1 -> FIN_WAIT2 39.963
ffff9882b00aaaaa 1462 java 10.14.40.x 8080 10.0.125.z 4286 FIN_WAIT2 -> CLOSE 0.006
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.97 4288 SYN_RECV -> ESTABLISHED 0.009
ffff9882b00aaaaa 896675 java 10.14.40.x 8080 10.0.125.97 4288 ESTABLISHED -> FIN_WAIT1 1281.720
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.97 4288 FIN_WAIT1 -> FIN_WAIT2 40.297
ffff9882b00aaaaa 0 swapper/0 10.14.40.x 8080 10.0.125.97 4288 FIN_WAIT2 -> CLOSE 0.005
As from above it is evident that since client (curl) closes the connection, the server goes from established to close wait (pretty quickly as well). But incase of server closing the connection server goes from establised to fin_wait1 in 21s.
How to fix this latency and/or force the browser client to close the connection ?