I have a query regarding pod-to-pod communication across nodes with different subnets in Kubernetes using Calico CNI.

In my Kubernetes cluster, I have one control plane node and two worker nodes, each hosting a single pod. Specifically, Pod1 is located on Worker1, while Pod2 resides on Worker2. These worker nodes operate within distinct subnets.

Worker1 Network configuration:

**Interface Configuration:**
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:44:7e:48 brd ff:ff:ff:ff:ff:ff
    inet 10.82.10.82/24 brd 10.82.10.255 scope global enp0s3
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe44:7e48/64 scope link
       valid_lft forever preferred_lft forever
5: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
    inet 192.168.160.195/32 scope global tunl0
       valid_lft forever preferred_lft forever
6: calie49fb8bfb10@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1480 qdisc noqueue state UP group default qlen 1000
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link
       valid_lft forever preferred_lft forever

**Routes:**
default via 10.82.10.94 dev enp0s3 proto static
10.82.10.0/24 dev enp0s3 proto kernel scope link src 10.82.10.82
blackhole 192.168.160.192/26 proto bird
192.168.160.196 dev calie49fb8bfb10 scope link
192.168.163.0/26 via 20.20.2.4 dev tunl0 proto bird onlink
192.168.243.64/26 via 10.82.10.81 dev tunl0 proto bird onlink

Worker2 Network Configuration:

**Interface Configuration:**
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:e4:e6:ec brd ff:ff:ff:ff:ff:ff
    inet 20.20.2.4/29 brd 20.20.2.7 scope global enp0s3
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fee4:e6ec/64 scope link
       valid_lft forever preferred_lft forever
5: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
    inet 192.168.163.2/32 scope global tunl0
       valid_lft forever preferred_lft forever
11: cali1bd73d02ed0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1480 qdisc noqueue state UP group default qlen 1000
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link
       valid_lft forever preferred_lft forever

**Routes:**
default via 20.20.2.6 dev enp0s3 proto static
20.20.2.0/29 dev enp0s3 proto kernel scope link src 20.20.2.4
192.168.160.192/26 via 10.82.10.82 dev tunl0 proto bird onlink
blackhole 192.168.163.0/26 proto bird
192.168.163.3 dev cali1bd73d02ed0 scope link
192.168.243.64/26 via 10.82.10.81 dev tunl0 proto bird onlink

When traffic originates from Pod1 on Worker1 towards Pod2 with 192.168.163.3 on Worker2, the encapsulation process occurs on the tunl0@NONE interface of Worker1. During encapsulation, the actual source IP address of Pod1 (192.168.160.196) is masqueraded with the IP address of the tunl0 interface (192.168.160.195). Subsequently, the packet is encapsulated with an outer IP header having a source IP of 10.82.10.82 and a destination IP of 20.20.2.4.

However, I have a query regarding the necessity of masquerading the source IP of Pod1 with the tunl0 interface’s IP. If the traffic reaches Worker2 with the source IP of 192.168.160.196, based on its routes, the return traffic can still be routed to Worker1 via 10.82.10.82.

I would appreciate insight into why Calico masquerades the actual source IP of the pod with the tunl0 interface’s IP address.

Tried to refer the documentation regarding Calico online to find the reason behind masquerading. But couldn’t find a reasonable one. Would like to understand why Calico masquerades the actual source IP of the pod with the tunl0 IP address.