VM contains several default routes

[vasileknik76]

Description

I'm testing vzNat and noticed strange behavior. After startup, the VM contains several default routes. So network connections are balancing between slirp and vnNat.

Lima Version: v0.21.0
Template: experimental/vz.

Steps:

• Run limactl start --name test --debug template://experimental/vz
• Run limactl shell test ip r. Output:

Actual output:

default via 192.168.5.2 dev eth0 proto dhcp src 192.168.5.15 metric 100
default via 192.168.105.1 dev lima0 proto dhcp src 192.168.105.2 metric 100
192.168.5.0/24 dev eth0 proto kernel scope link src 192.168.5.15 metric 100
192.168.5.2 dev eth0 proto dhcp scope link src 192.168.5.15 metric 100
192.168.105.0/24 dev lima0 proto kernel scope link src 192.168.105.2 metric 100
192.168.105.1 dev lima0 proto dhcp scope link src 192.168.105.2 metric 100

Expected output (without first route):

default via 192.168.105.1 dev lima0 proto dhcp src 192.168.105.2 metric 100
192.168.5.0/24 dev eth0 proto kernel scope link src 192.168.5.15 metric 100
192.168.5.2 dev eth0 proto dhcp scope link src 192.168.5.15 metric 100
192.168.105.0/24 dev lima0 proto kernel scope link src 192.168.105.2 metric 100
192.168.105.1 dev lima0 proto dhcp scope link src 192.168.105.2 metric 100

This comes from cloud-init, but I haven't found an option to disable this in network-config

[vasileknik76]

Off topic
Difference between user-v2 and vzNat is huge.
vzNat:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  64.9 GBytes  55.8 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  64.9 GBytes  55.8 Gbits/sec                  receiver

user-v2:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.75 GBytes  3.22 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  3.75 GBytes  3.22 Gbits/sec                  receiver

This is a test between the virtual machine and the host using iperf3

[balajiv113]

@AkihiroSuda
I believe we will always have our internal network (user-v2 or slirp) as default route right ??

[AkihiroSuda]

@AkihiroSuda I believe we will always have our internal network (user-v2 or slirp) as default route right ??

SGTM, but the plan may change depending on the performance and stability

[balajiv113]

Atleast with vzNAT i noticed that few cases of guest -> host network calls failing. This i noticed during very initial development of vz driver.

Maybe we can test and provide a option to override as per need basis

[vasileknik76]

Atleast with vzNAT i noticed that few cases of guest -> host network calls failing

Same thing for me. In my case problem related to VPN. By default nat is disabled between the vpn and bridge interface. I applied nat rules using pfctl and the network started to work stably.

I believe we will always have our internal network (user-v2 or slirp) as default route right

This is unexpected behavior for me. VzNat in this case seems unnecessary. Only for connect the VM at a own address?

[madalinignisca]

For a vm on vz, with first interface on user-v2, why would vzNAT on secondary interface add again default route?

Reading the docs, when using v2 and vzNAT makes me thing I use v2 like public interface and vzNAT as a host to VM possible connection.

I would have preferred vzNAT only if I could create separate vzNAT networks to separate my projects. Is there any way to configure vzNAT like user-v2 and the others are possible?

[nirs]

I think this is fixed by #2632. If you add additional network, it will be preferred over the user network.

[jandubois]

I think this is fixed by #2632. I

I think so too. Feel free to re-open with additional details if this is not the case!