How to ping a VM going through another VM in the same network without pinging directly?

[JordanFist]

I have 3 VMs on the same network using host-only adapters. We can call them VM1, VM2, and VM3. VM1 is already able to directly ping VM3. However, I would like VM1 to ping VM3 going through VM2. How can I change this behavior?
I tried to change the routes as follows:

Initially, when I type route -n in VM1:

Code: Select all

Destination       Gateway          Genmask           Flags    Metric  Ref     Use     Iface
169.X.0.0         0.0.0.0          255.255.0.0       U        1000     0        0     enp0s3
192.X.X.0         0.0.0.0          255.255.255.0     U        100      0        0     enp0s3

192.X.X.0 is the network where my 3 VMs run.
Then I added a route where 192.X.X.11 is VM2

Code: Select all

Destination      Gateway          Genmask               Flags.   Metric   Ref      Use     Iface
169.X.0.0         0.0.0.0           255.255.0.0         U        1000      0         0     enp0s3
192.X.X.0         192.X.X.11        255.255.255.255     UGH      0         0         0     enp0s3
192.X.X.0         0.0.0.0           255.255.255.0       U        100       0         0     enp0s3

VM2 is regarded as a switch thanks to OpenvSwitch (OVS) but when I try to ping VM3, VM1 ping directly VM3. I used traceroute and tracepath to check this result. How can I change the route to go through VM2?

[scottgus1]

I am not skilled in setting up routes, but we can first check that the networks are set up to allow communication of some sort.

Please post the text command output of:

vboxmanage list hostonlyifs

Also, please right-click each guest VM in the main Virtualbox window's guest list, choose Show in File Manager. Zip the guest's .vbox file, and post the zip file, using the forum's Upload Attachment tab.

Finally, I have been looking for a "switch OS" that can be installed in a Virtualbox guest. I looked at http://www.openvswitch.org/, downloaded it, and found it was a bunch of source code. I am also not skilled in building things from source code, but could you please tell me if OpenvSwitch acts like an operating system that gets installed directly in a VM, or is it an app or service that rides on a previously-installed OS?

[Martin]

Are you sure that your TCP/IP stack even cares about the additional route entry if you already have an active interface directly in that subnet?

[JordanFist]

Answer to scottgus1

After installing VirtualBox, this command vboxmanage list hostonlyifs prints nothing in the terminal.
You can find my VMs in the attachment and a diagram of the network I am trying to achieve attached. Host1 and Host2 are just regular Ubuntu18.04 VMs and the OVS VM is a Raspbian VM(RP1) with OVS installed.

Actually you can easily install OpenvSwitch as follows:

Code: Select all

sudo apt-get install openvswitch-switch -y
sudo apt-get install openvswitch-common
sudo apt-get install bridge-utils

Then you can use this command to add a bridge sudo ovs-vsctl add-br ovsbr0
To add a port in the bridge sudo ovs-vsctl add-port ovsbr0 eth1
To show your interface sudo ovs-vsctl show

And as you guessed OVS is just a service running over the OS pre-installed.

Answer to Martin
How can I check if this route is taken into account? I didn't manage to remove this line 192.X.X.0 0.0.0.0 255.255.255.0 U 100 0 0 enp0s3. However, the route I want to use should be prioritized because the metric of 0 is lower than 100.

[fth0]

VM1 is already able to directly ping VM3. However, I would like VM1 to ping VM3 going through VM2. How can I change this behavior?

By using two different IPv4 networks.

From your description, I assume that you are using only one IPv4 network. VirtualBox provides a virtual switch, and VM1, VM2, VM3 and the host are all connected to this switch. In consequence, VM2 cannot even see the packets that are directly switched between VM1 and VM3.

[scottgus1]

Thanks, Jordan, for the info on OVS, I'm going to try it.

Your guest network setups from the .vbox files are:

<Network>
<Adapter slot="0" enabled="true" MACAddress="08002739F99B">
<NAT/>
</Adapter>
<Adapter slot="1" enabled="true" MACAddress="0800274F94FC">
<DisabledModes>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<HostOnlyInterface name="vboxnet0"/>
</Adapter>
</Network>

<Network>
<Adapter slot="0" enabled="true" MACAddress="08002783A430" type="82540EM">
<DisabledModes>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<HostOnlyInterface name="vboxnet0"/>
</Adapter>
<Adapter slot="1" enabled="true" MACAddress="080027DA233A" type="82540EM">
<DisabledModes>
<InternalNetwork name="intnet"/>
<NATNetwork name="NatNetwork"/>
</DisabledModes>
<NAT/>
</Adapter>
</Network>

Both guests are using Host-Only "vboxnet0" for between-guest comms and NAT for internet. For a complete picture we'd need the Host2.vbox file. But if it is also Host-Only "vboxnet0" and NAT, then your guests are all in the same Host-Only network.

vboxmanage list hostonlyifs prints nothing in the terminal.

Sorry about that. I just recently learned that, on either Linux or Mac hosts (I think), if no guests using a Host-Only network are running, 'vboxmanage list hostonlyifs' returns nothing. It should return good data if at least one Host-Only-attached guest is running already. Kind of a bother, really, Windows hosts return good results regardless of guest status. OK, flame war off .

As fth0 states, your RP1 guest has no second network to isolate traffic between it and the Host2 guest. You need two isolated networks to test putting traffic on one network into another network. If Host2 is also attached to Host-Only "vboxnet0" then there is already a switch on the Host-Only network attaching the PC, Host1, RP1, and Host2 in the same network and IP range. OVS inside RP1 has no extra network to jump traffic over to. FWIW if OVS inside RP1 is making a mere switch and has no routing set up, then OVS's switch duplicates Host-Only's switch.

My recommendation is to add another network card to RP1, connect it to Internal 'intnet'. Also change Host2's network to Internal 'intnet' instead of Host-Only. If you don't need internet in the guests, disable NAT in each guest, to avoid confusion. RP1 and the Host2 Internal network cards will need static IP addresses in a different range than anything else in the Virtualbox network, and ensure they can ping each other over the separate Internal network. Then you will have two separate networks, Host-Only and Internal, both connected through RP1, and ready for routing or whatever else needs to be done.

[JordanFist]

Thank you very much for the time you spend there. Finally, I did something else but that looks close to what you have explained. Now, host1 is in the network vboxnet0 and I have created a second network called vboxnet1 where host2 is and I am using Host-Only for both. Then RP1 has 2 network cards which are vboxnet0 and vboxnet1 in Host-Only as well. I prefer working with Host-Only so I can ssh my VMs but normally if I understood your explanations well the important is to have 2 different networks so I think internal network or not should be the same, could you confirm that?

Now, both host1 and RP1 and host2 and RP1 can ping each other but of course, host1 cannot ping host2. So I added eth1(vboxnet0 network) and eth2(vboxnet1 network) to OVS bridge in RP1 and added a route for host1 saying if you don't know where to send your message go through RP1. Same thing for host2. Here are what route -n returns for hosts.

Both X.X.57.4 and X.X.58.4 are eth ports of RP1.

Host1(in the network X.X.57.0)

Code: Select all

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         X.X.57.4        0.0.0.0         UG    0      0        0 enp0s3
169.X.0.0       0.0.0.0         255.255.0.0     U     1000   0        0 enp0s3
X.X.57.0        0.0.0.0         255.255.255.0   U     100    0        0 enp0s3

Host2(in the network X.X.58.0)

Code: Select all

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         X.X.58.4        0.0.0.0         UG    0      0        0 enp0s3
169.X.0.0       0.0.0.0         255.255.0.0     U     1000   0        0 enp0s3
X.X.58.0        0.0.0.0         255.255.255.0   U     100    0        0 enp0s3

Finally, in the RP1 VM when you type the following command sudo ovs-ofctl dump-flows ovsbr0 you get this default entry table.

Code: Select all

 cookie=0x0, duration=14.668s, table=0, n_packets=29, n_bytes=4301, priority=0 actions=NORMAL

which means if you get a message from a port send it to all the other ports. So I decided to remove this table with sudo ovs-ofctl del-flows ovsbr0 and to be more specific(but to be honest it is exactly the same thing). So I added some routes with these commands sudo ovs-ofctl add-flow ovsbr0 priority=500,in_port=1,actions=output:2 and sudo ovs-ofctl add-flow ovsbr0 priority=500,in_port=2,actions=output:1 and now when I check the table I get

Code: Select all

 cookie=0x0, duration=3.939s, table=0, n_packets=0, n_bytes=0, priority=500,in_port=1 actions=output:2
 cookie=0x0, duration=1.098s, table=0, n_packets=0, n_bytes=0, priority=500,in_port=2 actions=output:1

Port1 and port2 are respectively X.X.57.4 and X.X.58.4

However, when Host1 tries to ping Host2 I get host unreachable. Maybe now the problem comes from OVS, could you give me your insight?

[scottgus1]

Unfortunately I have absolutely no experience in getting two different networks to talk to each other, unless it's through a typical network router that helps me get on the internet.

I think bridging the two adapters in RP1 might work. I don't know if that setup fights Virtualbox's Host-Only paradigm anywhere.

As a quick double-check that bridging in RP1 hasn't damaged communications over the adapters thus bridged, try to ping RP1's Host-Only vboxnet0 adapter from Host1, and RP1's Host-Only vboxnet1 adapter from Host2. (check RP1's firewall allows Ping.) If these pings work, then the individual Host-Only networks are still good, and the issue is likely in OVS's commands. If the pings don't work, take off OVS's bridge so the adapters go back to two separate un-connected adapters, and try the pings again.

[fth0]

However, when Host1 tries to ping Host2 I get host unreachable. Maybe now the problem comes from OVS, could you give me your insight?

Your mistake probably is that you are trying to bridge, instead of trying to route. I'll give you a generic example to explain what's happening:

Let's assume you have a PC A (192.168.1.101/24), a PC B (192.168.1.102/24), an Internet router R (192.168.1.1/24) and a Google DNS server G (8.8.4.4).

If you ping from A to B, the IP addresses of A and B are in the same IP network. A determines the Ethernet MAC address corresponding to the IP address of B, and finally sends an Ethernet frame containing the IP packet.

If you ping from A to G, the IP addresses of A and G are in different IP networks. A determines the Ethernet MAC address corresponding to the IP address of R, and sends an Ethernet frame containing the IP packet. R receives the Ethernet frame, determines the Ethernet MAC address corresponding to the IP address of the next intermediate gateway on the way to G, and finally sends an Ethernet frame containing the IP packet. This last step is called routing. If R would bridge instead, it would send the Ethernet frame unmodified on the next network segment, and it wouldn't reach the next intermediate gateway.

Transferring this explanation to your own scenario is left as an exercise for the reader.