TCP MSS unexpected increase by VirtualBox

[Johan Mazel]

The host OS is a Debian Testing 64 with 32Gb of RAM. I am using the VirtualBox provider in Vagrant. My VirtualBox version is 6.1.14_Debian r140239.
The guest OS is a Debian Buster 64 with 512Mb of RAM. I am using the Guest Additions on the guest. Its version is 6.1.6.

According to my tests, the MSS set in the SYN packet of the TCP handshake that leaves the host is the host's MTU, and not the guest's MTU. This happens even if the latter is set lower than the host MTU.

I run the following tests:
1) Guest and host default MTU

• * Settings:
  • - Guest eth0 MTU: 1500
    - Host eth0 MTU: 1500
  * Network traffic capture results:
  • - Guest eth0 SYN MSS: 1460
    - Host eth0 SYN MSS: 1460

2) Guest and host modified MTU

• * Settings:
  • - Guest eth0 MTU: 1400
    - Host eth0 MTU: 1400
  * Network traffic capture results:
  • - Guest eth0 SYN MSS: 1360
    - Host eth0 SYN MSS: 1360

3) Guest modified MTU and host default MTU

• * Settings:
  • - Guest eth0 MTU: 1400
    - Host eth0 MTU: 1500
  * Network traffic capture results:
  • - Guest eth0 SYN MSS: 1360
    - Host eth0 SYN MSS: 1460

4) Guest default MTU and host modified MTU

• * Settings:
  • - Guest eth0 MTU: 1500
    - Host eth0 MTU: 1400
  * Network traffic capture results:
  • - Guest eth0 SYN MSS: 1460
    - Host eth0 SYN MSS: 1360

For both tests 1 and 2, MTU and MSS are the same and there is no ambiguity to solve. So the observed behavior is the expected one.
The MSS clamping behavior for test 4 is expected because the host cannot provide the MSS that the guest expects.
I however think that the behavior of test 3 is incorrect because it increases the packet size received by the guest which may cause problems on the guest itself or in a virtualized network.

I already posted a bug report on the tracker about this problem but I read on this bug report that it is better to post on the forums.
This problem seems to be related to an existing bug report.
I am not using URLs here because I am not allowed to but I will try to edit my post later.

I can provide the Vagrantfile and/or the pcap files if needed.

[fth0]

Did you fully understand vushakov's comment in 15256? If not, you can ask questions here, and I'll try to explain.

What happens when you download something from within the guest (in the guest MTU < host MTU case, of course)? You can provide pcap.gz files, and if they exceed the 128 kB limit of forum attachments, the first few thousand packets will probably be sufficient.

[Johan Mazel]

My understanding of vushavok's comment is the following.
First, he is saying that:
* MSS is not negotiated but announced during handshake by both parties.
* Announced MSS may be based on MTU but a much lower MSS is also valid.

Second, he is explaining details about VirtualBox "automagic socks-like proxy" behavior.

If I understand correctly, in my case, VirtualBox cannot retrieve the custom MSS that the guest send in the SYN because of the networking API of the Linux Kernel. VirtualBox thus cannot propagate any customized/specific MSS value that the guest may be using.
Is this understanding correct?

I attached the PCAPs just in case.

[fth0]

First, he is saying that:
* MSS is not negotiated but announced during handshake by both parties.
* Announced MSS may be based on MTU but a much lower MSS is also valid.

Yes, that's correct. Routers between the two TCP endpoints may also reduce the MSS (e.g. VPN gateways), which is called MSS clamping. But VirtualBox doesn't act as a router.

Second, he is explaining details about VirtualBox "automagic socks-like proxy" behavior.

If I understand correctly, in my case, VirtualBox cannot retrieve the custom MSS that the guest send in the SYN because of the networking API of the Linux Kernel. VirtualBox thus cannot propagate any customized/specific MSS value that the guest may be using.
Is this understanding correct?

No, but similar. Their argumentation was as follows: The MSS value in the TCP SYN from the guest to the host could be used when creating the TCP connection between the host and the peer, but the MSS value in the TCP SYN/ACK coming back from the peer could not be determined and sent to the guest. Therefore, no MSS value is exchanged between the two TCP connections (guest <-> host, host <-> peer).

But they also expected the TCP data received from the peer to be sent to the guest in TCP data packets not exceeding the guest's MSS. Your PCAP files however show that this doesn't work as expected: The TCP data packets seem to be using an MSS of 1420 instead of 1360. So you seem to have the PCAP files that they wanted to see 5 years ago. I'd suggest to add them to your own ticket, and put a comment to the older ticket.

[Johan Mazel]

No, but similar. Their argumentation was as follows: The MSS value in the TCP SYN from the guest to the host could be used when creating the TCP connection between the host and the peer, but the MSS value in the TCP SYN/ACK coming back from the peer could not be determined and sent to the guest. Therefore, no MSS value is exchanged between the two TCP connections (guest <-> host, host <-> peer).

I have a related question.
Is the VirtualBox "automagic socks-like proxy" directly parsing packets from the guests (and can thus access TCP options such as MSS) and use Linux sockets to communicate with the peer (and cannot access the MSS option in SYN/ACK)?

But they also expected the TCP data received from the peer to be sent to the guest in TCP data packets not exceeding the guest's MSS. Your PCAP files however show that this doesn't work as expected: The TCP data packets seem to be using an MSS of 1420 instead of 1360. So you seem to have the PCAP files that they wanted to see 5 years ago.

Well, if the host send a SYN packet to the peer with an MSS option value of 1460 instead of 1360, how can they expect to receive packets consistent with an MSS of 1360?

I'd suggest to add them to your own ticket, and put a comment to the older ticket.

I will do this.

Thank you very much for your time and your explanations.

[fth0]

Is the VirtualBox "automagic socks-like proxy" directly parsing packets from the guests (and can thus access TCP options such as MSS) [...]

Yes, I definitely think so. This will get more obvious with my answer to the third question below.

[...] and use Linux sockets to communicate with the peer (and cannot access the MSS option in SYN/ACK)?

Yes.

Well, if the host send a SYN packet to the peer with an MSS option value of 1460 instead of 1360, how can they expect to receive packets consistent with an MSS of 1360?

They don't, and they don't need to. You've been missing the fact that not whole TCP packets are transferred from one TCP connection to the other, but only the TCP data. So the host creates the TCP packets it sends to the guest itself, and therefore could take the guest's MSS into account appropriately. Take a look at your PCAP files (compare both) to see for yourself.

PS: I haven't checked any of this in the VirtualBox source.

[Johan Mazel]

Well, if the host send a SYN packet to the peer with an MSS option value of 1460 instead of 1360, how can they expect to receive packets consistent with an MSS of 1360?

They don't, and they don't need to. You've been missing the fact that not whole TCP packets are transferred from one TCP connection to the other, but only the TCP data. So the host creates the TCP packets it sends to the guest itself, and therefore could take the guest's MSS into account appropriately. Take a look at your PCAP files (compare both) to see for yourself.

Oh, I see.

Can you confirm the text below?
The 6th packet in my host_eth0_an.dump has a size of 1514 bytes. This is consistent with a MSS of 1460 (=1514-(14+20+20) for Ethernet, IP and TCP headers) as seen in the SYN packet.
But VirtualBox actually send chunks of that original payload to the guest. Here, VirtualBox sends a 1474 byte packet to the guest which corresponds to a 1420 MSS. And then, VirtualBox, sends other packets whose size is at most 1474 bytes.
So the actual question should be: why is the packet sent to the guest consistent with a MSS of 1420 instead of the specified 1360?

[fth0]

Can you confirm the text below?
The 6th packet in my host_eth0_an.dump has a size of 1514 bytes. This is consistent with a MSS of 1460 (=1514-(14+20+20) for Ethernet, IP and TCP headers) as seen in the SYN packet.
But VirtualBox actually send chunks of that original payload to the guest. Here, VirtualBox sends a 1474 byte packet to the guest which corresponds to a 1420 MSS. And then, VirtualBox, sends other packets whose size is at most 1474 bytes.
So the actual question should be: why is the packet sent to the guest consistent with a MSS of 1420 instead of the specified 1360?

Yes, that's your incarnation of the bug. And since you have caught up with my understanding of what's happening, you can now reread:

But they also expected the TCP data received from the peer to be sent to the guest in TCP data packets not exceeding the guest's MSS. Your PCAP files however show that this doesn't work as expected: The TCP data packets seem to be using an MSS of 1420 instead of 1360. So you seem to have the PCAP files that they wanted to see 5 years ago. I'd suggest to add them to your own ticket, and put a comment to the older ticket.

[Johan Mazel]

Thank you very much for your time and your detailed explanations.

Bonus question: do you have any information for communication between guest and host for UDP? Is it only based on MTU? Is MSS reused for UDP?

[fth0]

There are fundamental differences between TCP and UDP: TCP is a stream-oriented protocol, often used to transfer large amounts of data, of which the next MSS bytes are put into the next TCP data packet. UDP is a message-oriented protocol, where the data is often packetized from the beginning, for example 20 bytes of voice data, and UDP packets are therefore rarely large. There is no equivalent to the TCP MSS.