No Ethernet Ubuntu/Windows guest VM's after computer wake from sleep

[fth0]

It took me awhile, but here is some detailed feedback to several aspects of the issue:

Set computer in sleep mode

How do you do that, and how is it configured? I'm asking because some users call all the different power saving modes (e.g. sleep, hibernate, hybrid sleep) sleep mode, and the differences could be important when trying to reproduce the issue.

The problem looks like located on Windows 10 host and not in the guest.

Agreed.

Are there any services on Windows 10 host which can be restarted or investigated?

Please reproduce the behavior and provide the (zipped) contents of the C:\Users\<username>\.VirtualBox folder. Especially of interest are the VBoxSVC.log files. The network filter driver for the bridged networking mode is named VBoxNetLwf.sys, and you could search the Windows Event Viewer (on the Windows host) for that.

I noticed that you provided an event log entry about the Intel(R) PROSet Monitoring Service not shutting down properly. Perhaps you can prevent it from starting at all for a test.

No Ethernet in/out trafic captured with Wireshark between host and VM.

When you wrote that, did you capture in the guest, in VirtualBox (--nictrace option) or on the host?

All Ethernet frames start with an unusual structured Ethernet header (two bytes inserted before the EtherType?).

Although I'm using Wireshark almost weekly, it's been a long time since I've captured with the "any" loopback interface, so I didn't recognize the Linux cooked-mode capture header right away.

Please find attached the Wireshark PCAP file in the VM

Thanks for the capture file. Although it doesn't contain the ARP packets, the ICMP Destination Host Unreachable packets made me realize that the ARP requests have not been answered. This means that either the ARP requests aren't sent or the ARP replies aren't received.

How far those packets get, can be investigated in at least 5 positions: The 3 Wireshark capture positions (indicated in one of my questions above), and two statistics counting positions. If you are a little adventurous, you could start your VM with VirtualBoxVM --startvm "VM name" --debug, unfold the /Devices/e1000#0 and /Drivers/IntNet-0 statistics in the right pane, unpause the VM in the VM window's Machine menu, reproduce the problem, and watch the statistic counters when trying to ping from within the guest OS. I don't know if the VirtualBox debug mode survives the host sleep, though.

[bertje]

Thanks for your feedback. Please find my answers below:

How do you do that, and how is it configured?

On the Windows computer, click Start (icon lower left corner) | Power | Sleep. I did not try other sleep modes.
The power management option such as power saving, normal, maximum performance does not make sense.

Today I've enabled option "c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" modifyvm "Windows 10" --nictrace1 on --nictracefile1 file.pcap (which is new for me), but see too many confidential Ethernet traffic, so can't share this file.

Please reproduce the behavior and provide the (zipped) contents of the C:\Users\<username>\.VirtualBox folder. Especially of interest are the VBoxSVC.log files.

I've included this in the attachment. I cleaned up the logs before running the test case.

Although it doesn't contain the ARP packets, the ICMP Destination Host Unreachable packets made me realize that the ARP requests have not been answered.

Correct.

Although I'm using Wireshark almost weekly, it's been a long time since I've captured with the "any" loopback interface, so I didn't recognize the Linux cooked-mode capture header right away.

Good to hear that you understand it.

Windows Event Viewer

I've cleaned the event log before the test case and did not see any VirtualBox related errors/warnings.

I've included screenshots of the error counters as you requested with VirtualBoxVM --startvm "VM name" --debug (also a new command for me, but useful). The red marked counters does not increment after computer wake up en seems to match with this Ethernet behavior.

Can you try to reproduce this problem?

[bertje]

Can someone confirm this bug?

[fth0]

Sorry for taking so long. Here's my current analysis:

Can you try to reproduce this problem?

Last weekend, I tried to reproduce the issue, but without success. On my Windows 10 host, everything worked as expected. But thanks to your log files, I could at least spot a significant difference:

00:00:00.198897          HostDnsMonitor: updating information
00:00:00.198914          HostDnsMonitor: old information
00:00:00.198914            no server entries
00:00:00.198916          HostDnsMonitor: new information
00:00:00.198916            server 1: 111.111.111.1

00:00:06.780492          Launched VM: 68236608 pid: 2028 (0x7ec) frontend: GUI/Qt name: Windows 10 Clone

00:00:38.158611 MainPower Host suspending: Paused 1 VMs

00:01:01.915492 MainPower Host resumed: Resumed 1 VMs

00:01:03.765380 dns-monitor HostDnsMonitor: updating information
00:01:03.765418 dns-monitor HostDnsMonitor: old information
00:01:03.765421 dns-monitor   server 1: 111.111.111.1
00:01:03.765430 dns-monitor HostDnsMonitor: new information
00:01:03.765433 dns-monitor   no server entries

Note when the changes of the DNS server happen: As expected, the DNS server is enabled right in the beginning. But then it is only disabled after the "Host resumed" message. In my working setup, it is disabled ~2 seconds before the "Host resumed" message and re-enabled ~2 seconds afterwards (all three events take place while the host is resumed). So we have at least some evidence of the problem, but I'm not sure how to go from there ...

Regarding the statistic counters and the packet captures: Where did you see the ARP requests after the host resumption (in the guest, in VirtualBox, or nowhere)? I noticed that /Device/e1000#0/uStatTxFrm has the same value as /Drivers/IntNet-0/Packets/Sent. I'll try to check if the former indicates that the guest OS isn't even trying to send the ARP requests, or if the equality is the result of sent packets only being counted after their confirmation via Tx-done interrupts.

What's also puzzling is that you experience the behavior on several devices, whereas it is rarely reported by other users. Do you have any idea what could be different in the host setup? Perhaps something influencing the timing of the host suspend/resume?

[arQon]

I've reported a new defect https://www.virtualbox.org/ticket/20922 to find a solution.

I'm not sure I'd call a 7-year-old bug "new"... https://www.virtualbox.org/ticket/14374

I investigated it at the time, but apparently discarded my notes after abandoning it and switching to NAT instead a few months later.
I do remember that the bug is specific to Windows hosts, and I have a fairly strong feeling I had a Filthy Hack (tm) that worked around it by disabling and re-enabling the adaptor in the VM (i.e. a pair of vboxmanage commands).

Yeah, I still have it lying around. Give this a shot:

Code: Select all

rem  see https://www.virtualbox.org/ticket/14374 as well as my thread on the forums

rem  since 5.xx, virtualbox has had a bug where guests using bridged networking
rem  permanently lose connectivity when the host sleeps.
rem  this hopefully restores it by reinitialising the guest NIC's "physical" connection.

VBoxManage controlvm <vm> setlinkstate1 off
timeout /T 1
VBoxManage controlvm <vm> setlinkstate1 on

I have a feeling it would occasionally need a couple of tries to work, but generally worked first time. No idea if it still does though: the date on the file is 2019, and I'm sure it's only that recent because I updated it for a VM rename then.

[bertje]

Thanks for your details.

Do you have any idea what could be different in the host setup?

All computers are Ryzen, not tested on Intel. On some computers VMWare is also installed, but Hyper-V is disabled on all hosts. All guests are bridged and not tested with NAT.
I assume there is no dependency on host DNS configuration. Can you test with a fixed DNS configuration on Windows host which cannot be reached on the network? This may influence the timing.

Another suggestion is to sleep/wake multiple times. Sometimes network in the guest works after a single sleep/wake.

It looks like the same as manually performing a link-up/link-down which does not work to recover guest network. However, I give it a try:

Code: Select all

VBoxManage controlvm <vm> setlinkstate1 off
timeout /T 1
VBoxManage controlvm <vm> setlinkstate1 on