VM freezes under heavy load

Discussions related to using VirtualBox on Linux hosts.
Post Reply
Dexon
Posts: 4
Joined: 30. Sep 2020, 19:06

VM freezes under heavy load

Post by Dexon »

Hello Community,

I installed Virtualbox 6.0.14 on a server with Cent OS 7. The guest OS is Windows 10. I tried to run a program (MaxQuant) on Windows 10 but after some time (more than 2 hours) the Windows freezes.
I can connect with RDP on VM but the Windows are totally frozen. So, I need to restart the VM from the server. I attached the log file. I don’t know what to do. I will appreciate if someone can help me.

Host server spects:
CPUs: 2 x Xeon Gold 6148
RAM: 376GB

Thank you in advance!
Attachments
Log.zip
(71.05 KiB) Downloaded 10 times
scottgus1
Site Moderator
Posts: 20945
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: VM freezes under heavy load

Post by scottgus1 »

Dexon wrote:after some time (more than 2 hours) the Windows freezes.
The log shows a guest that appears to run fine for 56 hours, then apparently starts flatlining for 2 hours then loses connection to its primary disk "NewVirtualDisk1.vdi":
58:10:36.947785 AHCI#0: Port 1 reset
58:10:37.035539 VD#1: Cancelling all active requests
<repeat ad infinitum>
This is just a guess, the log is not more specific on what might have been the problem:

You have two disks in the guest:
00:00:03.069538 File system of '/home/ilias/VirtualBox_VMs/Windows/NewVirtualDisk1.vdi' is ext4
00:00:03.074953 File system of '/home/ilias/VirtualBox_VMs/Windows/NewVirtualDisk3.vdi' is ext4
Both on the same host disk, if I read correctly. If there is a lot of disk activity with MaxQuant, you might want to put the 'NewVirtualDisk3.vdi' on a different physical disk.
fth0
Volunteer
Posts: 5677
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VM freezes under heavy load

Post by fth0 »

Dexon wrote:I don’t know what to do.
In the VM configuration, enable Use Host I/O Cache for the SATA controller, and see if the problem persists.
scottgus1 wrote:loses connection to its primary disk "NewVirtualDisk1.vdi"
Small correction, it's the secondary disk NewVirtualDisk3.vdi, which is connected to AHCI#0 Port 1 and VD#1 (both count from 0 to 7).
scottgus1
Site Moderator
Posts: 20945
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: VM freezes under heavy load

Post by scottgus1 »

fth0 wrote:it's the secondary disk NewVirtualDisk3.vdi
Ah, yes, good ol' base 0...
Dexon
Posts: 4
Joined: 30. Sep 2020, 19:06

Re: VM freezes under heavy load

Post by Dexon »

Hi again, I want to thank you again for your answers and interest.
fth0 wrote:
Dexon wrote:I don’t know what to do.
In the VM configuration, enable Use Host I/O Cache for the SATA controller, and see if the problem persists.

I tried to run MaxQuant with the parameter "Use Host I/O Cache" enabled but the result is the same, the VM freezed.
I think that something happens with the set up of disks on server. The server has 2 disks 1) 240GB for the host os (Cent OS) and 2) 30TB HDD
VM is on 30TB HDD. I thought that 30TB are in RAID mode but I didn't detect any RAID configuration. So I think that the 30TB disk is in a storage box which performs RAID configuration and this box is connected with the server.
So with this way the server detects it as a physical HDD of 30TB. Am I wrong? What is your opinion? I attached a screenshot with the disks configuration on server.
Thank you again for your help!
Attachments
hard disk set up.png
hard disk set up.png (91.37 KiB) Viewed 1932 times
fth0
Volunteer
Posts: 5677
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VM freezes under heavy load

Post by fth0 »

Hardware RAID controllers typically hide the individual drives and show them only with special commands or command variants, so I'd assume that /dev/sdb is already the RAIDed combination of two physical disk drives.

You could provide a zipped VBox.log file for the use case with Host I/O Cache, so that we can compare it to the previous VBox.log file.
Dexon
Posts: 4
Joined: 30. Sep 2020, 19:06

Re: VM freezes under heavy load

Post by Dexon »

fth0 wrote:Hardware RAID controllers typically hide the individual drives and show them only with special commands or command variants, so I'd assume that /dev/sdb is already the RAIDed combination of two physical disk drives.
You could provide a zipped VBox.log file for the use case with Host I/O Cache, so that we can compare it to the previous VBox.log file.
Hi again,
I attached the VBox.log file with "Host I/O Cache" enebaled.
If I understand right the problem with the hard disk persists
Also I detected the below rows inside the VBox.log:

81:53:28.639896 ***
81:53:28.639897 VCPU[17] hardware virtualization state:
81:53:28.639898 fLocalForcedActions = 0x0
81:53:28.639899 No/inactive hwvirt state
81:53:28.639899 ***

Is it possible the virtualization abilities to be disabled from the BIOS?
Thank you again for your help!
Attachments
VBox with Host IO Cache enabled.rar
(63.95 KiB) Downloaded 8 times
fth0
Volunteer
Posts: 5677
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VM freezes under heavy load

Post by fth0 »

Dexon wrote:I attached the VBox.log file with "Host I/O Cache" enebaled.
No, you didn't. ;)
Dexon
Posts: 4
Joined: 30. Sep 2020, 19:06

Re: VM freezes under heavy load

Post by Dexon »

Oh, sorry I uploaded the right file.
Attachments
VBox with IO cache.zip
(70.02 KiB) Downloaded 5 times
fth0
Volunteer
Posts: 5677
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: VM freezes under heavy load

Post by fth0 »

Ok. Some further ideas:

For the next VM run, note the wall clock times when you connect/disconnect to/from the VM, and on which connection you encounter which exact notable behavior. If necessary, try to keep the host from going to sleep.

Provide less vCPUs to the VM:
Your CPU has 20 physical cores (plus hyperthreading). If you provide 20 vCPUs to the VM, and the guest uses them near 100%, then the host maybe struggles to do disk I/O or anything else in behalf of the guest in a timely manner. Find out how much the host has to do, and reduce the number of vCPUs accordingly. For example, if there Is 80% calculation and 20% disk I/O, provide only 16 vCPUs to the VM.
Post Reply