Big Sur: VB works when SIP enabled, not when SIP disabled

Discussions related to using VirtualBox on Mac OS X hosts.
Post Reply
instalunch
Posts: 11
Joined: 17. Nov 2020, 10:59

Big Sur: VB works when SIP enabled, not when SIP disabled

Post by instalunch »

This is a weird one, so I'm hoping for this one to be a bug and not expected behaviour.

You might have noticed that in the pre-release Mac OS Hosts discussion was resolved by people discovering that on Big Sur, it seems VB now *requires* SIP to be enabled. Here's the discussion in question: (I can't seem to link , but it's the last page of the topic titled 'Big Sur Host Issues' on the pre-release Mac OS Hosts forum.)

This is an issue because in my case, I have another tool that requires SIP to be disabled, so I have to keep it disabled. However, that makes VB not work. The error is below. All the kexts are loading just fine both when SIP is enabled and disabled. By simply enabling or disabling SIP, I can switch from a working to a non-working state and vice versa. To be super clear, this is different from the 'VB doesn't work when SIP is enabled' issues I've seen on the forums. I'm seeing the exact opposite: VB *only* works when SIP is enabled. It does not work when SIP is disabled.

I have also replicated this behaviour on my iMac Pro and on my Macbook Pro as well. They both behave the same. SIP disabled means Virtualbox will fail with the exact same error below on both machines.

The SIP permission that appears to make a difference is the fs permission, so a custom SIP config where eveything else is enabled, but SIP Filesystem protection is disabled

Code: Select all

csrutil enable —without fs
will result in a non-working state. Unfortunately, that's also the exact same permission my other tool needs to have disabled, ha. :)

I'm running:
VB: Version 6.1.16 r140961 (Qt5.6.3)
Mac OS Big Sur: 11.0.1 (20B28)

This is not a beta or an otherwise developer-only release, this is the current, public version of Mac OS.

Code: Select all

The virtual machine '[redacted]' has terminated unexpectedly during startup with exit code 1 (0x1).

Result Code: 
NS_ERROR_FAILURE (0x80004005)
Component: 
MachineWrap
Interface: 
IMachine {85632c68-b5bb-4316-a900-5eb28d3413df}
I suspect this is not the expected behaviour, but I'm not sure if I should file a bug or not.
Steffen M.
Posts: 17
Joined: 12. Sep 2013, 16:56

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Steffen M. »

Hi all,
instalunch wrote:This is a weird one, so I'm hoping for this one to be a bug and not expected behaviour.
You might have noticed that in the pre-release Mac OS Hosts discussion was resolved by people discovering that on Big Sur, it seems VB now *requires* SIP to be enabled.
I just made exactly the same observation in macOS 11.0.1 (Big Sur). I need to disable SIP by doing "csrutil disable" in order to make my ASIX AX88179 USB3-/Ethernet driver work. When activating SIP, the LAN is gone. When deactivating SIP, VirtualBox 6.1.16 does not work anymore. This is very annoying. Of course, I think the main culprit is ASIX in my case as they were not able to deliver drivers fully-compatible with the new module format of Big Sur.

Nevertheless, does anybody have an idea why VirtualBox has a problem running in a SIP-disabled mode?

Kind regards,
Steffen
multiOS
Volunteer
Posts: 1743
Joined: 14. Sep 2019, 16:51
Primary OS: Mac OS X other
VBox Version: VirtualBox+Oracle ExtPack
Guest OSses: Windows, Linux, BSD
Location: United Kingdom

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by multiOS »

Isn't the behaviour exactly what Apple wants. It certainly doesn't want users disabling SIP to run software?

You could log the issue in the VirtualBox 'Public Bugtracker" but my guess is that you might well get the response that it's "by design", to conform to Apple's security requirements/expectations of developers.
Steffen M.
Posts: 17
Joined: 12. Sep 2013, 16:56

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Steffen M. »

multiOS wrote:Isn't the behaviour exactly what Apple wants. It certainly doesn't want users disabling SIP to run software?

You could log the issue in the VirtualBox 'Public Bugtracker" but my guess is that you might well get the response that it's "by design", to conform to Apple's security requirements/expectations of developers.
Interestingly, it only seems to affect VirtualBox. All other pieces of software I have on my MacBook seem to run well even with SIP being disabled...
mhwill55
Posts: 5
Joined: 21. Nov 2020, 15:45

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by mhwill55 »

I concur with Steffen M. VirtualBox is the only VM software that does not work with SIP disabled. With SIP disabled, Parallels and VMware Fusion work just fine.
instalunch
Posts: 11
Joined: 17. Nov 2020, 10:59

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by instalunch »

Yes, VirtualBox is not only the only VM software that does not work with SIP disabled in Big Sur, it's the *only software*, ever, that does not work with SIP disabled. Hence my belief (and hope!) that this is not intentional, and not by design. Mind that before Big Sur, VB used to work just fine with SIP disabled.

Disabling SIP is not like jailbreaking an iPhone, a Mac with SIP disabled is still a fully supported configuration of the OS by Apple. In fact, the tool to disable SIP is a part of Mac OS itself, and it tells you what the supported configurations are: fully enabled and fully disabled. The tool in question is also capable of generating 'inbetween' configurations like partial enables of certain components of SIP, but if you opt for it, the tool explicitly tells you that this is not a supported configuration, and likely to cause issues. However, SIP being fully disabled *is* a supported configuration, and it does not generate the 'unsupported' warning.

My best guess here is that disabling SIP removes a few APIs that Virtualbox uses to prompt the user to approve or enable certain kexts (with SIP disabled state needing no approval), and since the Mac installer relies on those APIs being present, it fails to fully enable or use certain kexts that way. That said, all the kexts that VB needs appear to be present in my system and loaded, so I wouldn't know.
Steffen M.
Posts: 17
Joined: 12. Sep 2013, 16:56

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Steffen M. »

instalunch wrote:Yes, VirtualBox is not only the only VM software that does not work with SIP disabled in Big Sur, it's the *only software*, ever, that does not work with SIP disabled. Hence my belief (and hope!) that this is not intentional, and not by design. Mind that before Big Sur, VB used to work just fine with SIP disabled.

Disabling SIP is not like jailbreaking an iPhone, a Mac with SIP disabled is still a fully supported configuration of the OS by Apple. In fact, the tool to disable SIP is a part of Mac OS itself, and it tells you what the supported configurations are: fully enabled and fully disabled. The tool in question is also capable of generating 'inbetween' configurations like partial enables of certain components of SIP, but if you opt for it, the tool explicitly tells you that this is not a supported configuration, and likely to cause issues. However, SIP being fully disabled *is* a supported configuration, and it does not generate the 'unsupported' warning.
I fully agree with you!
instalunch wrote: My best guess here is that disabling SIP removes a few APIs that Virtualbox uses to prompt the user to approve or enable certain kexts (with SIP disabled state needing no approval), and since the Mac installer relies on those APIs being present, it fails to fully enable or use certain kexts that way. That said, all the kexts that VB needs appear to be present in my system and loaded, so I wouldn't know.
This sounds very well possible. Interestingly, it seems to be related to the display function of a virtual machine. Starting the VM in headless mode works even with having SIP disabled, but the "Show" function does not work, though. Do you know whether there is a bug report in VirtualBox' Trac, yet? I didn't find one when searching yesterday evening.

Kind regards,
Steffen
instalunch
Posts: 11
Joined: 17. Nov 2020, 10:59

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by instalunch »

Yes, I have also noticed that I am able to use VB in headless mode — my Docker setup on Mac does in fact use VB as its virtualisation backend and it works fine. So at least Docker can spawn its own VB container just fine. However, I don't seem to be able to spawn anything, even in headless mode, through the Virtualbox GUI, nor through the VBoxManage terminal command. I'm not sure how my Docker is able to spawn a headless VM and I cannot.

No bug report yet, I'm not sure how to file one, I don't even know where the bug tracker is. Any chance you could, if you know how to?
Ee
Posts: 9
Joined: 18. Apr 2019, 18:20

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Ee »

Ticket here: https://www.virtualbox.org/ticket/20052

Oracle/VirtualBox, please treat this as a bug and not a feature.
peterbr
Posts: 5
Joined: 24. Feb 2011, 21:02
Primary OS: Mac OS X Leopard
VBox Version: OSE other
Guest OSses: windows7, windows xp

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by peterbr »

I have SIP enabled, no network connection possible either bridged or NAT, both host wired and wireless connections tried.

VB crash happens ( critical error has occurred while running virtual machine and the machine execution has stopped) when using DHCP in windows virtual client and NAT option set. Using a fixed set IP address in VB windows client stops the crash from happening but still no network access from windows client.

I have not seen any acknowledgement from the developers on the issue.
Same VB setup works flawless under catalina.
aeichner
Oracle Corporation
Posts: 205
Joined: 31. Aug 2007, 19:12

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by aeichner »

Everyone having trouble running VirtualBox with SIP disabled please try the latest 6.1 testbuild from https://www.virtualbox.org/wiki/Testbuilds. It contains fixes for this configuration.
Ee
Posts: 9
Joined: 18. Apr 2019, 18:20

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Ee »

I tried the test build on SIP disabled Big Sur. VirtualBox appears to be working!

Thank you, Oracle/VirtualBox!
instalunch
Posts: 11
Joined: 17. Nov 2020, 10:59

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by instalunch »

The test build is confirmed working on my side. Thank you!
Steffen M.
Posts: 17
Joined: 12. Sep 2013, 16:56

Re: Big Sur: VB works when SIP enabled, not when SIP disabled

Post by Steffen M. »

I can fully confirm that the test build solves the problem.
Post Reply