Can not start 2 VMs, why? SunRay.

[kebabbert]

I have a Solaris 11 Express (downloadable from today) server and one SunRay thin client. I have VirtualBox 3.2.10 and bridged NIC.

I start up one Windows XP on either the server or the SunRay client and that works fine. When I start another virtual machine (Linux or WinXP) on the other device it does not work. It says

"Failed to open a session for the virtual machine
Linux_CentOS_v55

Failed to open/create the internal network
'HostInterfaceNetworking-
e1000g0 - Intel PRO/1000 Gigabit Ethernet'
(VERR_PERMISSION_DENIED)

Unknown error creating VM
(VERR_PERMISSION_DENIED)

Details
Result Code: NS_ERROR_FAILURE (0x80004005)
Component: Console
Interface: IConsole {6375231a-c17c-464b-92cb-ae9e128d71c3}"




It seems that I can only have one VM running on either the server or the SunRay client. Why is that?

If I start bridged VM and then start a NAT VM it seems to work. But I can not start two bridged VMs. Why? Is this a bug? Do you want to see a core dump or something?

[jimklimov]

I am not certain if you can have several VMs bridged on one interface in Opensolaris (that is allowed in Solaris 10 though as it's the only way).

On my SXCE servers I used VNICs dedicated to the VMs, as documented in VBox user guide, and such VNICs are not even "plumbed" in the OS.
However, if you run the VMs in a local zone, they need to be passed into that zone - but also don't have to be plumbed.

And the VNIC's unique MAC address should be the same as one configured for its cetain VBox VM.

Hope this helps,
//Jim

[jimklimov]

By the way, do you run these VMs as root or as a user (2 different unprivileged users perhaps)?

See if the device file permissions or owner change when you start up a VM (/dev/net/e1000g0 or alike).

[kebabbert]

Ok, that could be the problem. Bridged NIC require a dedicated network card? If I start two VMs there will be a conflict of the NIC?

If so, one workaround could be to install a VM into a local zone, and then dedicate virtual nics, utilizing Crossbow. In that case, each zone will get a dedicated NIC. I should try this, but I noticed another problem now. VB v3.2.10 said (upon starting the second VM) it was low on memory. I have 4GB RAM. Maybe that could be the problem I earlier had; the entire computer went slow and paused. So I always rebooted after some time. Earlier I used VB 3.1.4 and did not get any error messages. With 3.2.10 I get more error messages - telling me what the problem is. This is good.

I run everything as non-root users.

[jimklimov]

Well, I wouldn't bluntly bet my head or arm that you can't have several bridged VNICs in OpenSolaris, but having dedicated VNICs separate from OS (or local zone) IP interfaces certainly does work for me. Documentation implies that the NIC in VBox VM and in the host OpenSolaris should have the same MAC addresses for everything to work properly (for me, communications between host and its guests worked even with mismatching MACs).

On the other hand, VMs running under Solaris 10 certainly have bridging which shares the the OS IP interface, and VM's MAC address is different from the physical NIC MAC. But I only have one networked VM on such a machine, so I can't say with proof whether two VMs would work or not.

As another proper disclaimer I just thought of - most of my installations are with VBox 3.0.12, maybe something has changed in 3.2.x in ways I don't know of.

[jimklimov]

Regarding the low RAM issue, did you follow generic suggestions to have on-disk swap space at least equal to RAM size, and perhaps to minimize the ZFS ARC cache max size, i.e. to 512M?

"Oracle VDI 3.2.1" is based on VBox 3.2.10 and claims in release notes to have finally not have these issues, but I'm not sure if this means that publicly available VBox 3.2.10 with any *Solaris host OS is also clear of them.

As for delegating VNICs dedicated for VMs into a local zone, here's a snippet from my zones' XML manifest (/etc/zones/ZONENAME.xml):

Code: Select all

<zone name="vboxes" zonepath="/zones/demo/vboxes" autoboot="true" ip-type="exclusive">
  <inherited-pkg-dir directory="/lib"/>
  <inherited-pkg-dir directory="/platform"/>
  <inherited-pkg-dir directory="/sbin"/>
  <inherited-pkg-dir directory="/usr"/>

  <network address=""  physical="vnic81198"/>
  <device match="/dev/net/vnic81198"/>

  <network address="" physical="vnic163020"/>
  <device match="/dev/net/vnic163020"/>
...
  <network address="" physical="vnic162020"/>
  <device match="/dev/net/vnic162020"/>

  <device match="/dev/vboxdrv"/>

  <device match="/dev/vboxnet0"/>
</zone>

Here the local zone manages its own routing and IPFilter, so it has IP-type exclusive (it is actually a static router with NAT for the VMs), and there are over a dozen Crossbow vNICs like "vnic162020" which are connected to several "etherstubs" (Crossbow virtual switches).

Overall, there are 4 etherstubs interconnecting certain interfaces of the VMs and occasionally of the zone which executes them, and the physical network accessible to the local zone (which is a NAT/router for the VMs) via "vnic81198". It is a nifty one-machine demo version of a setup we did with a blade chassis with its several servers and built-in VLAN-enabled switches.

However, *I may have been wrong to say above* that these VNICs were not plumbed. As the zone is IP-exclusive, I didn't see the "vnic16xyzz" from the global zone as I posted that. Apparently, these VNICs are plumbed but not UP, by virtue of a dozen empty /etc/hostname.vnic16xyzz files inside that zone. And files for VNICs used for the local zone's access into the etherstubs (as a router interfaces) are not empty, so these few VNICs are plumbed, UP, and have assigned virtual IP addresses.

Hope this helps
//Jim

[Ramshankar]

I am not certain if you can have several VMs bridged on one interface in Opensolaris (that is allowed in Solaris 10 though as it's the only way).

You most certainly can, multiple VMs of the same user can use the same interface.

On my SXCE servers I used VNICs dedicated to the VMs, as documented in VBox user guide, and such VNICs are not even "plumbed" in the OS.
However, if you run the VMs in a local zone, they need to be passed into that zone - but also don't have to be plumbed.

And the VNIC's unique MAC address should be the same as one configured for its cetain VBox VM.

This restriction only applies to VNICs as they form yet another abstraction MAC layer which VBox is already doing.

The problem the original poster is facing is different users trying to bridge their VMs on the same interface. This is a known security limitation as the first launched VM will create the trunk interface with appropriate flags and an explicit check is made when subsequent VMs joins a trunk connection, one of the checks being the user-id.

To work around this problem, either have the same user launch multiple VMs on the NIC or make sure each VM launch (from different users) sets the "RestrictAccess" to false. Something like:

Code: Select all

VBoxManage setextradata <vmname> VBoxInternal/Devices/pcnet/0/LUN#0/Config/RestrictAccess 0

or

Code: Select all

VBoxManage setextradata <vmname> VBoxInternal/Devices/e1000/0/LUN#0/Config/RestrictAccess 0

Depending on the guest network adapter type. Each VM must adhere to having this setting before being launched in order to circumvent the VERR_PERMISSION_DENIED error.

[jimklimov]

Wow, thanks. I don't know abut the original poster, but this clarified some things for me, certainly!
And one of my initial guesses was on-spot

[kebabbert]

Ramshankar, thanks for your help and clarification. Ive tried what you said, but it didnt work.

I first start up a bridged VM on the sunray.

Then, I have a WinXP differencing disk, based on an immutable image.Then I start up this differencing disk as a bridged image on my Solaris 11 Express server. I see this error message:

"Failed to open a session for the virtual machine
WinXP_test

Configuration error: Failed to get MAC adress
(VERR_CFGM_VALUE_NOT_FOUND).

Unknown error creating VM
(VERR_CFGM_VALUE_NOT_FOUND).

Details
Result Code: NS_ERROR_FAILURE (0x8000405)
Component: Console
Interface: IConsole {6375231a-c17c-464b-92cb-ae9e128d71c3}"


I then tried to switch to NAT and got the same error message. Earlier, I could at least start this as NAT on my Solaris server. Do you have any idea what I can try next?


EDIT: I can start another Linux NAT VM on my Solaris 11 Express server. It seems that the command I tried messed up my WinXP differencing disk virtual image?

[Ramshankar]

Run that VBoxManage command you did without the last value (i.e. leave out the "0"). This will remove the CFGM setting. I think the key I mentioned pertains to an older version of VirtualBox, the key it seems has changed but the approach is the same. I'll try find out the correct key and let you know.

And no, this does not in any way affect or touch the disk images. These are purely VM CFGM keys and you can remove them by omitting the value.

[otobyt]

I've been attempting to find a work-around to the same issue that you've been having kebabbert... working off of an Ubuntu host though. I'll look forward to your solution Ramshankar. Thanks for taking the time!

[kebabbert]

I've been attempting to find a work-around to the same issue that you've been having kebabbert... working off of an Ubuntu host though. I'll look forward to your solution Ramshankar. Thanks for taking the time!

Cool! Do you also have SunRay thin clients? If not, do you really have to use Bridged VM on two different users?

[otobyt]

I'm not running SunRay thin clients... I have a mix of Windows and Linux guests under different host users. I want them to grab an IP using my physical router's DHCP. This works fine for multiple guests run by one host user. However I get the same issue that you've run into if I try it from >1 host user. I could migrate all of my VMs to a single host user, but if there's an option not to... would just assume keep my setup how how I currently have it.

[kebabbert]

How is progress on this problem? I want to use bridged NIC on two or more VMs. Right now, only one VM can be bridged, the rest must be NAT.

Are there any plans to fix this problem? No plans at all, or later sometime, or is it already fixed?