Has anyone gotten vboxvnic_template0 to work?

[martyscholes]

I just upgraded to Solaris 11 and am grinding through all of the issues. So far I have Sun Ray server working.

My next step is to start validating the VirtualBox installations. I didn't move over all of my vnic definitions because I was suffering from vnic sprawl, where vnics were getting defined all over creation for various virtual machines and I wasn't sure if they were needed anymore. Instead, I was ever-so-hopeful that the new vnic templates would save me from vnic sprawl by having VB soft-provision the vnics on demand.

The documentation is unclear on how to set the guest, so I set my first guest to "bridged" against vboxvnic_template0. When it starts, I get the error: VERR_INTNET_FLT_VNIC_CREATE_FAILED. I am going to guess that VB lacks permission to provision vnics on the fly.

How do I fix this, or am I just misunderstanding the purpose of templates?

Many thanks,
Marty

[Ramshankar]

Can you give me /var/adm/messages and VBox.log and the output of "dladm show-link" "dladm show-phys" and "ifconfig -a" and "modinfo | grep vbox" ?

Please note VNIC templates are mainly useful only if you're using VLANs. If you want to just use a VNIC or a physical interface as a bridged interface pass the VNIC or physical interface directly.

[martyscholes]

Thanks so much for the reply. Output from /var/adm/messages:

Jan 20 11:15:02 v40z vboxdrv: [ID 585527 kern.notice] vboxbow:vboxNetFltSolarisCreateVNIC failed to create VNIC 'vboxvnic0' over 'aggr1000' rc=2 Diag=0
Jan 20 11:15:02 v40z vboxdrv: [ID 854045 kern.notice] vboxbow:vboxNetFltPortOsConnectInterface failed to create VNIC rc=-3605

Output from VBox.log:

00:00:03.022 IntNet#0: szNetwork={HostInterfaceNetworking-vboxvnic_template0 - VirtualBox Virtual Network Interface Template} enmTrunkType=3 szTrunk={vboxvnic_template0} fFlags=0x8000 cbRecv=325632 cbSend=196608 fIgnoreConnectFailure=false
00:00:03.025 VMSetError: /home/vbox/tinderbox/4.1-sol-rel/src/VBox/Devices/Network/DrvIntNet.cpp(1694) int drvR3IntNetConstruct(PDMDRVINS*, CFGMNODE*, uint32_t); rc=VERR_INTNET_FLT_VNIC_CREATE_FAILED
00:00:03.025 VMSetError: Failed to open/create the internal network 'HostInterfaceNetworking-vboxvnic_template0 - VirtualBox Virtual Network Interface Template'
00:00:03.026 VMSetError: /home/vbox/tinderbox/4.1-sol-rel/src/VBox/Devices/Network/DevE1000.cpp(6025) int e1kConstruct(PDMDEVINS*, int, CFGMNODE*); rc=VERR_INTNET_FLT_VNIC_CREATE_FAILED
00:00:03.026 VMSetError: Failed to attach the network LUN
00:00:03.026 PDM: Failed to construct 'e1000'/0! VERR_INTNET_FLT_VNIC_CREATE_FAILED (-3605) - Failed to create a virtual network interface instance.
00:00:03.054 ERROR [COM]: aRC=NS_ERROR_FAILURE (0x80004005) aIID={1968b7d3-e3bf-4ceb-99e0-cb7c913317bb} aComponent={Console} aText={Failed to open/create the internal network 'HostInterfaceNetworking-vboxvnic_template0 - VirtualBox Virtual Network Interface Template' (VERR_INTNET_FLT_VNIC_CREATE_FAILED).

bash-4.1$ dladm show-link
LINK CLASS MTU STATE OVER
net1 phys 1500 up --
net0 phys 1500 up --
aggr0 aggr 1500 up net1 net0
global0 vnic 1500 up aggr0
vboxnet0 phys 1500 up --
vboxvnic_template0 vnic 1500 up aggr0
bcus0 vnic 1500 up aggr0
bugzilla0 vnic 1500 up aggr0

bash-4.1$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet up 1000 full bge1
net0 Ethernet up 1000 full bge0
vboxnet0 Ethernet up 1000 full vboxnet0

bash-4.1$ ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
global0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.223.8 netmask ffffff00 broadcast 192.168.223.255
vboxnet0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 3
inet 192.168.56.1 netmask ffffff00 broadcast 192.168.56.255
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
inet6 ::1/128
global0: flags=20002004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 2
inet6 fe80::8fea5:3750/10

bash-4.1$ modinfo | grep vbox
283 fffffffff7fbafa0 2e3b0 306 1 vboxdrv (VirtualBox HostDrv 4.1.8r75467)
284 fffffffff7f0e548 d48 307 1 vboxnet (VirtualBox NetAdp 4.1.8r75467)
286 fffffffff816b000 3be8 308 1 vboxbow (VirtualBox NetBow 4.1.8r75467)
287 fffffffff7dda000 4a18 309 1 vboxusbmon (VirtualBox USBMon 4.1.8r75467)
288 fffffffff7d96000 75f8 310 1 vboxusb (VirtualBox USB 4.1.8r75467)

Thanks again. If you need more, let me know.

[Ramshankar]

Code: Select all

vboxbow:vboxNetFltSolarisCreateVNIC failed to create VNIC 'vboxvnic0' over 'aggr1000' rc=2 Diag=0

https://www.virtualbox.org/manual/ch14. ... wnProblems

Crossbow based bridged networking on Solaris 11 hosts does not work directly with aggregate links. However, you can manually create a VNIC (using dladm) over the aggregate link and use that with a VM. This technical limitation between VirtualBox and Solaris will be addressed in a future release.

[martyscholes]

Ramshankar, thanks for the reply.

Crossbow based bridged networking on Solaris 11 hosts does not work directly with aggregate links. However, you can manually create a VNIC (using dladm) over the aggregate link and use that with a VM. This technical limitation between VirtualBox and Solaris will be addressed in a future release.

That's interesting because on build 128a I was hitting the aggregate link directly from the guests, but I don't think I was using the Crossbow features of VB. Using Solaris 11, the vnic template is over the aggregate link. Also interesting is that I cannot seem to create a vnic over a vnic. What I have now is:

THIS WORKS
physical links => aggregate link => vnic bridged by VB

THESE FAIL
physical links => aggregate link => vnic template
physical links => aggregate link => vnic => vnic template

I can wedge a vlan link in there, but I can't set the tag to the default, which forces me actually to set up a vlan.

This isn't a big deal for me; I just won't use the templates.

Thanks again,
Marty

[Ramshankar]

Ramshankar, thanks for the reply.

Crossbow based bridged networking on Solaris 11 hosts does not work directly with aggregate links. However, you can manually create a VNIC (using dladm) over the aggregate link and use that with a VM. This technical limitation between VirtualBox and Solaris will be addressed in a future release.

That's interesting because on build 128a I was hitting the aggregate link directly from the guests, but I don't think I was using the Crossbow features of VB. Using Solaris 11, the vnic template is over the aggregate link. Also interesting is that I cannot seem to create a vnic over a vnic. What I have now is:

Build 128a doesn't have the required Crossbow kernel API that VirtualBox uses, so it'd be using the old STREAMS based network filter.

THIS WORKS
physical links => aggregate link => vnic bridged by VB

THESE FAIL
physical links => aggregate link => vnic template
physical links => aggregate link => vnic => vnic template

Yes, this is expected because the problem is that VirtualBox is unable to create a VNIC over the aggregate interface by itself (automatically). The VNIC template is just a VirtualBox concept. Even if you create a VNIC template over an aggregate interface & assign the template to a VM, VirtualBox will copy the properties of the template and try to create a VNIC over the aggregate interface (on which the template is made). This again will fail similar to how directly passing the aggregate interface fails. Both are doing the same thing from VirtualBox's point of view. This is infact a bug/limitation in the Solaris Crossbow kernel API which we'll try to address in some future Solaris 11 update/release.

[martyscholes]

Yes, this is expected because the problem is that VirtualBox is unable to create a VNIC over the aggregate interface by itself (automatically). The VNIC template is just a VirtualBox concept. Even if you create a VNIC template over an aggregate interface & assign the template to a VM, VirtualBox will copy the properties of the template and try to create a VNIC over the aggregate interface (on which the template is made). This again will fail similar to how directly passing the aggregate interface fails. Both are doing the same thing from VirtualBox's point of view. This is infact a bug/limitation in the Solaris Crossbow kernel API which we'll try to address in some future Solaris 11 update/release.

Thanks for the insight.

[martyscholes]

Ramshankar, I have had some more fallout from this issue. Curiously, HP Officejet software for Windows 7 will not install unless the guest is on the same LAN as the printer, i.e. no NAT. If the software was previously installed on a bridged OS image and the image is later switched to NAT (which is what I did), the software causes intermittent network connectivity drops in the guest.

Strange.

I am curious about the kernel API limitations. I browsed the code a little but couldn't find where it came off the rails or any discussion on the topic. Is there some discussion I could look through?

Again, just curious.

Thanks again,
Marty

[Ramshankar]

Ramshankar, I have had some more fallout from this issue. Curiously, HP Officejet software for Windows 7 will not install unless the guest is on the same LAN as the printer, i.e. no NAT. If the software was previously installed on a bridged OS image and the image is later switched to NAT (which is what I did), the software causes intermittent network connectivity drops in the guest.

If you're having issues with NAT, I suggest you open a new bug ticket in virtualbox.org with the appropriate details (VBox.log, setup etc.) This will have to be investigated separately.

Strange.

I am curious about the kernel API limitations. I browsed the code a little but couldn't find where it came off the rails or any discussion on the topic. Is there some discussion I could look through?

Again, just curious.

Thanks again,
Marty

Not sure why this is a problem for you. You can workaround this API bug by creating a VNIC manually (using dladm) over aggregate and passing that VNIC to VirtualBox (not VNIC template over aggregate). In any case, the API issue is this: the call to vnic_create (http://www.virtualbox.org/svn/vbox/trun ... -solaris.c in function vboxNetFltSolarisCreateVNIC) will fail with EINVAL for aggregate interfaces. The code for vnic_create in the Solaris 11 kernel is not available publicly.

[martyscholes]

Thanks for the reply.

f you're having issues with NAT, I suggest you open a new bug ticket in virtualbox.org with the appropriate details (VBox.log, setup etc.) This will have to be investigated separately.

The real issue is that the printer driver from HP requires that the machine be on the same subnet as the printer, which is not true with NAT. I have reached out to HP and don't really expect a response. The reason I mentioned it here is because this is a real world scenario where NAT does not work well.

Not sure why this is a problem for you. You can workaround this API bug by creating a VNIC manually (using dladm) over aggregate and passing that VNIC to VirtualBox (not VNIC template over aggregate).

I agree, and the point is that it's manual. The conversation goes something like this:

User: Hey, I tried to install the printer software install on one of my Windows sessions, but it keeps failing
Me: Oh yeah, I need to:

• provision a VNIC for you
  log into your Unix session
  shut down the Windows instance
  change the networking configuration to bridged for the VNIC
  set the MAC address to the new VNIC
  restart your windows session
  put your VNIC into the list of VNICs and associate with your Windows session
  periodically circle back and validate that you still use the VNIC

User: That seems like a lot of work. I just want to print this report. Why is this so hard?
Me: I don't know

[Ramshankar]

User: Hey, I tried to install the printer software install on one of my Windows sessions, but it keeps failing
Me: Oh yeah, I need to:

• provision a VNIC for you
  log into your Unix session
  shut down the Windows instance
  change the networking configuration to bridged for the VNIC
  set the MAC address to the new VNIC
  restart your windows session
  put your VNIC into the list of VNICs and associate with your Windows session
  periodically circle back and validate that you still use the VNIC

User: That seems like a lot of work. I just want to print this report. Why is this so hard?
Me: I don't know

OR
provision a VNIC permanently (skip using "-t" in dladm)
change network config from NAT to bridged using the VNIC without shutting down the guest

Changing the network config. from NAT to bridged does not require shutting down the guest. VirtualBox has been long capable of dynamically switching the network config. backend without powering off the VM.

I agree with you, it's definitely not as nice as automatically assigning an interface and it just works, but unfortunately this is not something I can fix because I don't have write access to the source, the fault lies in Solaris 11 and not VirtualBox. If it was in VirtualBox's code I can fix it. As for why NAT doesn't work in your setup is, at this point, anybody's guess. Since I'm not the NAT expert here I requested you to create a new report which will hopefully be looked into by one of our NAT experts.

[martyscholes]

OR
provision a VNIC permanently (skip using "-t" in dladm)

That was the plan all along and what I am doing.

Changing the network config. from NAT to bridged does not require shutting down the guest. VirtualBox has been long capable of dynamically switching the network config. backend without powering off the VM.

Did not know that. Thanks for the insight.

unfortunately this is not something I can fix because I don't have write access to the source, the fault lies in Solaris 11 and not VirtualBox

Understood and apologies for the rant. I have to wonder how dladm can make it work but VB can't. That seems strange to me.

As for why NAT doesn't work in your setup is, at this point, anybody's guess. Since I'm not the NAT expert here I requested you to create a new report which will hopefully be looked into by one of our NAT experts.

I'd rather not waste their time. The very definition of NAT is that the subnets are different. The HP printer software we use has a requirement that the printer and computer are on the same subnet. I think it a silly requirement of HP, but I am sure HP thinks that it is a sane requirement.

As always, thanks for all you do.

[Ramshankar]

Understood and apologies for the rant. I have to wonder how dladm can make it work but VB can't. That seems strange to me.

No need for apologies, I'm glad to clear things up as far as possible.

Regarding why it works with dladm and not using VirtualBox:
dladm uses userland calls into the kernel which then checks for appropriate privileges before creating & managing VNICs and this path is not usable (not feasible) from the kernel itself, as strange as it seems. With VirtualBox this would be mean we'd have to run VirtualBox with hightened privileges always or use ugly privilege escalating wrappers or impose more manual input from the user. All these alternatives range from impossible, inconvenient, time consuming to annoying potential security holes. We thus proposed and got delivered a kernel API for managing VNICs which VirtualBox uses. I believe, at this point, we're the only consumers of this small API. As a result, users no longer need to worry about creating VNICs, changing MAC address for them etc., we do it all automatically in the kernel. The downside as it turns out, VNIC creation over aggregate links isn't the typical use-case and wasn't really tested. When it was initially reported after a bit of debugging it turned out our Solaris API implementation is doing an incorrect check somewhere down the line before invoking the common code, but it was too close to Solaris 11's public release so changes weren't possible at that time... anyway, that's the long story behind this. I'll see if I can get the ball rolling over fixing this with the Solaris kernel networking team but I can't commit any deadlines for the fix.

[martyscholes]

dladm uses userland calls into the kernel which then checks for appropriate privileges before creating & managing VNICs and this path is not usable (not feasible) from the kernel itself,

Ah, that makes sense. I didn't realize VB was managing VNICs from kernel space.

[martyscholes]

Ramshankar,

You have been very helpful on this issue. I recently upgraded to 11.1 and tried out bridging to the aggregate link. The guest started and a VNIC was created. I started several VMs this way and the VNICs came and went as expected, but the network connectivity in the guest was intermittent. I have not done any diagnosing. Currently running 4.1.22.

Do you know if the new Solaris 11.1 has solved the limitation of bridging to aggregate links?

Many thanks,
Marty