Solaris Zone Guests - Bridged - Duplicate addresses

[susu.exp]

Host: SunOS 5.11 11.0 i86pc i386 i86pc
Guests: different flavors of 32 & 64 bit Linux bridged to Zone vnics, 1 guest per VBox instance per exclusive IP zone
VBox: 4.1.16+extpack+guest additions

All guests and Zone hosts have logs full of Duplicate Address warnings - "Someone advertises our address!" (guests), "Duplicate Address Detected!" (zone hosts), both ipv4 & 6 addys, with either DHCP(6) or static addressing on bridged guests.

The guests seem to perform well, but they don't route DHCPv6 addresses (even though Zone hosts are dual stack), and /var/log/messages gets new warnings every few seconds the guests are up.

Hints? Suggestions? Have I provided enough info?

Thanks.

[martyscholes]

I saw similar weirdness when I ran Linux appliances within zones. My solution was to allocate a separate NIC to each zone and dedicate that to the appliance.

The catch was that the NIC had to be declared and plumbed for the zone, but not used by the zone itself. I can probably find my notes on how to get it done if you'd like.

Did the messages about duplicate addresses come from the Solaris zone or the guest OS?

[susu.exp]

I saw similar weirdness when I ran Linux appliances within zones. My solution was to allocate a separate NIC to each zone and dedicate that to the appliance.

The catch was that the NIC had to be declared and plumbed for the zone, but not used by the zone itself. I can probably find my notes on how to get it done if you'd like.

Can I do that with Crossbow vnics?
Creating and attaching the vnic to a zone seems basic, but how to reserve the vnic for the VBox guest?

Did the messages about duplicate addresses come from the Solaris zone or the guest OS?

Both, but not as spammy in the zone host as the guest. I'm a little afraid to Wireshark the LAN...


Thank you for your help.

[martyscholes]

Can I do that with Crossbow vnics?
Creating and attaching the vnic to a zone seems basic, but how to reserve the vnic for the VBox guest?

That's the trick. I found my notes. I did it by trial and error and error and error. There probably is a better way. I had defined "net1" for the zone, but as you noted, the zone wanted to use it. I wanted both net0 and net1 assigned via DHCP so that I could manage all IP addresses at the DHCP server. What follows are my notes for setting up net1 in the zone "bugzilla." All of this was done within the zone itself.

Code: Select all

# here is the current state
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           online
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# now create a new NCP and tie in the NCU of the two networks
bugzilla@bugzilla:~$ sudo netcfg
netcfg> create ncp bugzilla
netcfg:ncp:bugzilla> create ncu phys net0
Created ncu 'net0'.  Walking properties ...
activation-mode (manual) [manual|prioritized]> 
link-mac-addr> 
link-autopush> 
link-mtu> 
netcfg:ncp:bugzilla:ncu:net0> end
Committed changes
netcfg:ncp:bugzilla> create ncu ip net0
Created ncu 'net0'.  Walking properties ...
ip-version (ipv4,ipv6) [ipv4|ipv6]> 
ipv4-addrsrc (dhcp) [dhcp|static]> 
ipv4-default-route> 
ipv6-addrsrc (dhcp,autoconf) [dhcp|autoconf|static]> 
ipv6-default-route> 
netcfg:ncp:bugzilla:ncu:net0> end
Committed changes
netcfg:ncp:bugzilla> create ncu phys net1
Created ncu 'net1'.  Walking properties ...
activation-mode (manual) [manual|prioritized]> 
link-mac-addr> 
link-autopush> 
link-mtu> 
netcfg:ncp:bugzilla:ncu:net1> end
Committed changes
netcfg:ncp:bugzilla> create ncu ip net1
Created ncu 'net1'.  Walking properties ...
ip-version (ipv4,ipv6) [ipv4|ipv6]> 
ipv4-addrsrc (dhcp) [dhcp|static]> 
ipv4-default-route> 
ipv6-addrsrc (dhcp,autoconf) [dhcp|autoconf|static]> 
ipv6-default-route> 
netcfg:ncp:bugzilla:ncu:net1> 
netcfg:ncp:bugzilla:ncu:net1> end
Committed changes
netcfg:ncp:bugzilla> commit
'commit' is not allowed at this scope
netcfg:ncp:bugzilla> end
Committed changes
netcfg:ncp:bugzilla> commit
'commit' is not allowed at this scope
netcfg:ncp:bugzilla> end
netcfg> list
NCPs:
        Automatic
        bugzilla
Locations:
        Automatic
        NoNet
        User
netcfg> exit

# now we have a new NCP which basically is the same as auto, so switch to it
bugzilla@bugzilla:~$ sudo netadm enable bugzilla
Enabling ncp 'bugzilla'

# that will kick us off since the network stacks are changing, so log back in
# first press tilde then dot to get the session to end
bash-4.1$ ssh bugzilla@bugzilla_z

# now we are back in and it looks like nothing has changed
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      disabled
ncp         bugzilla       online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           online
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# but we can now disable the ip portion of net1
bugzilla@bugzilla:~$ sudo netadm disable -c ip net1
Disabling ncu 'net1'
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      disabled
ncp         bugzilla       online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           disabled
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# this leaves us with a plumbed IP which the zone will not try to use
# phew

I hope that makes sense. Good luck and let us know how it works out!

[susu.exp]

Can I do that with Crossbow vnics?
Creating and attaching the vnic to a zone seems basic, but how to reserve the vnic for the VBox guest?

That's the trick. I found my notes. I did it by trial and error and error and error. There probably is a better way. I had defined "net1" for the zone, but as you noted, the zone wanted to use it. I wanted both net0 and net1 assigned via DHCP so that I could manage all IP addresses at the DHCP server. What follows are my notes for setting up net1 in the zone "bugzilla." All of this was done within the zone itself.

Code: Select all

# here is the current state
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           online
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# now create a new NCP and tie in the NCU of the two networks
bugzilla@bugzilla:~$ sudo netcfg
netcfg> create ncp bugzilla
netcfg:ncp:bugzilla> create ncu phys net0
Created ncu 'net0'.  Walking properties ...
activation-mode (manual) [manual|prioritized]> 
link-mac-addr> 
link-autopush> 
link-mtu> 
netcfg:ncp:bugzilla:ncu:net0> end
Committed changes
netcfg:ncp:bugzilla> create ncu ip net0
Created ncu 'net0'.  Walking properties ...
ip-version (ipv4,ipv6) [ipv4|ipv6]> 
ipv4-addrsrc (dhcp) [dhcp|static]> 
ipv4-default-route> 
ipv6-addrsrc (dhcp,autoconf) [dhcp|autoconf|static]> 
ipv6-default-route> 
netcfg:ncp:bugzilla:ncu:net0> end
Committed changes
netcfg:ncp:bugzilla> create ncu phys net1
Created ncu 'net1'.  Walking properties ...
activation-mode (manual) [manual|prioritized]> 
link-mac-addr> 
link-autopush> 
link-mtu> 
netcfg:ncp:bugzilla:ncu:net1> end
Committed changes
netcfg:ncp:bugzilla> create ncu ip net1
Created ncu 'net1'.  Walking properties ...
ip-version (ipv4,ipv6) [ipv4|ipv6]> 
ipv4-addrsrc (dhcp) [dhcp|static]> 
ipv4-default-route> 
ipv6-addrsrc (dhcp,autoconf) [dhcp|autoconf|static]> 
ipv6-default-route> 
netcfg:ncp:bugzilla:ncu:net1> 
netcfg:ncp:bugzilla:ncu:net1> end
Committed changes
netcfg:ncp:bugzilla> commit
'commit' is not allowed at this scope
netcfg:ncp:bugzilla> end
Committed changes
netcfg:ncp:bugzilla> commit
'commit' is not allowed at this scope
netcfg:ncp:bugzilla> end
netcfg> list
NCPs:
        Automatic
        bugzilla
Locations:
        Automatic
        NoNet
        User
netcfg> exit

# now we have a new NCP which basically is the same as auto, so switch to it
bugzilla@bugzilla:~$ sudo netadm enable bugzilla
Enabling ncp 'bugzilla'

# that will kick us off since the network stacks are changing, so log back in
# first press tilde then dot to get the session to end
bash-4.1$ ssh bugzilla@bugzilla_z

# now we are back in and it looks like nothing has changed
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      disabled
ncp         bugzilla       online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           online
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# but we can now disable the ip portion of net1
bugzilla@bugzilla:~$ sudo netadm disable -c ip net1
Disabling ncu 'net1'
bugzilla@bugzilla:~$ netadm list
TYPE        PROFILE        STATE
ncp         Automatic      disabled
ncp         bugzilla       online
ncu:phys    net0           online
ncu:ip      net0           online
ncu:phys    net1           online
ncu:ip      net1           disabled
loc         Automatic      online
loc         NoNet          offline
loc         User           disabled

# this leaves us with a plumbed IP which the zone will not try to use
# phew

I hope that makes sense. Good luck and let us know how it works out!

Wow, thank you! I'll take this up soon.
Are all these netadm commands run from a nonglobal zone? I ask because netadm in a zone returns an error due to my zone's DefaultFixed NCP.

Again, thanks!

[martyscholes]

Wow, thank you! I'll take this up soon.
Are all these netadm commands run from a nonglobal zone? I ask because netadm in a zone returns an error due to my zone's DefaultFixed NCP.

Again, thanks!

Yes, this is all done in the local zone. While I am not at all an expert here, I think Defaultfixed refers to static IP addresses in the zone. In my configuration, I had Automatic, which is the default for DHCP set addresses. I suspect if you do something similar to what I did, but copy Defaultfixed instead of Automatic, you will get close.

Does that make sense?

[susu.exp]

Wow, thank you! I'll take this up soon.
Are all these netadm commands run from a nonglobal zone? I ask because netadm in a zone returns an error due to my zone's DefaultFixed NCP.

Again, thanks!

Yes, this is all done in the local zone. While I am not at all an expert here, I think Defaultfixed refers to static IP addresses in the zone. In my configuration, I had Automatic, which is the default for DHCP set addresses. I suspect if you do something similar to what I did, but copy Defaultfixed instead of Automatic, you will get close.

Does that make sense?

Yes, and thank you for the help. I started the thread in the hope of perhaps finding a simple misconfiguration or bungle on my part. I tried your approach, got good IPV6 connectivity, but still get DAD warnings or similar in the VBox guests.

[This isn't confined to my Solaris hosts, or my networks either. All my bridged VBox guests have DAD or similar warnings in their respective logs, regardless of host OS or network. All my bare metal hosts can autoconfig with temporary/privacy addresses and/or use DHCPv6 on this LAN. NWAM on Solaris 11 is no exception, and neither is manual NCP. ]

On Solaris 11 with manual network config (default:physical NCP), I found a stable solution using 2 vnics in active IPMP with static and addrconf v6 addressing on each NGZone VBox Host. While I haven't yet gone through all the permutations of link-based IPMP and Zone IP addressing and of creating the extra vnic on the GZ or NGZ to settle on the best technique to use with VirtualBox, I've had fun learning as I step through some of the options available.