Page 1 of 1

Multi Host (two VMs on seperate ESX) networking with virtuabox

Posted: 13. Oct 2016, 11:42
by gurukraj
Hi team,

For our application use case which is distributed and multi-tier, we are exploring Docker with below configuration. I am using Docker 1.12. (latest) version. And we are using virtualbox network driver to configure overlay network. Here is the detail about setup and problem.

I have two seperate ESX hosts, one ESX has single VM running as docker host(say host-1). Another ESX has another VM running with docker host (say host-2). Both of these VMs are running with CentOS 7.x. I have installed docker engine and docker-machine on both of these hosts.

I have successfully configured the external K-V store using virtualbox network driver . I created the virtualbox on one of the VM (host1) using docker-machine create. Also installed the KV store container consul on one of the docker host. However since these VMs(both docker hosts) have static IP assigned and configured in vLAN236, the virtualbox network interface (vboxnet0) always creates the virtual-box and docker-machines using 192.168.x.x. I would like to configure the docker-machine and virtual box with vLAN236 (static IP) so that I can leverage centralized virtual box with docker machines running on host-1 as well as host-2. The goal I am trying to achieve is,
1) I would like to make the virtual box as common between host-1 and host-2.
2) I would like to create a swarm cluster of VMs configutred on host-1 and host-2 to centralized virtual box network.
3) Create a overlay network to make the swarm to communicate containers across two different hosts.
4) Make the containers communicate with external world as well.

What I did so far is:

1) I tried to configure the vboxnet0 interface created by virtualbox, with static IP. However when I tried to create a virtual machine using "docker-machine" with this virtual box, it always revert back to 192.168.x.x IP. This is making me difficult for communication from other docker host (host2).
2) I tried DHCP server by providing lower range and upper range using "vboxmanage" and tried to modify the vboxnet0 interface, however this also does not help me since virtualbox network driver always assigns the 192.168.x.x IP.
3) I tried modifying the vboxnet0 interface in "/etc/sysconfig/network-scripts/ifcfg-vboxnet0", by assigning static IP. However after reboot, the vboxnet0 still gets the IP range 192.168.x.x.

Please advice me with some examples on how this can be made successful communication to take the advantage of containers running on distributed networked cluster of docker hosts.

Re: Multi Host (two VMs on seperate ESX) networking with virtuabox

Posted: 13. Oct 2016, 14:31
by socratis
Docker is a program that relies on VirtualBox but modifies its configuration files in unknown ways to us. It is not supported on these VirtualBox user forums, they have their own support channels. If you are having this problem with a standalone version of VirtualBox, then we can continue this discussion.

Re: Multi Host (two VMs on seperate ESX) networking with virtuabox

Posted: 14. Oct 2016, 07:22
by gurukraj
Thank you for your response.

Let us assume there is "no docker", can we create a virtualbox network to cluster the two Hosts running on separate ESX boxes? I am leveraging the Docker to create a swarm cluster, however to do that, I need virtualbox can be used to clustering the two VMs running on separate ESX machines.

Re: Multi Host (two VMs on seperate ESX) networking with virtuabox

Posted: 14. Oct 2016, 08:23
by socratis
gurukraj wrote:can we create a virtualbox network to cluster the two Hosts running on separate ESX boxes?
You want to run VMWare's ESX using VirtualBox's network? I don't really know but wouldn't that be akin to creating an MS Server cluster using OSX's network driver? I'm afraid I wouldn't have the slightest idea if this would work or not. I'm only using VirtualBox, sorry. As I said already:
socratis wrote:If you are having this problem with a standalone version of VirtualBox, then we can continue this discussion.