Shared physical disk

[Øystein Amundsen]

Hi.

First off, my physical system setup:
I'm running a Windows server 2008 R2 on a Lenovo W510, with one 160Gb SSD and one 300Gb HHD. I've got an Intel i7 CPU and I'm running 12 Gigs of ram. My VM engine is VirtualBox 4.1.4, after painfully realising that Hyper-V is useless (the main reason for running Win 2008 server in the first place. But I'm too lazy to reinstall my windows 7).

This is my requirement:
I have set up a number of different VM's which all need to share data. I also wish the same data to be available to the host.

This is how I've tried to do this:
I've got a separate physical disk (D:) set up for data storage. I've created a vmdk for this using;

Code: Select all   Expand viewCollapse view

 VBoxManage internalcommands createrawvmdk -filename D:\DDisk.vmdk -rawdisk \\.\PhysicalDrive1

I then used the Virtual media manager to state that the medium is "Shareable", through "Modify medium attributes". The vmdk is attached to several VM's, as a secondary SATA drive.

Problem:
When running inside one of my VM's, any operation done on files (creating/deleting/modifying) in the shared vmdk (which I thought provided direct access to my physical drive) will not be reflected if I browse the same files from my host. All file changes done inside the VM are lost when the VM is restarted. This makes the shared vmdk unusable for my purpose. What can I do to gain absolute access to my physical disk inside my VM's (except adding them as shared network folders, because that just sucks as far as usability is concearned)?

[Perryg]

Why not just use an NFS or some other type of network share?

[Øystein Amundsen]

I am a developer, developing solutions for different platforms and for different customers. I wish to have a common code base as well as common user files for all my platforms. I therefore want to share all my user files between VM's, and found that espessially windows platforms does not do a good job of doing this over a network path (experiencing problems with indexing and offline files will simply replicate thus causing wasted space on my VHD's).

Virtual PC, Hyper-V and VMWare will all (as far as I understand it) allow access to all physical disks on the system and not isolate the VM completelly. But Hyper-V causes my system to loose the ability to hibernate. Virtual PC cannot take on 64bit images, and VMWare is expensive.

Why can't I gain such a complete access in VirtualBox as well?

[Perryg]

No one says you can't it is just that it is more than slightly dangerous. I would look at permissions between the different types of OSes as your issue.

[Øystein Amundsen]

Interesting. Please elaborate, as I'm not sure what this means or how this is done.

[Perryg]

Windows and Linux handle permissions differently. Windows likes to think it has total control over the universe and Linux really just wants to make sure it is the way you want it to be.
Check the permission levels of your share in all of the platforms and make sure that you have read, write. as a minimum.
Another issue you are going to need to deal with if you are running several guests and host is that you really should not access (directly - RAW) from multiple operating systems. They will not know what each other is doing and you can really corrupt the data and even the drive. VirtualBox uses ring0 and ring3 access so hopefully you can see where RAW could be an issue.

[Øystein Amundsen]

Windows and Linux handle permissions differently. Windows likes to think it has total control over the universe and Linux really just wants to make sure it is the way you want it to be.

Nicely put.

Check the permission levels of your share in all of the platforms and make sure that you have read, write. as a minimum.

Now you've lost me. My "share"? Are you talking network share now? I would actually like to access the disk in it's raw format, and not through a network share, as I stated earlier - there are a lot of problems with doing what I need to do through a network share.

Another issue you are going to need to deal with if you are running several guests and host is that you really should not access (directly - RAW) from multiple operating systems. They will not know what each other is doing and you can really corrupt the data and even the drive. VirtualBox uses ring0 and ring3 access so hopefully you can see where RAW could be an issue.

Here I need to correct myself. I actually only need to share the disk between all my windows environments. That is my host, and a couple of windows vm's setup slightly different for different customer needs.

Now that I look over what I've written, one question comes naturally to mind if I was somebody else: Do you really need to have several VMs? How come you cannot merge all your needs into one platform and get rid of VirtualBox completelly?
Well, as I stated; different customers have different needs and different demands as to how the development environment should be set up. But I would still like to have a common ground between all environments, sort of like several pc's all having the same external USB disk connected. My "home" folder would be the same on all systems, allowing me to share files between them.

And also, as far as a development platform goes, having it as a VM makes it all more easy to share accross developers. New developers don't have to use a lot of time to set up their system, they can just receive a VM and get going. And if all developers have their own "shared home disk", all their files are still accessible from every platform. In my mind this is beautiful, if I can get it to work.

[vbox4me2]

Reinventing the wheel... iScsi does this.

[Leak]

I've got a separate physical disk (D:) set up for data storage. I've created a vmdk for this using;

Code: Select all   Expand viewCollapse view

 VBoxManage internalcommands createrawvmdk -filename D:\DDisk.vmdk -rawdisk \\.\PhysicalDrive1

I then used the Virtual media manager to state that the medium is "Shareable", through "Modify medium attributes". The vmdk is attached to several VM's, as a secondary SATA drive.

Unless the file system on that disk is actually capable of being used from several machines at once (which means it's either read-only or some special filesystem that neither ext[234] nor NTFS is) this can't work, since nothing is going to synchronize concurrent accesses to the filesystem's structures and you'd quickly end up with a damaged or destroyed filesystem that way.

What you're probably looking for is listed under Shared Folders in the VirtualBox manual - it's a network provider (for Windows) or similar driver that allows you to mount directories on your host directly inside the guest without having to go through any actualy networking, as long as there exists a version of the VirtualBox guest additions for that OS and they're installed in the guest.

[vbox4me2]

Here design ready: http://www.snia.org/sites/default/files ... -04-13.pdf

[Øystein Amundsen]

Unless the file system on that disk is actually capable of being used from several machines at once (which means it's either read-only or some special filesystem that neither ext[234] nor NTFS is) this can't work, since nothing is going to synchronize concurrent accesses to the filesystem's structures and you'd quickly end up with a damaged or destroyed filesystem that way.

I'm not sure why you state that this cannot be done, when it is absolutelly possible for several systems to access a NAS formatted with NTFS concurrently. It is the same principle, only over network. I want the same access, just locally. I have to add that the disk to be shared between VM's, only contains data - not OS or host critical files.

What you're probably looking for is listed under Shared Folders in the VirtualBox manual - it's a network provider (for Windows) or similar driver that allows you to mount directories on your host directly inside the guest without having to go through any actualy networking, as long as there exists a version of the VirtualBox guest additions for that OS and they're installed in the guest.

This also proves my point. VirtualBox allows concurrent access to a hosts filesystem over network folders, but not as a raw disk?
And if such raw access actually were impossible, how come VirtualPC can allow total access to it's hosts filesystem?

[Perryg]

VirtualPC can allow total access to it's hosts filesystem

I admit I have not used VPC for sometime but the way I understood it Windows did not actually do RAW. It used the VPC program and hooked in via an API.
Probably what the others are doing as well. The Shared Folders of VirtualBox is doing the same thing. You do not need a network connection to make it work although it is mapped as a network share.

But none of this matters. You are a developer and will figure it all out. Here is the source code if you want to see what and how. You can if you want continue with the RAW access. All I needed to do was point out the ramifications.

[Leak]

I'm not sure why you state that this cannot be done, when it is absolutelly possible for several systems to access a NAS formatted with NTFS concurrently. It is the same principle, only over network. I want the same access, just locally. I have to add that the disk to be shared between VM's, only contains data - not OS or host critical files.

No matter if it's a simple network file system like SMB or NFS or if it's direct access like iSCSI, there's a strict protocol and a piece of hard- and software (the NAS) there to synchronize accesses. You don't get that if you just let several VMs directly access some physical or virtual partition - do the math.

This also proves my point. VirtualBox allows concurrent access to a hosts filesystem over network folders, but not as a raw disk?

Well, duh - of course not. For Windows, it implements a network provider (like SMB or Netware or whatever) inside the guest that calls back to the VM and simply translates all file and directory accesses to it into file operations on the local drive via the regular open/read/write/etc. file calls, so there's no need to do any heavy low-level lifting in the background - if two VMs try to concurrently access a shared file it's no different than two regular processes trying to do the same, which is something all modern OSes can easily handle on their own.

Or, in other words, VirtualBox (and VMWare, and VPC, and whatever other VM there is) is sharing files on the application level instead of sharing them on the block level you were trying to do. Much less of a hassle, and the result for the end user is the same - you'll get access to all the shared files from all VMs that you mount the share on and the host.

[mpack]

when it is absolutelly possible for several systems to access a NAS formatted with NTFS concurrently.

Nope. The clients only access the server software, via network comms. The server PC OS has sole ownership of and sole access to its own physical hard disk. The illusion the guest has at user level, that it can access files on the server drive... is just that, an illusion. In fact File I/O requests on the client get translated into network protocol messages by the client OS, and sent to the server OS, which does the reverse. At no time does the client directly access the hardware or filesystem of another PC.