Making existing VDI disk shareable

[mjf]

Hi,

can I change the the attribute of an existing VDI drive from "normal" to "shareable" without risking any of the data on the disk? According to section Special image write modes in the manual I should be able to use a shareable disk from two VMs running at the same time as long as I make sure they are not writing at the same time. Right now, the disk is attached to two VMs, but I cannot run those two simultaneously.

Thanks,
mjf

[socratis]

can I change the the attribute of an existing VDI drive from "normal" to "shareable" without risking any of the data on the disk?

I take it that you noticed the following from the user manual:

Shareable hard disks ... This makes them suitable for use by cluster filesystems between VMs and similar applications which are explicitly prepared to access a disk concurrently.

• Warning: This is an expert feature, and misuse can lead to data loss – regular filesystems are not prepared to handle simultaneous changes by several parties.

which pretty much excludes 99.99% of the normal use cases that you may have in mind.

And then you state:

as long as I make sure they are not writing at the same time

How exactly are you envisioning that? Like working with a data collection program in VM1, write the data to the disk, switch to VM2, analyze the data, write the results to the disk? Something like that?

Please start the description from where you want to go, not from how you're planning to go (somewhere where we don't know). State what your end goal is, not the means to get there.

[mpack]

Also mention why you don't just use a shared folder to communicate files between two running VMs?

[mjf]

Here is my scenario: On a Windows host, I'm using 32-bit and 64-bit Ubuntu guest VMs for software development, depending on the development environment requirements. Each set of source code, SDKs and Buildroots resides on its own (multi GB) VDI data disk. Besides their private root disk and their "primary" data VDI, the VMs mount other VDIs as "secondary" data disks for convenience (e.g. check or copy souce code from other projects). These "secondary" disks prevent running multiple VMs at the same time.

My intention is not to exchange data between muliplte VMs but to occasionally access data from multiple VMs. At least as long as its triggered by user action, each VM only writes to its "primary" data disk during development work. Unless there are background processes involved that I'm not aware of, there should be no data corruption caused by multiple VMs writing to the same disk at the same time.

I'm not using shared folders because I don't want the data to reside on the host. I need to preserve Linux file attribute and the option to move to a different host easily.

Yes, I'm aware of the warnings regarding shared disks in the manual section I included a link to in my original post. What I'd like to ask is
- Does changing the attribute of a VDI from "normal" to "shareable" by itself change any of the disks contents, i.e. would I risk loss of data just by changing the attribute?
- Are there automatic background processes in a reguluar Ubuntu system (besides a misconfigured fsck) that might try to write to my data disk?

[mpack]

IMHO that is not a valid scenario for shareable disks, and will almost certainly result in corrupted filesystems. Explicit user action is far from being the only reason an OS writes to a disk.

It doesn't sound to me like you need shareable disks at all. "Shareable" means that VirtualBox allows multiple running VMs to access it simultanously. This would require each participating OS to recognize that the disk is shared. It doesn't seem to me that you have understood this requirement.

If you don't need the "simultanously" bit, then you don't need shareable disks.

[mjf]

Explicit user action is far from being the only reason an OS writes to a disk.

In my scenario, what reasons for the OS to write to any of the "secondary" data disks do you envision?

It doesn't sound to me like you need shareable disks at all.

Then what setup would you suggest instead?

"Shareable" means that VirtualBox allows multiple running VMs to access it simultanously. This would require each participating OS to recognize that the disk is shared. It doesn't seem to me that you have understood this requirement.

It seems to me that one simple way of honoring that a disk is shared is not to write to it automatically. As long as there is only one user causing a write (and waiting until it is finished), there should be no overlapping writes.

If you don't need the "simultanously" bit, then you don't need shareable disks.

May be we have different concepts of "simultanously", but I tried to explain that I would like to run different VMs at the same time that mount the same VDI.

Do you know whether I risk any data corruption by changing a VDIs attribute from "normal" to "shareable"?

[ChipMcK]

Only if volume is mounted as Read-Only on both systems, then there is no chance of data corruption.

This is not network-share (SMB) where the server expects concurrent (simultaneous) access/update of the volume's data. You will not have a server to act as "traffic cop" for the data.

The volume may be Writable from ONLY one system (a server) in order to ensure data integrity.

[mjf]

Yes, that's true. Mounting the disks read-only on all but only one VM should be the safe way to go. But it would be somewhat of a limitation because I'd always have to run multiple VMs even if I wanted to work with data on those disks purely sequentially.

I think I understand the dangers of writing to a (shareble) VDI from multiple VMs concurrently. There are no transactions for accessing file-systems on shareable disks in VirtualBox. But as long I am sitting in front of the host system, actually using only one guest system at a time, I still don't see what might cause overlapping writes (or reads).

So let me rephrase my initial question: Can is change a VDI's attribute from "normal" to "shareable" after the disk has been already been used, i.e. after it has been formatted and written to, without losing any data on it? Or must I declare a VDI "shareable" right when I create it?

[socratis]

I think I understand the dangers of writing to a (shareable) VDI from multiple VMs concurrently.

No, I don't think you do. You keep getting warning after warning, caution after caution and danger after danger. I don't think we're playing the scare-the-new-guy game.

So let me rephrase my initial question: Can is change a VDI's attribute from "normal" to "shareable" after the disk has been already been used

Technically, yes. Try it. Make a backup of your VDI (which I assume you have already) and give it a shot. Theoretically you will fail (that's what everybody and their mother are telling you), but, you seem to be willing to take the risk. I'm all for it! Let us know how it went...

[mjf]

No, I don't think you do. You keep getting warning after warning, caution after caution and danger after danger. I don't think we're playing the scare-the-new-guy game.

Well, I have asked you before to give an example of a background process in a standard Linux guest that might cause concurrent writes to a shared non-system disk. That would convince me easily.

[mpack]

Actually you asked me to do that, not Socratis, but I don't consider it my role to give tuition on how modern OS's work. If you don't accept the advice then by all means continue at your own risk.

[socratis]

give an example of a background process in a standard Linux guest that might cause concurrent writes to a shared non-system disk.

It doesn't have to be concurrent. Strictly sequential will work as well (as in screw your filesystem big time). If you know what a table of contents is in a book, think of something analogous in the file system. If one OS re-arranges the TOC, the other OS (which assumes that it is the sole one controlling the TOC) will be out of sync. Try to spell disaster.

If you can think of any real-life similar situation where you have one HD attached to two computers, then you'll have figured out the answer.

[mjf]

Actually you asked me to do that, not Socratis ...

Oh, sorry socratis.

... but I don't consider it my role to give tuition on how modern OS's work. If you don't accept the advice then by all means continue at your own risk.

I'm surprised that you are willing to share your knowledge on VirtualBox but not your knowledge on operating systems. Especially when the latter would help people like me to understand how to use VirtualBox.

May be someone else can help out with in example?

[socratis]

I gave you one a couple of minutes before your answer. Maybe you were typing your answer and didn't notice it. Hence this "bump"...

[mjf]

Yes, thanks. Actually, I disovered your answer after I had finished my post.