What is the 'PROPER' way of backing up and or copying a vm to another location for storage or usage or for just backing up in case of disaster? I currently use VMware which is quite easy to copy, move or reuse. VBox has some features that I would like to try, but for various reasons I am having trouble moving and or copying the vm's. So before I trust any work to VBox I need to make sure I can move, copy or restore them like I can with VMWare.
MCP
Backup and or Copy
-
TerryE
- Volunteer
- Posts: 3572
- Joined: 28. May 2008, 08:40
- Primary OS: Ubuntu other
- VBox Version: PUEL
- Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
- Contact:
I am a little confused as to the difference between VMware and Vbox in this regard. If you look at the Preferences in the GUI manager, you will see the default location for your VDIs. I move mine to D:\VirtualBox\VDI on my set up. The machines are in a sister directory. I just shut down my machines and use 7Zip to compress the VDIs. I also split my configurations into a system VDI and an Apps VDI. I keep my system configs pretty much fixed and sometimes move them to immutable, and that way I only need to routinely backup the Apps disk.. And yes, it is very sensible to move these backups off onto a separate machine, NAS or removable media.
One trick that I also do before doing a compressed backup is to boot in init 1 mode and run the zerofree utility (search this forum and you'll find a post describing this.)
Why do you think VBox is fundamentally different to VMware?
One trick that I also do before doing a compressed backup is to boot in init 1 mode and run the zerofree utility (search this forum and you'll find a post describing this.)
Why do you think VBox is fundamentally different to VMware?
-
mcp
- Volunteer
- Posts: 132
- Joined: 12. May 2007, 23:55
- Primary OS: PCLinuxOS
- VBox Version: PUEL
- Guest OSses: Windows XP Pro
TerryE,
Thanks for your insights. I said what I did because of my experience with VMware and VBox. I discovered that I had posted this in the wrong place because I am using Xandros linux as a host and not Windows. I'm beginning to think that my problem is not VBox but Xandros. I assume that you are using Windows as your host os. Is that true? And may be why you do not have the same problem that I'm experiencing. It appears to be linked to a networking issue.
Thanks again,
MCP
Thanks for your insights. I said what I did because of my experience with VMware and VBox. I discovered that I had posted this in the wrong place because I am using Xandros linux as a host and not Windows. I'm beginning to think that my problem is not VBox but Xandros. I assume that you are using Windows as your host os. Is that true? And may be why you do not have the same problem that I'm experiencing. It appears to be linked to a networking issue.
Thanks again,
MCP
-
TerryE
- Volunteer
- Posts: 3572
- Joined: 28. May 2008, 08:40
- Primary OS: Ubuntu other
- VBox Version: PUEL
- Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
- Contact:
MCP,
You have two broad strategies for backing up VMs: brute-force and application/system specific. Each has its merits. The brute-force approach is the total backup of the HDD state, much as you would do full disk backups in the old days on systems. This is what I discussed above.
The application/system specific approach is more of a precision mechanism which is tailored to the specific application mix on your system. Yes more precision, but harder to configure, though if you want your VMs to run pretty much non-stop you will be forced down this route. What you are doing here is essentially no different than if your VM was running on bare metal. Using VBox NAT for these sorts protocols can be very problematic because of the port mapping, however if you bridge your network, then you should be fine.
You have two broad strategies for backing up VMs: brute-force and application/system specific. Each has its merits. The brute-force approach is the total backup of the HDD state, much as you would do full disk backups in the old days on systems. This is what I discussed above.
The application/system specific approach is more of a precision mechanism which is tailored to the specific application mix on your system. Yes more precision, but harder to configure, though if you want your VMs to run pretty much non-stop you will be forced down this route. What you are doing here is essentially no different than if your VM was running on bare metal. Using VBox NAT for these sorts protocols can be very problematic because of the port mapping, however if you bridge your network, then you should be fine.
-
mcp
- Volunteer
- Posts: 132
- Joined: 12. May 2007, 23:55
- Primary OS: PCLinuxOS
- VBox Version: PUEL
- Guest OSses: Windows XP Pro
TerryE,
I appreciate your comments but I'm specifically talking about moving and or copying the entire vdi file. So far I have not been successful. If I move or copy a working vdi it becomes useless. I only get a BSOD when I try to start it up. So I am hesitant to use it for work. And in case your wondering why, I have had to move them so as to provide enough disk space as it grows. Any suggestions on why this is happening?
MCP
I appreciate your comments but I'm specifically talking about moving and or copying the entire vdi file. So far I have not been successful. If I move or copy a working vdi it becomes useless. I only get a BSOD when I try to start it up. So I am hesitant to use it for work. And in case your wondering why, I have had to move them so as to provide enough disk space as it grows. Any suggestions on why this is happening?
MCP
-
TerryE
- Volunteer
- Posts: 3572
- Joined: 28. May 2008, 08:40
- Primary OS: Ubuntu other
- VBox Version: PUEL
- Guest OSses: Ubuntu 10.04 & 11.10, both Svr&Wstn, Debian, CentOS
- Contact:
I've actually been looking into this issue, and the only way that you can understand this product is to go through the source code.
All VDIs have essentially the same structure. The underlying disk image is divided into 1Mb image blocks. The VDI itself has three parts: a standard header descriptor (~512 bytes) followed by an image block map, followed by the binary image blocks which make up the virtual disk. The descriptor contains the information that you see when you do a VboxManage showvdiinfo plus internal housekeeping and disk geometry being emulated. The block map contains a 4 byte per block index which maps the virtual image block to the physical block in the file. Hence this vector takes up 4Kbytes for each Gb in the VDI. Because the image blocking works on 1Mb (1,048,576 byte) chunks, the image must be an integral number of Mb.
The only difference between static and dynamic disks is that with dynamic disks all block indexes are set to zero to denote that the block is zero and yet to be allocated, and therefore none are allocated; static files have all blocks allocated and pre-filled with zeroed sectors, hence the index is (1,2,3, …). Immutable and snapshot sets largely work the same way except that ‘dirty’ image blocks are written to the new file. Again this works on a 1Mb grain-size so writing 1 byte in the middle of a file will create a new 1Mb dirty image block which has 1,048,575 bytes copied from the original.
As you write non-zero byte streams to a dynamic VDI, the block map is used to translate the virtual address to physical on the host VDI file. If the block has not been allocated then a new one is allocated at the end of the file, its contents zeroed, and the block map updated. Hence unlike static disks which store the image blocks in virtual order, the dynamic disks store them in allocated order.
Both NTFS and Ext3 use a 8K allocation unit for typical size disks and as you use the file system it will slowly fragment. Every time the guest OS file system touches an allocation unit in a previously unused 1Mb window, the host VDI will increase by 1Mb.
For Windows guests the way to minimise this growth is to regularly:
All VDIs have essentially the same structure. The underlying disk image is divided into 1Mb image blocks. The VDI itself has three parts: a standard header descriptor (~512 bytes) followed by an image block map, followed by the binary image blocks which make up the virtual disk. The descriptor contains the information that you see when you do a VboxManage showvdiinfo plus internal housekeeping and disk geometry being emulated. The block map contains a 4 byte per block index which maps the virtual image block to the physical block in the file. Hence this vector takes up 4Kbytes for each Gb in the VDI. Because the image blocking works on 1Mb (1,048,576 byte) chunks, the image must be an integral number of Mb.
The only difference between static and dynamic disks is that with dynamic disks all block indexes are set to zero to denote that the block is zero and yet to be allocated, and therefore none are allocated; static files have all blocks allocated and pre-filled with zeroed sectors, hence the index is (1,2,3, …). Immutable and snapshot sets largely work the same way except that ‘dirty’ image blocks are written to the new file. Again this works on a 1Mb grain-size so writing 1 byte in the middle of a file will create a new 1Mb dirty image block which has 1,048,575 bytes copied from the original.
As you write non-zero byte streams to a dynamic VDI, the block map is used to translate the virtual address to physical on the host VDI file. If the block has not been allocated then a new one is allocated at the end of the file, its contents zeroed, and the block map updated. Hence unlike static disks which store the image blocks in virtual order, the dynamic disks store them in allocated order.
Both NTFS and Ext3 use a 8K allocation unit for typical size disks and as you use the file system it will slowly fragment. Every time the guest OS file system touches an allocation unit in a previously unused 1Mb window, the host VDI will increase by 1Mb.
For Windows guests the way to minimise this growth is to regularly:
- Keep a tight control on the growth that tends to occur in windows files systems anyway. You'll get posts on this in windows forums, but I have a CMD script which clears out all of the various temp directories, caches, recycled bin, etc. I also delete all those $NtUninstall...$ directories that get created every Patch Tuesday. (I've never heard of anyone who sensibly uninstalled one anyway!)
- Run defrag on the guest to defragment both files and space.
- Clear down unused file system space to zeros. For NTFS under a Windows guest OS, the best tool to use is the System Internals (now part of Microsoft) tool sdelete (Windows Sysinternals — Sdelete). You run this at command prompt with the -c option to tell sdelete to clear the unused space, hence the following command cleans out the C and D partitions:
- sdelete –c C
sdelete –c D
- sdelete –c C
- Shutdown the guest, then run VBoxManage modifyvdi xxxx.vdi compact from your host OS where xxxx.vdi is the VDI-file of your target drive.