Backup Up Vdi files, how should I do this with tar and tape?

[k-bl]

Host: Ubuntu 8.1, Guest: Win 2003;

I've been looking everywhere and I just can't find the answer. How do I attempt to backup a vdi file using tar commands to a tape? I know how to run the backup, but what if the machine is running? Is it safe to run the machine at the same time you are running the backup?

PLEASE HELP ME GUYS

Any additional insight on the matter would be great. If you have any knowledge on how to automatically samba (the vdi systems are win xp/2003) to the network shares in a Active Directory Environment, let me know as well. Also if I should be doing this differently, let me know. If I could mount the tape from the win 2003 machine, that'd be great, if anyone has any experience with that let me know also.

[TerryE]

Gosh, I didn't think that SMEs used much tape anymore, not with SATA drives starting at roughly $100 per Tb. However to your Q. There are three parts to the Q that you should be asking (i) what files comprise a complete backup set for a VM; (ii) what do I want to back for; and (iii) how do I back a bunch of files [to tape]. This last Q is a pure Linux one and nothing to do with VBox. It is straight forward and pretty well covered on the Ubuntu and other Linux forums, and I have so let me focus on the first two.

• What files comprise a complete backup set for a VM?
  There are a bunch of files that record the state of a VM: the machine.XML file in the machine directory, the contents of the snapshot directory and any VDIs (other disk image files) mapped into the VM. The backup is simplest if you have no snapshot and the machine is properly powered down, and in this case the machine.xml and the VDI(s) are a complete backup set. These will ultimately be encapsulated by the OVF export command and format, though there are still some teething problems in V.2.2.0, so for now what I do is to use the VDI(s) plus a set of VBoxManage openmedium / createvm / modifyvm which I create manually. If you are using snapshots then you need to include these VDI files and if you want to save a running machine then you will need the SAV files as well. However, again restore is convolved and you need to know how the VBox registry works as some manual editing is required.
• What do I want the backup for
  Whilst there are a range of answers here, I would roughly divide the types of backup into two categories: a baseline backup is one you take at a major milestone and is something that you might want to distribute or roll back to for a different branch of your project. In this case I would recommend that you keep this as clean as possible by avoiding snapshots. It is also well worthwhile preparing the VDIs before doing a baseline backup by doing a full defrag then use the MS Sysinternal sDelete tool to zero out any unused clusters. Once you've done this you will also probably want to compress the image on the fly during the backup. Of the standard Linux options bzip2 (the tar -j option will give the best compression, with a size that is typically less than half the used blocks in the NTFS file system. I discuss this in further detail in my tutorial All about VDIs
  The second case is a working backup and in this case I believe the best thing to do is to snapshot the baseline and then all you need to do is to backup the snapshot -- again using a compressed tar.

Really IMHO, this whole area isn;t very user friendly. There is no simple "one button stop-shop" here. You need to think about what you are doing and you need to dress-rehearse recovery before your disasters to make sure that you understand and have a working process.

[k-bl]

Thanks for your reply. My question though is more so the actual commands. I mean, do you guys really manually run all of your backups? That can become very time consuming. I will manage Milestone backups differently than per-say daily backups. I really just want to know how to schedule the correct commands and which ones I should run, OR how can I get windows to recignize the tape and then take care of the backup using ntbackup?

I even have a ide to usb drive that I plugged up to the tape drive, and windows sees it, but it expects info through the ide/serial port, so it doesn't work.

So, in short, how would I:
Pause the machine/drive (the virtual drive isn't a machine).
Back it up to tape.
Unpause the machine and schedule ubuntu to do this daily?

If you guys could just shoot me the commands, that'd be great, in the mean time, I will be trying to find the solutions.

[k-bl]

How about I use "cron" to schedule some of the following commands.

VBoxManage controlvm computername savestate

mt -f /dev/st0 rewind

tar -cf /dev/st0 MyVdiFile

tar -tf /dev/st0

VBoxManage startvm computername

And then if I need to restore:

tar -xf /dev/st0 "MyVdiFile" /restore_directory

I would schedule this to run daily. Is there any reason that this wouldn't work? Let me know if I should do this differently guys!

[TerryE]

I go back to the Qs that I asked and what your requirements are. For example, there's no point in backing up just the VDI if you are using snapshots (in fact the base VDI won't be changing so there's no point anyway. Also if your backup set is large then the tape backup could take hours. Do you want your vm offline for that length of time? If you use LVM to manage your partitions then you can do a savestate, create a snapshot partition and restart the VM. You can then backup the VDIs etc from the snapshot partition to tape and then remove the snapshot partition. This way the VM will be off-line for less than a minute.

The KISS approach might work, but it might also prove a disaster. If you shapshot the baseline and use LVM snapshots then a typical delta tgz file will be under a Gb. I would do these to disk initially. If your tape is locally mounted then this isn't much of a DR approach is it? At home, I use a tiny 500Gb NAS device in an out-house for my backups: disk-to-disk then disk-to-disk over the LAN. If that and my main machine are toast, then I'll probably be toast as well

[k-bl]

The reason: the vdi that I'll be backing up isn't a machine, it is the company's data as a virtual hard-drive , it only weighs in at maybe 10 gig. Is there another way to clone the drive while it is mounted by the virtual guest?

[TerryE]

The reason: the vdi that I'll be backing up isn't a machine, it is the company's data as a virtual hard-drive , it only weighs in at maybe 10 gig.

If it is 10Gb and you have a regular scheduled task to defrag it then run sDelete, a tar.gz of it should only be about 3Gb. That's significantly less than 10 min copy time if you are running a 100Mbit LAN. All the more reason to avoid tape IMHO -- too often I've found tape to prove write many, read never (when you need it).

Is there another way to clone the drive while it is mounted by the virtual guest?

Yes, as I said read up on using LVM. It's really very good. You allocate one or more physical partitions as give them to LVM which can then allow you to create and to extend logical partitions using RAID-1, 3 or RAID-5 if you so wish. You can also create snapshots at a partition level which allow you create a frozen copy of a logical partition at the snapshot point. In fact the LVM drivers use the same sort of tricks as VBox does with its snapshots, but this works at the partition level on the host OS. If the base partition is an ext3 FS, you then mount the snapshot as a readonly ext3 FS. You can then back this up at your leisure. When you are done, you then dismount the copy FS and delete the snapshot. The only reason that you need to take the VM down at all is to flush the FS to disk. One of the frequent posters here doesn't even bother doing that. He just creates a snapshot on the fly. If he then recovers from this, it's no worse than allowing NTFS to recover itself after a BSOD. Why not try googling LVM snapshot backup or the like and having a read?