Backing up a running VM - AOS 503 (eCS) as guest

Discussions about using non Windows and Linux guests such as FreeBSD, DOS, OS/2, OpenBSD, etc.
Post Reply
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Backing up a running VM - AOS 503 (eCS) as guest

Post by massimo-ecs »

(mod edit: split from viewtopic.php?f=4&t=106559)
massimo-ecs wrote:the VM is of course under bkup using VSS technology (daily and weekly *full* bkups)
scottgus1 wrote:
massimo-ecs wrote:the VM is of course under bkup using VSS technology (daily and weekly *full* bkups)
If these backups are being done in the host OS while the VM is running, then they're corrupted and not reliable. Such a host backup of a running VM does not let the VM OS know it is being backed up, so the VM OS cannot prepare its databases, file writes, etc. Data will be corrupted and dirty.

The only safe methods to backup a VM are:

A. with the VM completely shut down, not save-stated. Then host backups will be OK, if all the VM files are backed up.

B. if the VM must be running, then the OS inside the VM must handle the backup using native or 3rd-party software installed inside the VM capable of telling the VM OS to prepare itself for live backup. Then save the backups out through a Bridged or Host-Only network onto other media besides the disk holding the VM's files. The VM's files still need to be backed up via method A every so often.

Please start a new topic if you want to discuss this more.
no, it use VSS service
and the bkups are not corrupted

thanks you
Attachments
Immagine 2022-07-27 235914.jpg
Immagine 2022-07-27 235914.jpg (53.5 KiB) Viewed 14666 times
Last edited by scottgus1 on 28. Jul 2022, 16:01, edited 3 times in total.
Reason: changed topic title
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Guru meditations with AOS 503 (eCS) as guest

Post by scottgus1 »

Per "Please start a new topic if you want to discuss this more." I split this off to a new topic. :D
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by scottgus1 »

You have some 3rd-party host software that claims to be able to back up a running VM. It mentions a "snapshot", which if it means what it says, it probably uses the good ol' process suggested many times on the forum:

take a VM snapshot, which closes the base disk.
copy the base disk
delete the snapshot

Now, whether this method is used or another method, trying to back up a running VM from the host will leave dirty data. The reason is that the host cannot tell the VM OS that it is going to be backed up, so the VM OS cannot prepare itself (*). The VM OS does not know that a snapshot has been taken, or that the disk it's writing to has actually been split to two disks, or some other method.

Here's an example: The VM writes the English alphabet to disk, during which the backup software backs up the VM:

abcdefghijklmno|pqrstuvwxyz

the pipe | is the point during the write where the backup software took the backup.

So what's in the backup?

abcdefghijklmno

The rest of the alphabet is missing: data is incomplete. A single missing bit will corrupt an important file. OS's do this kind of constant writing all the time. And databases need to be closed properly: that's why power cuts on computers can be so bad.

A snapshot-based or other non-VM-OS-aware host backup takes a copy of a VM that is writing along swimmingly, then all of a sudden cuts off with nothing else there. For all intents and purposes, it's just like one pulled the power on a computer then backed up the drive. Dirty databases, unfinished files, lost data. Sure, the VM OS may boot again, but the data being written when the backup was taken is lost, and you'll have no idea what was lost.

The live-backup-from-the-host backup routine is fraught with data-loss possibilities. It will bite you one day.

( * there is a possibility that host software "could" tell a VM OS that it's going to be backed up, if it has a utility installed inside the VM that uses communication through Guest Additions to tell the VM OS to start its Volume Shadow Service too. If the backup software does not have such a utility, or the VM does not have Guest Additions, then no such communication is happening, the VM OS does not know it's being backed up, and the VM's data will be split-written badly.)
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by massimo-ecs »

thanks you
but Handy bkup claim that it use VSS so it can copy the VMs live with consistency

anyway see that we are talking about a system with small VMs that runs on a Samsung pro 980 4th gen 16X PCIe NVME
that dumps on a RAID with Samsung pro SSDs

the bkup just takes some seconds, i really doubt that there could be corruption in the file systems

instead if you mean that a user upload an image via ftp while the bkup start..
it's the same that if he upload a file when his internet connection go down, the same situation
he will upload again the image some seconds later

about DB, data etc. i've also application side bkup, i mean mysql that close down, it get zipped and bkup on other storage, so
just in case i could get a db corrupted, i've the application bkup too
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by scottgus1 »

I wouldn't do it that way, because of the problems that can happen. But those problems are known now, so you can know what happened when the users have lost data.
massimo-ecs
Posts: 29
Joined: 25. Feb 2020, 22:48

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by massimo-ecs »

which kind of the explanation about application side bkup is not clear?

i have rsync, i've zip bkups daily, weekly and monthly etc. etc. etc.
i manage servers since 1994
i guess you are bit exaggerating

thanks you
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by scottgus1 »

It's your computer, and you can do what you want with it.
But trying to read someth
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Backing up a running VM - AOS 503 (eCS) as guest

Post by scottgus1 »

There, see? I was typing my post and the Submit button got clicked while I was typing. The same thing happens when the host reads the disk off to a "backup" while the VM is still writing to it. No amount of VSS or snapshotting on the host can prevent this.
Post Reply