What happens if trim command applied to differencing vdi?

[940607]

Hi.
I heard that if you use the "--nonrotational on --discard on" options, the "trim" command sent by guest makes the VDI file smaller. I don't understand what happens when snapshots are used.

Does the differencing vdi format support marking some areas as containing zeros without actually writing zeros?

[mpack]

None of the experienced users here use snapshots, so it will be difficult to answer that question. Back up the VM and try it. Presumably you're aware that it only works on SATA and SCSI virtual drives (not IDE).

p.s. I suspect you can't just enable the feature and trim. Trim will eliminate blocks marked for trimming by the guest OS as files are deleted. That marking process will not start until after the feature is enabled (and that's assuming that the guest OS starts using the feature right away: not all guests OS's understand SSD, and fewer still will expect a magnetic disk to suddenly transform into an SSD).

[940607]

None of the experienced users here use snapshots

Why?

[mpack]

Google for "snapshot problem site:forums.virtualbox.org".

[940607]

I installed RHEL68 minimal; initial size 1Gb. I made the first snapshot and booted the guest.

First I tried the following:

Code: Select all

dd if=/dev/urandom of=/bigfile bs=1M count=1000

The snapshot vdi size became 1Gb. Then I removed the file and issued the trim command:

Code: Select all

rm -f /bigfile
fstrim /

The snapshot vdi shrunk to 20Mb.

After that I re-created /bigfile, but then made another snapshot. Removed the file and issued `fstrim` again. As expected, none of the vdi files shrunk this time. The total size of the VM was 2Gb.

Then I created the full clone of the VM's current state. Unfortunately, the size of the clone was also 2Gb.

A quick glance on the dynamic VDI file format explains that: the block map holds the offsets. (-1) means "unallocated" in root image or "inherited" in a differencing image. There's no special value (e.g. -2) to indicate a discarded block.

Looks like somebody had the same thoughts and proposed an extension to the vdi file format:
https://github.com/hakito/SlimVDI/blob/ ... ucts.h#L35

[ChipMcK]

None of the experienced users here use snapshots

Why?

Additional reason

When a virtual disk is first created for a new virtual machine, it is considered as the base disk for the guest - data for the guest is read from and written to that disk image.

The differencing disk records changes sector-by-sector to the whole disk image, not changes to any file in the disk. VirtualBox does not know what file system is employed on the disk image and therefore can not access any individual file of/on the disk image; only the guest OS is aware of that information.

First SnapShot creates a differencing disk for read/write access while the base disk becomes read-only - as the guest modifies its data, the data is written to the differencing disk and the base disk is untouched.

Second SnapShot creates another, new, differencing disk for read/write access while the first differencing disk becomes read-only along with the base disk.

Subsequent SnapShots create additional differencing disks, with the preceding differencing disk joining the hierarchy (pecking order/chain) of read-only disks.

Keep in mind that access to/from the virtual disks is sector-by-sector, not file-by-file.

When the guest requests that a sector be read, the latest SnapShot is read first. If the sector is not found there (Sector-Not-Found is returned), the next SnapShot in the chain (youngest to oldest) is read, until the base virtual disk is reached. Then the sector on/in the base virtual disk is either read or Sector-Not-Found is returned.

[mpack]

Looks like somebody had the same thoughts and proposed an extension to the vdi file format:
https://github.com/hakito/SlimVDI/blob/ ... ucts.h#L35

Eh? That's my code (or someone's unchanged fork of it), and I wasn't proposing any format change. What made you think it did?

The VDI_PAGE_ZERO magic value isn't a proposal, it documents an existing flag which is used when a block is all zeros and hence doesn't need to be stored in the file. This marker is created by "VBoxManage clonehd" (and CloneVDI) when it notices that a block it was about to copy is filled with zeros.

Filled with zeros doesn't mean that the block has been deallocated! A zero byte is perfectly valid data.

[940607]

I see. The current implementation is Deterministic TRIM (DRAT): All read commands to the LBA after a TRIM shall return the same data.
There's an alternative: Deterministic Read Zero after TRIM (RZAT): All read commands to the LBA after a TRIM shall return zero. Both are legal.

Filled with zeros doesn't mean that the block has been deallocated!

Would it harm if it did? I mean, if VirtualBox wrote (-2) instead of (-1) to the block map as the response to the TRIM command there would be RZAT instead of DRAT.

Cloning such differencing VDI would create an initially compacted image.

[mpack]

Filled with zeros doesn't mean that the block has been deallocated!

Would it harm if it did?

Not meaningful: the block has already been trimmed during the clone operation and replaced by that marker, so there is nothing else to do.

[940607]

the block has already been trimmed during the clone operation and replaced by that marker

This only happens if reading from a source image returns a zero block. However, trimming a block from a guest doesn't make it look like a zero block. Instead, it starts to return data from the parent image.

[mpack]

I'm losing track of what your proposal is. Yes, if all sectors in an 1MB block were zeroed then it would be possible to replace the blockmap entry with the flag. However what do you propose is done with the block? [belay that: the existing trim mechanism handles that]. And why would the sectors have been zeroed? Are you proposing modifying the guest OS to zero fill every deleted file?

[940607]

I propose marking trimmed blocks with -2 instead of -1. Reading such blocks will returns zeros instead of parent data. The advantage is that a clone of such differencing image will have smaller size, because it won't contain parent data.