Can I compact VDI this way?
-
- Posts: 18
- Joined: 24. Aug 2015, 21:02
Can I compact VDI this way?
I recently made the rash error of increasing the size of one of my guest VDIs due to the 5.2.6 GA bug.
Long story short: I don't want the extra space anymore.
From what I've read online, folks have made use of a software known as Zerofree.
Is is not possible to simply shrink the main partition via GParted (leaving unallocated free space on the drive) and run vboxmanage modifymedium disk.vdi -compact?
I'm new to this procedure so I'd like some feedback (please).
TIA
- T.R.G.
Long story short: I don't want the extra space anymore.
From what I've read online, folks have made use of a software known as Zerofree.
Is is not possible to simply shrink the main partition via GParted (leaving unallocated free space on the drive) and run vboxmanage modifymedium disk.vdi -compact?
I'm new to this procedure so I'd like some feedback (please).
TIA
- T.R.G.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Can I compact VDI this way?
First a question: why did you increase the size as a response to the 5.2.6 GAs issue?
Second, your thinking is almost correct, but I would do the following:
Second, your thinking is almost correct, but I would do the following:
- Run the "zerofree" command using a live CD like "SystemRescueCd"
- Run the "VBoxManage modifymedium "<YourVDI> --compact" on your host.
- Resize/shrink the partition with GParted.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Posts: 18
- Joined: 24. Aug 2015, 21:02
Re: Can I compact VDI this way?
To answer the question:
Running Linux Mint, I attempted to logon and was greeted with a message displaying possible causes for the error. One of them happened to be insufficient disk space. I purposely created the VDI to hold just the minimum amount of disk space necessary for my purposes (learning). The reason I thought of increasing the disk size was because I recently encountered an issue where I literally ran out of disk space for normal operation and was forced to use GParted to reallocate the newly added space on the drive. I clearly wasn't thinking correctly this time around as I knew that I hadn't taken up the entirety of the drive upon installation of the buggy 5.2.6 GA. For whatever reason, I panicked and decided to give the increase a go. After that had failed, I came on here and started looking to see if anyone had the same issue. Eventually, I was able to download the more recent GA and resume things normally.
I was really out of it yesterday...
Back to my question:
I do not yet have Zerofree installed on my guest machine. From what you've mentioned thus far, I can assume I don't need to as long as I use that particular live-cd?
May I ask why GParted wouldn't be run before shrinking the drive?
Running Linux Mint, I attempted to logon and was greeted with a message displaying possible causes for the error. One of them happened to be insufficient disk space. I purposely created the VDI to hold just the minimum amount of disk space necessary for my purposes (learning). The reason I thought of increasing the disk size was because I recently encountered an issue where I literally ran out of disk space for normal operation and was forced to use GParted to reallocate the newly added space on the drive. I clearly wasn't thinking correctly this time around as I knew that I hadn't taken up the entirety of the drive upon installation of the buggy 5.2.6 GA. For whatever reason, I panicked and decided to give the increase a go. After that had failed, I came on here and started looking to see if anyone had the same issue. Eventually, I was able to download the more recent GA and resume things normally.
I was really out of it yesterday...
Back to my question:
I do not yet have Zerofree installed on my guest machine. From what you've mentioned thus far, I can assume I don't need to as long as I use that particular live-cd?
May I ask why GParted wouldn't be run before shrinking the drive?
Last edited by socratis on 19. Jan 2018, 20:29, edited 1 time in total.
Reason: Removed unnecessary verbatim quote of the whole previous message.
Reason: Removed unnecessary verbatim quote of the whole previous message.
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Can I compact VDI this way?
I see no "shrink drive" instruction, whether before or after GParted. In fact VirtualBox has no such feature, because of the obvious destructive potential.Random_Consultant wrote:May I ask why GParted wouldn't be run before shrinking the drive?
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Can I compact VDI this way?
You don't want 'zerofree' installed in your guest. In fact I'm not sure if 'zerofree' can work on a drive that it is located on and it is the boot drive. That's why I said run it from a live CD, because your guest's drive can be modified at will.Random_Consultant wrote:I do not yet have Zerofree installed on my guest machine. From what you've mentioned thus far, I can assume I don't need to as long as I use that particular live-cd?
Just like mpack said, no drive shrinking, only partition "rearranging". And I'm not sure what you mean running X before X (does what it does). Doesn't compute...Random_Consultant wrote:May I ask why GParted wouldn't be run before shrinking the drive?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Posts: 18
- Joined: 24. Aug 2015, 21:02
Re: Can I compact VDI this way?
Good catch!mpack wrote:I see no "shrink drive" instruction, whether before or after GParted. In fact VirtualBox has no such feature, because of the obvious destructive potential.Random_Consultant wrote:May I ask why GParted wouldn't be run before shrinking the drive?
Also, thanks for clarifying. In my mind, that is what it was doing; that is why I asked what I did.
I wasn't sure how it was doing the "compacting".
-
- Posts: 18
- Joined: 24. Aug 2015, 21:02
Re: Can I compact VDI this way?
I'm wondering why the VDI can be compacted before using GParted. Why is the recommended procedure compacting the VDI before running GParted? Is compacting not shrinking the VDI?socratis wrote:Just like mpack said, no drive shrinking, only partition "rearranging". And I'm not sure what you mean running X before X (does what it does). Doesn't compute...Random_Consultant wrote:May I ask why GParted wouldn't be run before shrinking the drive?
I ask because to me, that does not compute.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Can I compact VDI this way?
Let me "sort of" draw the process. Warning: Oversimplification.
In the following ASCII-art representation of the state of the VDI:
In the following ASCII-art representation of the state of the VDI:
- the letters are occupied sectors,
- the spaces " " are empty/unallocated sectors,
- the dashes "-" are sectors that were allocated at some point, but not anymore,
- the stars "*" are sectors that have been marked as "zeroed out" from zerofree/sdelete/other,
- and finally, the "||" is where you divide your disk with GParted/DiskUtility to have a smaller accessible partition.
After running 'zerofree/sdelete/other' all the previously used sectors are marked as zero sectors:+-----------------------------------------------------------------+ | l d --- od--- skkjs gsosgg ds dl k sl ks f po ss-- | | j s k l---j--sd v k sd j s dl kj s d sd g jk ls d js | | d l---s j o ; s d j lk-s---j-sl---a-s al sd sd hk l as 5 | | has d h ig----w 2 4 pu t g df b j; af s j--as-l-as--b | +-----------------------------------------------------------------+
After running 'VBoxManage modifymedium "<VDI>" --compact' all the zero sectors are emptied:+-----------------------------------------------------------------+ | l d *** od*** skkjs gsosgg ds dl k sl ks f po ss** | | j s k l***j**sd v k sd j s dl kj s d sd g jk ls d js | | d l***s j o ; s d j lk*s***j*sl***a*s al sd sd hk l as 5 | | has d h ig****w 2 4 pu t g df b j; af s j**as*l*as**b | +-----------------------------------------------------------------+
After shrinking the partition with GParted/DiskUtility all the data are moved to the front, and they are restricted onto their own, smaller partition, and you don't allow anything to access the "No Access" partition:+-----------------------------------------------------------------+ | l d od skkjs gsosgg ds dl k sl ks f po ss | | j s k l j sd v k sd j s dl kj s d sd g jk ls d js | | d l s j o ; s d j lk s j sl a s al sd sd hk l as 5 | | has d h ig w 2 4 pu t g df b j; af s j as l as b | +-----------------------------------------------------------------+
+---------------------------------------++------------------------+ | ldodskkjsgsosggdsdlkslksfposs || | | jskljsdvksdjsdlkjsdsdgjklsdjs || N O A C C E S S | | dlsjo;sdjlksjslasalsdsdhklas5 || | | hasdhigw24putgdfbj;afsjaslasb || | +---------------------------------------++------------------------+
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Can I compact VDI this way?
Q. What does zerofree do?
A. It can be used to zero-fill unused parts of a selected filesystem.
Q. What does VBoxManage compact do?
A. It deallocated zero blocks from the VDI (1MB blocks filled with zeroes).
Q. What does GParted do?
A. It changes what fraction of the disk is covered by a filesystem.
Note that GParted doesn't shrink or erase anything, it just writes a new data structure on the drive.
So now :-
Q. What would happen if I shrank a partition using GParted before running zerofree?
A. Then the filesystem would cover less of the disk when you ran zerofree, the remainder would be unreachable to zerofree, and those high blocks would hence not be zeroed and not be removed by VBoxManage compact.
p.s. The recommended procedure is not ideal either. Running gparted after zerofree causes data to be written to some of the blocks you just zeroed, lowering the effectiveness of the compaction step. It's a limitation of the official approach. The unofficial approach is to use CloneVDI, which doesn't require that you run zerofree, and doesn't care at which point you run GParted, because it discards unused blocks from the whole drive, not zero blocks from one part of it. CloneVDI will also be very much faster.
A. It can be used to zero-fill unused parts of a selected filesystem.
Q. What does VBoxManage compact do?
A. It deallocated zero blocks from the VDI (1MB blocks filled with zeroes).
Q. What does GParted do?
A. It changes what fraction of the disk is covered by a filesystem.
Note that GParted doesn't shrink or erase anything, it just writes a new data structure on the drive.
So now :-
Q. What would happen if I shrank a partition using GParted before running zerofree?
A. Then the filesystem would cover less of the disk when you ran zerofree, the remainder would be unreachable to zerofree, and those high blocks would hence not be zeroed and not be removed by VBoxManage compact.
p.s. The recommended procedure is not ideal either. Running gparted after zerofree causes data to be written to some of the blocks you just zeroed, lowering the effectiveness of the compaction step. It's a limitation of the official approach. The unofficial approach is to use CloneVDI, which doesn't require that you run zerofree, and doesn't care at which point you run GParted, because it discards unused blocks from the whole drive, not zero blocks from one part of it. CloneVDI will also be very much faster.
-
- Posts: 228
- Joined: 1. Jan 2017, 09:16
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Ubuntu 16.04 x64, W7
Re: Can I compact VDI this way?
It can, but it requires the system to be in single-user mode (ie init 1) and the drive to be mounted ro. (viewtopic.php?f=3&t=82623 has the details). As you say though, it's a lot easier to just use a live CD for stuff like this, especially if you're going to use tools like GParted as well anyway.socratis wrote:You don't want 'zerofree' installed in your guest. In fact I'm not sure if 'zerofree' can work on a drive that it is located on and it is the boot drive.
(and as mpack says, CloneVDI is the best option of all, provided you can run it).
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Can I compact VDI this way?
Pretty much everyone can, as it runs natively on Windows hosts, and is just as happy under Wine on Linux and OS X hosts.arQon wrote:CloneVDI is the best option of all, provided you can run it).
-
- Posts: 18
- Joined: 24. Aug 2015, 21:02
Re: Can I compact VDI this way?
@mpack @socratis
Thanks guys!
Thanks guys!
-
- Posts: 57
- Joined: 27. Apr 2008, 15:19
Preparing to compact: FreeBSD-CURRENT host
With FreeBSD-CURRENT, I seem to gain free space through preparing to compact virtual disk images.
Today, for example:
… .vdi reduced from 63.58 GB to 39.73 GB but no change in the amount of free space.
Maybe a peculiarity of the host system using ZFS.
Today, for example:
- I recall having 160-something G free before booting systemrescuecd-amd64-6.1.2.iso and running zerofree
- I have 180G free after the run of zerofree.
Code: Select all
grahamperrin@momh167-gjp4-8570p:~ % date ; uname -v
Sun 29 Mar 2020 19:01:16 BST
FreeBSD 13.0-CURRENT #51 r357746: Tue Feb 11 03:54:19 GMT 2020 root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG
grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' virtualbox-ose virtualbox-ose-kmod
emulators/virtualbox-ose 5.2.34_1 FreeBSD
emulators/virtualbox-ose-kmod 5.2.34 poudriere
grahamperrin@momh167-gjp4-8570p:~ % zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
copperbowl 448G 268G 180G - - 55% 59% 1.00x ONLINE -
grahamperrin@momh167-gjp4-8570p:~ % time VBoxManage modifymedium --compact 1e3306a9-3506-4bcc-9f98-d1cece4293d3
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
8.205u 4.794s 35:06.61 0.6% 936+176k 24+0io 10pf+0w
grahamperrin@momh167-gjp4-8570p:~ % zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
copperbowl 448G 268G 180G - - 55% 59% 1.00x ONLINE -
grahamperrin@momh167-gjp4-8570p:~ %
Maybe a peculiarity of the host system using ZFS.
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Can I compact VDI this way?
Neither zerofree nor compaction gives you more free space.
"zerofree" literally means to zero already free sectors, so how could that add free sectors?
Compaction is a host operation, the guest shouldn't be affected at all (not unless it was "hiding" data in unused parts of the drive).
"zerofree" literally means to zero already free sectors, so how could that add free sectors?
Compaction is a host operation, the guest shouldn't be affected at all (not unless it was "hiding" data in unused parts of the drive).