How would I shrink a macOS guest VDI?
-
- Posts: 74
- Joined: 14. Nov 2014, 18:54
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: OPENSTEP, Mac OS X, FreeBSD, Windows
How would I shrink a macOS guest VDI?
Hi there,
I have a virtual macOS installation (10.12.6) which uses 36,8 GB of it's hard disk space (VDI is of type "dynamically growing" until 2TB) but the VDI now has a size of 74GB, since there was once more stuff (older Xcode betas) on the disk which I deleted.
Unfortunately deleting stuff inside a VDI does not shrink it in size. So I googled how to do this, but found only how tos for Windows or Linux guests.
How would I defragment the VDI and erase the free space (can this be done using disk utility inside the guest?)?
thanks in advance,
IOOI
I have a virtual macOS installation (10.12.6) which uses 36,8 GB of it's hard disk space (VDI is of type "dynamically growing" until 2TB) but the VDI now has a size of 74GB, since there was once more stuff (older Xcode betas) on the disk which I deleted.
Unfortunately deleting stuff inside a VDI does not shrink it in size. So I googled how to do this, but found only how tos for Windows or Linux guests.
How would I defragment the VDI and erase the free space (can this be done using disk utility inside the guest?)?
thanks in advance,
IOOI
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
Avoid Google, use the User Manual .IOOI wrote:So I googled how to do this, but found only how tos for Windows or Linux guests.
Ch. 8.23. VBoxManage modifymedium:
I had to ping the manual maintainers a couple of times to change that, because Apple kept changing the way they're doing things, but I found that this command works from 10.4 to even the 10.13b.For Mac OS X guests, use the "diskutil secureErase freespace 0 /" command line from an elevated Terminal.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Site Moderator
- Posts: 39134
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: How would I shrink a macOS guest VDI?
Shrinking the partition (with a guest partition manager) will limit how large the VDI can grow, as it only grows when disk sectors are first used.IOOI wrote: Unfortunately deleting stuff inside a VDI does not shrink it in size.
-
- Posts: 72
- Joined: 15. Oct 2019, 22:11
Re: How would I shrink a macOS guest VDI?
I would like to warn readers of this particular thread NOT to attempt using the following, at least not on 10.14.x Mojave:
"diskutil secureErase freespace 0 /"
This hoarked my Mojave .vdi completely.
It may be that the issue is the volume is APFS, and using / is simply too much -- perhaps you have to specify something like /dev/disk1, or /dev/disk0s1.
In any case, take great care!
"diskutil secureErase freespace 0 /"
This hoarked my Mojave .vdi completely.
It may be that the issue is the volume is APFS, and using / is simply too much -- perhaps you have to specify something like /dev/disk1, or /dev/disk0s1.
In any case, take great care!
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
Do you have an alternative solution? I haven't installed 10.14 yet, so I haven't studied the ins and outs of APFS. Is this another case where Apple screwed up again? As usual?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Posts: 72
- Joined: 15. Oct 2019, 22:11
Re: How would I shrink a macOS guest VDI?
No, APFS is definitely NOT a screw-up. It's a major improvement on HFS+, in terms of performance, security, and reliability.
It is, however, optimized for SSDs. So typically, you would never ever need to do a secure erase of unused space, because the built-in encryption is plenty of security. But repeatedly writing zeros to an SSD would wear out the SSD, so I think that option is normally not available.
But you CAN do it with diskutil.
I think the idea of using the volume root "/" with diskutil secureErase is just too dangerous -- I think that might include all those special APFS component / partitions that are magically stitched together.
But without doing that, I don't know how the volume can be compacted.
Stephen
It is, however, optimized for SSDs. So typically, you would never ever need to do a secure erase of unused space, because the built-in encryption is plenty of security. But repeatedly writing zeros to an SSD would wear out the SSD, so I think that option is normally not available.
But you CAN do it with diskutil.
I think the idea of using the volume root "/" with diskutil secureErase is just too dangerous -- I think that might include all those special APFS component / partitions that are magically stitched together.
But without doing that, I don't know how the volume can be compacted.
Stephen
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
APFS was definitely a screw-up when introduced. Untested, unproven, bugs riddled.
BTW, I don't know how many times I have to say that: a 10.14/15 guest is NOT supported.
Please tell that to my external hard drive, that it should upgrade itself to an SSD...SteveMacGeek wrote:It is, however, optimized for SSDs
You actually didn't tell how...SteveMacGeek wrote:But you CAN do it with diskutil.
Then someone hasn't done their homework... hint: Apple.SteveMacGeek wrote:I think the idea of using the volume root "/" with diskutil secureErase is just too dangerous
BTW, I don't know how many times I have to say that: a 10.14/15 guest is NOT supported.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Posts: 72
- Joined: 15. Oct 2019, 22:11
Re: How would I shrink a macOS guest VDI?
APFS was road-tested on iPhone and Apple watch for a couple years before being introduced on macOS. The biggest issue with the introduction of APFS on macOS was the forced conversion of Apple's hybrid drives (part SSD, part hard disk). In my own development I had to accommodate the changes in API due to Apple's changing / eliminating API calls that it had deprecated years earlier. So, like me, a lot of other software vendors may have been caught flat-footed.APFS was definitely a screw-up when introduced. Untested, unproven, bugs riddled.
Sorry I wasn't clear. What I meant was, diskutil will allow you to execute a secure freespace command from the terminal. It's just...deadly as proposed in section 8.24:You actually didn't tell how...
Code: Select all
diskutil secureErase freespace 0 /
OR -- maybe Section 8.24 should be revised to say "Don't try this on macOS 10.14 or greater"Then someone hasn't done their homework... hint: Apple.
Do you not want forum members to explicitly not post questions for 10.14 Guests?BTW, I don't know how many times I have to say that: a 10.14/15 guest is NOT supported.
Stephen
-
- Volunteer
- Posts: 5677
- Joined: 14. Feb 2019, 03:06
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Linux, Windows 10, ...
- Location: Germany
Re: How would I shrink a macOS guest VDI?
IMO the example command in the VirtualBox User Manual is simply wrong ok*. and should be corrected. The example commands for the other OSs do not specify the (necessary) disk device, so the easiest change would be to simply remove the trailing '/', so that none of the commands work right out of the box.
*The commands all expect a device parameter. The file system root '/' could either be interpreted as no device or as all devices mounted somewhere in the file system hierarchy. The consequence would be either no operation taking place, or all devices being affected. I think both are usually not wanted. The device parameter can be given in form of a mount point, so the file system root '/' specifies the device mounted at this mount point.
*The commands all expect a device parameter. The file system root '/' could either be interpreted as no device or as all devices mounted somewhere in the file system hierarchy. The consequence would be either no operation taking place, or all devices being affected. I think both are usually not wanted. The device parameter can be given in form of a mount point, so the file system root '/' specifies the device mounted at this mount point.
Last edited by fth0 on 27. Oct 2019, 23:55, edited 1 time in total.
-
- Posts: 72
- Joined: 15. Oct 2019, 22:11
Re: How would I shrink a macOS guest VDI?
@fth0 - agreed. So the question is, where is the main device for the user's boot volume on APFS?The file system root '/' could either be interpreted as no device or as all devices mounted somewhere in the file system hierarchy. The consequence would be either no operation taking place, or all devices being affected. I think both are usually not wanted.
Perhaps a safer way to zero out the sector data would be a more low-rent app that doesn't try to use the system hard disk utility to do its work. I recall one such app -- years ago -- that simply kept allocating a sectors in a huge file, then wrote out zeros to that file, then deleted the file. While that might not be enough for a "secure" erase on a non-ssd hard disk, it should do the job for he purposes of allowing VirtualBox to compact the filesystem space.
Stephen
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
[color=#00AA00]socratis[/color] wrote:a 10.14/15 guest is NOT supported.
[color=#105289]SteveMacGeek[/color] wrote:OR -- maybe Section 8.24 should be revised to say "Don't try this on macOS 10.14 or greater"
[color=#105289]SteveMacGeek[/color] wrote:Do you not want forum members to explicitly not post questions for 10.14 Guests?
The "not supported" part means that the User Manual stays "as is". You can't have instructions on the User Manual for something that's not supported. If you find a proper workaround/solution, and 10.14/15 become officially supported, then the User Manual may be corrected to reflect that aspect.[color=#3366CC]fth0[/color] wrote:IMO the example command in the VirtualBox User Manual is simply wrong* and should be corrected.
How do I know? Because that's exactly what happened 7 years ago, and amended 3 years later, after Apple once again changed their tune: #11175: Add documentation for zeroing Mac OS X guests before using the 'modifyhd --compact' operation...
Be sure that I'm going to be the first that wakes that ticket again, once a proper solution is found. Since I haven't installed 10.14/15 yet, I can't personally do the testing. It's on my "to-do list" for the longest of times...
@SteveMacGeek
You already opened a new thread about this, Strategies for compressing huge .vdi for upload/download, which practically talks about the same thing. Can you focus on that thread and leave this one (that was resolved in 2017) sleep? You gave a warning for 10.14 guests, fine. But let's keep the discussion about the way to compress an OSX 10.14 VDI in your new, more relevant thread.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Volunteer
- Posts: 704
- Joined: 3. Mar 2015, 07:27
- Primary OS: Mac OS X other
- VBox Version: OSE other
- Guest OSses: Linux, macOS, Windows
Re: How would I shrink a macOS guest VDI?
I have just manged to compact a Mojave boot drive using the following commands.
I kept a copy of the original disk before doing any of this - before/after comparisons below
Following this - I ran the VM (using the compacted disk) and it 'just worked'.
- Run the VM and login as the admin user
- Use Disk Utility to find the device ID of the volume you wish to compact - please note - you need the volume ID, not the whole disk
2a. Alternatively use 'diskutil list' from the command line to find the device ID of the volume - In the Terminal, 'sudo su -' to give yourself elevated privileges.
- diskutil secureErase freespace 0 "device ID from 2 - without quotes"
- Wait until this completes - it may take some time depending on the type of disk (SSD vs HDD) and the size of the disk
- Shutdown the VM cleanly - i.e. a complete shutdown and not "saved state" or "power off"
- In the VirtualBox Manager, right click the VM entry in the left column and select 'Show in Finder'. This will open a Finder window showing your VM.
- Open Terminal - do NOT elevate privileges
- Type 'VBoxManage modifymedium --compact ' then drag the .vdi file from the Finder window to complete the command. Please note that there should be a space character between 'compact' and the beginning of the file name. The normal rules of escaping spaces in paths are done automatically if you use drag and drop. If you choose to type the path manually, please remember to take into account any spaces in the path.
- Press 'return' and wait until the command completes.
I kept a copy of the original disk before doing any of this - before/after comparisons below
Code: Select all
bryan@number9 BuildInkscape % VBoxManage modifymedium --compact ./BuildInkscape.vdi
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
bryan@number9 BuildInkscape % ls -l
total 136300576
-rw------- 1 bryan staff 4299 28 Oct 08:49 BuildInkscape.vbox
-rw------- 1 bryan staff 4299 25 Oct 10:35 BuildInkscape.vbox-prev
-rw------- 1 bryan staff 26684162048 28 Oct 08:59 BuildInkscape.vdi
-rw-------@ 1 bryan staff 43088084992 24 Oct 08:49 BuildInkscape.vdi.save.vdi
drwx------ 6 bryan staff 192 28 Oct 08:24 Logs
bryan@number9 BuildInkscape %
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
granada29 wrote:Use Disk Utility to find the device ID of the volume you wish to compact - please note - you need the volume ID, not the whole disk
granada29 wrote:That's it. It seems the documentation in Chapter 8 is misleading and it is much safer to specify the deviceID rather than the root path "/"
- It's not misleading because ... it works with ALL supported OSX guests.
- If this finding of yours works with previous guests, from 10.16[1] 10.6 to 10.13, and it works for 10.14/15 as well, then it's something to definitely consider! It's going to make the instructions in the User Manual a tad more awkward, but who cares...
I take it that the ".vdi.save.vdi" is the original VDI. That's quite some savings there!granada29 wrote:-rw------- 1 bryan staff 26684162048 28 Oct 08:59 BuildInkscape.vdi -rw-------@ 1 bryan staff 43088084992 24 Oct 08:49 BuildInkscape.vdi.save.vdi
[1]: Thanks to 'rpmurray' for spotting the typo!
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Site Moderator
- Posts: 27329
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: How would I shrink a macOS guest VDI?
I think I'm going to reverse that decision, since there's no actual back and forth in that other, newer thread. I'm going to lock the other thread, and redirect people here.socratis wrote:You already opened a new thread about this, Strategies for compressing huge .vdi for upload/download, which practically talks about the same thing. Can you focus on that thread and leave this one (that was resolved in 2017) sleep? You gave a warning for 10.14 guests, fine. But let's keep the discussion about the way to compress an OSX 10.14 VDI in your new, more relevant thread.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Posts: 72
- Joined: 15. Oct 2019, 22:11
Re: How would I shrink a macOS guest VDI?
@granada29 -- So -- to be clear -- in the following diskutil list, APFS Volume Prezdrive is my mounted boot volume. Its Device ID is disk1s1 -- what Disk Utility refers to as the "BSD device node", correct?
Stephen
Code: Select all
$ diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_APFS Container disk1 1.0 TB disk0s2
/dev/disk1 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +1.0 TB disk1
Physical Store disk0s2
1: APFS Volume Prezdrive 925.2 GB disk1s1
2: APFS Volume Preboot 42.8 MB disk1s2
3: APFS Volume Recovery 510.4 MB disk1s3
4: APFS Volume VM 9.7 GB disk1s4