How to migrate VMs from the old HD to a new HD

[satimis]

Hi all,

Host - Ubuntu 16.04 Gnome desktop on 1TB SSD
VMs - about 30 VMs also on the same SSD
VirtualBox also installed on the same SSD

The 1TB SSD is nearly full. I'm prepared purchasing a 2TB SSD.

I'll install Ubuntu 18.04 Gnome desktop on the new 2TB SSD as Host and the latest version of VirtualBox as well, to be download on Oracle VirtualBox website.

Please advise what will be the easy way to migrate all VMs to the new 2TB SSD from the old 1TB SSD. Thanks in advance.

Regards
satimis

[socratis]

Howtos and Tutorials » Generic Advice » Moving a VM.

[WFV]

i recently did this - replaced /home drive with a larger one. If you are familiar with rsync, assuming you have the new drive formatted and installed, rsync the entire contents of old /home/ to new drive /*you*/ - then rewrite fstab to mount the new drive for /home. The guests should all work then. There is a lot of information you leave out though, so follow the tutorials that fit your system build.
The example I give here is assuming / is on a separate drive, and that you are mounting the drives via fstab, and that you are working from a desktop (not laptop) pc that has room to add drives internally, although rsync can also be used for USB connected drives - I've found that hdd docking stations may be problematic with rsync, but haven't had any problems installing hdd's in usb cases and rsync'ng for external connections. Note /*you*/ on the new drive is your user folder, don't need a /home folder on new drive as /home is where you will mount the new drive once the copy is done and you verify the contents copied. All of my VMs worked soon as booted up computer with new /home drive. Didn't have to change anything in or do anything to VirtualBox. The example is this, format the new drive the way you need it, then mount it somewhere (in this example its mounted in /mnt/temp), then run rsync:

Code: Select all

rsync -aAXhv --delete --progress /home/<you>  /mnt/temp/

this will copy the entire /home/<you> directory to /mnt/temp/ whereas <you> might be "/home/satimis" - then unmount it when rsync completes, and do the rest to make it your new /home drive. If both / and /home are on that one SSD, then there are a few more operations that have to be performed to move /home to the new drive after the rsync step is finished.

[satimis]

Howtos and Tutorials » Generic Advice » Moving a VM.

Hi,

Thanks for your links

Regards
satimis

[satimis]

i recently did this - replaced /home drive with a larger one. If you are familiar with rsync, assuming you have the new drive formatted and installed, rsync the entire contents of old /home/ to new drive /*you*/ - then rewrite fstab to mount the new drive for /home. The guests should all work then. There is a lot of information you leave out though, so follow the tutorials that fit your system build.
The example I give here is assuming / is on a separate drive, and that you are mounting the drives via fstab, and that you are working from a desktop (not laptop) pc that has room to add drives internally, although rsync can also be used for USB connected drives - I've found that hdd docking stations may be problematic with rsync, but haven't had any problems installing hdd's in usb cases and rsync'ng for external connections. Note /*you*/ on the new drive is your user folder, don't need a /home folder on new drive as /home is where you will mount the new drive once the copy is done and you verify the contents copied. All of my VMs worked soon as booted up computer with new /home drive. Didn't have to change anything in or do anything to VirtualBox. The example is this, format the new drive the way you need it, then mount it somewhere (in this example its mounted in /mnt/temp), then run rsync:

Code: Select all

rsync -aAXhv --delete --progress /home/<you>  /mnt/temp/

this will copy the entire /home/<you> directory to /mnt/temp/ whereas <you> might be "/home/satimis" - then unmount it when rsync completes, and do the rest to make it your new /home drive. If both / and /home are on that one SSD, then there are a few more operations that have to be performed to move /home to the new drive after the rsync step is finished.

Hi,

Thanks for your detail advice and time spent helping me.

In the past I have done many VM migrations between PCs as well as export VMs and import them to another virtualizer, without much problem. Also I have run rsync quite often moving files/sub-directories/directories between PCs across network with following command

Code: Select all

rsync -r -a -v -e "ssh -l satimis" Transfer_Directory_folders/* ip_address:/path_to_remote_pc_folder_for_temp_storage/

But this time I have to migrate 30 VMs collectively in one time. Actually most VMs are for testing and learning IT such as AI, Machine learning in Python, Machine learning in R, Programming, C, Python, etc. Only a few VMs are for production such as cloned websites.

I have considered to dump all data from the old SSD to the new SSD but wonder whether it works on SSD/HD of different storage capacities? Even I succeed moving all VMs to the new SSD, still I have to re-create them one-by-one on the VM virtualbox-manager. Whether I have a wrong concept?

Regards
satimis

[WFV]

The quantity of VMs shouldn't matter so long as the new drive has enough room - I'm not sure I follow you as I didn't have to do anything in VirtualBox when I rsync'd /home to a new drive - so as mentioned, the example I gave is if you leave root on the original drive and have a separate drive for /home - but even if you move the whole enchilada to a new drive (root and home) - rsync will handle it (unless I'm unaware of rsync capacity limit?) - be patient - then edit your fstab (or whatever is mounting your OS) to the new UUID's and you should be good to go. 30 VMs is a lot, but it really comes down to how much data rsync can copy and if you have enough room point a to point b. Then the usual adjustments to make your new drive your OS. If you only want to move the VMs and nothing else, I'd rsync the

Code: Select all

../VirtualBox\ VMs

folder to new drive, then (assuming you have good backup also? or at least confirm the files exist on the new drive) delete the originals, and mount the new drive in /home/../VirtualBox\ VMs

[satimis]

Hi,

Thanks for your further advice.

It is a little bid complicate on this PC, a daily working PC with 8-cores, 32G RAM and dualboot system controlled with BIOS

SSD 1TB (say PC1) - for OS, VirtualBox and VMs
WD HD 2TB (say HD1)
1) for data storage of PC1
2) also for VMs of PC1
3) for Guests of PC2 below
Free space available about 1.1TB

SSD 250G (say PC2) - for OS and another virtualizer
WD HD 1.5TB (say HD2)
for data storage of PC2
Free space available about 1TB

I'm prepared purchasing a 2TB SSD
(Samsung 860 EVO Series MZ-76E2T0BW 2TB 2.5" SATA3 6Gb/s SSD)
and a 4TB WD HD
(WD Gold WD4002VYYZ 4TB SATA3 6Gb/s /128MB)

New configuration:
SSD 2TB (PC1) - for OS, VirtualBox and VMs
new WD 4TB HD
1) for data storage of PC1
2) for VMs, in case the 2TB SSD is full

SSD 250G (PC2) - for OS and another virtualizer
WD 2TB (HD1)
1) for guest storage
2) for data storage of PC2

The old 1TB SSD and old 1.5TB WD HD will be installed on a spare PC, with AMD 8-core CPU and 8G RAM, also running VirtualBox.

A few months ago I have an idea building a new AMD Ryzen 7 1700X box with 64G RAM to replace my daily working PC which has been running for about 10 years. Some folks on Internet advised me keep on using it until it stops working, no need to waste money. It is already a powerful PC.

I'll come back after purchasing the new 2TB SSD and 4TB WD HD. I won't delete anything on the old SSD and old HD until the new PCs are running without problem. Thanks

Regards
satimis

[WFV]

Hello satimis,
Yes that is complex system, so the 30 VMs are distributed across two pc's?
and you want to get them all on one hdd?
and looking at PC2 it is using something other than VirtualBox?
so, rsync can work over the network - you'd have to read man pages, maybe here if you want to put them all on one drive in the same directory (I've no experience rsync'ing over the network), and not use the "delete" operand in rsync on the first runs. And if you want them all to open in VirtualBox you might have to convert the "other virtualizer" images... however I'm thinking you want PC1 and PC2 to access the VMs as they do now only with all of the VMs on one drive (the new drive) - there are other options - if I were doing this, I'd partition the new drive in two, rsync the VirtualBox VMs folder contents into one partition, likewise with the "other virtualizer" folder contents into the other partition, delete the originals folders contents but leave the original folders intact - mount the new partitions in their original respective folders - you might still end up having to

Code: Select all

vboxmanage

resetting UUIDs of the VMs although I doubt it, and I have no idea what you'd have to do with the "other virtualizer" stuff. My system is nothing in comparison, FX8350, 32GB ram, one linux pc, 5 VirtualBox guests - so this really as far as I can go in my own experience (having moved these from drive to drive more than once in past 6+ yrs).

[satimis]

Hi all,

Now I have received the new Samsung 860 EVO 2TB SSD ordered previously.
(Samsung SSD 860 EVO SATA III 2.5 " 2TB (MZ-76E2T0BW))

I'll use it to replace the old 1TB SSD which has been running for about 10 years, almost fully occupied and leaving about 90GB free space.

Host - Ubuntu 18.04 gnome desktop (OS)
VirtualBox

1)
Installation of OS and Virtualbox

1a)
The installer will be on an USB drive. I suppose it doesn't need to format the new 2TB SSD manually. The installer will format it before installing the OS? If I'm wrong please correct me. Thanks

(Remark:
Sorry, I haven't built new PC for at lease 10 years)

1b)
I shall not manually create partitions on the 2TB SSD, except LVM created by the installer. The entire new SSD will be used for running OS, VirturalBox, VMs as well as for storage of daily working files. The old files will be stored on a 2TB WD Black Label HD on the same PC.

1c)
I shall follow the steps listed on;
Debian-based Linux distributions
https://www.virtualbox.org/wiki/Linux_D ... tributions

to install VirtualBox, download on Oracle website

I'll download and install "Oracle_VM_VirtualBox_Extension_Pack-5.2.16.vbox-extension pack" on;
https://www.virtualbox.org/wiki/Downloads

If I'm wrong please advise me. Thanks in advance

2)
For safety reason, before installation, I shall only connect the new 2TB SSD on the PC and all other drives will be disconnected.

Although following commands will list all HDs on the PC

Code: Select all

⟫ hwinfo --disk

and

Code: Select all

⟫ sudo fdisk -l

But to avoid making mistake, in acident, I prefer disconnecting all HDs leaving only the new 2TB SSD connected.

However I learned a bitter lessen before with this arrangement. All other drives after connected will not be mounted automatically at booting the PC. After booting up the PC, I have to mount them manually with following step;

On the (left vertical) menu
right-click (each drive) -> Open

Is there any solution?

3)
Migration of all VMs from the old 1TB SSD to the new 2TB SSD

Either
3a)
Export all VMs and import them to the VirtualBox Manager on the new 2TB SSD

OR
3b)
Copy/Paste or rsync all VMs including the folder "VirtualBox VMs" from the old 1TB SSD to the new 2TB SSD. Afterwards add the VMs to VirtualBox Manager manually, one by one?

On VirtualBox Manager
Machine -> Add

Which steps, 3a) or 3b) will be more reliable? Thanks

3c)
If selecting 3b) above, do I need to make any change on;
VM.vbox-prev
?

Please advise. Thanks

Any other advices would be appreciated.

Regards
satimis