Problem Cloning Oracle Linux 9.2 - lvm corrupted when cloning

[griff366]

Cloning is causing errors with devices and LVM

Steps to reproduce
Virtual Box 7.0.12 - Host is Windows 11 Pro
Created a brand new Oracle Linux 9.2 guest.

• 4CPU, 8192 GB memory, EFI bios (not secure)
• 200GB VDI drive - not fully allocated

Install OL 9.2 with defaults including Oracle Linux default LVM Partitioning.
Default LVM Partitioning for this size disk creates two logical volumes - "/" and "/home"
Create a admin user account with admin privileges (wheel)
Apply all available patches and install Virtual Box Guest Additions
Everything running fine and dandy....
Open a terminal and check lvm

Code: Select all

$ lsblk
NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda                 8:0    0   200G  0 disk 
├─sda1              8:1    0   600M  0 part /boot/efi
├─sda2              8:2    0     1G  0 part /boot
└─sda3              8:3    0 198.4G  0 part 
  ├─ol_ol9u2-root 252:0    0    70G  0 lvm  /
  └─ol_ol9u2-home 252:1    0 128.4G  0 lvm  /home

Code: Select all

$ sudo pvscan
  PV /dev/sda3   VG ol_ol9u2        lvm2 [198.41 GiB / 0    free]
  Total: 1 [198.41 GiB] / in use: 1 [198.41 GiB] / in no VG: 0 [0   ]

Code: Select all

$ sudo lvmdevices
  Device /dev/sda3 IDTYPE=sys_wwid IDNAME=t10.ATA_VBOX_HARDDISK_VB9465053e-2aab3486 DEVNAME=/dev/sda3 PVID=0mF9C4eeQPky4xzfHwDTd3iB7R82KO83 PART=3

Shut it down and clone it - full clone - new MAC addresses.
Boot up the clone - booting takes an unexpectedly long time.
Login screen shows up eventually after a couple of minutes
Trying to login as the admin user fails - no error - just fails
Login as root instead - this works
Open up a terminal and check lvm
Logical volume for "/home" is missing

Code: Select all

# lsblk
NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda                 8:0    0   200G  0 disk 
├─sda1              8:1    0   600M  0 part /boot/efi
├─sda2              8:2    0     1G  0 part /boot
└─sda3              8:3    0 198.4G  0 part 
  └─ol_ol9u2-root 252:0    0    70G  0 lvm  /

Code: Select all

# sudo pvscan
  Devices file sys_wwid t10.ATA_VBOX_HARDDISK_VB9465053e-2aab3486 PVID 0mF9C4eeQPky4xzfHwDTd3iB7R82KO83 last seen on /dev/sda3 not found.
  No matching physical volumes found

Code: Select all

# sudo lvmdevices
  Devices file sys_wwid t10.ATA_VBOX_HARDDISK_VB9465053e-2aab3486 PVID 0mF9C4eeQPky4xzfHwDTd3iB7R82KO83 last seen on /dev/sda3 not found.
  Device none IDTYPE=sys_wwid IDNAME=t10.ATA_VBOX_HARDDISK_VB9465053e-2aab3486 DEVNAME=/dev/sda3 PVID=0mF9C4eeQPky4xzfHwDTd3iB7R82KO83 PART=3

End Result
Have a bootable OS, that can be logged into. but trying to use any LVM tools returns error messages about missing device
System is therefore not safe to use, since LVM maintenance is not possible.

Unable to find any fixes and Workarounds
Searching online for fixes has failed so far.
I've also tried a couple of things to workaround it, but no joy

• Using Legacy BIOS instead of EFI - same result
• Using default 20GB disk instead of 200GB only creates 1 logical volume "'/" but pvscan and lvmdevices still give same error, so it's still corrupted and still not usable.

[scottgus1]

Cloning may change the UUIDs for the disks (all disks need unique UUIDs) and if the LVM in Linux uses the disk UUID as the pointer to the disk, then the changed UUID will up-end the LVM. Changing Linux to point to /dev/sd# or something else (I think there's something else, I'm no Linux guru) instead of disk UUIDs to find the disks may help get around this problem.

[griff366]

I've just tried cloning with option "Keep Hardware UUIDs" checked.
It doesn't make any difference.

[scottgus1]

I believe that setting only applies to main UUIDs like the "motherboard" which has to stay the same if software is to stay licensed and activated.

All individual disks need their own uniquid UUID, no getting around that. So a cloned disk will be required to have a different UUID.

I'm no Linux guru. What exactly is LVM complaining of, in Linux noob terms? What metadata does LVM use to find the disks and/or partitions?

[griff366]

During the boot this the step that fails

[ERROR] A start job is running for /dev/mapper/ol_ol9u2-home

So it is something to do with devices and mappings.

i.e. device mappings are getting lost/corrupted correctly during the cloning process.

This in turn causes a problem with LVM

[scottgus1]

Device mappings sounds like the LVM is still looking for the old UUIDs. You need to find the new drive UUIDs, or change how the devices are mapped.

You can find the drive UUIDs in the Virtual Media Manager, select the cloned drive, see the Information tab at the bottom, select the UUID, Copy to the clipboard, paste where needed.

I believe that is the UUID you need, but I may be wrong.

Or change the way the devices are mapped in LVM to \dev\sd# or the other method Linux has.

[griff366]

look at the contents of /dev/mapper

On original:

Code: Select all

$ ls -lAp
total 0
crw-------. 1 root root 10, 236 Nov  5 19:28 control
lrwxrwxrwx. 1 root root       7 Nov  5 19:28 ol_ol9u2-home -> ../dm-1
lrwxrwxrwx. 1 root root       7 Nov  5 19:28 ol_ol9u2-root -> ../dm-0

On clone:

Code: Select all

# ls -lAp
total 0
crw-------. 1 root root 10, 236 Nov  5 19:28 control
lrwxrwxrwx. 1 root root       7 Nov  5 19:28 ol_ol9u2-root -> ../dm-0

Look for /dev/dm-1 on the clone and it is missing - cloning process has lost it......

--

[scottgus1]

The contents of /dev/mapper are not something Virtualbox would read, let alone edit, during the clone process. Virtualbox never touches the contents of the disks. The disk UUID that gets changed is in Virtualbox's metadata headers for the disk, not in the disk sectors themselves.

In all likelihood LVM tried to interpret things and messed up the /dev/mapper file. You'll need to get help on an LVM forum on how to re-align the mapper to what exists in the clone.

[griff366]

UUID of Original's disk is: {9465053e-3ffc-498e-b41a-092e8634ab2a}
UUID of Clone's disk is: {44f01c45-de04-4dad-8b13-2b4968ab57f8}

What/where should I update in the Clone with the clone's UUID?

[scottgus1]

What/where should I update in the Clone with the clone's UUID?

You'll need to get help on an LVM forum on how to re-align the mapper to what exists in the clone.

Must have missed that last part.

[griff366]

I did.


I would say it's a problem in Virtual Box.
It updates the VDI disk image during the cloning process, and whatever it is doing is causing the problem.

LVM forums might help fix it, But I'm kind of ready to give up and wait for Oracle to notice and fix it.
I don't have any more time to spend on this.

Thank you for the assistance.

[scottgus1]

It's like when one moves: One changes their home address. But now they have to tell all their correspondents that they live at a new address. Or their mail doesn't show up.

Virtualbox does not edit the contents of the files on the disk. So it will not edit the mapper file to point to the new disks.

Whether this is something Virtualbox should do requires Virtualbox to be up on whatever addressing methods exist now or may exist in the future. Virtualbox does not adjust the addressing inside the VM's OS now.

You may post an enhancement ticket on the Bugtracker if you wish, to ask Virtualbox to edit the disk pointer files (and boy can I foresee the forum posts about that one...).

But the way to get around the problem now is, Get LVM's addressing to point at the new disk UUIDs. Or point it using the other methods used to find disks in Linux. Personally, I don't know how to do that.

[griff366]

One more thing.

It also happens with clone of Rocky Linux 9.2, but not with a clone Oracle Linux 8.8.

So, I'm going to assume is a generic problem affecting all variants of RedHat 9.

I'm going to have to drop back to RedHat/OL/Centos/Rocky Linux 8.

Either that or not use VirtualBox Clones or Appliances until it is fixed.

[griff366]

One more. one more thing.

I'm not even sure it is LVM that needs fixing - I think the LVM errors are a symptom.

[fth0]

FWIW, your issue is probably related to a known issue that was discussed a few weeks ago, see [SOLVED] After Clone virtual guest gets into emergency mode for the solution and a discussion of the technical details.