Handling of encryption of individual virtual disks

[scottgus1]

Are you really sure that the KeyStore can be saved in the VirtualBox.xml file?

Fairly sure. I saw this thread in its early days and tried encrypting an unattached vdi with the vboxmanage command, for I was sure that soon as the vboxsvc died off the encryption key would die off, too, like I had also previously found here: viewtopic.php?f=1&t=83217#p393802. However, this time the key didn't disappear. So I hunted it up and found the keys in the XML file.
I tried it with 6.0.something, though, not 5.2.#, that might be the difference.
That host is offline now, but I'll check up on it in the morning and respond back with what I have.

[socratis]

I tried it with 6.0.something, though, not 5.2.#, that might be the difference.

Yes, that is the difference.

[scottgus1]

The devs responded to a bugtracker ticket: https://www.virtualbox.org/ticket/16784 that they saw something needed to be done about what happens when a disk is encrypted with vboxmanage, but it would take them a bit of time. In 6.0 they got it figured out.

So I tried it again, using 6.0.14 and the vboxmanage command to encrypt a disk file. Here are my Virtualbox.xml & .xml-prev, to show what happened.

In .xml-prev, this is the "Media Registry" tag:

[quote]    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{f3c21c4c-28cb-4644-a8ce-99bc52efa798}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS 6.22 A/3DOS-disk001.vdi" format="vdi" type="Normal"/>
        <HardDisk uuid="{566ea56c-92e7-465b-b980-1deaf7e2cd7f}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS 6.22 A Clone/3DOS-disk002.vdi" format="vdi" type="Normal"/>
        <HardDisk uuid="{ee8e28d5-161c-4b5b-8843-dfc5ccaa5c14}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS_6.22_B/3DOS-disk003.vdi" format="vdi" type="Normal"/>
      </HardDisks>
    </MediaRegistry>[/quote]

The main Virtualbox window was closed and no Virtualbox processes were running.
I made a clone of a vdi using CloneVDI, so Virtualbox would have no previous knowledge of the new vdi. I did not attach the vdi to any guest.
I ran the 'VBoxManage encryptmedium' command on the new cloned vdi. Here is what Virtualbox.XML now looked like:

[quote]    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{f3c21c4c-28cb-4644-a8ce-99bc52efa798}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS 6.22 A/3DOS-disk001.vdi" format="vdi" type="Normal"/>
        <HardDisk uuid="{566ea56c-92e7-465b-b980-1deaf7e2cd7f}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS 6.22 A Clone/3DOS-disk002.vdi" format="vdi" type="Normal"/>
        <HardDisk uuid="{ee8e28d5-161c-4b5b-8843-dfc5ccaa5c14}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS_6.22_B/3DOS-disk003.vdi" format="vdi" type="Normal"/>
        <HardDisk uuid="{5b87f1c1-720e-4dc6-a26e-47a730e455a3}" location="C:/Users/Public/VirtualBox VMs/DOS/DOS 6.22 A/doscloneenc.vdi" format="VDI" type="Normal">
          <Property name="CRYPT/KeyId" value="testenc"/>
          <Property name="CRYPT/KeyStore" value="U0NORQABQUVTLVhUUzI1Ni1QTEFJTjY0AAAAAAAAAAAAAAAAAABQQktERjItU0hB
MjU2AAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAE6G0WB3FXCWKwDxEFFyhxCb+fsJ7H
gxb6DBubxc8p/iAAAADbk/9BGe4owKJPRkSv5/d5KS0cRgrD6TIpEUB4CluaECBO
AACh6/2ZMmhvryRCakZDx4/Io7DHc00rM2xIRY4mnSJcgMDUAQBAAAAAkPyJjd4w
Lu9hly3W/deP1cC4NYsxri0AcIKyWO5dGiVZM/XX/3PvYatMpu5Iu4C8y3o0R3DG
98kf65qRYnyLWg=="/>
        </HardDisk>
      </HardDisks>
    </MediaRegistry>[/quote]

So, post-6.0, when a non-attached disk file is encrypted, the the enceyption key goes into Virtualbox.XML.

When I later attached this encrypted orphaned vdi to a guest, the key disappeared from Virtualbox.XML and went into the guest's .vbox file. The key stayed in the .vbox file when I later released the vdi from the guest. Removing the vdi from Virtualbox altogether removes the key from the .vbox file, but does not replace it in the Virtualbox.XML. They key goes to /dev/null, as the Linux guys like to say. The key is gone and so is decrypting or running the disk file. Removing the encrypted vdi from Virtualbox = Data death.

[socratis]

The devs responded to a bugtracker ticket: https://www.virtualbox.org/ticket/16784

Ah, the famous #16784. I remember that discussion now...

Removing the encrypted vdi from Virtualbox = Data death.

<WearsTheNapoleonHat>

• And that's what happens when MY ideas don't get implemented!

</WearsTheNapoleonHat>

Seriously now, when we talked about the standalone encrypted VDI in #16784, I had an idea (based on your proposal), to have each VDI accompanied by a .DEK file which would contain the keys. The VDI and DEK files would go together, as a pair, freeing both the VBOX and the XML files from storing the keys, making the whole thing portable, backupable and standalone. I guess it didn't pass muster...

[scottgus1]

each VDI accompanied by a .DEK file

This would really be the best way. There's way too many (forum-documented) ways to lose a DEK for a disk (not the least of which is new users thinking "Oh cool! I can encrypt!" I don't feel good about the move-the-DEK-from=XML-to-.vbox, either. Computers glitch, drives don't write when they are thought to, power fails, etc. Remove the disk from Virtualbox, the DEK goes south forever...eek. Of course a backup would prevent data loss if the encryption fails, but the whole idea is to not leave unencrypted data around...

Maybe you and I should grab a few million from our investment portfolios and buy an influencing number of Virtualbox licenses, then we could get this pushed through.

[fth0]

Thanks for investigating @scottgus1 and @socratis.

Instead of a separate DEK file, I'd prefer to enhance the VDI file format and keep the DEK inside the VDI file (like TrueCrypt/VeraCrypt do), so that the VDI file is self-contained and complete. But I can imagine this to be a lot of work ...

[scottgus1]

keep the DEK inside the VDI file

Thought about this once, too, myself, but I don't think they'll do it.

The devs could certainly do this, with an format enhancement for vdi's. But they want to be able to encrypt vmdk's, vhd's, etc., too, and those formats' standards are beyond the devs' control. So this would involve two different ways of handling the DEK depending on the drive file's format.

Truecrypt will let the key be stored in a different file, too, if desired.

[fth0]

Truecrypt will let the key be stored in a different file, too, if desired.

AFAIK, only for backup purposes:

The encrypted volume header containing the master keys can be backed up to and restored from an external file, but cannot be used from there. Not to be confused with the optional keyfiles, which are used together with the optional passwords to decrypt the volume header and master keys.

[chrismin]

Thanks for above input from everyone.

I've been struggling to handle encrypted media (vdi image or rawdisk vmdk) in consistent way following above hints and guides.
Please find what I found and which seems to be working for me.

My setup is headless Arch host ( VirtualBox v 6.1.8 ) and variety of VMs running on it.
I prefer CLI but I have also phpVirtualBox, where I can check and confirm (or not ) actions.

I've been trying to find right way of attaching/detaching encrypted images having in mind that encrypted media may need to be moved to a different host machine / re-attached to other VM(s) from image file backup.
This is also to make sure, in case of some failure, data can be decrypted and extracted from encrypted drive or mounted in some other way (at least for fixed size VDIs).

Here is what I found:
1. create VDI independently of the VM (before even target VM existed in the list of VMs):

Code: Select all

vboxmanage createmedium disk --filename encrypted.vdi --size 4000 --variant Standard

Medium created. UUID: ebfb0df5-d456-47b1-9282-dbccf51d9efb

2. encrypt created VDI:

Code: Select all

vboxmanage encryptmedium ebfb0df5-d456-47b1-9282-dbccf51d9efb --newpassword - --cipher AES-XTS128-PLAIN64 --newpasswordid prikey

Type password when prompted....

3. VB has now put new image into list with below data:

Code: Select all

vboxmanage showmediuminfo ebfb0df5-d456-47b1-9282-dbccf51d9efb

UUID: ebfb0df5-d456-47b1-9282-dbccf51d9efb
Parent UUID: base
State: created
Type: normal (base)
Location: /srv/vbox/encrypted.vdi
Storage format: VDI
Format variant: dynamic default
Capacity: 4000 MBytes
Size on disk: 2 MBytes
Encryption: enabled
Cipher: AES-XTS128-PLAIN64
Password ID: prikey
Property: AllocationBlockSize=1048576
CRYPT/KeyId=prikey
CRYPT/KeyStore=U0NORQABQUVTLVhUUzEyOC1QTEFJTjY0AAAAAAAAAAAAAAAAAABQQktERjItU0hB
MjU2AAAAAAAAAAAAAAAAAAAAAAAAACAAAACtprBQRMnClzwnvXB5bXLMVA7Pos9e
auuNjJyjsBZOjyAAAACJ4kFrFwm9SYuNyKiOaTD9gJSzUQimukyoJZkEaAS/LCBO
AAD/EOzWmBsDaDvpnYFftWW5diIkYY0qMJ9lExbphW0+LkB+BQAgAAAAFn2cRpQJ
YRwxXgtwkEMh5X94dcCZzzPuVwg80O9f07sAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAA==

Please backup this output into separate file (perhaps with password used during creation) in safe place.
Image is listed in (web)GUI Virtual Media Manager or CLI: vboxmanage show hdds (here you can also see line Encryption: enabled)

4. Remove this image in Virtual Media Manager (or CLI closemedium), but don't delete the image file. Check manager or list again that it doesn't list.

5. Now attach the image file (by selecting the image file) to a VM which you want to use to access the data (perhaps even on a different host machine).

6. In my case image becomes attached to selected VM but CRYPT information seem to be lost. Image is basically attached as regular unencrypted image.

Code: Select all

vboxmanage list hdds

UUID: ebfb0df5-d456-47b1-9282-dbccf51d9efb
Parent UUID: base
State: created
Type: normal (base)
Location: /srv/vbox/encrypted.vdi
Storage format: VDI
Capacity: 4000 MBytes
Encryption: disabled

and same output from boxmanage showmediuminfo ebfb0df5-d456-47b1-9282-dbccf51d9efb

7. Now THE MAGIC that works for me:

Code: Select all

vboxmanage mediumproperty disk set encrypted.vdi CRYPT\/KeyId prikey
vboxmanage mediumproperty disk set encrypted.vdi CRYPT\/KeyStore U0NORQABQUVTLVhUUzEyOC1QTEFJTjY0AAAAAAAAAAAAAAAAAABQQktERjItU0hBMjU2AAAAAAAAAAAAAAAAAAAAAAAAACAAAACtprBQRMnClzwnvXB5bXLMVA7Pos9eauuNjJyjsBZOjyAAAACJ4kFrFwm9SYuNyKiOaTD9gJSzUQimukyoJZkEaAS/LCBOAAD/EOzWmBsDaDvpnYFftWW5diIkYY0qMJ9lExbphW0+LkB+BQAgAAAAFn2cRpQJYRwxXgtwkEMh5X94dcCZzzPuVwg80O9f07sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

If you have not made a mistake copying/pasting keyid and keystore from your backup notes you should see that check media information now shows correct fields for Cipher and Encryption, as below:

UUID: ebfb0df5-d456-47b1-9282-dbccf51d9efb
Parent UUID: base
State: created
Type: normal (base)
Location: /srv/vbox/encrypted.vdi
Storage format: VDI
Format variant: dynamic default
Capacity: 4000 MBytes
Size on disk: 2 MBytes
Encryption: enabled
Cipher: AES-XTS128-PLAIN64
Password ID: prikey
Property: AllocationBlockSize=1048576
CRYPT/KeyId=prikey
CRYPT/KeyStore=U0NORQABQUVTLVhUUzEyOC1QTEFJTjY0AAAAAAAAAAAAAAAAAABQQktERjItU0hBMjU2AAAAAAAAAAAAAAAAAAAAAAAAACAAAACtprBQRMnClzwnvXB5bXLMVA7Pos9eauuNjJyjsBZOjyAAAACJ4kFrFwm9SYuNyKiOaTD9gJSzUQimukyoJZkEaAS/LCBOAAD/EOzWmBsDaDvpnYFftWW5diIkYY0qMJ9lExbphW0+LkB+BQAgAAAAFn2cRpQJYRwxXgtwkEMh5X94dcCZzzPuVwg80O9f07sAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

Please try for yourself and let me know of this works for earlier VirtualBox versions?

NOTE: above seems to work with encrypted physical raw disks described with VMDK file.
NOTE2: please mind escape character in my code in case you work in windowz host.
NOTE3: you will also see ENCRYPTION tab in a VM will "kick in" automatically once the image properties are set.
I hope this helps.