Strange handling of encryption keys in case of encrypted media vs. encrypted VM.

[mpack]

I'm not challenging the point about encryption not working with unregistered VDI's - that subject doesn't interest me at all. It's blatantly obvious that it can't work.

What I asked for clarification about (and indicated by selective quoting) was your statement the DEKs are not stored, that the system works a la password protected zips. You seemed to be saying that all one has to do is remember the password, and no .vbox metadata is required. Did I misunderstand the point you were making?

[socratis]

No, I wasn't trying to make a point at all. I was simply trying to read the tea leaves of the manual.

It seems to me, according to the documentation, that encrypting/decrypting a stand-alone medium is something that the developers intended(?). Otherwise why would you be able to encrypt/decrypt a single medium file, independent of the VM? There is even a standalone command and a devoted chapter to this!

And the language indicating clearly that the only thing required is the password. So, if the DEK is stored as a property in the VDI (does the format support that?) it could be deduced that it could work as a stand-alone password-protected ZIP/RAR/other.

I'll need to hit the sources to see what in the seven kingdoms their intentions were...
That is if I can decipher the code (pun intended).

[Martin]

It seems to me, according to the documentation, that encrypting/decrypting a stand-alone medium is something that the developers intended(?). Otherwise why would you be able to encrypt/decrypt a single medium file, independent of the VM? There is even a standalone command and a devoted chapter to this!

And the language indicating clearly that the only thing required is the password. So, if the DEK is stored as a property in the VDI (does the format support that?) it could be deduced that it could work as a stand-alone password-protected ZIP/RAR/other.

The main paragraph in chapter 9.31 tells you "The DEK is stored encrypted in the medium properties .... Since the DEK is stored as part of the VM configuration file, it is important that it is kept safe. "
The topics 9.31.2 and 9.31.4 are subtopics to that chapter...

[socratis]

the DEK is stored encrypted in the medium properties

EXACTLY! The VM configuration file, where the DEK is stored in your typical GUI-encrypt scenario, has absolutely nothing to do with the medium properties. That's where the confusion (mine alone?) comes in.

Here's a quick test. Launch VirtualBox. Create a VDI from the command-line, do not attach it to any VM, just let it sit there:

$ VBoxManage encryptmedium /Users/socratis/test.vdi --newpassword - --cipher "AES-XTS256-PLAIN64" --newpasswordid "testenc"
Enter new password:
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Go to media manager. "test.vdi" is listed as not-attached, encrypted. Where in the name of the seven gods is the DEK? Under what configuration file?

$ VBoxManage showmediuminfo test.vdi 
UUID:           09a01235-3556-4ad3-bc9c-f0057339647a
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /Users/socratis/test.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       100 MBytes
Size on disk:   2 MBytes
Encryption:     enabled
Cipher:         AES-XTS256-PLAIN64
Password ID:    testenc
Property:       CRYPT/KeyId=testenc
Property:       CRYPT/KeyStore=U0NORQABQUVTLVhUUzI1Ni1QTEFJTjY0AAAAAAAAAAAAAAAAAABQQktERjItU0hB
MjU2AAAAAAAAAAAAAAAAAAAAAAAAAEAAAABOv9w49/LIPpanvSENxN+i8hqLe3cW
GYdbtNYnITXdMiAAAAAdxKwpjzSlg/sLBZPvnf+V85h92YvQLgY9uT2sNWjy8CBO
AABdTB5TaUJLpDaOmFbVx9xXPgrGiDB1PHsw6bSAKuWbeGDqAABAAAAA/hUvDQct
IVlWORD48wEcw2bhSgl6uQhzAAX+1eIXPX4joruODGSGHVbZN7kW+LO/AnYIiTMa
C1LMTKCa6cz8JA==

In the medium properties? Could it? No, of course not, that would be too easy. I copied the VDI to another host:

$ VBoxManage showmediuminfo test.vdi 
UUID:           09a01235-3556-4ad3-bc9c-f0057339647a
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       U:\test.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       100 MBytes
Size on disk:   2 MBytes
Encryption:     disabled

WHAT???

Here comes the kicker. Quit VirtualBox (on your original computer, the one that the VDI was created and encrypted). Let VBoxSVC die a quiet death. Relaunch Media Manager. Your original "test.vdi", the one that was registered but not attached? It's gone!!! And guess what happens if you try to get its properties from the command-line:

$ VBoxManage showmediuminfo test.vdi 
UUID:           09a01235-3556-4ad3-bc9c-f0057339647a
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /Users/socratis/test.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       100 MBytes
Size on disk:   2 MBytes
Encryption:     disabled

So, to me it seems that this is an "encryption-in-memory-as-long-as-VBoxSVC-is-active", type of encryption. That could explain everything we've been trying to decipher/interpret so far. Everything, except the logic behind the decision.

[scottgus1]

this is an "encryption-in-memory-as-long-as-VBoxSVC-is-active", type of encryption.

That's what I have deduced. As long as you can keep VBoxSVC.exe alive (even if you run the command line encrypt with no Virtualbox processes running and then get the main Virtualbox window open in time under the same VBoxSVC.exe) then the disk is known as encrypted and is usable. The VBoxSVC.exe holds the DEK in memory for a guest .vbox file to store when you pick a guest for the disk.

If the VBoxSVC.exe that encrypted the disk dies before the disk is attached to a guest, then the in-memory DEK is lost and the disk is unusable.

Personally I don't think it was an active decision to have it this way. It is probably an unexpected side-effect. Like Microsoft's SMBv1 security hole, which has been there since the beginning of time, and no one noticed until the NSA saw it. Except that the NSA kept it secret, then it was leaked, then we have WannaCry....

[socratis]

I don't think it was an active decision to have it this way. It is probably an unexpected side-effect.

You don't create a specific command-line command and a chapter in the documentation for a side-effect, no. There must have been a customer need that dictated that; teleporting, on-the-fly attachments or something else.

I have a couple of how-to posts that I'm working on. After I'm done, I'll try some tests, like attaching an on-the-fly-encrypted disk to one or more (that's going to be fun) VMs and see what happens. I did try to attach an encrypted disk to a VM, filling it with data and then detaching it. That ruined a perfectly good VDI

[scottgus1]

I agree, the command line was definitely an active decision. I just think it was an oversight to allow encrypting of unregistered drives from the command line while there's no pre-existing VBoxSVC.exe that could carry the DEK torch into further usage of the drive. Once the VBoxSVC.exe that is generated by that command line dies just a few seconds after the command completes, the disk is dead if it hasn't been registered to a guest. In fact, no matter how long the VboxSVC.exe had been previously running, if one fails to attach a command-line-encrypted disk to a guest, and then allows the all-day-long-running VBoxSVC.exe to die, then the disk dies too. That part probably wasn't actively designed. But that's just my 0.02.

One could theoretically use a batch file to encrypt the unregistered drive then immediately attach it to a guest. Then the VBoxSVC.exe would stay active and store the DEK in the guest's .vbox. But if there's a syntax error or some other error in the batch file that causes the registration to fail, then the DEK goes south and the disk is dead. Even an unfortunately-synchronized accidental delay in executing such a batch file with no errors could allow death of the DEK and disk, if a delay of a few seconds happened between encrypt and register, and the VBoxSVC.exe was allowed to die off.

A quick fix would be to register the disk with a guest first then encrypt. And a solution would be to reprogram the command line to require disks to be registered with a guest before they are encrypted.