Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

This is for discussing general topics about how to use VirtualBox.
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

I am writing a Python script and need to process VirtualBox.xml without any dependencies.

I have the following code

Code: Select all

import os
# ...
vbox_dir = fr"C:\Users\{os.getlogin()}\.VirtualBox"
config_file_object = open(os.path.join(vbox_dir, "VirtualBox.xml"))
How would I be able to get a list of names/UUID of VMs from this file into a Python List [] and get the UUID/Name and path to the .vbox file of a VM by Name/UUID?

Me thinks make a script that lists VMs and you can press 1 to select the first one, 2 for the second, etc then put the UUID, VM Name, and path to.vbox file into variables or a dict{}...

There might be other ways to get this information without processing the XML (like using the VboxManage command from Python) but if anyone knows how to process the XML file in Python, I would still like to learn how to read XML files in Python.

The code at https://www.tutorialspoint.com/python/p ... essing.htm is specific to the example and might be overkill, but mainly the way the XML there is structured seems to be different than VirtualBox.xml...
viewtopic.php?t=8208
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by scottgus1 »

I only footled with Python once so I can't answer all of your questions. But Virtualbox.xml is a deterministic text file, so parsing it with basic text functions should get you the VM list.

Each VM registered has a line starting with " <MachineEntry " and lists the VM's UUID and path to the VM's .vbox file. The .vbox file also is a text file and can be parsed for the VM's name.

There is a problem with parsing these files, though, and is pointed out in the beginning of each file: The VboxSVC service might overwrite any of them at any time. 'vboxmanage list vms' gets VboxSVC's active list with names and UUIDs. And 'vboxmanage showvminfo "VM name" ' will provide the path to the .vbox file and a lot of other info.

Though you don't want dependencies, vboxmanage will be on every host that Virtualbox is on, so that may be a tolerable prerequisite. Unless you want to read Virtualbox.xml away from a Virtualbox host.
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by mpack »

VirtualBox xml files are intended to be used by VirtualBox only. The VirtualBox devs do not guarantee that the structure of xml files is constant, so IMO it would be bad practice to assume otherwise. At the very least you should check that the version indicator at the top of the xml is what you expect.

You may instead want to look at "VBoxManage list vms".
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

scottgus1 wrote:I only footled with Python once so I can't answer all of your questions. But Virtualbox.xml is a deterministic text file
What is a deterministic text file?

scottgus1 wrote:There is a problem with parsing these files, though, and is pointed out in the beginning of each file: The VboxSVC service might overwrite any of them at any time.
I would load the XML contents into memory and close the file as soon as that's done. I do not intend to write the file, that would be silly.
scottgus1 wrote:'vboxmanage list vms' gets VboxSVC's active list with names and UUIDs. And 'vboxmanage showvminfo "VM name" ' will provide the path to the .vbox file and a lot of other info.
The next stop would be to get the folder containing the VDI file, which in most cases is the same folder as the .vbox file.
scottgus1 wrote:Though you don't want dependencies, vboxmanage will be on every host that Virtualbox is on, so that may be a tolerable prerequisite.
That would be acceptable. I try to do things in native Python when possible, but that could still be considered native python.
mpack wrote:VirtualBox xml files are intended to be used by VirtualBox only. The VirtualBox devs do not guarantee that the structure of xml files is constant, so IMO it would be bad practice to assume otherwise.
OK I will not assume the structure is similar. If the format were to change, would that break compatibility with different versions of Vbox and the format of the xml file?
viewtopic.php?t=8208
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by mpack »

operation420.net wrote:If the format were to change, would that break compatibility with different versions of Vbox and the format of the xml file?
I'm not aware of any guaranteed compatibility between different versions of VirtualBox, that is why the .vbox file has a version indicator near the top. VirtualBox normally keeps the old file if there is a breaking change when you upgrade versions.
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

mpack wrote:
operation420.net wrote:If the format were to change, would that break compatibility with different versions of Vbox and the format of the xml file?
I'm not aware of any guaranteed compatibility between different versions of VirtualBox, that is why the .vbox file has a version indicator near the top. VirtualBox normally keeps the old file if there is a breaking change when you upgrade versions.
So would it be possible for VirtualBox to be unable to open a .vbox file made in an older version? Worse case scenario, just create a new VM with desired settings and attach the VDI image...
viewtopic.php?t=8208
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by scottgus1 »

operation420.net wrote:What is a deterministic text file?
I've heard the term 'deterministic' used in conjunction with computers, if I remember correctly, meaning (I think) that there is a determinable reason for what the computer does, as in it's not magic or borrowers causing a bug, etc. :D

I might be using the term wrong, but I meant that Virtualbox.xml, even if not technically XML-standard-aligned, still has certain sections written in certain formats established by Virtualbox, so the VM registry should always look the same in a particular version, and being text can be accessed quite predictably without special routines needed.
operation420.net wrote:I would load the XML contents into memory and close the file as soon as that's done.
A problem with this is that VboxSVC carries its own understanding of Virtualbox.xml and all .vbox files in RAM, and the files may be outdated, even by a very short time, which is the other side of that warning message. Your parser may then get stale information. Vboxmanage can get this information always up-to-date from VboxSVC, and is not limited by what version the Virtualbox.xml is written in.
operation420.net wrote:the VDI file, which in most cases is the same folder as the .vbox file.
That's the default, but they can be elsewhere, and from many years of reading the forum I'll mention it's surprising how many times they are elsewhere. Depending on the VM's disk usage there are times where the disk file has to be elsewhere.

'vboxmanage showvminfo "VM name" ' also shows the disks attached to the VM and the path to the disk files.

'vboxmanage' will always provide up-to-date info from VboxSVC, which can be more recent than Virtualbox.xml or the .vbox files. 'vboxmanage' will get the correct info every time even if no Virtualbox processes are running, since 'vboxmanage' calls up its own fresh VboxSVC at that time to properly parse all the XML files.
operation420.net wrote:would it be possible for VirtualBox to be unable to open a .vbox file made in an older version?
Older versions than the installed Virtualbox should be compatible. Running a later VM on an earlier Virtualbox version may have problems.
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

I was thinking of splitting lines by <space>{ and stripping the } at the end.

Code: Select all

"MX Linux" {1e8f8580-3b38-409a-8899-1d4a07ae033c}
" test" {207d1c3c-6794-4b2c-b1cd-29d8852c0096}
""MX Linux" {1e8f8580-3b38-409a-8899-1d4a07ae033c}" {2b38157b-a678-48b3-a3c6-9f22546f4565}
" '{"  }" {90f2de2e-7d94-41ed-a862-2a1532deb05c}
But, not that I would, but if VM names contain spaces, 'quotes', or {curly braces} (or even UUIDs) that could be problematic...
viewtopic.php?t=8208
mpack
Site Moderator
Posts: 39156
Joined: 4. Sep 2008, 17:09
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Mostly XP

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by mpack »

XML lines have to be parsed properly, using syntax rules. If you don't know what this means then don't do it.

And as far as I'm aware, VirtualBox XML files are fully XML compliant, but XML is concerned only with the syntax not the content, and it's the content the OP is after.
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

mpack wrote:XML lines have to be parsed properly, using syntax rules. If you don't know what this means then don't do it.
I'm familiar w/ XML Syntax. I want to learn how to to that in Python, but looks like I'm better off to follow the example XML given and discuss on a site like Stack Overflow as that's beyond Vbox forum's scope. Before I thought I would try and learn XML while writing this script.
mpack wrote:And as far as I'm aware, VirtualBox XML files are fully XML compliant, but XML is concerned only with the syntax not the content, and it's the content the OP is after.
Hehe before I knew OP stood "Original Poster" I shorted my site name to OP420, then just OP. So the OP in this thread (me) is OP.

The code in my previous post was output from VBoxManage, not XML in case I didn't make everyone know.

Code: Select all

import subprocess
_vbm=r"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe"#Get x-platform later
output = subprocess.check_output([_vbm,"list","vms"]).decode().strip().split("\r\n")
vms = {}        ;       id = 0
for vm in output:
    id += 1
    UUID = vm.split(" {")[-1]    ;   UUID = "{}{}".format("{", UUID)
    # VboxManage.exe showvminfo {6d1c84b2-0821-482d-b95a-b86c67c7337f}
    for line in subprocess.check_output([_vbm, "showvminfo", UUID]).decode().split("\r\n"):
        try:
            key, value  = line.split()
            print(key, value)
        except:
            #print(line)
            pass
        #if
    ###
    record = dict(id=id, uuid=UUID)
    vms[id] = record
    #print(record)
viewtopic.php?t=8208
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by scottgus1 »

operation420.net wrote:I was thinking of splitting lines by <space>{ and stripping the } at the end.
.....
But, not that I would, but if VM names contain spaces, 'quotes', or {curly braces} (or even UUIDs) that could be problematic...
.....
The code in my previous post was output from VBoxManage
Here's an output from one of my hosts, with spaced and single-word VM names:
C:\Users\Kids>vboxmanage list vms
"ubuntubudgie" {3819db5f-7f44-4ee8-955b-8d4c3963b3d7}
"DOS" {834529f6-7dbf-4c36-ae47-14f7bdd7910d}
"XP" {5ff40456-5306-4e7d-a37d-9d698522b64b}
"win7" {47eda8af-0e9e-4651-bb41-307df1d6c19c}
"Linux Mint" {992958e0-5e90-4842-ad33-de6e1fd3924b}
"Linux Mint 2" {d145c705-eb21-4e32-a994-d77a8368cf79}
"Win11" {10db6635-abd6-4793-96ea-5808a5ecc362}
Double quotes around the VM name and braces around the UUID seem common to both spaced and single-word VM names.

To get the VM name I'd try splitting on the open brace {, then replace double quotes with spaces, then 'trim' spaces on beginning & end. Depending on how the VM name would be used, keeping the double quotes might be better, since double-quotes are necessary when using spaced VM names in vboxmanage, and double-quoting a single-word VM name also works with vboxmanage.

To get the UUID, I'd 'instr' the open brace {, 'mid' the input from 'instr' number + 1, then remove the final character after confirming it is the closed brace }.

To get both VM name and UUID in one swoop, 'instr' the open brace, then take 'left' of of 'instr' number -1, as well as 'mid of 'instr' + 1, then modify the outputs as needed.

(the above is from a VBscript background; PHP/Python-ize as needed. :D)
operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

I dunno if 'instr' is a Python thing. I am able to use the VboxManage command with UUIDs for a more 'deterministic' commands.

I have the output of showvminfo, I just need to get the data from it into my dicts...
viewtopic.php?t=8208
scottgus1
Site Moderator
Posts: 20965
Joined: 30. Dec 2009, 20:14
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Windows, Linux

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by scottgus1 »

operation420.net
Posts: 60
Joined: 28. May 2020, 02:17
Primary OS: MS Windows 10
VBox Version: PUEL
Guest OSses: Linux, Windows
Location: The hood
Contact:

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by operation420.net »

In the output of "VBoxManage.exe showvminfo ..." the VM name there on a line starting with "Name:" but the output for shared folders also lists each folder on a line starting with "Name:" I dunno if it would be safe to assume the VM name is on the first line? Or at least before the shared folder entries...
viewtopic.php?t=8208
fth0
Volunteer
Posts: 5668
Joined: 14. Feb 2019, 03:06
Primary OS: Mac OS X other
VBox Version: PUEL
Guest OSses: Linux, Windows 10, ...
Location: Germany

Re: Processing %HOMEPATH%\.VirtualBox\VirtualBox.xml in python

Post by fth0 »

Hint: VBoxManage showvminfo --machinereadable "VM name" doesn't have this problem. ;)
Post Reply