Updating an ISO with persistent data?

[ThisIsMe]

I'm using VirtualBox on Linux Mint 20, and I'm new to it, but I have a question about how to do something. If I'm running live mode of an ISO in VirtualBox, I'm not sure whether there's a way to persist data for that, but more importantly, can I have it store the changes directly back into the ISO, so that if I then burn it onto a USB drive it will run in a regular, non-persistent live mode, but with the changes that I'll have already made to it?

Or if I can't save directly to the ISO then how would I take the file that it does save and cram all of its data into the ISO?

I know there are some tools that can allow me to edit an ISO like that, but they seems to only allow me to use the terminal and don't actually have a virtual machine for graphical interaction. There's a lot of stuff that I must do, and it might be possible to do it all in the terminal, but I'd really be more comfortable if I could interact using the GUI, and then just have it save all of my changes back into the ISO.

[scottgus1]

An ISO is a disk image of a CD-ROM, and a CD-ROM can't be written to after it's been made. So you are not normally able to write data from a LiveCD ISO back to the ISO for it to show up next time.

You can add a disk to the VM to store the data you want to be persistent, but there's no way for the ISO to know when you reboot that the data would be there.

To put persistent data into the ISO you would have to either edit the ISO or unpack the ISO to a folder structure on the host disk and add your data then make a new ISO. Be careful you don't lose the ability to boot the new ISO. Web-searching ISO editors or ISO makers will get you programs that can help.

Being in a virtual machine does not change any of the above. A VM is just like a regular PC, and typically anything you can or cannot do on a regular PC you are enabled or limited the same in a VM.

[ThisIsMe]

Well I know about the tools and how they differ from a VM (see my third paragraph in the first post), but that's why I want to be able to edit the ISO after the fact, by inserting the persistent data from the file generated by VirtualBox, so that I won't have to resort to the text-only interface of those other tools, because I'm not sure that would be sufficient (like for example, suppose I had to install and uninstall things using Synaptic Package Manager, or other programs - I'm not sure how to do that with only text commands, and there may be other things that wouldn't even be possible).

[scottgus1]

It sounds like what you want to do is get an OS installed the way you like it then port it back to an ISO so it installs in the customized fashion the next time you install it.

That falls under copying the contents of an ISO onto the host PC, editing the copy's scripts to download and install desired items or recognize and use new items added to the folder structure in the copy when it is installed again, then making a new ISO of the edited contents.

You could also web-search backporting an already-installed Linux OS to a LiveCD ISO. If this is possible it should be doable on a real PC. "Make a LiveCD installer out of a pre-installed Linux OS" shows some interesting results.

[scottgus1]

Also lest this be an "XY problem", can you please explain what the ultimate goal of the edited ISO is? Make your own Linux distro? Be able to make copies of pre-installed guests without having to configure them fresh each time?

[ThisIsMe]

Yes, that's basically what I'm trying to do, but like I said, using the other tools doesn't seem to give me a graphical interface for doing what I want, and it seems like only a VM could do that, but since it stores persistent data and has the ability to merge that into the OS when it runs it, I figured, why can't there also be a way for me to extract all of that data from the persistence file(s) and merge them into the ISO, thus upgrading it?

I checked your link about the "XY" problem, and I guess I was describing what I was trying to do specifically, rather than the purpose that I'm trying to achieve, but that's because it's a long and convoluted explanation, but I'll summarize:

- I need to have Linux (preferably Mint 20 unless it's easier to get a different version to work better for this) put onto a USB stick so that ultimately I'll be able to make however many exact duplicates I want and give them to people to use.

- It doesn't matter whether it's installed onto the USB stick or whether it's just an ISO that runs in live mode, but either way, I must be able to customize it how I want (and if it's live mode then it must no longer have the installer available to install further onto the HD from there).

- I need to be able to install whatever software I want onto it, which would include at least Mono and a version of Java, as well as some custom software, and do do all of that I need to first install the driver for my WiFi adapter (which is on my hard drive) so that I can access the internet to download and install the rest of the stuff.

- After installing and configuring everything I must be able to then permanently disable any internet connection from that USB stick, for security reasons (I guess I could uninstall the driver and all other potential drivers, protocols and anything having to do with network connectivity, unless you have a better way?), so that it will be for all intents and purposes air-gapped.

- This USB stick must be extremely compatible with all computers so that it can boot independently without requiring a boot loader to be installed on the hard drive or anywhere else other than the USB stick itself (so it must show up in the boot menu), and it must be able to run in BIOS/Legacy or UEFI mode with or without Secure Boot.

- It must automatically mount other drives such as the hard drive, to make them accessible when running from the OS installed on the USB stick, because the people who will use it don't necessarily know anything about Linux or much about any advanced computer usage. They'll be using it for a simple purpose only.

If I can do all of this then I don't really care much how it gets accomplished, as long as I can check each and every one of those things off of the list. The reason I posted it here is because I'm working on about 4 or 5 potential solutions and hitting dozens of road blocks for every one of them, so I thought that trying to indirectly do it through the persistent data of a VM would be worth a shot.

[scottgus1]

That does make more sense, thanks. I would abandon the whole Edit-the-ISO thing, since booting from a USB does not use ISOs directly.

Web-search "Make a LiveCD installer out of a pre-installed Linux OS" and follow those methods. If what you want is possible, it's ben done before.

Virtualbox is really just another "PC manufacturer" and should not help or hinder the project.

[ThisIsMe]

Yes, I realize that a VM is not directly related to what I'm doing, but I just wanted graphical capabilities, but I suppose that if I can take an already installed OS and then cram that into an ISO, I suppose that would achieve the goal, except that I'd have to make sure that it becomes read-only at that point. And I hope it's not anything way too difficult, because I'm sort of a Linux newbie and I really just want to be able to do this relatively quickly.

EDIT: Except that I did just think of a possible issue with that. Every time I install Linux onto a USB stick, I can't get it to boot on its own, without using the GRUB on the hard drive (which is one of the reasons why I haven't been able to use that method, along with the fact that I can't figure out how to make it read-only). So if I install it, and then convert that whole installation to an ISO then even though it should become read-only (I think), I'm afraid that it still may not be able to boot independently, which would be a deal-breaker, since the computers on which it will run almost certainly won't already have GRUB or anything like that installed on them. In any case, I still don't see why I can't take the data that VirtualBox generates and then just shove that into the ISO, since while it's running, it's using all of that data integrated together, and it obviously extracts everything out of the ISO, so how hard would it be to put it back inside, with some additions?

[fth0]

That sounds to me as if a normal installation on a USB stick would do (just like on an internal or external hard disk). Then you can modify this installed Linux as you see fit, and duplicate the USB stick when you're finished. If Linux Mint isn't compatible enough, Knoppix is known to be very versatile.

[ThisIsMe]

But as I said, I can NOT get it to boot independently without relying on the GRUB that I have installed on my HD to boot it indirectly! It's an absolute requirement that it MUST be able to boot on its own, and that it must work in BIOS/Legacy or UEFI mode with or without Secure Boot.

I've tried installing a BIOS partition and a UEFI partition in addition to the Linux partition (3 partitions in total) and I followed instructions of a tutorial written by someone who generally really seems to know what he's doing.

But when I use the 3 partition method, not only will it not boot at all unless I use the GRUB on my HD, but when I do boot it that way, it runs extremely slowly, even on USB 3.1!

[multiOS]

This subject seems to me to be going way beyond the scope of a VirtualBox Forum, apart from the possibility of building/maintaining a 'base ''Virtual Machine' from which to produce Live USB Images' for distribution.

VirtualBox doesn't have any tools to directly merge persistent data into a read only ISO so a better start point might be to look at tools already developed for this purpose. One example is at https://www.linux-live.org/#home or perhaps look at Fedora Linux which seems to have had the capability and tools for producing Live CD/USB Images for several years - https://github.com/livecd-tools/livecd-tools/. If Debian/Ubuntu has access to any similar software tools, then so will Linux Mint.

[ThisIsMe]

Well that seems to require aufs, which evidently isn't supported by Mint.

[mpack]

Most guest OSs will buffer the directory of a CD-rom and only refresh it after a disk change event. So even if you found a tool, it would be wasted on most guests IMO.

Also, this sounds like a job for virtual ISOs (VISOs), but even they can't be dynamically updated - the guest OS simply isn't looking for that, so you'd need something inside the guest to force/fake a disk change.

[fth0]

But as I said, I can NOT get it to boot independently without relying on the GRUB that I have installed on my HD to boot it indirectly! It's an absolute requirement that it MUST be able to boot on its own, and that it must work in BIOS/Legacy or UEFI mode with or without Secure Boot.

How do you intend to address that the UEFI or legacy BIOS of a modern PC, where the USB stick is connected to, will not automatically boot from a USB stick? Especially a UEFI BIOS may allow to boot from selected USB stick partitions.

[ThisIsMe]

mpack, I'm not sure what you mean by a "guest OS". Are you talking about the one being run in the VM? I'm not sure what a VISO is, but I need something which can be dynamically updated, until I flip a switch so that it can't any more.

fth0, are you referring to it not being configured by default to prioritize booting a USB drive first? Well that's easy to fix, or the boot menu can be used.