virtualbox.org

(a.k.a. coffee break system installation)

This is the implementation. For Research Paper, see this:
viewtopic.php?f=10&t=46816&p=211275#p211275

Install instructions: (Debian Linux)
1. New Dependencies:
# apt-get install p7zip-full mtools genisoimage
2. Download & install patched VirtualBox-unattended-OSE for Linux: (4.1.29)
http://www.qumble.org/Downloads/Virtual ... r89849.run
2.a. [OPTIONAL] MD5SUM:
2e22e1568a49c0585ab7d4dee6eec50e /home/alexey/1/VirtualBox-4.1.29_unattended-OSE-r89849.run
3. Make sure you use Oracle's guest additions:
Download http://download.virtualbox.org/virtualb ... 4.1.28.iso
cp [path]/VBoxGuestAdditions_4.1.28.iso /opt/VirtualBox/additions/VBoxGuestAdditions.iso

Good news !
I have finally achieved first milestone - first working version !
This project lets you to install any supported guest OS in 10 minutes !

Supported Guest OSes:
------------------------------------
1. Windows NT 5 (tested with Windows XP, 32bit)
2. Windows NT 6 (tested with Windows Vista, 32bit)
3. Red Hat kickstart (tested with RHEL 3/4/5, 32+64bit)
4. Debian preseed (tested with Debian 6.0 DVD, both 32+64bit)
5. SUSE autoyast (tested with openSUSE 11.3/11.4, 32+64bit)
------------------------------------

Following the topics:
viewtopic.php?f=9&t=26005
and
viewtopic.php?f=9&t=42926

I decided to write a small program, that does exactly this work:
named 'vbox-unattended'

Why I write 'vbox-unattended' ?

Because:
Installing Windows XP by coffee-drinking people takes about 30 min
Installing Windows XP by active geek, clicking instantly takes about 15 min
Installing Windows XP by Unattended Install, takes about 10 min, letting people to drink coffee in first place.
(on standard Core2-class hardware with HDD)

Also because:
*no reason to let VMware to steal whole market (but that's secondary reason)

-----------------------------------------------------------
Design Goals:
-----------------------------------------------------------
The system should:
1. run on any host OS supported by VirtualBox, (I try to avoid host-OS-specific tricks, if possible, but for now only Linux + FBSD host is implemented)
2. be simple to implement,
3. be offline (i.e. not require an Internet connection),
4. be fast (i.e. generate only minimal remastered boot CD image, rather than whole OS CD image)
5. support both Windows guests and several major Linux guest families (Debian, Red Hat, SUSE).
6. be Free Software (dependency on non-free software is forbidden, support is 'OK')

Target: Desktop users. (but also useful for: sysadmins, VBox QA, distro-hoppers, newbies)
-----------------------------------------------------------
vbox-unattended was written on Debian GNU/Linux 6.0 and tested with VirtualBox 4.1.
It currently supports only Linux-hosts with Windows guests (NT 5x and NT 6x families: Windows 2000/XP/2003/Vista/7/2008 guests, tested with WinXP/Vista, 32-bit), Red-Hat-like Linux guests. (Tested with RHEL3/4/5 32-bit)), Debian guests (tested with Debian 6), and SUSE guests (tested with openSUSE 11.3/11.4, 32-bit).
Due to it's beta quality the software may not function properly as advertised.
Because the basic design is OS-neutral, porting it from Linux to Windows hosts should be possible later...

TODO:
-more tests and stability
-pushing upstream

How to use: (python version)
Python version works like a 3rd party application, with standard VirtualBox v4.1.x.
1. edit vboxunattended.py - last 10 lines - this will configure your stuff.
2. run it:
$ python ./vboxunattended.py

Release Notes:
1. It works !
2. Full VirtualBox GUI and Main API integration ! (for C++ version)
3. To install Debian 6 guest, you need the first DVD, 1 of 8. (32-bit or 64-bit Squeeze)
4. Linux / FreeBSD hosts only
5. working from physical CD-ROM is not supported. (only with ISO images)
6. for each run, you must change VM name inside "vboxunattended.py" - in last 10 lines. (python version only)
7. RHEL6 and Fedora guest OS support are broken.
8. there is no cleanup code, so you can study the leftovers... (@_@)

Changelog:
-v0.1 - 2011-12-19: First release. supports Linux hosts and Windows NT 5.x and NT 6.x guests (python only)
-v0.2 - 2011-12-20: added support for Red-Hat-like Linux guests (via Red Hat Anaconda kickstart backend)
-v0.3 - 2011-12-24: added support for VirtualBox Guest Additions + improved stability.
-v0.4 - 2011-12-29: added support for Debian Linux guests (via Debian preseed backend)
-v0.5 - 2011-12-31: added support for SUSE Linux guests (via AutoYast backend) + improved code arch.
HAPPY NEW YEAR 2012 !
v0.5.1-2012-01-01: hugely improved support for Debian guests.
v0.6.0-2012-01-10: Complete rewrite in C++ ! Features VirtualBox GUI integration !
v0.6.1-2012-01-12: First BETA ! Basic functionality is mostly stable. C++ and python combined release !
v0.6.2-2012-01-17: Fixed bug with folders with spaces (hosts) and workaround for Debian Bug #655841. (Debian guests)
v0.6.3-2012-01-17: added support for FreeBSD hosts and small fix for Debian guests. (thanks to "nox-")
v0.7 - 2012-02-01: Near complete rewrite in COM C++, integrated into VBox Main API.
v0.7.1 - 2013-10-18: Refactored patch to apply on top of VirtualBox OSE 4.1.28. + First binary build that you can download and run !

v0.6.1. details:
1. Combined release in C++ and python. (they have similar function names,
and similar features). GUI is provided with the C++ version.
2. Critical bug fixed: libunattended.cpp in file read code. (C++ only)
3. Red Hat guest support went throught a major rewrite:
- added support for Red Hat Enterprise Linux guests (C++ version)
- added "password encrypter" (this solves the problem, that
Red Hat requires encrypted passwords by crypt(3) POSIX function.
- added better granularity to IGuestOSType (C++ only)
The bad news: Introduction of RHEL guests forced me to make incompatible changes in VirtualBox.
The v0.6.1 C++ version is not 100% compatible with the original.
4. As per <Teknomancer>'s recommendation, I replaced system("mkdir")
with "IPRT RTDirCreate()" (C++ only)
5. Debian 6.0 guest found to be non-deterministic, swapping
between /dev/cdrom and /dev/cdrom1 at random.
Problem hopefully solved by using /dev/sr0, which seems reliable.
6. python code got much better abstraction, and much code removed.

v0.7:
vbox-unattended has now integrated much deeper into the core of VirtualBox.
+Introduces New Main API: IMachine::IUnattended
+customizable Unattended pipeline. You can hook into the Unattended pipeline and change things on the fly.
+3rd near complete rewrite. (1st was in python, 2nd in Qt C++)
+GUI: First Run Wizard is now disabled for unattended machines.
+Visually and functionally it is a carbon copy of v0.6.3.

License: MIT/X11.
====
-Technologov

New GUI design: (C++ version)

The standard 'NEW VM' GUI will be modified at step #2 to add 'unattended' option. (added on 07.Jan.2012) If the new 'Unattended Install' option was chosen, then new GUI wizard will take place, that asks for 'Unattended' specifics.
In addition the CD-ROM media is moved into this step (instead of 'First Run' Wizard)
Ideas / Q&A:
Q: Why I think it should be part of 'New VM' Wizard ?
A: Because it seems to fit best into this class code-wise.

Q: Why 'Unattended Wizard' cannot be part of 'First Time Wizard' ?
A: Reason: The unattended scripts are pretty hard-coded and cannot support endless number of possible virtual hardware configurations, that the user might configure in 'VM Settings', before starting the VM. IMO it is impossible to write unattended scripts to support unknown hardware reliably (SATA vs. IDE vs. SATA; Primary Master vs. Primary Slave, etc...).

I just sat and read trough all your code and I just needed to reply to this and say...

WAUV, lots of hours in this. I'm sure this will come in handy for people and it looks very easy to customize to your own setup.

Thumbs up from here.

yea, thanks.

Now the big problem is future direction of this project:
Two options:
A. keep 'vbox-unattended' as-is - separate project from VirtualBox core.
For me, it would be sad, because newbies (whom gain the most) won't be able to find it.
I can port it to Windows and write GUI in this case. Windows version will be *very* ugly -- it will weight > 100 MB due to huge dependencies. (python+pyQt+Java)
Linux version will weight around 30 KB (kilobytes !)

B. somehow attempt to integrate this into VirtualBox proper.
I lack the technical skill to make quality integration, although ugly patches are possible.
Quality integration requires good skills in C/C++, which I lack, plus good GUI design.
Also a big question is GUI design -- it is hard to design is useful GUI that will integrate with my new engine and fit into current VBox design.
In this case porting to Windows host and separate GUI need not to be written.
Quality integration of my technology into VirtualBox will increase it's size, by, perhaps 100-200 KB. It is that small.

I have not decided yet as to which route is preferable.

Jeremy Grossman wrote: leader of GNS3 project wrote:
<grossmj> the GUI part looks good
<grossmj> I would prefer an integration to vbox, option B

Couple of questions.

• 1) what about Solaris and MAC? I doubt this will be included in the main trunk if it does not support at least Solaris, but that is just my 2cents.
  2) error control, IE. grayed out if the GUI detects that you are installing a non-supported guest. Or when you input a wrong string?
  3) how are you doing regression testing? older versions that work but are not totally supported.
  4) deb and rpm should be able to understand the depends and install the missing ones, have you taken that into account?
  5) any idea how much more weight this is going to add to the DL/install files or the total installed package?

1) what about Solaris and MAC? I doubt this will be included in the main trunk if it does not support at least Solaris, but that is just my 2cents.

Windows hosts:
I need some cross-platform FAT12 tools for them; on Linux I use 'mtools'.
Windows hosts lack free, small, native tools to do this job. (I know 2 tools, one is dependent on cygwin, other on Java, so both are ugly, and won't be accepted by Oracle)
The recommended solution, is to port Qemu's FAT12 code to VirtualBox, which will allow us to support all host OSes, and solve at fundamental level. Some experienced C developer will have to do this task. It is MIT-licensed, so it's possible.

For now, the plan is to enable this feature only on Linux hosts, and see how someone solve this later. (hopefully for v4.2.0)
My feature *can* work on Windows hosts without the FAT12 code, but will be limited to Red Hat/SUSE guests. Windows and Debian guests will not work.

Solaris/Mac/BSD hosts: Someone has to research the option to use 'mtools' there, and if not, then wait for someone to port Qemu's code over here.

Mac/BSD guests:
I don't know if automating those is possible. Never researched.

Solaris guests:
Solaris provides "Jump Start" technology, so writing a backend is possible. It is similar in concept to Red Hat's "Kick Start", but different in detail.
I have no plans to write such thing, however.

2) error control, IE. grayed out if the GUI detects that you are installing a non-supported guest. Or when you input a wrong string?

Yes, I will do _some_ error control, based on Guest OS Type.
There is no way to do _complete_ control, because the Guest OS Type is not granular enough: i.e. you can specify "Fedora", but not "Fedora 7", etc...
In general, the situation will be much like with VBox Linux Additions. (i.e. it will mostly work, but not guaranteed to work on ancient versions of Linux guest OSes, or on too new versions). In reality it will work well on stable OSes, such as RHEL and Debian, and will work poorly for fast-changing OSes, such as Fedora.
I plan to test certain Linux guests, and they will be fully supported. The test results will be published.

With Windows guests, the quality control will be 100%, because VBox provides granularity (correct GuestOS Type) for Windows guests.
I plan to support everything from Windows 2000 and up.

3) how are you doing regression testing? older versions that work but are not totally supported.

The same way, as I test Guest Additions, by using my stability matrix.

4) deb and rpm should be able to understand the depends and install the missing ones, have you taken that into account?

ofc.
I take advantage of automatic dependency resolver.

5) any idea how much more weight this is going to add to the DL/install files or the total installed package?

If done properly, it will add about 100 kilobytes of code. (not much vs. the 300+ MB of code VBox already has...)

+new dependencies:

• 7-zip
• genisoimage
• geteltorito
• mtools -or- Qemu's FAT12 code porting.

The new dependencies will make the Windows version more bloated by few MB, while Linux version will stay the same. (because Linux OS has those)

-Technologov

OK now I am totally confused.

My feature *can* work on Windows hosts without the FAT12 code, but will be limited to Red Hat/SUSE guests. Windows and Debian guests will not work.

Is this a private project or is it something that you are working on for the Oracle/VirtualBox project? As it stands this will never make it into the trunk and from the looks of it you probably will never make it in unless you know C++, The DEVs are too busy to handle this project from what I understand.

Just for my edification when you say we, who are you talking about?

>Is this a private project or is it something that you are working on for the Oracle/VirtualBox project?
private project.

>Just for my edification when you say we, who are you talking about?
"me", no "we".
"we" as in "community", if someone wishes, to... If not, then Linux hosts only.

I have clarified this a bit, in my original text.

-Technologov

OK Thanks.

>you probably will never make it in unless you know C++,

I already have some C++ code, and I think I will _make_ it working in few weeks.
I write 2 compoents:
1. libunattended.cpp - generic unattended library in C++, with few specifics towards VirtualBox (only Guest Additions); This is similar to my current libunattended.py library, so the core logic is tested.
2. I have modified the VirtualBox GUI code already.

So the next step is to glue the two components together, and test the whole.
_I_ believe, that I will have a working C++ solution in few weeks. Linux hosts only.

-Technologov

update: I have decided to make it slightly earlier, and released the C++ version _today_.

v0.6. patch against VirtualBox.

First experimental patch of my new lovely feature, now re-written in C++ !

Release Notes: (v0.6)
1. It works ! (tested with 32-bit guests: Windows XP, Vista, openSUSE
11.4, Debian 6)
2. Full VirtualBox GUI integration !
3. you must symlink all template* files to /tmp/ (I have not found a
way retrieve vbox locations properly)
3.a. symlink GA.iso into "/tmp/VBoxGuestAdditions.iso" (for same reason)
4. Linux hosts only
5. Red Hat guests not supported in this release.
6. patch must be applied vs. vbox-svn-2012-01-07
7. working from physical CD-ROM is not supported. (only with ISO images)

Dependencies:
7z (7-zip) - used to extract data from ISOs.
mtools - used to copy data to floppy images.
genisoimage & geteltorito

It would be nice to stabilize it, and include into 4.2.0.

License: core logic under BSD, all my vbox-patches under MIT. (I can
change if needed)

Happy Hacking !

Main API: IMachine::IUnattended:

Currently I am aware of 5 projects trying to achieve Unattended installs with VirtualBox, to varying degrees (openSUSE openQA, Technologov's vbox-unattended, JClouds, pallet, vagrant-VeeWee)
In order to remove duplication of effort, it is a good idea to implement this at the Main API level.

/* Attributes: */
BSTR user (R/W) <-- create a new user, named <user>.
BSTR password (R/W) <-- password is both for normal user and for Administrator / 'root' accounts.
BSTR ProductKey (R/W) <-- must have notation: "12345-12345-12345-12345-12345". Windows guests only.
BSTR ISOpath (R/W) <-- a full path to original OS ISO.
BOOL InstallGuestAdditions (R/W) <-- add an option to enable / disable automated VirtualBox additions install.

/* Functions: */
/**
* Generates Guest-OS-specific unattended scripts (depending on IMachine::OSTypId),
* and puts them into the MachineFolder/...
*
* The parameters are replaced according to IUnattended attributes.
*
* The API user can edit, customize or completely replace those scripts,
* before final packaging.
* The unattended scripts are text files, that have Guest-OS-specific format.
*
* What gets generated ?
*
* Windows NT 5.x:
* MachineFolder/winnt.sif
* Windows NT 6.x:
* MachineFolder/autounattend.xml
* Red Hat:
* MachineFolder/ks.cfg
* MachineFolder/isolinux.cfg
* Debian:
* MachineFolder/preseed.cfg
* MachineFolder/isolinux.cfg
* SUSE:
* MachineFolder/autoinst.xml
* MachineFolder/isolinux.cfg
*/
constructUnattendedScript() <-- will edit scripts automatically, and put them into correct places ( MachineFolder/... )

/**
* This function finalizes the task of the Unattended pipeline.
* Packages Guest-OS-specific unattended scripts into CD bootable ISOs,
* and into floppies, depending on IMachine::OSTypId.
*
* Windows guests automatically read their unattended config files from
* a floppy disk, and so do not require custom bootable medium.
*
* Linux guests:
* Because Linux guests do not automatically read their unattended scripts,
* We must generate bootable CD on the fly for them.
* In order to make CDs bootable, we first extract boot files from Linux OSes,
* namely: Linux kernel, initrd, isolinux bootloader. Read: [boot contents]
* Plus we extract LinuxAdditions from the Guest Additions ISO, and
* also package this together on the bootable ISO.
*
* What gets generated ?
*
* Windows NT 5.x:
* MachineFolder/floppy_script.img
* Windows NT 6.x:
* MachineFolder/floppy_script.img
* Red Hat:
* MachineFolder/redhat_boot.iso
* MachineFolder/bootiso/[boot contents]
* Debian:
* MachineFolder/debian_boot.iso
* MachineFolder/bootiso/[boot contents]
* MachineFolder/floppy_script.img
* SUSE:
* MachineFolder/suse_boot.iso
* MachineFolder/bootiso/[boot contents]
*/
packageUnattendedScript() <-- will copy into floppy / CD images, and put them into correct places ( MachineFolder/... )

NOTE: between those 2 stages, it is possible for the API user to edit those files, either manually (in case of VBoxManage CLI user) or via stream editor, such as "sed". This will give us pretty customizable solution.

Status: DONE in v0.7

Work Complete. v0.7 released !

First binary release !!! VBox-unattended-OSE 4.1.29 (patch 0.7.1)

Finally ! The unattended patch from VirtualBox v4.1.x (patch 0.7.x) was ported to VirtualBox 4.3.x (patch 0.8.x) !
It was a major rewrite, and it is less stable than 0.7.1.

vbox-unattended-0.8-alpha1.patch - Dec.2014