Page 1 of 1

Building a VirtualBox "appliance" by script

Posted: 12. Feb 2011, 02:13
by mmayer
I am wondering if the following is already possible or what would be needed in order to make it happen. Would it be possible to build the necessary tools fairly easily or would there a more involved development effort be required?

I am looking for a way to build a VDI file from a directory tree without running the VirtualBox graphical user interface. I am trying to do this on a Linux machine (Debian) and I am trying to package a mini Linux distribution, which has been built and created on said Debian machine, into a VDI.

The purpose for this is supposed to be that one can say "make virtualbox" and the system will go, build everything that's needed, and then it'll generate a VDI file that contains the entire system it just built. The resulting virtual machine could then be loaded into any VirtualBox and be executed without any further need to configure or download anything. Of course, I'll have to make sure that the directory hierarchy the image is to be created from contains everything that is needed to boot (Linux kernel, /sbin/init, startup scripts, etc.)

So, I am looking for something along the lines of a shell-based "createvdi" command that reads all the required parameters either from a config file or the command line and generates a VirtualBox appliance without user-interaction.

Thanks,
-Markus

Re: Building a VirtualBox "appliance" by script

Posted: 12. Feb 2011, 11:04
by HubTou
Hello,

Have a look at this video and you'll find that it's possible and the necessary tools already exist: http://www.projet-hev.org/dist/hev1.flv

The principle is fairly simple, you write a script that:
- download the install CD/DVD of the OS you want to install (your favourite Linux distribution)
- create some blank virtual disks
- have the virtual machine boot on CD/DVD
- launch the VM headless
- use my "Generateur" component (cf. http://www.projet-hev.org/dist/generateur1-1.02.tar.bz2) to drive the OS installation from the host
(you'll need some adaptations if you don't use my vboxhost component on a HeV system, or you can simply use the Script/genereateur.libsh shell library)
- close the VM
- and there you are, you have a fully automated OS installation into a VM.

The first time you start the VM with remote display (VNC or VRDP), and build your install script step by step (by moving the ENVOYER_SCANCODES=1/0 sections as you can see below).

An example :

Code: Select all

#!/bin/sh
# @(#) Générateur HeV phase 1 - hev1.sh v1.02 (10/08/2010) / Hubert Tournier

################################################################################

VM=hev1

VERSION_FREEBSD=8.1
URL_CD="ftp://ftp.fr.freebsd.org/pub/FreeBSD/releases/amd64/ISO-IMAGES/${VERSION_FREEBSD}/FreeBSD-${VERSION_FREEBSD}-RELEASE-amd64-disc1.iso"

# Action par défaut (montrer l'aide en ligne) :
ACTION=HELP

# Tailles de disques et partitions en mega octets :
TAILLE_VDI=20480
TAILLE_SWAP=1024

# Géométrie du disque dur (couples classiques : 16/63, 16/32, 255/63) :
TETES=16
SECTEURS=63

# Déverminage :
DEBUG=0
TEMPORISATION=2

################################################################################

CD=`basename ${URL_CD}`

if [ -f /usr/local/etc/vboxhost.conf ]
then    . /usr/local/etc/vboxhost.conf
elif [ -f ./vboxhost.conf ]
then    . ./vboxhost.conf
else    echo "`basename ${0}`: Error: Configuration file not found."
        exit 1
fi

. Scripts/generateur.libsh

export PATH=${PATH}:.

InstallerPartition( )
{
ENVOYER_SCANCODES=1

[ "${ENVOYER_SCANCODES}" -ge "1" ] && sleep 45

AfficherMessage "Country Selection: 232 United States -> 75 France + Enter"
EnvoyerScancodes ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_PGUP} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_ENTER}

AfficherMessage "System Console Keymap: French ISO (accent) + Enter"
EnvoyerScancodes ${SC_ENTER}

AfficherMessage "FreeBSD/amd64 8.1-RELEASE - sysinstall Main Menu: Usage -> Custom + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_ENTER}


AfficherMessage "Choose Custom Installation Options: X Exit -> 3 Partition + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_ENTER}

AfficherMessage "FDISK Partition Editor: g (G = set Drive Geometry)"
EnvoyerScancodes `ConvertirCaractereEnScancode g`

CYLINDRES=`expr \( ${TAILLE_VDI} \* 2048 \) / \( ${TETES} \* ${SECTEURS} \)`
AfficherMessage "Value Required: 41610/16/63 -> ${CYLINDRES}/${TETES}/${SECTEURS} + Enter"
EnvoyerScancodes ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} `ConvertirCaracteresEnScancodes "${CYLINDRES}/${TETES}/${SECTEURS}"` ${SC_ENTER}

AfficherMessage "FDISK Partition Editor: c (C = Create Slice)"
EnvoyerScancodes `ConvertirCaractereEnScancode c`

if [ "$1" = "Secours" ]
then	AfficherMessage "Value Required: 41943040 -> 1g + Enter"
	EnvoyerScancodes ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} `ConvertirCaracteresEnScancodes "1g"` ${SC_ENTER}
elif [ "$1" = "Principale" ]
then	AfficherMessage "Value Required: (remaining space) + Enter"
	EnvoyerScancodes ${SC_ENTER}
fi

AfficherMessage "Value Required: 165 + Enter"
EnvoyerScancodes ${SC_ENTER}

AfficherMessage "FDISK Partition Editor: s (S = Set Bootable)"
if [ "$1" = "Secours" ]
then	EnvoyerScancodes ${SC_UP} `ConvertirCaractereEnScancode s`
elif [ "$1" = "Principale" ]
then	if [ `expr \( ${TAILLE_VDI} \* 2048 \) % \( ${TETES} \* ${SECTEURS} \)` = "0" ]
	then	EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} `ConvertirCaractereEnScancode s`
	else	EnvoyerScancodes ${SC_UP} `ConvertirCaractereEnScancode s`
	fi
fi

AfficherMessage "FDISK Partition Editor: q (Q = Finish)"
EnvoyerScancodes `ConvertirCaractereEnScancode q`

AfficherMessage "Install Boot Manager for drive ad4?: Standard + Enter"
EnvoyerScancodes ${SC_ENTER}


AfficherMessage "Choose Custom Installation Options: 3 Partition -> 4 Label + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_ENTER}

if [ "$1" = "Secours" ]
then	AfficherMessage "FreeBSD Disklabel Editor: c (C = Create)"
	EnvoyerScancodes `ConvertirCaractereEnScancode c`

	AfficherMessage "Value Required: 2096577 + Enter"
	EnvoyerScancodes ${SC_ENTER}
elif [ "$1" = "Principale" ]
then	AfficherMessage "FreeBSD Disklabel Editor: Partition name: ad4s1 -> Partition name: ad4s2 + c (C = Create)"
	EnvoyerScancodes ${SC_DOWN} `ConvertirCaractereEnScancode c`

	AfficherMessage "Value Required: 2088387 + Enter"
	TAILLE_PARTITION=`expr ${TAILLE_VDI} - 1024 - ${TAILLE_SWAP}`
	EnvoyerScancodes ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} `ConvertirCaracteresEnScancodes "${TAILLE_PARTITION}m"` ${SC_ENTER}
fi

AfficherMessage "Please choose a partition type: FS (a file system) + Enter"
EnvoyerScancodes ${SC_ENTER}

AfficherMessage "Value Required: / + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode /` ${SC_ENTER}

if [ "$1" = "Secours" ]
then	AfficherMessage "FreeBSD Disklabel Editor: s (S = Toggle SoftUpdates)"
	EnvoyerScancodes ${SC_DOWN} `ConvertirCaractereEnScancode s`

elif [ "$1" = "Principale" ]
then	AfficherMessage "FreeBSD Disklabel Editor: Partition name: ad4s2 + c (C = Create)"
	EnvoyerScancodes `ConvertirCaractereEnScancode c`

	AfficherMessage "Value Required: (remaining space) + Enter"
	EnvoyerScancodes ${SC_ENTER}

	AfficherMessage "Please choose a partition type: FS -> Swap + Enter"
	EnvoyerScancodes ${SC_DOWN} ${SC_ENTER}

	AfficherMessage "FreeBSD Disklabel Editor: s (S = Toggle SoftUpdates)"
	EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} `ConvertirCaractereEnScancode s`
fi

AfficherMessage "FreeBSD Disklabel Editor: q (Q = Finish)"
EnvoyerScancodes `ConvertirCaractereEnScancode q`


AfficherMessage "Choose Custom Installation Options: 4 Label -> 5 Distributions + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_ENTER}

AfficherMessage "Choose Distributions: X Exit -> B Custom + Space"
EnvoyerScancodes `ConvertirCaractereEnScancode b` ${SC_SPACE}

AfficherMessage "Select the distributions you wish to install.: X Exit -> base + Space"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_SPACE}

AfficherMessage "Select the distributions you wish to install.: base -> kernels + Space"
EnvoyerScancodes ${SC_DOWN} ${SC_SPACE}

AfficherMessage "Select the operating system kernels you wish to install.: X Exit -> GENERIC + Space"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_SPACE}

AfficherMessage "Select the operating system kernels you wish to install.: GENERIC -> X Exit + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode x` ${SC_ENTER}

AfficherMessage "Select the distributions you wish to install.: kernels -> lib32 + Space"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_SPACE}

AfficherMessage "Select the distributions you wish to install.: lib32 -> man + Space"
EnvoyerScancodes ${SC_DOWN} ${SC_SPACE}

AfficherMessage "Select the distributions you wish to install.: man -> X Exit + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode x` ${SC_ENTER}

AfficherMessage "Choose Distributions: B Custom -> X Exit + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode x` ${SC_ENTER}


AfficherMessage "Choose Custom Installation Options: 5 Distributions -> 7 Commit + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_DOWN} ${SC_ENTER}

AfficherMessage "Choose Installation Media: 1 CD/DVD + Enter"
EnvoyerScancodes ${SC_ENTER}

AfficherMessage "User Confirmation Requested: Yes (Are you SURE you want to continue the installation?) + Enter"
EnvoyerScancodes ${SC_ENTER}

if [ "$1" = "Secours" ]
then	AfficherMessage "User Confirmation Requested: Yes (No swap device found... Continue anyway?) + Enter"
	EnvoyerScancodes ${SC_ENTER}
fi

[ "${ENVOYER_SCANCODES}" -ge "1" ] && sleep 55

AfficherMessage "User Confirmation Requested: Yes (Visit the general configuration menu?) + Enter"
EnvoyerScancodes ${SC_LEFT} ${SC_ENTER}

AfficherMessage "FreeBSD Configuration Menu: X Exit -> Networking + Enter"
EnvoyerScancodes ${SC_PGDN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_DOWN} ${SC_ENTER}

AfficherMessage "Network Services Menu: X Exit -> Interfaces + Enter"
EnvoyerScancodes ${SC_DOWN} ${SC_ENTER}

AfficherMessage "User Confirmation Requested (IPv6 configuration): No + Enter"
EnvoyerScancodes ${SC_ENTER}

AfficherMessage "User Confirmation Requested (DHCP configuration): Yes + Enter"
EnvoyerScancodes ${SC_LEFT} ${SC_ENTER}

[ "${ENVOYER_SCANCODES}" -ge "1" ] && sleep 10

# l'opération suivante ne fonctionne pas toujours du premier coup, donc on fait 3 tentatives...
AfficherMessage "Network Configuration: Domain -> (blank)"
EnvoyerScancodes ${SC_ENTER} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_ENTER} ${SC_UP}
EnvoyerScancodes ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_ENTER} ${SC_UP}
EnvoyerScancodes ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_ENTER} ${SC_UP}

AfficherMessage "Network Configuration: Host -> freebsd"
EnvoyerScancodes ${SC_UP} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} ${SC_BS} `ConvertirCaracteresEnScancodes "freebsd"` ${SC_ENTER}

AfficherMessage "Network Configuration: OK + Enter"
EnvoyerScancodes ${SC_ENTER} ${SC_ENTER} ${SC_ENTER} ${SC_ENTER} ${SC_ENTER} ${SC_ENTER} ${SC_ENTER}

AfficherMessage "Network Services Menu: Interfaces -> X Exit + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode x` ${SC_ENTER}

AfficherMessage "FreeBSD Configuration Menu: Networking -> X Exit + Enter"
EnvoyerScancodes ${SC_PGUP} ${SC_UP} ${SC_UP} ${SC_UP} ${SC_UP} ${SC_UP} ${SC_ENTER}

AfficherMessage "Choose Custom Installation Options: 7 Commit -> X Exit + Enter"
EnvoyerScancodes `ConvertirCaractereEnScancode x` ${SC_ENTER}

AfficherMessage "FreeBSD/amd64 8.1-RELEASE - sysinstall Main Menu: Custom -> X Exit Install"
EnvoyerScancodes `ConvertirCaractereEnScancode x`

AfficherMessage "User Confirmation Requested: No -> Yes (Are you sure you wish to exit?) + Enter"
EnvoyerScancodes ${SC_LEFT} ${SC_ENTER}

AfficherMessage "Message (Be sure to remove the media from the drive): OK + Enter"
EnvoyerScancodes ${SC_ENTER}

[ "${ENVOYER_SCANCODES}" -ge "1" ] && sleep 10

ENVOYER_SCANCODES=0
}
 
start( )
{
	if [ -f ${REPERTOIRE_BACKUPS}/FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi.7z ]
	then	echo "The requested image already exist!"
		exit 1
	fi
	if ! VmEnregistree
	then	vboxhost register ${VM}
	fi
	if ! VmEnFonctionnement
	then	vboxhost start ${VM}
	fi

	InstallerPartition Secours

	if VmBloquee
	then	vboxhost poweroff ${VM}
		echo "Waiting 30s for VirtualBox to release the CD ISO image..."
		sleep 30
		VBoxManage -q storageattach "${VM}" --storagectl "IDE" --port 0 --device 0 --type dvddrive --medium ${CD} > /dev/null
		vboxhost start ${VM}
	fi

	InstallerPartition Principale

	if VmBloquee
	then	stop
	fi
}

stop( )
{
	if VmEnFonctionnement
	then	vboxhost poweroff ${VM}
		echo "Waiting 30s for VirtualBox to release the CD ISO image..."
		sleep 30
	fi
	if VmEnregistree
	then	VBoxManage -q storageattach "${VM}" --storagectl "IDE" --port 0 --device 0 --medium none 2> /dev/null
		VBoxManage -q closemedium dvd ${RACINE_VIRTUALBOX}/${REPERTOIRE_DISQUES}/${CD} 2> /dev/null
		if [ "${ACTION}" = "START" ]
		then	# Tout s'est bien passé, on peut cloner le disque virtuel
			VBoxManage -q modifyvm ${VM} --boot1 disk
			[ "${DEBUG}" = "1" ] && echo "DEBUG: cloning the resulting VDI"
			VBoxManage -q clonehd ${VM}.1.vdi FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi
			cd ${REPERTOIRE_DISQUES}
			[ "${DEBUG}" = "1" ] && echo "DEBUG: compressing the resulting VDI"
			7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on ../${REPERTOIRE_BACKUPS}/FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi.7z FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi
			rm -f FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi
			cd ..
			if [ "${DEBUG}" = "1" ]
			then	echo "DEBUG: Done!"
				ls -l ${REPERTOIRE_BACKUPS}/FreeBSD-${TETES}h-${SECTEURS}st-${VERSION_FREEBSD}-RELEASE-amd64.vdi.7z
			fi
		fi
		vboxhost unregister ${VM} 2> /dev/null >&2
		rm -f ${REPERTOIRE_DISQUES}/${VM}.1.vdi nohup.out
	fi
	[ -f VBoxHeadless.core ] && rm VBoxHeadless.core
}

AfficherSyntaxe( )
{
	cd ${RACINE_VIRTUALBOX}
	what `basename $0`
	echo "options:"
	echo "	[-g]				to start or continue VM execution"
	echo "	[-x]				to stop VM execution"
	echo "	[-d]				to use debug mode"
	echo "	[-t timeout_in_sec(${TEMPORISATION})]		to change timeout between keyboard injections"
	echo "	[-H heads(${TETES})]"
	echo "	[-S sectors_per_track(${SECTEURS})]	to change disk geometry"
	echo "	[-v VDI_size_in_GB(`expr ${TAILLE_VDI} / 1024`)]"
	echo "	[-s swap_size_in_GB(`expr ${TAILLE_SWAP} / 1024`)]		to change default partitions sizes"
}

PARAMETRES=`getopt gxdt:H:S:v:s: $*`
if [ $? -ne 0 ]
then	AfficherSyntaxe
	exit 1
fi
set -- ${PARAMETRES}
for i
do	case "$i" in
		-g)	# activation du mode START
			if [ "${ACTION}" = "STOP" ]
			then	AfficherSyntaxe
				exit 1
			fi
			ACTION=START
			shift ;;

		-x)	# activation du mode STOP
			if [ "${ACTION}" = "START" ]
			then	AfficherSyntaxe
				exit 1
			fi
			ACTION=STOP
			shift ;;

		-d)	# activation du mode debug
			DEBUG=1
			shift ;;

		-t)	# temporisation en secondes entre deux envois de scancodes
			if [ "$2" -ge "1" ]
			then	TEMPORISATION=$2
			else	AfficherSyntaxe
				exit 2
			fi
			shift ; shift ;;

		-H)	# nombre de têtes pour la géométrie de disque
			if [ "$2" -ge "1" -a "$2" -le "255" ]
			then	TETES=$2
			else	AfficherSyntaxe
				exit 3
			fi
			shift ; shift ;;

		-S)	# nombre de secteurs par piste pour la géométrie de disque
			if [ "$2" -ge "1" -a "$2" -le "63" ]
			then	SECTEURS=$2
			else	AfficherSyntaxe
				exit 4
			fi
			shift ; shift ;;

		-v)	# taille du disque VDI en Go
			if [ "$2" -ge "3" ]
			then	TAILLE_VDI=`expr $2 \* 1024`
			else	AfficherSyntaxe
				exit 5
			fi
			shift ; shift ;;

		-s)	# taille en Go de la tranche de swap dans la partition principale
			if [ "$2" -ge "1" ]
			then	TAILLE_SWAP=`expr $2 \* 1024`
			else	AfficherSyntaxe
				exit 6
			fi
			shift ; shift ;;

		--)
			shift; break;;
	esac
done
if [ "${DEBUG}" = "1" ]
then	echo "DEBUG mode is on"
	echo "DEBUG: timeout between keyboard injections is ${TEMPORISATION} seconds"
	echo "DEBUG: VDI size = ${TAILLE_VDI} MB"
	echo "DEBUG: swap slice size in main partition = ${TAILLE_SWAP} MB"
	echo "DEBUG: disk geometry with ${TETES} heads"
	echo "DEBUG: disk geometry with ${SECTEURS} sectors per track"
fi
cd ${RACINE_VIRTUALBOX}
case "${ACTION}" in
	START)		start
			;;
	STOP)		stop
			;;
	*)		AfficherSyntaxe
esac
exit 0
I'm considering writing a small utility to tap into the VirtualBox Mouse Control API in order to do the same thing with graphical OS installations...

I've made a serie of imbricated scripts (just like a matriochka Russian doll), in order to avoid making the same operations over and over again, when they have already been done correctly once. I start with the highest level script. It it doesn't detect the VM resulting from the previous script, it just calls it, and so on.

That's for creating the base OS installation and some configuration.

After that, I have other components that configure a whole FreeBSD system (that's what I use).

For example, in order to automatically build a VM containing that: http://www.projet-hev.org/fr/telecharge ... guest.html
The required source code is only that:

Code: Select all

#!/bin/sh
# @(#) Configurateur FreeBSD - Guest pile FAMP v1.01 (30/01/2011) / Hubert Tournier

. ../Modeles/Configurer.Guest.libsh

export ADRESSE_IP_DYNAMIQUE=oui
export PAS_DE_X_WINDOW=oui
export CONNECTEURS_LDAP=oui

export OPTIMISATIONS="-mtune=k8 -mmmx -msse -msse2 -msse3 -mfpmath=sse"

export MDP_ROOT=mdpinit
export MDP_MYSQL=mdpinit

ConfigurationMachineVirtuelle
ConfigurerNoyauSurMesure
InstallerNoyauSurMesure
ProfilServeurFAMP
InstallerPHPMyAdmin
InstallerZendFramework
ProfilServeurAllege
Best regards,

Hubert