How to build VirtualBox, that will match published version

[eduma]

Hi,

I want to be sure that I am making same VirtualBox for my IPS repository, with same options as published package.

From my tests I understand that they are using Solaris express version datad 2010.11, since in configuration options it check for python2.4 to be compatible with solaris10.
I have tested on solaris 10, but all libs there are too old.
So on Solaris express I suppose they need next packages to be installed:
pkg:/developer/gcc-3 SUNWgcc SUNWsvn SUNWhea FSWxorg-headers SUNWgnome-common-devel SUNWaudh SUNWusbu SUNWugenu SUNWugen SUNWagph SUNWcurl SUNWxorg-mesa SUNWj6dev SUNWj6dvx header-math doxygen
and slim_install to have python2.4 and 2.6.

Also according to Solaris%20build%20instructions there should be qt4, since link not works, I found other link.
./configure -v -platform solaris-g++-64 -thread -shared -stl -largefile -tablet -sm -qt-libjpeg -qt-libpng -qt-libmng -qt-zlib -prefix /opt/vboxose-qt -I/usr/include -I/usr/X11/include -I/usr/X11/share/include -I/usr/sfw/include

and patched with
--- src/3rdparty/webkit/JavaScriptCore/wtf/MathExtras.h.orig 2010-07-09 13:23:51.817496928 +0200
+++ src/3rdparty/webkit/JavaScriptCore/wtf/MathExtras.h 2010-07-09 13:26:40.436662765 +0200
@@ -190,7 +190,7 @@
inline float rad2grad(float r) { return r * 200.0f / piFloat; }
inline float grad2rad(float g) { return g * piFloat / 200.0f; }

-#if !COMPILER(MSVC) && !COMPILER(RVCT) && !OS(ANDROID) && !COMPILER(WINSCW)
+#if !COMPILER(MSVC) && !COMPILER(RVCT) && !OS(ANDROID) && !COMPILER(WINSCW) && !OS(SOLARIS)
using std::isfinite;
using std::isinf;
using std::isnan;


and virtualbox compiled with './configure --disable-docs --with-qt-dir=/opt/vboxose-qt/'

But still my files are in 2 or more times bigger that original one.
Is there any trick I don't know?

Thank you
Yevhen

[eduma]

Also in build result I didn't see libQt files or files related to vrdp.

Regards
Yevhen

[klaus]

You'd be the first to prepare an IPS package. We made lots of experiments, but so far didn't manage to produce an IPS package which provides the same user install experience as the SysV PKG. There's not much to share at the moment, since there's no point in publishing the obvious (just packing the files and let the admin reboot the system (!) after install)...

Be our guest, we'd love to be told that we simply misunderstood everything.

[eduma]

Hello klaus,

Next is the script I have done that translate virtualbox pkg into IPS
It downloads pkg from virtualbox site, translate, make smf etc, also it make run-once smf service to run postinstall script.

As I told before, it use ready pkg, but I'd like to have virtualbox compuled from sources, and only after that create ips packege.

Regards
Yevhen

Code: Select all

#!/bin/bash
### Variables, need to review and change:
temp="/temp"
repopath="/export/repo"
reponame="yevhen"
pkgserver="pkg/server"

#getting version number
#next lines - we get list of all versions, sort them and get last one
echo "Getting virtualbox versions from server ..."
tempfile='/tmp/vbox.temp'
wget -O $tempfile -o /dev/null http://download.virtualbox.org/virtualbox/
version=`cat $tempfile  | awk '{print $2}' | cut -d'=' -f2 | grep -e "[0-9].[0-9]" | grep -v "_" |sed  's#/##g'| sed 's#"##g'| sort --version-sort | tail -1`
allversions=`cat $tempfile  | awk '{print $2}' | cut -d'=' -f2 | grep -e "[0-9].[0-9]" | grep -v "_" |sed  's#/##g'| sed 's#"##g'| sort --version-sort`
#User can choose - download latest version or other one
echo "Latest VirtualBox version - $version. Create IPS package for v.${version}? [Y or ENTER/n]"
read answer
if [ "$answer" != "y" -a "$answer" != "Y" -a x"$answer" != "x" ];
then
	echo "Choose available version:"
	echo $allversions
	read version
fi
#since we know needed version we can build link to download vbox
#getting download link
wget -O $tempfile -o /dev/null http://download.virtualbox.org/virtualbox/${version}
#File should contain work sunos, so we get all files and take only with sunos in the name
filename=`cat $tempfile | awk '{print $2}' | grep -i "Sunos" | cut -d'=' -f2 | grep -e "[0-9].[0-9]" | grep -v "_" |sed  's#/##g'| sed 's#"##g'`
link="http://download.virtualbox.org/virtualbox/${version}/${filename}"
#This will be working directory, where script will download vbox, and create package
tempfolder="VirtualBox-`date +%s`"
mkdir -p ${temp}/${tempfolder}
echo "Downloading VirtualBox ... "
wget -o /dev/null -O ${temp}/${tempfolder}/${filename} $link
cd ${temp}/${tempfolder}
#untar downloaded file
pkgfile=`tar -zxvf $filename | grep ".pkg"`
echo "Uncompressing $filename ..."
mkdir ips
#this will transform/unpack pkg file 
pkgtrans $pkgfile ${temp}/${tempfolder}/ips/ all
cd ips
#pkg contain postinstall script, but philosophy of ips - no more postinstall scripts that cause problems, so we need to delete them before create ips
deletefiles=`ls SUNWvbox/install`
for deletefile in $deletefiles
do
sed "/${deletefile}/d" SUNWvbox/pkgmap > SUNWvbox/pkgmap.tmp
mv SUNWvbox/pkgmap.tmp SUNWvbox/pkgmap
done
rm -rf SUNWvbox/install
#here will be part for postinstall script
mkdir -p ${temp}/${tempfolder}/ips/SUNWvbox/smf/
#variables for paths
smffile="${temp}/${tempfolder}/ips/SUNWvbox/smf/virtualbox.xml"
#manually, just echo data info xml file
echo "<?xml version=\"1.0\"?> " > $smffile
echo "<!DOCTYPE service_bundle SYSTEM \"/usr/share/lib/xml/dtd/service_bundle.dtd.1\">">> $smffile
echo "<service_bundle type='manifest' name='VirtualBox:run-once'>">> $smffile
echo "<service">> $smffile
echo " name='system/virtualbox/run-once'">> $smffile
echo " type='service'">> $smffile
echo " version='1'>">> $smffile
echo "<single_instance />">> $smffile
echo "<dependency">> $smffile
echo " name='fs-local'">> $smffile
echo " grouping='require_all'">> $smffile
echo " restart_on='none'">> $smffile
echo " type='service'>">> $smffile
echo "<service_fmri value='svc:/system/filesystem/local:default' />">> $smffile
echo "</dependency>">> $smffile
echo "<dependent">> $smffile
echo " name='myapplication_self-assembly-complete'">> $smffile
echo " grouping='optional_all'">> $smffile
echo " restart_on='none'>">> $smffile
echo "<service_fmri value='svc:/milestone/self-assembly-complete' />">> $smffile
echo "</dependent>">> $smffile
echo "<instance enabled='true' name='default'>">> $smffile
echo "<exec_method">> $smffile
echo " type='method'">> $smffile
echo " name='start'">> $smffile
echo " exec='/opt/VirtualBox/ipsinstall.sh --ips'">> $smffile
echo " timeout_seconds='0'/>">> $smffile
echo "<exec_method">> $smffile
echo " type='method'">> $smffile
echo " name='stop'">> $smffile
echo " exec=':true'">> $smffile
echo " timeout_seconds='0'/>">> $smffile
echo "<property_group name='startd' type='framework'>">> $smffile
echo "<propval name='duration' type='astring' value='transient' />">> $smffile
echo "</property_group>">> $smffile
echo "<property_group name='config' type='application'>">> $smffile
echo "<propval name='assembled' type='boolean' value='false' />">> $smffile
echo "</property_group>">> $smffile
echo "</instance>">> $smffile
echo "</service>">> $smffile
echo "</service_bundle>">> $smffile

echo "Generating ips package ..."
#next lines will create p5m file and publish into repository
echo "set name=pkg.fmri value=pkg://${reponame}/service/virtualbox@${version},5.11-1.0" > ./SUNWvbox/virtualbox.p5m
pkgsend generate ./SUNWvbox | pkgfmt >> ./SUNWvbox/virtualbox.p5m
echo "file smf/virtualbox.xml path=var/svc/manifest/system/virtualbox.xml owner=root group=root mode=644 restart_fmri=svc:/system/manifest-import:default" >> ./SUNWvbox/virtualbox.p5m
#publish
pkgsend publish -d ./SUNWvbox -s $repopath ./SUNWvbox/virtualbox.p5m
#End of vbox package generation

# Generationg VirtualBox Extension Pack
#next lines will download same ext file as version of virtualbox
echo "Downloading VirtualBox Extension Pack ..."
vboxpath="root/opt/VirtualBox"
mkdir -p VBoxExt/${vboxpath}
mkdir -p VBoxExt/smf
wget -O $tempfile -o /dev/null http://download.virtualbox.org/virtualbox/${version}
filename=`cat $tempfile | awk '{print $2}' | grep -i "vbox-extpack" | cut -d'=' -f2 | grep -e "[0-9].[0-9]" | tail -1 | sed  's#/##g'| sed 's#"##g'`
link="http://download.virtualbox.org/virtualbox/${version}/${filename}"
rm $tempfile
wget -o /dev/null -O ${temp}/${tempfolder}/ips/VBoxExt/${vboxpath}/${filename} $link
#variables for paths
smffile="${temp}/${tempfolder}/ips/VBoxExt/smf/virtualbox-ext.xml"
p5mfile="${temp}/${tempfolder}/ips/VBoxExt/virtualbox-ext.p5m"
echo "Preparing files to packege virtualbox-ext... "
#creating smf
#manually, just echo data info xml file
echo "<?xml version=\"1.0\"?> " > $smffile
echo "<!DOCTYPE service_bundle SYSTEM \"/usr/share/lib/xml/dtd/service_bundle.dtd.1\">">> $smffile
echo "<service_bundle type='manifest' name='VirtualBox-Extension:run-once'>">> $smffile
echo "<service">> $smffile
echo " name='system/virtualbox-ext/run-once'">> $smffile
echo " type='service'">> $smffile
echo " version='1'>">> $smffile
echo "<single_instance />">> $smffile
echo "<dependency">> $smffile
echo " name='fs-local'">> $smffile
echo " grouping='require_all'">> $smffile
echo " restart_on='none'">> $smffile
echo " type='service'>">> $smffile
echo "<service_fmri value='svc:/system/filesystem/local:default' />">> $smffile
echo "</dependency>">> $smffile
echo "<dependent">> $smffile
echo " name='myapplication_self-assembly-complete'">> $smffile
echo " grouping='optional_all'">> $smffile
echo " restart_on='none'>">> $smffile
echo "<service_fmri value='svc:/milestone/self-assembly-complete' />">> $smffile
echo "</dependent>">> $smffile
echo "<instance enabled='true' name='default'>">> $smffile
echo "<exec_method">> $smffile
echo " type='method'">> $smffile
echo " name='start'">> $smffile
echo " exec='/opt/VirtualBox/VBoxManage extpack install /opt/VirtualBox/${filename}'">> $smffile
echo " timeout_seconds='0'/>">> $smffile
echo "<exec_method">> $smffile
echo " type='method'">> $smffile
echo " name='stop'">> $smffile
echo " exec=':true'">> $smffile
echo " timeout_seconds='0'/>">> $smffile
echo "<property_group name='startd' type='framework'>">> $smffile
echo "<propval name='duration' type='astring' value='transient' />">> $smffile
echo "</property_group>">> $smffile
echo "<property_group name='config' type='application'>">> $smffile
echo "<propval name='assembled' type='boolean' value='false' />">> $smffile
echo "</property_group>">> $smffile
echo "</instance>">> $smffile
echo "</service>">> $smffile
echo "</service_bundle>">> $smffile

### Creating p5m
echo "set name=pkg.fmri value=pkg://${reponame}/service/virtualbox-ext@${version},5.11-1.0" > $p5mfile
echo "set name=pkg.summary value=\"Oracle VM VirtualBox\" ">> $p5mfile
echo "set name=pkg.description value=\"A powerful PC virtualization solution\" ">> $p5mfile
echo "set name=pkg.send.convert.email value=info@virtualbox.org" >> $p5mfile
echo "dir  path=opt/VirtualBox owner=root group=bin mode=0755" >> $p5mfile
echo "file root/opt/VirtualBox/${filename} path=opt/VirtualBox/${filename} owner=root group=bin mode=0755" >> $p5mfile
echo "file smf/virtualbox-ext.xml path=var/svc/manifest/system/virtualbox-ext.xml owner=root group=root mode=644 restart_fmri=svc:/system/manifest-import:default" >> $p5mfile
echo "depend type=require fmri=pkg://${reponame}/service/virtualbox@${version},5.11-1.0" >> $p5mfile

### Genetating vbox-ext package
echo "Genetating ips package ..."
pkgsend publish -d ./VBoxExt -s $repopath ./VBoxExt/virtualbox-ext.p5m

pkgrepo refresh -s  $repopath
svcadm restart $pkgserver
echo "IPS package created and published. $pkgserver restarted."
rm -rf ${temp}/${tempfolder} > /dev/null 2>&1
rmdir ${temp}/${tempfolder}/ips > /dev/null 2>&1
rm -rf ${temp}/${tempfolder} > /dev/null 2>&1

[noteirak]

@eduma : do not post such long text without the [ code ] [ / code ] tags, or you can also attach it as zip.

[Ramshankar]

It downloads pkg from virtualbox site, translate, make smf etc, also it make run-once smf service to run postinstall script.

1. Does it automatically run the entire postinstall as root and install and bring up drivers and services *without* any extra steps from the user installing the package including not rebooting or running any commands manually?

2. Does it work when you upgrade packages to properly unload and reload all the drivers and services of the previous versions before updating to the new version without any extra manual intervention from the admin and without any reboots?

[eduma]

Hello Ramshankar,

I suppose you can try and check it)))
1) yes, it make run-once service that run once and installs drivers. (same as was in postinstall script)
2) I am not sure, I haven't test this case.

But I'd like to back to my question:
Even running kmk packing I receive files of same size but still different.
Is it possible to know exact environment that was used for compilation of version 4.1.12 and 4.2.10 for example?
Also I have tried with solaris 11 express (booting into 32bit mode) and no luck.

Thank you
Yevhen

[eduma]

Also on upgrade as I remember smf will do next:
1) delete all files, so all drivers will be deleted
2) install new drivers
3) run-once script will load new modules (old one I suppose will be loaded untill next reboot

[klaus]

But I'd like to back to my question:
Even running kmk packing I receive files of same size but still different.
Is it possible to know exact environment that was used for compilation of version 4.1.12 and 4.2.10 for example?
Also I have tried with solaris 11 express (booting into 32bit mode) and no luck.

I wonder if your expectations are wrong... the typical reason for differences between binaries are things like timestamps (both in the actual code and in the debug information). If we'd build the same source code twice we'd also get different results due to those reasons. There's simply no way to achieve 100% identical binaries, even if all build utilities are identical and the source code is identical.

[Ramshankar]

Also on upgrade as I remember smf will do next:
1) delete all files, so all drivers will be deleted
2) install new drivers
3) run-once script will load new modules (old one I suppose will be loaded untill next reboot

Unfortunately, that's not an acceptable solution for us - which is why we don't really publish any IPS packages. In an upgrade, the old modules must be fully unloaded and inactive, and the new modules must be loaded and active. Running newer VirtualBox binaries with older drivers is just asking for trouble.

While I appreciate the effort you're taking, it would only be practical for us to implement it if the initial installation, upgrade and uninstallation all worked without any extra manual steps from the admin (including not having to reboot at any stage).

[eduma]

I suppose it's possible to on disable step for run-once smf its possible to add steps you need (sorry I had just quick view on run-once smf).

But I'd like to back to my question:

I am trying to be sure that my compilation is same as your for solaris hosts, but still my files (even with same size) different from original one, and I didn't receive files related vrdp.

Thank you
Yevhen

[eduma]

I wonder if your expectations are wrong... the typical reason for differences between binaries are things like timestamps (both in the actual code and in the debug information). If we'd build the same source code twice we'd also get different results due to those reasons. There's simply no way to achieve 100% identical binaries, even if all build utilities are identical and the source code is identical.

Thank you klaus for your answer.

Yes, that's correct, but I still wondering how you get i386 files (I suppose running oracle solaris 11 express in 32bit mode, changs in grub/menu.lst file?)
and about vrdp, still have no luck, and at least one test was very unstable.

Thank you
Yevhen

[Ramshankar]

If your Solaris has support for compiling 32-bit applications, and you've booted Solaris kernel in 64-bit mode, you can cross compile for Solaris using:

Code: Select all

kmk KBUILD_TARGET_ARCH=x86

After having both 32 and 64-bit binaries built, you would have to do combined packing of 32 and 64-bit using:

Code: Select all

kmk VBOX_WITH_COMBINED_PACKAGE=1 packing

The combined package will then be in the 32-bit out.../bin folder (iirc).

[eduma]

Thank you Ramshankar,

And last question about vrdp - is there some specific keys? Since I didn't get any of next files
rdesktop-vrdp-keymaps
rdesktop-vrdp.tar.gz
rdesktop-vrdp
And same with libQt* files.

Thank you
Yevhen

[klaus]

Did you somehow disable building rdesktop? It should be done as part of a normal build, since the make variable VBOX_WITH_VRDP_RDESKTOP is always set by default. I wonder if the packaging uses the wrong conditionals to disable packing the rdesktop stuff on Solaris... it's checking VBOX_WITH_QTGUI (which should be set too).

Do you set VBOX_HEADLESS anywhere? Are you passing --build-headless to the configure script? This shouldn't be done, it's disabling everything except the headless parts!