Page 7 of 13

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 21. Jun 2009, 18:38
by csavery
Actually I was simply pointing out that if you're going to make a HOWTO and label it as such then it would be good that it have the minimal info needed so that it will get someone going in the simplest case. I'm using VirtualBox in it's out of the box simplest configuration, as a non-root user with no desire for other users to gain access. After reading and trying to get this working in this simplest use scenario people here start coming back and asking if I did this or did that. Well, no. I didn't because it's not in the HOWTO and furthermore it doesn't even mention that maybe users should or may need to do this or that other things.

I've spent many hours of my time contributing back to the community by endlessly answering questions on the Ubuntu forums. I do appreciate that you put in time to make this available as it is something that is useful and needs to be out there. From what I can tell other approaches don't work with dynamic VDI images. The HOWTO here says this works with any vdi which I took to mean it does work with dynamic ones. So I consider this important and a worthwhile community addition.

I just think that if you are going to put it there and write up a HOWTO then it would be good for it to be helpful in at least the simplest average users expectations. Perhaps it just hasn't had time to mature from other users feedback. While I have been a programmer as a career for 20 years I'm not inclined to start into the source of tools, and figure them out from the inside, to get the most basic functionality to work. It was obviously a mistake to read your HOWTO and think that I could "just use" the tool. Forgive me for trying.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 23. Jun 2009, 12:28
by Hubbitus
Thank you very much guys!
And additional thanks to patch in viewtopic.php?p=80524#p80524 now I can mount it.

But I have strange behavior (I think this is incorrect caching in implementation):
Code: Select all   Expand viewCollapse view
# LANG=C ls -l /mnt/ttt
total 14776140
-rw------- 1 pasha users 7571767296 Jun 23 13:56 EntireDisk
-rw------- 1 pasha users 7559000064 Jun 23 13:56 Partition1


So, accessible only one from this file which was be accesed first after mount! So, say I mount it and do (please note, I;m do it as root, to do not believe access-denied messages):
Code: Select all   Expand viewCollapse view
# file Partition1
Partition1: x86 boot sector, code offset 0x52, OEM-ID "NTFS    ", sectors/cluster 8, reserved sectors 0, Media descriptor 0xf8, heads 255, hidden sectors 63, dos < 4.0 BootSector (0x80)

All cool.
But second file:
Code: Select all   Expand viewCollapse view
# file EntireDisk
EntireDisk: writable, executable, regular file, no read permission


If I first access to EntireDisk, then next access to Partition1 return this error.


P.S. And one more question. What license of this code? Can I, as Fedora maintainer add it to Fedora official repository?

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 23. Jun 2009, 15:54
by TerryE
h2o or I need to expand the HowTo in the light of the feedback that we've got. It's on my list but unfortunately there's a few higher priority jobs on at the moment.

If you look at the code the EntireDisk and Partition views interlock. That was a KISS solution to the integrity problem of filesystem vs. raw access: you can't do both on the same mount. You can simply reset this by umounting and remounts the VDI.

As to licence I am pretty relaxed, but I remember the guy who wrote SQLite talking about "no licence" can be a real hassle for the distributors. Any attributed FLOSS licence would be OK, and given that the bulk of the code is released under GPL2 then it's probably simplest for that to apply here as well, even though I think that some of the GPL2 terms are unnecessarily burdensome. I'll contact h2o to see if he want to add his real name, add the patches and copyright notice and repost the version.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 28. Jun 2009, 20:24
by bmwiedemann
There is actually another way to mount dynamic VDI files and VMDK on linux hosts.
This involves a qemu-nbd to export the image as (quasi raw) Network Block Device and a nbd-client to be able to use it as raw block device for the disk. That might still need some fiddling with mount -o loop,offset=xx to get to the partitions.

Of course, you will need a version of Qemu, that is patched to read/write VDI images such as mine (auto-built for openSUSE) http://download.opensuse.org/repositori ... SUSE_11.1/

example calls:
Code: Select all   Expand viewCollapse view
$ qemu-img info /tmp/lenny.vdi
image: /tmp/lenny.vdi
file format: vdi
virtual size: 2.0G (2147483648 bytes)
disk size: 825M

$ qemu-nbd /tmp/lenny.vdi &
$ sudo nbd-client localhost 1024 /dev/nbd0
Negotiation: ..size = 2097152KB
bs=1024, sz=2097152
$ sudo fdisk -lu /dev/nbd0
Disk /dev/nbd0: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x0001ea41

     Device Boot      Start         End      Blocks   Id  System
/dev/nbd0p1   *          63     3887729     1943833+  83  Linux
/dev/nbd0p2         3887730     4192964      152617+   5  Extended
/dev/nbd0p5         3887793     4192964      152586   82  Linux swap / Solaris

# mount Partition 1 that starts at sector 63
$ sudo mount -o loop,offset=`expr 512 \* 63` /dev/nbd0 /mnt
$ df /mnt
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/loop0             1913244    356100   1459956  20% /mnt

Can somebody create an initrd.img for this?

PostPosted: 28. Jul 2009, 01:56
by chronoboy
An interesting idea here...

Now that we are able to mount the VDI perfectly fine in Linux, would it be possible to use kboot and an initrd.img to boot the VDI on the host's hardware natively, if the VBox guest additions have not been installed? Distros like Debian do not store a hardware database and do re-detections on each boot.

initrd.img's can use loopback devices, however I'm not sure if they can use fuse. Another neat idea would be to build this type of functionality into the Linux device mapper. Of course, by doing this, it will no longer be cross-platform, however it will provide the ability to mount VDIs via dev/mapper and use VDI for many purposes beyond simply file copies. dd would work almost instantly if a version of vdfuse was made for devmapper. devmapper is also very powerful and can increase what you can use vdfuse for. Using phyical device nodes rather than loop devices is more of a preference for me, as you can do a lot more with device nodes than a loop device.

Now, if only it were possible to boot Windows natively like this, as in the next version of Windows, Microsoft is adding a feature to boot directly into VHDs. Perhaps the GRUB developers may have some insite on how this can be done.

Re: Can somebody create an initrd.img for this?

PostPosted: 28. Jul 2009, 23:11
by ksmyth
chronoboy wrote:Now that we are able to mount the VDI perfectly fine in Linux, would it be possible to use kboot and an initrd.img to boot the VDI on the host's hardware natively, if the VBox guest additions have not been installed? Distros like Debian do not store a hardware database and do re-detections on each boot.


Ubuntu has an installer called 'Wubi' that creates a disk image on an NTFS filesystem and configures the windows bootloader to use it. I think the initrd image uses fuse (i.e. ntfs-3g) to mount the NTFS filesystem. Seems like a good starting point.

https://help.ubuntu.com/community/Wubi

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 2. Aug 2009, 06:26
by jagdpanther
I am getting the "invalid argument" error that I have seen earlier in this thread. I successfully compiled vdfuse-v50 using gcc-4.3.2 on a 64-bit Gentoo based system. (Kernel: 2.6.29.6) Any ideas on how to prevent this "invalid argument" error? ('user_allow_other' is set in /etc/fuse.conf)

Code: Select all   Expand viewCollapse view
#  ./vdfuse-v50  -v -r -f /sdb8/vbox-drives/Ubuntu.vdi /tmp/vmount
disktype is VDI
fuse: mount failed: Invalid argument


Thanks

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 2. Aug 2009, 13:30
by TerryE
Use the debug flags. Does the mount point /tmp/vmount exist? Fuse doesn't automatically create mount points.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 2. Aug 2009, 15:47
by jagdpanther
Yes, the mount point exists. I ran 'mkdir /tmp/vmount' just before running the command.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 2. Aug 2009, 15:51
by jagdpanther
I re-ran the command with the debug flag set. I get the same output ...

Code: Select all   Expand viewCollapse view
./vdfuse-v50  -v -d -r -f /sdb8/vbox-drives/Ubuntu.vdi /tmp/vmount
disktype is VDI
fuse: mount failed: Invalid argument

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 3. Aug 2009, 01:03
by jagdpanther
Problem solved. I tried the following patch posted on the last page :

Code: Select all   Expand viewCollapse view
- fuse_opt_add_arg (&fuseArgs, "");
+ fuse_opt_add_arg (&fuseArgs, "vdi");


Now vdfuse-v50 works on my Gentoo based system.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 25. Aug 2009, 11:50
by kh92
Did anyone compile vdfuse under MacOSX? I'd be interested in a working shell script for building.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 26. Aug 2009, 01:45
by TerryE
Not to my knowledge, but MacOS is biuld on a BSD shell which supports Fuse, so in principle it should work. Try it.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 26. Aug 2009, 09:48
by kh92
TerryE wrote:Not to my knowledge, but MacOS is biuld on a BSD shell which supports Fuse, so in principle it should work. Try it.


That's my intention, I just wanted to make sure first that I am duplicating an existing effort. It should be possible, as you said, because Fuse is available, but I still expect some modifications to be necessary because Fuse is a framework on the Mac, and because dynamical libraries work quite differently.

Re: Discuss: Mount any VBox-compatible disk image on the host

PostPosted: 26. Aug 2009, 11:27
by kh92
kh92 wrote:
TerryE wrote:Not to my knowledge, but MacOS is biuld on a BSD shell which supports Fuse, so in principle it should work. Try it.

That's my intention, I just wanted to make sure first that I am duplicating an existing effort. It should be possible, as you said, because Fuse is available, but I still expect some modifications to be necessary because Fuse is a framework on the Mac, and because dynamical libraries work quite differently.


Here is the Mac version of the build script. I didn't make much of an effort to detect where things are installed, given that 99.99% of Mac users will have both MacFUSE and VirtualBox installed in the default places, so these are hard-coded.

This will mount a hard disk image just fine, but this will just give raw access to the disk contents at the byte level. Since the Mac can't handle ext3 natively, another tool will be needed. My plan is to look at the ext2 implementation for Fuse, but that's for another day.

Code: Select all   Expand viewCollapse view
#!/bin/sh

# Environment variables (all optional)
# CFLAGS - flags for gcc
# NOSTRIP - don't strip output
# INSTALL_DIR - vbox install directory

INSTALL_DIR=/Applications/VirtualBox.app/Contents/MacOS

if [ $# -ne 2 ]; then
   echo "Usage: $0 include-dir vdfuse.c"
   exit 255
fi

if [ -z "${CFLAGS}" ]; then
   CFLAGS="-pipe"
fi

incdir="$1"
infile="$2"
outfile="${infile%.c}"

if ! [ -e "${infile}" ]; then
   echo "${infile} not found."
   exit 1
fi

oldvboxhdd=""
if [ -e "${incdir}/VBox/VBoxHDD-new.h" ]; then
   oldvboxhdd="-DOLDVBOXHDD"
elif ! [ -e "${incdir}/VBox/VBoxHDD.h" ]; then
   echo "Invalid include directory. Make sure that it has the VBox directory inside."
   exit 1
fi

gcc "${infile}" \
        ${INSTALL_DIR}/VBoxDDU.dylib \
        -o "${outfile}" \
        -v -Wl,-v \
   -I"${incdir}" \
   -I/usr/local/include/fuse \
   -Wall ${oldvboxhdd} \
        ${CFLAGS} \
        -L/usr/local/lib -l fuse


if [ -z "${NOSTRIP}" ]; then
   strip -x "${outfile}"
fi

if [ $? -eq 0 ]; then
   echo "Success!"
else
   echo "Compile Failed!"
fi