Using Virtual drive as ram

[tadajin]

i think the one biggest features that virtual box is missing that could make it a world-wide leader in operating system virtualisation is the ability to somehow create a virtual drive that can be used as ram (somethin like a linux or sun 'Swap Space')? this would result in faster speed for virtualised operating system, less strain on the computer hardware.

[TerryE]

?? This is standard Linux funxtionality. Why reimplement it in VBox?

[tadajin]

it will enable users to use their hard drive as RAM, instead of taking up system RAM

[fixedwheel]

hard drive as RAM, instead of taking up system RAM

??! this is what standard swap space is for, and using hdd instead of RAM is slowing down by substantial order of magnitude

?? This is standard Linux funxtionality.

[tadajin]

using hard drive space will allow users to run potentially unknown ammount virtual pcs at the same time. instead of using the systems physical ram, it uses hard drive space, using that as its ram. that way, you won't be limited in how many virtual pcs you can run at one given time

[fixedwheel]

using hard drive space will allow users to run potentially unknown ammount virtual pcs at the same time. instead of using the systems physical ram, it uses hard drive space, using that as its ram.

this can be done by giving swap space to the host: standard linux functionality!

that way, you won't be limited in how many virtual pcs you can run at one given time

that way your machine(s) will be terribly slowed down. And RAM is quite cheap now ...

[mark rumsey]

I can sort of see this from both sides. For standard installs, simulating RAM with any form of drive (HDD, solid state or USB stick) is pointless. Simulated ram would be considerably slower than physical ram, probably 100 times slower (maybe more for the latest modern hardware). There's also the problem of how the guest OS would use it. The guest would not know if it was writing to real RAM or simulated RAM, which means you could end up with the ridiculous scenario of the guest OS swapping data from simulated RAM (which it believes is real RAM) into its own swap file to make room for running other applications. Clearly, this would be nothing more than moving data from one part of the disk to another for no benefit, which would reduce performance.

Now, if there was not simulated RAM the guest OS would be swapping more to its own swap file, but this would be moving in and out of real RAM where real things are done, so in fact having simulated RAM using files on disk could actually make a guest OS run slower! The only scenario I can see where simulated ram would be useful is for testing a new guest OS on a machine that doesn't have enough physical RAM to support it. For example, I wouldn't be able to run Windows7 as a guest on my PC because I only have 512MB of RAM, which means I can't really allocate more than 256mb to a VM. If I had the ability to create simulated RAM I would be able to run Windows7, but I suspect it would be extremely slow, possibly to the point it would be unusable and impossible to evaluate.

[TerryE]

There are current architectural reasons why this is not feasible. The guest OS kernel essentially treats pages in one of two categories: either pages are in memory or they are swapped out to some pagefile. There are all sorts of time critical assumptions made in the guest OS code and drivers assuming that if a page is in memory then it will take sub-?S (not multiple mS) to access it. If you breach this assumption then the guest OSs will tend to panic / BSoD and die. Hence VBox honours this rule, and locks any pages that the guest VM believes is in memory into the host memory. All of the mainstream virtualisation products such as VMware, MS and Xen do exactly the same.

VMware ESX does have some sophisticated algos to save memory: one is to scan for common pages in the VMs (e.g. Windows OS code) and allocate these as shared read, clone on write pages. Another is to extend the guest additions with a "balloon" process which detects if the guest VM is idling and slowly balloons up occupying guest memory and forcing other processes to flush to the pagefile; this process then tells the host that these balloon pages are special filler pages and so the VBox VMM doesn't actually need to allocate memory for them.

However both of these tricks really start to pay dividends if you have lots of similar VMs running, and if you read the VMware support forums there are lots of bitches about tuning these systems so that your host system doesn't run to a standstill from page faulting.

[emperos]

One comment
*******Ramdisk!!!!!!!!!!!

Ram disks can be used to utilized unmanaged ram. im running a 32bit xp operating system as my primary host os. if i coud use my unamanged ram using a ramdisk to be used as my guest os's ram. this would literally make all ram needed to be run by my guest os on my unamaged ram. allowing for seamless integration to barely even effect my host os.. other then video cards and cpu usage
p.s. dont just say use 64bit os. im useing a 64bit ubuntu for the heavy virtualization of servers. my 32bit is for gaming but i love virtualbox and the seamless integration ability, nothing like easily having all of my lovely linux software right there for my use

[mark rumsey]

emperos, what this thread is talking about is doing it the other way around, i.e. simulating ram using a file on a drive where there is not enough physical ram to support the host/guest combination.

[emperos]

while this is true, they would both utilize the same function, using a virtual drive potentially under drive letter X or using a folder as a virtual drive. even though we want it for different reasons, i want the same feature because it would fill in a gap for my x86 architecture operating system. so far it sounds as if the feature will not be implemented, so i decided to add my two cents about how this feature could be good for the community.

(decided not to open another tread requesting the same feature, for one it would of had the same name.)

[mark rumsey]

Let me see if I understand you correctly. You have a 32bit XP host on a PC with 4GB or greater of RAM installed. XP itself is only seeing about 3.2GB leavng you with a chunk of memory that cannot be accessed. What you are suggesting is using this RAM somehow with VirtualBox. Ideally VirtualBox itself would make that unmanaged memory directly available to the guest as RAM or, as you suggest, the RAM would be used to create a RAMDisk using suitable software that can get at the RAM XP can't use (such software does exist), then use that RAMDisk in VirtualBox to simulate real memory that can be used by the host. This would then allow a system with, say, 8GB RAM to support a guest using 3-4GB RAM even on a 32 bit host.

Its an interesting idea, but I can see a couple of possible flaws. Taking the first scenario (VirtualBox directly accesses the unmanaged memory) I think the problem here (and VBox developers please correct me if I'm wrong) is that VirtualBox itself can only use the memory presented to it by the operating system, and if that is only 32bit the memory available to VirtualBox will be limited to sub 4GB figures regardless of how much is physically installed.

In the second scenario (RAMDisk) I suspect there could be a significant performance overhead. The problem is the RAMDisk could realistically only be used as a paging/swap file by VirtualBox to swap out chunks of 'real' memory from the guest to give it more breathing space. This might sound simple, but it probably isn't. The problem as I see it (again can the VBx developers please tell me if I'm talking nonsense here) is down to memory addressing. I do not know how an operating system handles memory addresses when a swapfile is used, but I suspect it is probably as simple as moving the data from one address (in RAM) to a new address (in the swapfile) and ensuring the applications that might need that data in the future know about the move and adjust their addresses accordingly. I don't believe this method could be made to work for a virtual machine. Instead I think the entire address structure covering the real and swapfile memory would need to be created in (for want of a better word) memory, then the physical RAM in the machine would get moved around this memory structure, changing its address as it goes, writing data to the swapfile and reading data back as the window the swapile occupies is shifted aroud. This is not only going to involve a fair performance overhead as things get shunted between physical memory and the swapfle and moved between locations in the physical memory. There will also be a further performance hit from the memory manager that has to determine what moves where, not to mention the physical memory VirtualBox itself will need to manage this and possibly cache it, which means less real RAM for the guest, increasing the amount of swapping further still.

I can see a positive benefit for certain specific tasks, such as testing where time and performance are unimportant aren't important, or where the guest has to be run alongside an old piece of software that will only work on a 32bit operating system and that software cannot be (or will not) run inside a virtual machine.

[emperos]

Wow, now that is a well thought out reply. you are completely correct about what i was aiming for. i do hope the VBox developers can say you are "talking nonsense" because i have yet to research how the swapfile is truly used, (or i forgot the research that ive done) that would also give hope to my current idea. so far im using ramdisc for my pagefile(does slow down initial startup, but it appears to boost overall performance... i think), and for caching frequently used files for games that use the harddrive constantly. having virtualdisk-ram for virtualbox would be an interesting feature that i would love to have, but in the end it would see very little use, at least for me.(little use because of current setup, but if it worked nicely allot of use. that is if your second point is wrong)(current setup is pageing file in ram, would have to restart to change)

also if anything in the reply is strange or misstated i apologize i'm currently running a fever of ~102.4 so its a bit hard to gather my thoughts.

[mitio2]

I have 4GB limit on my laptop. if I could use virtual drive to replace RAM I would test a 8GB system without using a bit of RAM. Wouldn´t it be good ?

[mark rumsey]

The 4GB limit is more a limitation of 32bit computing. In simple terms, a 32 bit OS or processor simply cannot access memory beyond 4GB because it runs out of numbers to use as addresses (2^32 = 4294967296 which is the biggest number a 32 bit processor can handle and is the number of bytes in 4GB). For this reason all 32 bit PCs were built with a maximum memory capacity of 4GB, and until recently most 64 bit processor based PCs also had this limit because they predominantly ran 32 bit operating systems. It is only the most recent PCs that can take more than 4GB, and even then they can only make use of it with a 64bit operating system.

Simulating RAM using hard drive space as already mentioned is going to be extremely slow. Hard drives, even the best solid state drives, are 100s of times slower than real memory. To give you an idea of the performance if a machine was set up to use virtual ram was created from a hard disk file, imagine this. A typical Windows XP installation takes about 2 minutes to boot and settle down. Using virtual ram from a hard disk file, that same XP installation would take over 3 HOURS to boot, and that's assuming you use a top quality solid state drive. Use a standard hard drive and it could take 36 HOURS to boot! Now imagine trying to do anything with it. As I said, being able to use hard drive space to simulate memory might be useful for those situations where performance is not important, and as you can see, I really do mean not important!