Inconsistent information about memory in real-mode

[prograholic]

I need memory region for my application before OS (Windows) was started, so i intercept BIOS function INT15h E820 and modify memory map:
Before modification memory map (only 1st segment):
1. 0x00000000:0x0009FC00, type 1 - free memory

After modification:
1. 0x00000000:0x00093000, type 1 - free memory
2. 0x00093000:0x0000C800, type 2 - reserved
3. 0x0009F800:0x00000400, type 1 - free memory

After such modifications Windows cannot boot (it crashes somewhere in kernel).

I found that problem is raised when windows starts using 3rd region in memory map.

Windows loader uses only INT15h E820 for obtaining memory map, it does not use BDA memory limit. So windows loader assume that segment 3 is free, and starts using it, and we get crash. Also i suppose that problem still exists without any modifications of memory map (future versions of boot loaders may use more memory).

I think that BIOS function INT15h E820 should be synchronized with BDA memory limit. For example it can generate following segments:
1. 0x00000000:<BDA_LIMIT>, type 1
2. <BDA_LIMIT>:(0x0009FC00 - <BDA_LIMIT>), type 2
3. 0x0009FC00:...

[michaln]

Two things I don't understand. You say that "windows loader assume that segment 3 is free" but you show the map reporting segment 3 as free memory.

You also say that the memory map should be synchronized with the BDA, but you show the map as declaring 0-9FC00h to be free memory, which covers 639K without the BDA. The BDA is not reported as free, so I don't quite get he problem?

You also never mentioned which version of Windows you're talking about. Is there only one?

[prograholic]

Sorry, i did not provide BDA limit.

BDA returns 0x027E - which is equal to 0x0009F800
E820 returns fragment 0x00000000:0x0009FC000, type 1.

Such state can be seen right after BIOS transferred execution to MBR (at the very early stage).

So there is a mismatch between BDA and E820. Furthermore if i modify BDA memory limit it does not update memory map in E820 - this is the problem.

My modifications of memory map only shows that problem affects windows loader.

p.s. I reproduce this effect on Windows7 x64

[michaln]

Sorry, i did not provide BDA limit.

BDA returns 0x027E - which is equal to 0x0009F800
E820 returns fragment 0x00000000:0x0009FC000, type 1.

Ah, your VM is probably configured to use AHCI... in which case yes, the BDA will be bigger. You are probably right that this should be reflected in the E820 map. I need to look up the relevant specs.

Furthermore if i modify BDA memory limit it does not update memory map in E820 - this is the problem.

I'm afraid that if you move the BDA around, you are ultimately responsible for adjusting the E820 memory map. Keep in mind that the BDA can be anywhere, it does not have to occupy the last N kilobytes of the 640K conventional memory. I don't think it's the BIOS's job to keep BDA modifications (beyond what the BIOS itself does) into account for E820 reporting, but again I'll do a bit of research.

Although if I understand you correctly, you are actually not moving or resizing the BDA? You are only adding another "reserved" region just below it?

My modifications of memory map only shows that problem affects windows loader.

Actually, am I understanding correctly that your modified memory map is in fact clashing with the real BDA? Is it the Windows loader that's crashing or is it the BIOS itself?

You could test with IDE instead of SATA, that ought to use only a 1K BDA.

[prograholic]

Yes, the problem with AHCI, i'l try to test with IDE.

Actually, am I understanding correctly that your modified memory map is in fact clashing with the real BDA?

Yes

Is it the Windows loader that's crashing or is it the BIOS itself?

Actually crash occurs in ntoskrnl.exe while initializing kernel memory manager. I do not have much information, but is suppose that real memory corruption occurs either in bootmgr (1st stage) or in winload (2nd stage), because these two stages work heavily with INT15 E820 and INT13h 42 (disk IO)

Also i do not change the location of BDA, it still located at initial position, i only modify E820 memory map, and change memory limit at address 0x00000413

[michaln]

To be clear, I think you're right that the VirtualBox BIOS can return incorrect information in the E820h memory map.

But my memory has fooled me... it's not the BDA that is bigger. There is additional conventional memory "stolen" from below the BDA, and the memory size at 40:13 is reduced accordingly.

[prograholic]

To be clear, I think you're right that the VirtualBox BIOS can return incorrect information in the E820h memory map.

But my memory has fooled me... it's not the BDA that is bigger. There is additional conventional memory "stolen" from below the BDA, and the memory size at 40:13 is reduced accordingly.

Ok, that sounds good, but i want to clarify meaning of BDA. According to this page http://wiki.osdev.org/BDA#BIOS_Data_Area_.28BDA.29 BDA is located at address 0x400 (40:00), and EBDA is located somewhere else. For example on virtualbox it is started from address 0x9FC00.

So when i say BDA i mean address 0x400.

[michaln]

Sorry! When I said BDA I really meant the EBDA, or Extended BIOS Data Area. This is the variable-size (but at least 1KB big) memory area that the word at 40:13 points to, normally located in the last kilobyte (or more) just below 640KB.

Could you please explain what precisely you mean by "BDA memory limit"? I'd rather avoid guessing.

[prograholic]

Sorry! When I said BDA I really meant the EBDA, or Extended BIOS Data Area. This is the variable-size (but at least 1KB big) memory area that the word at 40:13 points to, normally located in the last kilobyte (or more) just below 640KB.

Could you please explain what precisely you mean by "BDA memory limit"? I'd rather avoid guessing.

I mean memory limit which is located at 0x0413 (40:13). It is the part of BDA, so i called it BDA memory limit

[michaln]

OK, that's what used to be called simply "memory size", later maybe "base memory size". Anyway, it's now crystal clear what you mean and yes, INT 15h/E820h should take that into account instead of hardcoding some values. We'll fix that.

I'm still curious though, did your code work when you switched the VM to use IDE instead of SATA? Because then the INT 15h/E820h memory map should be consistent with actual memory usage.

[prograholic]

I'm still curious though, did your code work when you switched the VM to use IDE instead of SATA? Because then the INT 15h/E820h memory map should be consistent with actual memory usage.

Yes, with IDE controller everything is OK (SATA controller should be removed from VM configuration).