Inconsistent information about memory in real-mode
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Inconsistent information about memory in real-mode
I need memory region for my application before OS (Windows) was started, so i intercept BIOS function INT15h E820 and modify memory map:
Before modification memory map (only 1st segment):
1. 0x00000000:0x0009FC00, type 1 - free memory
After modification:
1. 0x00000000:0x00093000, type 1 - free memory
2. 0x00093000:0x0000C800, type 2 - reserved
3. 0x0009F800:0x00000400, type 1 - free memory
After such modifications Windows cannot boot (it crashes somewhere in kernel).
I found that problem is raised when windows starts using 3rd region in memory map.
Windows loader uses only INT15h E820 for obtaining memory map, it does not use BDA memory limit. So windows loader assume that segment 3 is free, and starts using it, and we get crash. Also i suppose that problem still exists without any modifications of memory map (future versions of boot loaders may use more memory).
I think that BIOS function INT15h E820 should be synchronized with BDA memory limit. For example it can generate following segments:
1. 0x00000000:<BDA_LIMIT>, type 1
2. <BDA_LIMIT>:(0x0009FC00 - <BDA_LIMIT>), type 2
3. 0x0009FC00:...
Before modification memory map (only 1st segment):
1. 0x00000000:0x0009FC00, type 1 - free memory
After modification:
1. 0x00000000:0x00093000, type 1 - free memory
2. 0x00093000:0x0000C800, type 2 - reserved
3. 0x0009F800:0x00000400, type 1 - free memory
After such modifications Windows cannot boot (it crashes somewhere in kernel).
I found that problem is raised when windows starts using 3rd region in memory map.
Windows loader uses only INT15h E820 for obtaining memory map, it does not use BDA memory limit. So windows loader assume that segment 3 is free, and starts using it, and we get crash. Also i suppose that problem still exists without any modifications of memory map (future versions of boot loaders may use more memory).
I think that BIOS function INT15h E820 should be synchronized with BDA memory limit. For example it can generate following segments:
1. 0x00000000:<BDA_LIMIT>, type 1
2. <BDA_LIMIT>:(0x0009FC00 - <BDA_LIMIT>), type 2
3. 0x0009FC00:...
-
- Oracle Corporation
- Posts: 2973
- Joined: 19. Dec 2007, 15:45
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Any and all
- Contact:
Re: Inconsistent information about memory in real-mode
Two things I don't understand. You say that "windows loader assume that segment 3 is free" but you show the map reporting segment 3 as free memory.
You also say that the memory map should be synchronized with the BDA, but you show the map as declaring 0-9FC00h to be free memory, which covers 639K without the BDA. The BDA is not reported as free, so I don't quite get he problem?
You also never mentioned which version of Windows you're talking about. Is there only one?
You also say that the memory map should be synchronized with the BDA, but you show the map as declaring 0-9FC00h to be free memory, which covers 639K without the BDA. The BDA is not reported as free, so I don't quite get he problem?
You also never mentioned which version of Windows you're talking about. Is there only one?
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Re: Inconsistent information about memory in real-mode
Sorry, i did not provide BDA limit.
BDA returns 0x027E - which is equal to 0x0009F800
E820 returns fragment 0x00000000:0x0009FC000, type 1.
Such state can be seen right after BIOS transferred execution to MBR (at the very early stage).
So there is a mismatch between BDA and E820. Furthermore if i modify BDA memory limit it does not update memory map in E820 - this is the problem.
My modifications of memory map only shows that problem affects windows loader.
p.s. I reproduce this effect on Windows7 x64
BDA returns 0x027E - which is equal to 0x0009F800
E820 returns fragment 0x00000000:0x0009FC000, type 1.
Such state can be seen right after BIOS transferred execution to MBR (at the very early stage).
So there is a mismatch between BDA and E820. Furthermore if i modify BDA memory limit it does not update memory map in E820 - this is the problem.
My modifications of memory map only shows that problem affects windows loader.
p.s. I reproduce this effect on Windows7 x64
-
- Oracle Corporation
- Posts: 2973
- Joined: 19. Dec 2007, 15:45
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Any and all
- Contact:
Re: Inconsistent information about memory in real-mode
Ah, your VM is probably configured to use AHCI... in which case yes, the BDA will be bigger. You are probably right that this should be reflected in the E820 map. I need to look up the relevant specs.prograholic wrote:Sorry, i did not provide BDA limit.
BDA returns 0x027E - which is equal to 0x0009F800
E820 returns fragment 0x00000000:0x0009FC000, type 1.
I'm afraid that if you move the BDA around, you are ultimately responsible for adjusting the E820 memory map. Keep in mind that the BDA can be anywhere, it does not have to occupy the last N kilobytes of the 640K conventional memory. I don't think it's the BIOS's job to keep BDA modifications (beyond what the BIOS itself does) into account for E820 reporting, but again I'll do a bit of research.Furthermore if i modify BDA memory limit it does not update memory map in E820 - this is the problem.
Although if I understand you correctly, you are actually not moving or resizing the BDA? You are only adding another "reserved" region just below it?
Actually, am I understanding correctly that your modified memory map is in fact clashing with the real BDA? Is it the Windows loader that's crashing or is it the BIOS itself?My modifications of memory map only shows that problem affects windows loader.
You could test with IDE instead of SATA, that ought to use only a 1K BDA.
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Re: Inconsistent information about memory in real-mode
Yes, the problem with AHCI, i'l try to test with IDE.
Also i do not change the location of BDA, it still located at initial position, i only modify E820 memory map, and change memory limit at address 0x00000413
Yesmichaln wrote:Actually, am I understanding correctly that your modified memory map is in fact clashing with the real BDA?
Actually crash occurs in ntoskrnl.exe while initializing kernel memory manager. I do not have much information, but is suppose that real memory corruption occurs either in bootmgr (1st stage) or in winload (2nd stage), because these two stages work heavily with INT15 E820 and INT13h 42 (disk IO)michaln wrote:Is it the Windows loader that's crashing or is it the BIOS itself?
Also i do not change the location of BDA, it still located at initial position, i only modify E820 memory map, and change memory limit at address 0x00000413
-
- Oracle Corporation
- Posts: 2973
- Joined: 19. Dec 2007, 15:45
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Any and all
- Contact:
Re: Inconsistent information about memory in real-mode
To be clear, I think you're right that the VirtualBox BIOS can return incorrect information in the E820h memory map.
But my memory has fooled me... it's not the BDA that is bigger. There is additional conventional memory "stolen" from below the BDA, and the memory size at 40:13 is reduced accordingly.
But my memory has fooled me... it's not the BDA that is bigger. There is additional conventional memory "stolen" from below the BDA, and the memory size at 40:13 is reduced accordingly.
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Re: Inconsistent information about memory in real-mode
Ok, that sounds good, but i want to clarify meaning of BDA. According to this page http://wiki.osdev.org/BDA#BIOS_Data_Area_.28BDA.29 BDA is located at address 0x400 (40:00), and EBDA is located somewhere else. For example on virtualbox it is started from address 0x9FC00.michaln wrote:To be clear, I think you're right that the VirtualBox BIOS can return incorrect information in the E820h memory map.
But my memory has fooled me... it's not the BDA that is bigger. There is additional conventional memory "stolen" from below the BDA, and the memory size at 40:13 is reduced accordingly.
So when i say BDA i mean address 0x400.
-
- Oracle Corporation
- Posts: 2973
- Joined: 19. Dec 2007, 15:45
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Any and all
- Contact:
Re: Inconsistent information about memory in real-mode
Sorry! When I said BDA I really meant the EBDA, or Extended BIOS Data Area. This is the variable-size (but at least 1KB big) memory area that the word at 40:13 points to, normally located in the last kilobyte (or more) just below 640KB.
Could you please explain what precisely you mean by "BDA memory limit"? I'd rather avoid guessing.
Could you please explain what precisely you mean by "BDA memory limit"? I'd rather avoid guessing.
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Re: Inconsistent information about memory in real-mode
I mean memory limit which is located at 0x0413 (40:13). It is the part of BDA, so i called it BDA memory limitmichaln wrote:Sorry! When I said BDA I really meant the EBDA, or Extended BIOS Data Area. This is the variable-size (but at least 1KB big) memory area that the word at 40:13 points to, normally located in the last kilobyte (or more) just below 640KB.
Could you please explain what precisely you mean by "BDA memory limit"? I'd rather avoid guessing.
-
- Oracle Corporation
- Posts: 2973
- Joined: 19. Dec 2007, 15:45
- Primary OS: MS Windows 7
- VBox Version: PUEL
- Guest OSses: Any and all
- Contact:
Re: Inconsistent information about memory in real-mode
OK, that's what used to be called simply "memory size", later maybe "base memory size". Anyway, it's now crystal clear what you mean and yes, INT 15h/E820h should take that into account instead of hardcoding some values. We'll fix that.
I'm still curious though, did your code work when you switched the VM to use IDE instead of SATA? Because then the INT 15h/E820h memory map should be consistent with actual memory usage.
I'm still curious though, did your code work when you switched the VM to use IDE instead of SATA? Because then the INT 15h/E820h memory map should be consistent with actual memory usage.
-
- Posts: 6
- Joined: 21. Aug 2015, 05:54
Re: Inconsistent information about memory in real-mode
Yes, with IDE controller everything is OK (SATA controller should be removed from VM configuration).michaln wrote:I'm still curious though, did your code work when you switched the VM to use IDE instead of SATA? Because then the INT 15h/E820h memory map should be consistent with actual memory usage.