VirtualBox 4.3.4 Aborts While Importing Large File Into DB

[Alpha47]

Hi,

I’ve been attempting to import a large file (30 GB compressed and 400 GB uncompressed) into my database but the import fails when VirtualBox aborts during the Process: Way stage of the import (I’m building an OpenStreet Map planet tile server). I’ve asked this question on the OpenStreet Map forum but I’m asking here in case it’s a VirtualBox issue. Details:

• Machine
o Windows 7
o 32 GB RAM
o Intel i7-3740QM 2.70 GHZ processor
o 236 GB of disk space
o 64-bit
• VirtualBox
o Version 4.3.4
o Linux 2.6 (64-bit) running CentOS 6.3
o PostgreSQL 8.4
o 27 (of 32) GB RAM
o 620 GB of disk space (external drive)
• VirtualBox log attached as txt. Key entries (?):

Code: Select all

21:33:23.887382 AHCI#0P1: Write at offset 122159951360 (4096 bytes left) returned rc=VERR_EOF

21:33:23.899808 AHCI#0P1: Write at offset 122159955456 (4096 bytes left) returned rc=VERR_EOF

21:33:24.023324 

21:33:24.023326 !!Assertion Failed!!

21:33:24.023327 Expression: pSgBuf->cbSegLeft <= 32 * _1M && (uintptr_t)pSgBuf->pvSegCur >= (uintptr_t)pSgBuf->paSegs[pSgBuf->idxSeg].pvSeg && (uintptr_t)pSgBuf->pvSegCur + pSgBuf->cbSegLeft <= (uintptr_t)pSgBuf->paSegs[pSgBuf->idxSeg].pvSeg + pSgBuf->paSegs[pSgBuf->idxSeg].cbSeg

21:33:24.023330 Location  : D:\tinderbox\win-4.3\src\VBox\Runtime\common\misc\sg.cpp(55) void *__cdecl sgBufGet(struct RTSGBUF *,unsigned __int64 *)

21:33:24.041889 pSgBuf->idxSeg=0 pSgBuf->cSegs=1 pSgBuf->pvSegCur=000000009c79d000 pSgBuf->cbSegLeft=4096 pSgBuf->paSegs[0].pvSeg=0000000000000000 pSgBuf->paSegs[0].cbSeg=0

I’m not sure what the errors at the end of the log indicate. Logs from previous attempts have also indicated a substantial time lag between my machine and VM. Any help or advice would be greatly appreciated!

Thanks,

Alpha

[Perryg]

Is the external drive USB or eSATA?
Does it fail at the same spot each time you try to import it?

Can you explain a bit about the import process you are using?

[Alpha47]

Hi Perryg,

Thanks for your response! After talking with some people on an OpenStreet Map forum it seems likely that the problem is from my old postgres version and some outdated import commands, not VirtualBox.

To answer your questions:

1) USB
2) Correct. It fails at the Processing: Node section of an OpenStreet Map planet file import (wiki openstreetmap org wiki Osm2pgsql benchmarks #No_hstore.2C_flat_file_node_cache). This is likely because I was using an outdated command and the number of nodes has increased significantly since this command was optimal.
3) Certainly - Using osm2pgsql (wiki openstreetmap org / wiki / Osm2pgsql) I was attempting to import a 30 GB file of the planet into my postgres database. If you're interested more details can be found here: switch2osm org serving-tiles manually-building-a-tile-server.

Like I said I strongly suspect this stemmed from my mistakes. If I find out more information about any VirtualBox problems I'll circle back and update this ticket.

Thanks again!

Apologies for the sloppy urls. I'm a new member and don't have permission yet.

[Alpha47]

Update:

The problem here wasn't VirtualBox but my use of the most up-to-date version of a converter for osm to pgsql files. I switched to an older version of osm2pgsql (0.81) and was able to get around the problem.