Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re^7: Reading huge file content

by GrandFather (Saint)
on Dec 09, 2007 at 07:54 UTC ( #655942=note: print w/replies, xml ) Need Help??


in reply to Re^6: Reading huge file content
in thread Reading huge file content

I couldn't get the inline C version of your code to work on my system so I rewrote the whole thing in C. I then ran the program twice, increasing the page file size by 1 GB between runs with the following result:

Global Memory Status: Total Physical: 401981440 bytes Available Physical: 32329728 bytes Total PageFile: 1569083392 bytes Available PageFile: 575242240 +bytes Total Virtual : 2147352576 bytes Available Virtual : 2139758592 + bytes Allocated: 16000 pages [65536000 bytes] at 0x00420000 Allocated: 26000 pages [106496000 bytes] at 0x00420000 Allocated: 36000 pages [147456000 bytes] at 0x00420000 Allocated: 46000 pages [188416000 bytes] at 0x00420000 Allocated: 56000 pages [229376000 bytes] at 0x00420000 Allocated: 66000 pages [270336000 bytes] at 0x10320000 Allocated: 76000 pages [311296000 bytes] at 0x10320000 Allocated: 86000 pages [352256000 bytes] at 0x10320000 Allocated: 96000 pages [393216000 bytes] at 0x10320000 Allocated: 106000 pages [434176000 bytes] at 0x10320000 Allocated: 116000 pages [475136000 bytes] at 0x10320000 Allocated: 126000 pages [516096000 bytes] at 0x10320000 Allocated: 136000 pages [557056000 bytes] at 0x10320000 Failed to allocate 146000 pages [598016000 bytes] C:\Documents and Settings\Peter\My Documents\Visual Studio 2005\Projec +ts\vmtest\vmtest\Debug>v Global Memory Status: Total Physical: 401981440 bytes Available Physical: 21180416 bytes Total PageFile: -1811525632 bytes Available PageFile: 1483104256 + bytes Total Virtual : 2147352576 bytes Available Virtual : 2139758592 + bytes Allocated: 16000 pages [65536000 bytes] at 0x00420000 Allocated: 26000 pages [106496000 bytes] at 0x00420000 Allocated: 36000 pages [147456000 bytes] at 0x00420000 Allocated: 46000 pages [188416000 bytes] at 0x00420000 Allocated: 56000 pages [229376000 bytes] at 0x00420000 Allocated: 66000 pages [270336000 bytes] at 0x10320000 Allocated: 76000 pages [311296000 bytes] at 0x10320000 Allocated: 86000 pages [352256000 bytes] at 0x10320000 Allocated: 96000 pages [393216000 bytes] at 0x10320000 Allocated: 106000 pages [434176000 bytes] at 0x10320000 Allocated: 116000 pages [475136000 bytes] at 0x10320000 Allocated: 126000 pages [516096000 bytes] at 0x10320000 Allocated: 136000 pages [557056000 bytes] at 0x10320000 Allocated: 146000 pages [598016000 bytes] at 0x10320000 Allocated: 156000 pages [638976000 bytes] at 0x10320000 Allocated: 166000 pages [679936000 bytes] at 0x10320000 Allocated: 176000 pages [720896000 bytes] at 0x10320000 Allocated: 186000 pages [761856000 bytes] at 0x10320000 Allocated: 196000 pages [802816000 bytes] at 0x10320000 Allocated: 206000 pages [843776000 bytes] at 0x10320000 Allocated: 216000 pages [884736000 bytes] at 0x10320000 Allocated: 226000 pages [925696000 bytes] at 0x10320000 Allocated: 236000 pages [966656000 bytes] at 0x10320000 Allocated: 246000 pages [1007616000 bytes] at 0x10320000 Allocated: 256000 pages [1048576000 bytes] at 0x10320000 Allocated: 266000 pages [1089536000 bytes] at 0x10320000 Allocated: 276000 pages [1130496000 bytes] at 0x10320000 Allocated: 286000 pages [1171456000 bytes] at 0x10320000 Allocated: 296000 pages [1212416000 bytes] at 0x10320000 Allocated: 306000 pages [1253376000 bytes] at 0x10320000 Allocated: 316000 pages [1294336000 bytes] at 0x10320000 Allocated: 326000 pages [1335296000 bytes] at 0x10320000 Allocated: 336000 pages [1376256000 bytes] at 0x10320000 Allocated: 346000 pages [1417216000 bytes] at 0x10320000 Allocated: 356000 pages [1458176000 bytes] at 0x10320000 Allocated: 366000 pages [1499136000 bytes] at 0x10320000 Allocated: 376000 pages [1540096000 bytes] at 0x10320000 Allocated: 386000 pages [1581056000 bytes] at 0x10320000 Failed to allocate 396000 pages [1622016000 bytes] C:\Documents and Settings\Peter\My Documents\Visual Studio 2005\Projec +ts\vmtest\vmtest\Debug>

Assuming the code is behaving correctly it seems to be allocating a larger block than my system has physical RAM - in fact about 4 times more!

The C code:

#include <windows.h> #include <stdio.h> #define PAGE 4096 void *virtualAlloc (long size) { return VirtualAlloc (0, size, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE +); } BOOL virtualFree (void *a) { return VirtualFree (a, 0, MEM_RELEASE); } void main () { MEMORYSTATUS stat; GlobalMemoryStatus (&stat); printf ( "Global Memory Status:\r\n" "Total Physical: %d bytes\tAvailable Physical: %d bytes\r\n" "Total PageFile: %d bytes\tAvailable PageFile: %d bytes\r\n" "Total Virtual : %d bytes\tAvailable Virtual : %d bytes\r\n", stat.dwTotalPhys, stat.dwAvailPhys, stat.dwTotalPageFile, stat.dwAvailPageFile, stat.dwTotalVirtual, stat.dwAvailVirtual ); for (long nPages = 16000; nPages <= 520000; nPages += 10000) { long allocSize = nPages * PAGE; void *addr = virtualAlloc (allocSize); if (! addr) { printf ("Failed to allocate %d pages [%d bytes]\r\n", nPages, +allocSize); break; } printf ("Allocated: %d pages [%d bytes] at 0x%08x\r\n", nPages, al +locSize, addr); virtualFree (addr); } }

Perl is environmentally friendly - it saves trees

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://655942]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (5)
As of 2022-05-22 20:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (81 votes). Check out past polls.

    Notices?