Re^2: mmaping a large file

Ť If you have a large dataset in a file: and a) only need access to small bits of it in any given run; b) you can find those bits without reading through the whole file from the beginning; then mapping can be an effective way on minimising the number of pages read from disk. ť

yes, that's exactly the use case.

I've just tried:

use Sys::Mmap;
new Sys::Mmap my $f, 8192, q(bigfile);
[download]

but now I get an error during cleaning:

(in cleanup) munmap failed! errno 22 Invalid argument

This is not going to be easy, is it?

Comment on Re^2: mmaping a large file Download Code

Replies are listed 'Best First'.

Re^3: mmaping a large file
by BrowserUk (Patriarch) on Aug 24, 2012 at 00:26 UTC

Warning: I don't use *nix, so I cannot test anything I'm about to say.

The first thing I notice is that the POD for Sys::Mmap uses

new Mmap my $f, 8192, q(bigfile);
[download]

not

new Sys::Mmap my $f, 8192, q(bigfile);
[download]

I would have expected you to receive a compile-time error message from that, which suggests you aren't using strict/warnings. It might be a good idea to start.

The other thought is that mapping a 2GB file through a 8k window is going to involve a lot of shuffling.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

[reply]
[d/l]
[select]


"be consistent"
	PerlMonks