http://qs321.pair.com?node_id=944885


in reply to Displaying/buffering huge text files

8 million bytes? Surely on modern computers that isn't a lot, is it? It may be a lot on small embedded systems, but I doubt you have a need to interactively edit files hundreds of Mb large. And 8 bytes for each offset? You expect files that large? Your file has to exceed 64 Pb before it needs 8 byte offsets. (1 byte offset: up to 256 bytes in a file; 2 byte offset: 64k; 3 byte offset: 16 Mb; 4 byte offset: 4 Gb; 5 byte offset: 1 Tb; 6 byte offset: 256 Tb; 7 byte offset: 64 Pb; 8 byte offset: 16 Eb)

Anyway, if storing all the offsets in an index makes the index too large, why not store the offset of every 100th line? That makes your index 99% smaller. You do have to read up to 100 lines if your user jumps to a certain line, but unless you're writing a line editor, you want to read multiple lines on jumps anyway.