http://qs321.pair.com?node_id=433903


in reply to Displaying/buffering huge text files

Extremely large text files do not give up useful information readily. Even if the tools used to index them or jump to a particular location are fast, it's still a chore for users to get information out of them. Databases are much better at storing and using huge amounts of data. Nobody looks at a huge text file and thinks "I'd like to jump to line 150,000 at this point" they just think "what a useless large text file- how on earth am I going to find the information I want here??" Enforcing a fixed limit on the size of text file that your GUI application can handle would not be unreasonable. Or handling huge files with less functionality is another option- for example the Linux "less" paging program gives you the option to abort line number calculation when it reads in huge files, giving you the option of getting the file up faster at the cost of losing the "go to line 235,000" functionality.