comment on

Also, when the data file has been modified, and has increased in size, check the consistancy of the last dozen indexes. If they are consistent, only index the new records appended rather than reindexing the whole file.

Actually, checking say log(n) (where n=no of lines) or so randomly selected lines spread across the file for consistancy is probably (probabilistically) sufficient to detect changes, given that any 1-byte change in the length of any line will invalidate the offsets of all subsequent records until an exactly opposite change of length re-syncs them.

Ignoring the possibility of deliberate tampering, any of the maths wizards feel like calculating the odds of a randomly re-written file matching more than say log(n) randomly chosen index offsets from the previous contents consistantly?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re: Proof of concept: File::Index (consistancy) by BrowserUk
in thread Proof of concept: File::Index by davido

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Perl-Sensitive Sunglasses
	PerlMonks