Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Re: Proof of concept: File::Index (detect updates)

by tye (Sage)
on May 12, 2006 at 16:45 UTC ( #549060=note: print w/replies, xml ) Need Help??

in reply to Proof of concept: File::Index

I think this is great. I'd add the following features...

When you create the index, allow the user to specify "only appends expected". Record in the index the inode of the indexed file. When the file is re-tied, if the file has been modified more recently than the index file has been modified (alternately, store a timestamp in the index file rather than relying on the file system's timestamp for it), then the index might need to be recreated...

When recreating the index, if "only appends expected" is true and the inode of the indexed file hasn't changed, then seek to the character before the Nth-to-last line and verify that it is "\n". Repeat for the Nth last lines. If all of that passes, then just scan the end of the file starting at the last indexed line and add indices for any additional lines (and update the index's timestamp).

When grabbing the Nth line, if N > 0, actually read starting from one character before the Nth line and then verify that the first character is "\n". Verify that the last character is either "\n" or is the last character of the file. If either of these tests fail, reindex the file from the beginning. Strip the leading "\n" before returning the line, of course.

You could also index the file lazily. That is, if they ask for the Nth line, then only index up to the Nth line. Next week, they could tie the file, the module would note that nothing had changed, then they ask for the 2*Nth line and the module would then index the N+1st line through the 2*Nth line.

Yes, the checks for updates aren't bullet-proof and users need to be able to force a re-index manually, of course, but I think the logic I outlined could be very valuable.


- tye        

  • Comment on Re: Proof of concept: File::Index (detect updates)

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://549060]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (3)
As of 2022-01-26 21:41 GMT
Find Nodes?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:

    Results (70 votes). Check out past polls.