Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Delta-encoded permanent items in the nodecache

by JayBonci (Curate)
on Mar 21, 2012 at 05:46 UTC ( [id://960724]=monkdiscuss: print w/replies, xml ) Need Help??

Hey folks,

I am speccing out a change to E2, but wanted to run it past some perl-heads so I figured I'd toss it out here for a few ideas. I'm looking for a way to buy myself some more database cycles with which to expand, and doing some nodecache analysis, I think that we can tailor both of our sites by changing the way that the nodecache works to perform delta-encoded update checking, rather than performing an albeit cheap but uberfrequent query on the database

First, we'd create a table, say called delta_nodeversion. This would look like a HEAP table with timestamp,node_id as a key. Everytime we call Everything::NodeCache::incrementGlobalVersion and the type exists in a pre-defined perma-cache list, we would drop the node_id into delta_nodeversion.

Next, every time the page is run (inside of resetCache), it will check the delta_nodeversion for any changes since its last check time it has run. If it has found them, it will immediately go and tombstone those queue items, and let the page run normally. The Everything::NodeCache::isSameVersion then returns 1 for anything in the permanent queue map. When there's nothing in the queue, a big update day would only harm performance a bit, or one could restart your webheads in a controlled fashion and trim the queue afterwards if you were worried about it.

For performance, one might prune the table for entries older than a day, since it's unlikely that all of the apache threads have not all run once in a day.

htmlcodes,htmlpages,superdocs,maintenances,settings,containers,nodetypes,themes and themesettings might all be good candidates for this type of treatment, depending on how much load you want to push from your database onto your memory stack. The basis of this performance increase is based on the theory that it is cheaper to check one table that you can keep small, with a slightly more expensive query, than to check another table 15 times per page.

I'd welcome the feedback on it, thanks


    --jaybonci

Replies are listed 'Best First'.
Re: Delta-encoded permanent items in the nodecache
by jdporter (Paladin) on May 16, 2012 at 21:58 UTC

    Sounds cool to me, jaybonci. thank you. maybe you could submit patches here to make those changes?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: monkdiscuss [id://960724]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2024-03-29 05:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found