http://qs321.pair.com?node_id=227295


in reply to Re: Recent slowness and outage
in thread Recent slowness and outage

jaybonci sent me patches to speed up group membership testing (which I don't think we do that much of compared to everything2.com where it was a huge difference). But it looks like ar0n might be applying those soon.

The simple search is not a potential huge problem like it used to be, but it looks like it is still too much load. I think fixing it requires that the first stab at search be much simpler and then you get the option to "try harder" if it didn't find what you wanted.

Good caching of "other users" and all of the places that make use of that could be a big win. Not an easy problem.

The "navigation nodelet" keeps showing up on my radar. The solution there may just be to turn it off (or at least the parts that MySQL does a poor job of optimizing, despite them appearing extremely simple).

I've done a lot of work toward better caching of chatter. I'm less convinced that is a big source of slowness at the moment, but it would also lead to better features.

There is tons of room for performance improvements on the web servers (like caching the compiled code from templates), but that isn't where the bottleneck is (and adding a third webserver is a good solution since I'm not paying for it). (update: that's a joke)

I think the newest nodes ticker never got optimized.

There are probably others I don't recall at the moment.

Update: Moving each user's scratchpad to a separate node. This is a performance improvement and makes features easier (like a "d/l code" link for scratchpads).

Hide the fact that user nodes contain a "settings" and "document" records. Probably a relatively big performance improvement.

                - tye
  • Comment on Re^2: Recent slowness and outage (other speed-ups)

Replies are listed 'Best First'.
Re: Re^2: Recent slowness and outage (other speed-ups)
by theorbtwo (Prior) on Jan 16, 2003 at 00:09 UTC

    Thanks for the list. Can you give us some idea of /how/ bad these things are? Would turning off the navigation and other users nodelets help things appreciably, or at least improve /my/ page load times? Would it be possible to run the DB server to log all requests for a few hours, to get some profiling info to know where to look? How work do you think it would be to go to a split-DB system, like livejournal uses?


    Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).