Re: Does the monastery want a threaded newest nodes?

Normally I don't interject, because I really don't see the point of some changes suggested, but since you ask...

As it stands, the monastery has turned into a monolith - several hundred db queries per page I believe. That's crazy - whatever happened to KISS?

The site is already crawling on several (3 now, isn't it?) servers.

I would only say no problem if each article thread was cached, and the DB was only hit for each menu only once every couple of minutes. Here's a RL example. We have "live" news feeds that our client's can embed in their pages that are called through SSI. The feed article DB is updated hourly. Templates are designed by customers, and populated once an hour, creating static files. This way, each feed template builds a live page once an hour. When the SSI is called, if the populated template is less than 60 minutes old, it's served, else it's rebuilt from the DB. This results in each designed feed template resulting in one hit on the DB per hour. This way, we can safely offer the feature to all 1000 odd domains on the server without having to worry about the overhead. Now, I know Newest Nodes is different, but is each view generated on the fly, or shown from a cache? Would anyone *really* be bothered if these threaded indices were cached and only rebuilt once every 2-3 minutes? Just something to think about...

I can't believe that this much hardware is needed to serve a couple of hundred people at a time. Unfortunately, I don't think this can be fixed without a complete rethink of the underlying design philosophy.

demerphq. I can understand you wanting to improve the monastery. You are a great coder, but I get a gut feeling that the inmates are running the asylum (that's meant to be a ;-) not a :( - ok! :)

I would love to see you using your (seemingly unbounded :) energy towards making each page load twice as fast. I can't see why each page needs more than a dozen or so DB queries.

Or, to start with a clean slate and create a "PM lite", with pages that load a lot faster, with compromises made on customizations. I would use it, probably most of the time. In the same way that I read slashdot with images turned off.

Or maybe I'm just turning into a grumpy old man (well, I *am* nearly 34 :)

.02

cLive ;-)

Comment on Re: Does the monastery want a threaded newest nodes?

Replies are listed 'Best First'.

Re^2: Does the monastery want a threaded newest nodes?
by demerphq (Chancellor) on Sep 23, 2004 at 07:54 UTC

I think theres some misconceptions in your node. My code actually will do less queries than NN. Yes it could profit from some performance tuning, but overall its actually fairly conservative with how it interacts with the DB. Constructing the trees is fairly cheap. Im not going to go into why PM needs so many queries, suffice it to say that there is a fairly good caching system in play so that commonly used nodes are almost always in cache. OTOH, the caching system means every time we "use" a node we have to do version check on the cached value. Since everything is a node in Everything pretty much this means a lot of version checking and because of the nondeterministice way it happens you cant even batch the checks up. So all together we arent reducing the number beyond a marginal amount without totally different architecture.

When and where we see optimization possibilities we do our best to exploit them, but theres only so much we can do in this framework.

---
demerphq

_{First they ignore you, then they laugh at you, then they fight you, then you win.

-- Gandhi

Flux8}

[reply]