http://qs321.pair.com?node_id=769974

Site seems slow. Is this normal?

It depends what you mean by "slow". It has become fairly common for many pages at PerlMonks to take several seconds to load, which indeed may seem rather slow compared to a lot of other web sites. This is at least partially due to the fact that the technology upon which the site was built tends to do thousands of SQL queries in the process of loading a typical page. Some long-running improvements on this front (reducing the number of queries per page load) are proving difficult to deploy but we still hope to launch them Real Soon Now.

So a page taking 10 seconds to load may mean that the page is doing something even more strenuous than a typical page would or that the site is a bit busier than usual or some other fairly minor complication has befallen your request. Unfortunately, we also have some recurring problems which can cause pages to take significantly longer than that or even cause pages to just time out.

One such problem is too many, too big Apache subprocesses making (one of) the web server(s) very slow to respond. Periodic tuning to prevent processes from growing too large and/or to limit the number of subprocesses has reduced how often this happens but, over time, re-tuning is required and so this still happens.

Another is a recurring problem with MySQL getting very little done for an extended period even though it has not run out of resources (such as CPU or memory). The original problem began dramatically when we moved from Linux to FreeBSD and suddenly a single long-running query was quite likely to bring the entire site performance to a crawl. Over the years we have nearly completely eliminated the possibility of long-running queries. But we still see such periods of "get nothing done" from MySQL and try to collect the data and find the time to try to figure out such problems and work around them. We are even trying to move to Postgresql but, as usual, this is taking a lot longer than one would initially expect it to.

There are also several significant performance problems which have actually been fixed. We made major improvements to how search engine web crawlers index the site such that they have significantly less impact on site performance. (This also improved the quality of the resulting indexing and more quality improvements are planned.)

Pair.com graciously upgraded our database server which just didn't have enough memory. (A similar upgrade to a much bigger web server from two smaller web servers is in the works but is stalled, not due to any fault of pair.com.)

A long-running effort at identifying performance problems found and fixed several mis-behaved pages which were being loaded by web spiders. After that effort (which made several other improvements, some noted above) the site actually achieved consistently reasonable performance for several months.

There have also been quite a few improvements such as different types of caching implemented when trouble spots have been identified.

Alas, new and old performance problems continue to crop up faster than the site's absentee landlords can find the resources to correct them.

So, our advice is to stop, close your eyes, and meditate on the fact that It's normal for monastery to be slow ;)


Back to the PerlMonks FAQ