But, from what I know of the systems, their individual servers are not what I would call "really high performance".

Your definition is therefore not worth much to me. I'll most likely never work on a system that has to stand up to more load than those boxes!

Come to my talk at YAPC or OSCON this summer and I'll be happy to talk about some of the techniques we use to build systems like this.

Do you really use a cooperative multi-threading system at Yahoo? I would be facinated (and very surprised) to hear about that!

-sam

I was speaking of individual machines, not of the whole server farm. The total load served by Amazon or TM is quite impressive, however they use a lot of machines to do it. From what I know (more about TM than Amazon) though, the query rate handled by a single machine is not huge.

Yes, we really do use cooperative multi-blah systems at Yahoo, at least for some applications. Some of these more resemble a threaded programming model, some are more like event-based state machine models.

The most common place that I can think of where you'd like to use a design like this is when you have a service-oriented architecture and the server that originally receives a user request does little more than analyze the request and then make a number of sub-requests to other services. An event-driven state machine is generally the most efficient and scalable way to multiplex really large numbers of simultaneous connections. With the Apache pre-forked process model, each child is tied up for the whole time that you are waiting for the subrequests to return. There's a fairly low limit on how many active processes you can have before your machine ends up spending more time context switching than doing useful work.