Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Possible perlmonks mirrors?

by Jaap (Curate)
on Feb 13, 2003 at 23:31 UTC ( [id://235133]=monkdiscuss: print w/replies, xml ) Need Help??

As stated in previous posts i am quite concerned with the speed of perlmonks.org
Now i am wondering if it is possible to mirror perlmonks.org and especially the database. I know MySQL has replication since version 3.23.15.

Is it possible and wise to mirror perlmonks.org?

If perlmonks want a mirror (in amsterdam, the netherlands), i am willing to donate the hosting (up to 50GB/month) at coloheaven.nl and part of the serverhardware.

2006-06-12 Retitled by Corion, as per Monastery guidelines
Original title: 'Mirror'

Replies are listed 'Best First'.
Re: Possible perlmonks mirrors?
by Abigail-II (Bishop) on Feb 13, 2003 at 23:47 UTC
    Yeah, sure. MySQL has replication. one-way replication. But if you are going to mirror, you need two-way replication. Which is hard to get it right. Believe me, I've done so, and it took us 3 months to build it, with a $300,000 price tag. (One of the most interesting projects I've ever done). Not that it would even be remotely possible to use the same technique with MySQL.

    You could set up a contruct were the database is mirrored, and you use the mirror as a query database, but still use the master to perform updates on. But then you get into a master-slave setup, with an assymmetric relationship. And you'd need synchronic writes to the replicated database - or else you get strange effects (you reply to an article, but then it disappears for a while - until the data has replicated). Synchronized writes are slooooooooooow.

    Abigail

      Yes one-way replication indeed. You could then only post to the master.
      If the slave site cannot find a page you could perhaps redirect to the master?

      But surely perlmonk's major load in terms of query time is read operations, not updates.

      $h=$ENV{HOME};my@q=split/\n\n/,`cat $h/.quotes`;$s="$h/." ."signature";$t=`cat $s`;print$t,"\n",$q[rand($#q)],"\n";
Re: Possible perlmonks mirrors?
by BrowserUk (Patriarch) on Feb 14, 2003 at 00:59 UTC

    If it is difficult to split the nodes database between two machines--though maybe a split along the lines of nodes older than (say) the last 10,000 could be acheived?--then how about splitting off transient data from the main machine to another. Stuff like the CB, Other users, Tick-Tock nodelets and possibly some other stuff could be embedded in IFRAME tags much as the advert is now and the load drawn upon another machine without much in the way of changes to the existing DB and few (I think) changes to the HTML.

    Even the preview pages might be offloadable in this way with only the final submit being back to the main site for storage?

    Just another random, based-on-guess-work thought.:)


    Examine what is said, not who speaks.

    The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.

      Yes this could be a good idea.
      Have one part of the site on server 1 and one completely seperate part on server 2. But for this you don't need a server at a different location.
Re: Possible perlmonks mirrors?
by belg4mit (Prior) on Feb 14, 2003 at 03:00 UTC
    Well you could always use thepen, or visit off-peak ;-) (I think all are concerned about speed, and those with the power certainly put in time and effort to do what they can).

    --
    I'm not belgian but I play one on TV.

Re: Possible perlmonks mirrors?
by krujos (Curate) on Feb 14, 2003 at 07:07 UTC
    I know MySQL won't do two way replication, so that option is out. Another option to improve performance would be clustering. I did a brief search at mysql.com and found that there are cluster solutions available on Solaris, AIX and SCO. I also did a google search but I didn't come up with much. Anyone know if there is a BSD + MySQL clustering solution? If anyone found one I would be more than willing to help out.
    I know that we run on BSD now but if we ever switched to Linux there are two(warning that’s a pdf) solutions we could use.
    <disclaimer>
    1. This is more of a curiosity question than anything else, as I think a cluster / the effort required to maintain one would be overstepping our bounds with the gracious service that pair provides us.
    2. Just to keep everything on the up and up, I do work for one of the companys I linked to. But am I in no way suggesting that PM buy into a proprietary, closed source, (maintenance nightmare) solution.
    </disclaimer>
      The link to mysql.com discusses a HA-cluster. That's a cluster where you have at least two nodes in a cluster, but a package (which contains one of more applications) will only run on one node at the time. That's very different from load balancing. Also the PDF file from Red Hat talks about a HA cluster. I can't get to the Veritas site, but our company gets hired to do consulting jobs for Veritas, and I've done their course about their cluster solution. That's also a high availability cluster.

      I think it should be doable to do load balancing on the front end (the webserver), as I don't think I get the impressions there's any session handling going on. (You can still do load balancing with session handling, but that requires specialized software). But load-balancing on the database end is a totally different ballpark. If you have lots of money, you could buy Oracle Parallel Server, that is capable of running more than one server on the same database. But MySQL isn't that smart by a long shot.

      Abigail

        Every cluster is an HA cluster (crevat: a beowulf cluster, but that's a totally different ballpark), that's the most basic, and arugably the best reason to use one.
        Load ballancing could happen in the cluster. The cluster server would control it and ferry requests to the approiprate node.
        It is true that MySQL would need to be able to handle running in an active active configuration before this would work. There is some software that would allow it to do it. I found this which appears to use some trickery to do it.
        Josh

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: monkdiscuss [id://235133]
Approved by PodMaster
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (4)
As of 2024-03-29 00:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found