Re: Using Google to search perlmonks.org
by Arunbear (Prior) on Mar 15, 2006 at 21:24 UTC
|
Because perlmonks is not indexed by Google. This is what our robots.txt looks like:
# sorry, but misbehaved robots have ruined it for all of you.
User-agent: *
Disallow: /
| [reply] [Watch: Dir/Any] [d/l] |
|
Wouldn't misbehaved robots ignore robots.txt?
| [reply] [Watch: Dir/Any] |
|
Thank you, ikegami and Arunbear for the info. It figures that this site has a snarky comment in its robots.txt:)...
| [reply] [Watch: Dir/Any] |
Re: Using Google to search perlmonks.org (changes)
by tye (Sage) on Mar 16, 2006 at 01:11 UTC
|
We had thepen and it was a much better place for google to index and so it was a good thing for google to not index PerlMonks directly (it got confused because (www.)?perlmonks.(org|com|net) are all the same site and so would index each page as many as 6 separate ways, it would preserve random, old chunks of ChatterBox conversation for longer than many felt comfortable with, etc.). But thepen silently went away not too long ago.
In the mean time, work was done on making a robot-friendly face for PerlMonks so that we could have google (et al) index us directly, but that work is not finished, unfortunately.
So we are left with no google index. I hope that will be resolved soon, but these things always take longer than most would think they'd be likely to.
| [reply] [Watch: Dir/Any] |
|
Your comments make sense. I am not unhappy that the site blocks search engines. I haven't spent a lot of time yet mastering Super Search, so I know I could have found more posts by using that more expertly.
Obviously, it makes sense that having a search engine friendly site will increase the exposure of this site specifically and Perl generally. So it would be good to have this project done, as you say.
I totally agree with the desire to avoid the problems with search engines that you mention. Obviously, folks use the Chatterbox with the expectation that they can chat freely without having their conversations recorded.
Thanks to you and the rest of the site maintainers for your efforts.
Scott
| [reply] [Watch: Dir/Any] |
|
I haven't spent a lot of time yet mastering Super Search, so I know I could have found more posts by using that more expertly.
Note that Super Search sucks at what google is best at while Super Search is good at things google sucks at and can do things google refuses to do (and vice versa). So the two complement each other (and google is probably easier to like for searching here, as Super Search takes a bit of care to use effectively).
So we want google searching of PerlMonks as an alternative to Super Search and for the reasons you cited. (:
| [reply] [Watch: Dir/Any] |
|
I am a bit embarrassed to admit that I only recently stumbled across Using the Simple Search. In any case, it looks like SiteDocClan has some work to do there in light of comments made earlier in this thread... ;-)
planetscape realizes she is tempting the gods with these comments... ;-)
HTH,
| [reply] [Watch: Dir/Any] |
Re: Using Google to search perlmonks.org
by ikegami (Patriarch) on Mar 15, 2006 at 21:08 UTC
|
I think perlmonks.thepen.com is a searchable mirror.
Alternatively, Super Search for Numerical integration and put some junk (e.g. "|") in the "separate strings with" field. That will search for the two words together.
Update: Since it's the exact title of a node, a search (as opposed to Super Search) for Numerical integration will open the node you want automatically.
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
I think perlmonks.thepen.com is a searchable mirror.
I wouldn't rely on it. When last I talked with blakem about thepen.com (4 months or so ago), it was running on way old hardware (a Cobalt Cube!), and was no longer providing a full, searchable mirror. I suspect that it hasn't moved off of his back-burner.
| [reply] [Watch: Dir/Any] |
|
The "Fastest Rising Monks" on his home node stopped being updated a while ago (2005-04-04) as well.
| [reply] [Watch: Dir/Any] |