Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Using Google to search perlmonks.org

by Scott7477 (Chaplain)
on Mar 15, 2006 at 21:02 UTC ( [id://536941]=monkdiscuss: print w/replies, xml ) Need Help??

I was reading a thread titled Numerical integration and wanted to see if there were any related posts on perlmonks.org.

I first went to Super Search and put "Numerical integration" (without the quotes) into the "Match titles containing" box
and hit the search button. The result was the three posts contained in the original thread. Out of curiosity, I went to Google
Advanced Search and did the following query: "numerical integration" site:perlmonks.org. The result I got there was Your
search - "numerical integration" site:perlmonks.org - did not match any documents.


My question is this: why did this Google search turn up nothing on a thread that was posted in December?

Replies are listed 'Best First'.
Re: Using Google to search perlmonks.org
by Arunbear (Prior) on Mar 15, 2006 at 21:24 UTC
    Because perlmonks is not indexed by Google. This is what our robots.txt looks like:
    # sorry, but misbehaved robots have ruined it for all of you. User-agent: * Disallow: /
      Wouldn't misbehaved robots ignore robots.txt?
      Thank you, ikegami and Arunbear for the info. It figures that this site has a snarky comment in its robots.txt:)...
Re: Using Google to search perlmonks.org (changes)
by tye (Sage) on Mar 16, 2006 at 01:11 UTC

    We had thepen and it was a much better place for google to index and so it was a good thing for google to not index PerlMonks directly (it got confused because (www.)?perlmonks.(org|com|net) are all the same site and so would index each page as many as 6 separate ways, it would preserve random, old chunks of ChatterBox conversation for longer than many felt comfortable with, etc.). But thepen silently went away not too long ago.

    In the mean time, work was done on making a robot-friendly face for PerlMonks so that we could have google (et al) index us directly, but that work is not finished, unfortunately.

    So we are left with no google index. I hope that will be resolved soon, but these things always take longer than most would think they'd be likely to.

    - tye        

      Your comments make sense. I am not unhappy that the site blocks search engines. I haven't spent a lot of time yet mastering Super Search, so I know I could have found more posts by using that more expertly.

      Obviously, it makes sense that having a search engine friendly site will increase the exposure of this site specifically and Perl generally. So it would be good to have this project done, as you say.

      I totally agree with the desire to avoid the problems with search engines that you mention. Obviously, folks use the Chatterbox with the expectation that they can chat freely without having their conversations recorded.

      Thanks to you and the rest of the site maintainers for your efforts.

      Scott
        I haven't spent a lot of time yet mastering Super Search, so I know I could have found more posts by using that more expertly.

        Note that Super Search sucks at what google is best at while Super Search is good at things google sucks at and can do things google refuses to do (and vice versa). So the two complement each other (and google is probably easier to like for searching here, as Super Search takes a bit of care to use effectively).

        So we want google searching of PerlMonks as an alternative to Super Search and for the reasons you cited. (:

        - tye        

        I am a bit embarrassed to admit that I only recently stumbled across Using the Simple Search. In any case, it looks like SiteDocClan has some work to do there in light of comments made earlier in this thread... ;-)

        planetscape realizes she is tempting the gods with these comments... ;-)

        HTH,

        planetscape
Re: Using Google to search perlmonks.org
by ikegami (Patriarch) on Mar 15, 2006 at 21:08 UTC

    I think perlmonks.thepen.com is a searchable mirror.

    Alternatively, Super Search for Numerical integration and put some junk (e.g. "|") in the "separate strings with" field. That will search for the two words together.

    Update: Since it's the exact title of a node, a search (as opposed to Super Search) for Numerical integration will open the node you want automatically.

      I think perlmonks.thepen.com is a searchable mirror.

      I wouldn't rely on it. When last I talked with blakem about thepen.com (4 months or so ago), it was running on way old hardware (a Cobalt Cube!), and was no longer providing a full, searchable mirror. I suspect that it hasn't moved off of his back-burner.

        The "Fastest Rising Monks" on his home node stopped being updated a while ago (2005-04-04) as well.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: monkdiscuss [id://536941]
Approved by tye
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2024-03-28 23:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found