My search engine, if it can actually be called that, though, does not use full text search, or any actual text search whatsoever. Except possibly site description. It basically ignores all text and just focuses on structured metadata.

The closest thing I ever came across in terms of an IDEAL search engine was the custom site search for Wiser.org

Over 100,000 groups and organizations and unnumbered individuals, worldwide networked and organized through this social network, which would have been impossible without the unique multifaceted search interface.

What happened to this social network? One day, it was simply announced that the site was shutting down. All that remains, it seems, is some of the non functional static pages archived on the Wayback Machine.

https://en.m.wikipedia.org/wiki/Wiser.org

Here is an Internet Archive page showing the deceptively simple search interface:

https://web.archive.org/web/20120910002106/http://www.wiser.org/all/search?phrase=

It had conceptual indexing of facets such as; "Solutions" (to world problems, issues and concerns) along with Organizations, Groups, People, Events, Resources, etc. Also these facets could be simultaneously searched by language, location and if desired, key word. I really loved that search engine.

I may be a wee bit paranoid or something, but it seems, nearly every trace of the original free, open source WiserEarth API, and all documentation has been scrubbed from the internet. Including the Internet Archive. If anyone has a tip where it can still be found, I'd appreciate that.

So, this brings to the foreground, one of the problems of centralized indexing. If a well organized, worldwide, social activist community becomes problematic, it is all to easy to take out a central server. Or maybe the maintainers of the site just got tired of maintaining it. Either way, something hundreds of thousands of world betterment groups, organizations and individual activists depended, really depended on, vanished.

What essentially pulled all these groups and organizations together was a database with a functional search engine geared towards real human needs.

Tom

Your "conceptual indexing" sounds a lot like what's today called "social bookmarking", which tries to apply a similiar process to webpages as used in libraries. The Wikipedia page has a section "Comparison with search engines".

The Wiser.org Search API probably was derived from (or the same as) the WiserEarth API, which (still) is in the Internet Archive (FAQ and Documentation)

I don't think there's active scrubbing going on, the "normal" entropic force is strong enough already, especially if the information in question needs active maintenance.