[aksbhat]

Interesting article!

A lot of commenter here are scared of using Magik code.

However note that Search/Information Retrieval is a hard problem. Unlike other problems, developing a generalized full text search engine is difficult.

Testing search algorithms is even more difficult e.g. NIST organizes TREC conference http://trec.nist.gov/pubs/call2011.html in which a major emphasis is on evaluation of search algorithms.

In fact Search is as what my advisor calls it, an AI-Complete problem, i.e. creating a perfect search engine would amount to creating a Human like artificial intelligence capable of understanding your query and the corpus.