« Amazon Can Do It. So Can Westlaw | Main | Elearning as a KM Application »
November 24, 2003
Making Your Spider Outperform Google
Even the best search engine can fail. But when it falls short of finding the most commonly requested information, whether on an external web site or an intranet, that's a real problem.
That's why I found Rich Wiggins' Internet Librarian presentation so interesting. He discussed in detail how he enhanced the search engine at Michigan State University in a way that no doubt improves the user experience.
First, he analyzed the search logs to identify the most common search terms, creating what he calls an "accidental thesaurus" from those keywords and phrases. He then stored them in a database, mapping them to the URLs that were most likely to provide the information being sought.
When a search is run on the MSU site, the "best bets" database is queried first, then the standard search engine, in this case, Google. The best bets, if any, appear at the top of the page followed by the standard search engine results. It is similar way that sponsored links are displayed on a commercial search engine. Rich can add search terms and/or URLs at any time to tweak the database for better search results.
He also created a browsable directory from the accidental thesaurus, providing another easy to use finding tool.
According to Rich, the best bets database typically works well for common searches, while Google's relevancy ranking works well for uncommon searches. So the searcher gets the best of both worlds.
You can try it out for yourself at the University web site.
Now why didn't I think of that?
Comments
Post a comment
Thanks for signing in, . Now you can comment. (sign out)
(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)
