Searching for a Site Search Engine

March 11, 2005 — by Per Christensson

Search is a complex thing. I found this out firsthand by attempting to code my own search engine for Sharpened.net. As some veteran visitors of the website may recall, there was a search section for Sharpened.net in the previous design. The section allowed you to search the Glossary, the Help Center, or other parts of Sharpened.net. While it basically did the job, I will admit that the search feature was mediocre at best.

So with the new website design introduced in February, I fixed the mediocre search engine -- by getting rid of it. Talk about an easy fix. I figured the search feature wasn't all that important, so I just left it out. However, after a month of testing and updating the site, I began to realize how helpful a search feature would be. So I decided to bring it back. Except this time, I had a little, OK a lot of help from Google.

Creating a search engine involves several steps, including communicating with a database, comparing the search string with the text in the database, and displaying relevant results. Sounds easy enough, right? Well, not quite. A quality search involves more than just comparing text to a database of documents. It involves parsing the text, eliminating common words, matching similar words, catching misspellings, and ranking results in the most relevant order. Creating algorithms that do this well can take weeks, even months to create. Google has been in the search business for years, and they are still finding ways to improve their search. The point is, I felt that this task was, let's say, too impractical to take on by myself. So I enlisted the help of Google to provide a way for users to search the site.

Now the Glossary and the Help Center have a search box at the top of each page allowing you to search the website for certain computer terms or questions. While the results are not specific to a certain area of the site, they are highly relevant and will give you a good list of pages that relate to your query. It was difficult for me to outsource the search feature, since I like to write my own code. But after using the new search feature, I think you'll agree I made the right choice. =)