Evaluating the Genealogy Search Engines - Evaluation Criteria

One of the problems that many genealogists, including my society colleagues and many commenters on blogs and message boards, cite about genealogy database providers is that the Search engines are poor and don't find the results expected by the users.

I posted What do you want in a search engine? and Testing different search engines - my perspective recently as an introduction to this series of posts. I laid out four Search criteria to evaluate in Testing different search engines - my perspective for these comparisons. The four Search criteria were:

* Exact search vs. Fuzzy search ("Fuzzy" meaning variations in names, dates and locations that might result in a match even if the record was enumerated or indexed poorly, e.g., Soundex).
* Use of a specific database vs. many or all available databases
* Use of Wild cards in names
* Use of dates and locations as search criteria

In the present series of posts, I'm also trying to compare the ease and accuracy of finding a record for a specific person. I think this is a typical search by a relative beginner and an experienced genealogy researcher. The questions I'm going to try to answer include:

* Does the Search find the record I'm seeking?
* How many clicks does it take to find the sought record?
* Does the sought record provide a source citation?
* Was the process easy to use? (This is difficult to assess by me - I am an experienced searcher, and have my own favorite search processes and biases about how to best search databases.)

My target process is to find an 1860 Census record for the family of Isaac Seaver (1823-1901), born in Massachusetts. He is my great-great-grandfather. I chose 1860 because I should be able to find the 1860 census record in the databases of four genealogy providers - Ancestry, Footnote, FamilySearch and HeritageQuestOnline (which requires a participating library card login).

My purpose here is to demonstrate how easy or hard it is to use these Search engines -not to complain about the Search process or the lack of results. If I find what I think are problems with the Search process, I will point them out.

While I am using Census records for this evaluation, the Search process is similar for most of the Historical Records (e.g., vital records, census, military, immigration, etc.) that are available in online databases. It may not be representative of newspapers, books, stories, maps, directories and family trees.

I welcome comments about the process, the criteria and the results. My hope is to complete this initial series using the 1860 US census in one week - doing essentially one post each day. I'll wrap up each series with a summary post at the end.