Thursday, April 2, 2015

FamilySearch Responses to Questions About IGI Extracted Sources

I wrote FamilySearch is Adding IGI Extracted Source Data to Family Tree Profiles on 30 March 2015, and asked some questions at the end of the post because I was confused by what the website told me about the issue.



Robert Kehrer, the FamilySearch Senior Product Manager for Search and Hinting, has provided responses to my questions and permitted me to share them with my readers.  Here are my questions (in red and bold) and Robert's answers (in purple):

1)  Why were they [the IGI sources] added, but, more importantly, what should I do with them! 

The new.familysearch (nFS) persons were created when FamilySearch put the AF, PRF, Indexed and Contributed IGI and other things all into one big merge process. At that time a mapping was created and maintained as to which indexed IGI personas were used to create which tree persons. That mapping was also maintained as nFS was shut down and transitioned to Family Tree. 

In as much as all the data on an indexed IGI persona comes from a single indexed document - these are no different from any other indexed historical document (there is nothing “conclusionary” or opinion’ish about them). In fact all of the indexed IGI personas are found today in the Historical Records database organized into their pertaining collections by location, record type and date. 

Since the release of Hinting, we are seeing users attach over a quarter million records personas to people in the Family Tree each day. Approximately 450 million of the 5.2 billion Historical Records personas hosted on Family Search are part of the indexed IGI. If one assumes it takes a user 5 minutes to find and attach one of these historical record personas to the corresponding person in the Family Tree, that represents ~4280 years of user time. That is user effort to recreate connections between the historical records personas and tree persona that we already have in our mapping. 

To save users time and to help users understand how the Family Tree persons were created, we are attaching the indexed IGI personas. NOTE: We are not attaching the contributed IGI persons. 

What should you do with them? Evaluate whether the persona in the record is the same as the person in the Family Tree. If it is not (if it was badly merged), you should identify what data on the Family Tree person might have come from that persona and correct the data on the tree person. You can detach that record persona and add a reason statement as to why it is not the correct person.

2)  How does FamilySearch know that profiles were wrongly merged? 

We don’t know with any certainty when a Family Tree person was created out of sources representing different real historical people. We do know that the merge algorithms sometimes made mistakes. 

3)  How can I judge that the profiles were correctly merged? 

Users will be alerted that FamilySearch attached the sources from which the tree person was originally created through two means. First they will be informed of these attachments in the weekly “Changes” email for the people they are watching. Second they will see a yellow note when they click on an attached source that this source was used to automatically create the tree person. 

To determine if the persona in the historical document really is the same real historical person represented by the person in Family Tree the user will use standard genealogical proof standard analysis and comparison with all other sources, event and relationship data. 

4)  Who is responsible for separating improperly merged persons?

FamilySearch can make it clear to the user what actually happened in the process of tree person creation by attaching the related sources. Using those sources, researchers will need to identify those, hopefully rare instances, where there were flaws in the creation process and sources were badly merged. 

I followed up with several more questions:

5)  Is it fair to say that "if the IGI Hint is correct for the person profile, leave it there and don't mess with it?”

Yes the contributed IGI records are simply historical records that we happened to know where they should be attached to. There is nothing terribly unique about them than makes them different from a census record or military immigration record except that they were used to automatically create people int he tree at one time. If we’ve attached one of these, most of the time you should just consider yourself lucky that we just added a source document to your ancestor without you having to search for it and use the attach to to add it. 

6)  Were the IGI Hints attached to correct person profiles also?  That would make sense - not every profile was merged incorrectly and many were not merged at all, i'm thinking.

They are ALWAYS attached to the person that was created from them. Sometimes in the merge process multiple records were brought together that didn’t actually represent the same unique historical real person (just like a user might erroneously do today when they attach a record person to the wrong tree person). This should be rare, but can happen.

7)  If an IGI persona was attached incorrectly, do you expect us to find the correct person profile to attach it to?  What happens to IGI personas that get detached and not attached to another profile?

If the record persona that was part of the merge does not represent the tree person it is attached to (and was used to create) you should take the same steps that you do whenever you find that one of the sources you have gathered doesn’t pertain to the person you are researching. First, make sure that the data from the erroneous record is not included in any of your conclusions - IOW correct any data on the tree person that may have come from the erroneous source. Second, remove the source from the set of sources that your are correlating and forming conclusions from - IOW detach it from the tree person and make the appropriate notes as to why you are making the “negative conclusion” that it is not really your ancestor. 

Once it is detached from your ancestor it may get attached to another person in family tree. That is fine. It may not be a source for your guy, but the record persona does represent someone who really lived and another user might conclude that it represents their ancestor and use it to document their ancestor’s life. 

8)  Can we detach IGI personas from IOUS people?  I would guess so...since they are attached to only one profile.

Yes.

I really appreciate Robert's responses to my sometimes obtuse questions and will try to work with the IGI Sources as he recommends.

You know, we always think that a software program or an online tree is, or should be, "finished" and "perfect."  But usually that's not the case, is it?  FamilySearch Family tree has been in development for several years now, and open to the public for over two years, and is still being developed.  The progress has been steady and inspirational.  My opinion has always been that this "one tree" concept has the potential to be the very best online tree system when completed.  But it's not done yet, and it may take several years to work out the problems in the IOUS profiles.

NOTE:  There are several comments about the IGI sources attached at the end of my earlier post that illustrate some of the problems that other researchers have faced.

The URL for this post is:  http://www.geneamusings.com/2015/04/familysearch-responses-to-questions.html

Copyright (c) 2015, Randall J. Seaver





3 comments:

Ted Johnson said...

Thanks for sharing the questions and answers. This is most helpful!

I was surprised at a couple of things:

(1) The statement that wrongly attached sources "should be rare." (Question 6 response.) That has not been my experience. Of all the IGI sources I have examined so far, I would say something in the range of one-third are attached to the wrong person.

(2) The response to question 7 suggests that all we need to do is detach wrongly attached sources from the tree person. That seems to be backing off from the "Learn More" document which seems to suggest pretty strongly that it is our responsibility to find the correct person in the tree and attach the source to that person:

https://familysearch.org/ask/salesforce/viewArticle?urlname=Dividing-incorrectly-combined-records-in-Family-Tree&lang=en

(3) The response to question 4 doesn't really address the issue how wrongly combined persons are going to be separated. Even though we detach the wrongly attached sources, the underlying record still contains improperly combined persons. It seems like that is going to have be sorted out at some point.

T said...

It's in no one's best interests to have their tree edited by anyone else, especially without their permission. Having a tree at familysearch gives them that permission. I will never have a tree there. There are plenty of messed up trees on line without creating more on purpose. Bad idea. Very bad idea. My guess is the people who think these things up do not do research or they tried it one day and it was too hard.

OurFamily said...

Ted,

The rate of error seen in the merge algorithms that were used years ago to create the Family Tree persons from the AF, PRF, contributed and indexed IGI datasets will vary based on which individual in the tree is being viewed. Persons with common names may see a higher rate of erroneous merges.

The Indexed IGI records that are being attached are simply historical documents that were used to created the tree people. If you know where the correct person in the Family Tree is, by all means, detach the historical record from the wrong person and attach it to the correct one. This, and a good reason statement explaining why the record person is not the one you are detaching it from, will keep someone else from reattaching it to the wrong person again. If you do not know who the correct person is in the Family Tree, you can detach it and it will go back into the set of 5.2 billion unattached records until a descendant of the correct person comes along to attach it to their ancestor correctly.

I do not understand your third point. The information displayed on the Family Tree person represents conclusions made by patrons based on the attached historical document sources. If you detach an erroneous source, and also edit the data displayed on the family tree person so that it no longer reflects erroneous data derived from that source, you have separated out the improperly combined records and data.

-Robert Kehrer