Login to participate
  
Register   Lost ID/password?

Louis Kessler's Behold Blog

Misleading Double Entries in FamilyTreeDNA Data - Thu, 7 Jul 2016

(This article was revised 11 Aug 2016 to fix some incorrect statements)

Be careful if you’re triangulating at FamilyTreeDNA. I just found out they can match twice on a segment.

If you look in your Chromosome Browser Results file which is downloadable from the Chromosome Browser page, you may find matches with a second person that overlap. For instance, look at this match my uncle has with David:

clip_image002

On chromosome 5, 7 and 12, there are three matches that overlap. The matches on chromosome 7 are identical. This would seem to indicate that one half of my uncle’s chromosome matches with one half of David’s chromosome and the other half of my uncle’s chromosome matches with the other half of David’s chromosome.

You don’t notice this when you use the chromosome browser. It will show just one of the matches:

SNAGHTML245e42a2

This doesn’t happen often. There are only 198 overlapping matches out of the 178,955 matches in my uncle’s file. But that’s often enough to worry about.

The match of my uncle with David is reported on the Chromosome Browser as having 26 shared segments totalling 87.36 cM. On the Family Finder it is reported as 107.87 cM, and in the chromosome match file I downloaded, after including the 3 overlapping segments shown above and 2 others, there are 59 matching segments totalling 203.56 cM. So what is going on here?

The overlapping matches in the Chromosome match files are not separate matches by FamilyTreeDNA on the two halves of the genome. They don’t do that. Any overlap would look like just one match over the both genomes.

What most (if not all) of those overlapping segments are from are from the incorrect way Family Tree Maker is listing people in the chromosome match file. They are being merged by person name and then by chromosome number and then by location on the chromosome. If two people have identical names, their information is being put together as one in the chromosome match file. This is incorrect and needs to be fixed by FamilyTreeDNA. What they need to do is incorporate the kit number into the matching, so that three John Smith’s are not put together.

See also my recent post: FamilyTreeDNA’s Chromosome Match File for more problems with the file that FamilyTreeDNA needs to fix.

And, if you hadn’t noticed, FamilyTreeDNA made some major changes today and updated their Family Finder interface. They now phase your relatives and show which matches are on your fathers side, mothers side, or both sides. Of course you need more than just one person tested and some known relationships entered before a paternal and maternal side can be assigned:

image

I also notice they changed the ordering. Your matches are now ordered first by relationship range and then by shared centimorgans. It used to be ordered first by relationship and then by largest segment. As a result, all the matches changed order significantly. But it seems that the relationships and cM values did not change.

For more information about this set of FamilyTreeDNA changes, see Roberta Estes’ post: Family Tree DNA Introduces Phased Family Finder Matches

Help Needed for DMT - Thank You! - EAST Part 3 - Sat, 2 Jul 2016

I have the basics of my Double Match Triangulation program working, but before I can release it to the world (as freeware!), I must put it through its paces and test it with some real data and ensure that it will correctly analyze and display the data and relationships.

Since I’ve only DNA tested my 93 year old uncle Harry, and since two people’s Chromosome match files are needed for the program to work on, I cannot do this by myself. So I contacted several of the people listed as matches on my uncle’s Family Finder page at FamilyTreeDNA to see if they would help out with my research.

I was overwhelmed by the enthusiastic response. Everyone, myself included, is looking to find some way to make some sense out of their autosomal matches, and then there’s the potential promise that true triangulation made easy by my DMT program could save loads of time and help us figure out how some of our matches are related.

My uncle’s match list (which is growing daily as FamilyTreeDNA finds new matches) is currently up to 7,865 matches and still only has one confirmed relative.

The one confirmed relative is Joel, who is my 3rd cousin, and my uncle’s 2nd cousin once removed on my uncle (father’s brother’s) mother’s father’s side. Joel is 3rd out of 7,865 on my uncle’s match list with 134.8 cM shared. Joel and I have been communicating for years working with several other cousins on that common side of our families. Joel sent me his chromosome match file.

Then I found Seth, whose ancestral surname was Braunstein (the same as my uncle), whose family originated in a town in Romania less than 100 km from where my uncle’s Braunstein ancestors came from. He didn’t show up prominently in my uncle’s matches until FamilyTreeDNA’s recent algorithm update. Seth moved up from a 5th to remote cousin sharing 127.1 cM to a 2nd to 3rd cousin sharing 130.7 cM. I’m very hopeful we’ll find the connection between Seth and my uncle because we know it will be on both our paternal lines. Seth sent me his chromosome match file.

Another person high up on my uncle’s match list was Erika, listed as a 2nd to 3rd cousin at 160.0 cM. She caught my attention when I was putting all my Pikholz connections together in preparation for my day at the Ontario Genealogical Society Conference where Israel Pickholtz was going to speak. About half the people in Israel’s book: Endogamy, One Family, One People were on my Uncle’s match list. But Erika was the closest of anyone with a Pickholtz connection. I contacted George, Erika’s cousin who was administering her account and is himself listed as a 3rd to 5th cousin at 92.2 cM shared. George sent me both his and Erika’s chromosome match files.

Then there’s a FamilyTreeDNA project for an area of Ukraine that i joined on behalf of my Uncle. Four people from there, Sandy, Barbara (2nd-4th cousin, 102.5 cM), Bruce (2nd – 4th cousin, 97.0 cM) and Mark who have connections to my uncle, sent me their chromosome match files. Barbara and Bruce also each sent me two more of the files they administer. Sandy sent me 23 files in total covering quite a few relatives in her family, many of which are among my uncle’s matches. Sandy has considerable experience in triangulation and has given talks on her analysis using it. I look forward to working with Sandy to help figure out her/our families.

Last but not least is Arnold Chamove who has been a Behold user for almost a year. He and I have had many good talks since then about Behold and what it does and should do. So it was a bit surprising when I found 6 people whose DNA Arnold administers listed in my Uncle’s match lists, the closest of whom is his cousin Roger (2nd – 3rd cousin, 144.2 cM). Arnold has given me access to 23 of the chromosome match files that he administers. It will be fun helping Arnold put his families together and finding out what our connection is.

It is very interesting that I can’t offhand connect yet to any of these 2nd to 4th cousins except for Joel. Most Ashkenazi lines only go back about 5 generations, and due to endogamy, 2nd to 4th cousins can mean 3rd to 6th cousins, even though FamilyTreeDNA says they try to correct for this.

I’ll be taking these 58 Chromosome Match files and use them for testing and to determine how best to analyze, interpret and present the triangulation data.

Most of the chromosome match files are from full Ashkenazi heritage with all its endogamy. These files range from 8 MB to 14 MB in size and the largest have more than 200,000 chromosome segment matches to 8,000 people. Non-Jewish chromosome files I’ve been sent seem to be about one tenth that size.

9 x 8 = 72 combinations both waysAnd the DMT program does Double Match Triangulation, meaning it needs two match files for a comparison. I will do every pair of comparisons. That will be 58 times 57 or 3,306 comparisons both ways. The program takes about 5 seconds per comparison (comparing two files of 200,000 lines each), so once I get the automated selector working, I’ll let it run for several hours to do them all.

There was one person I asked who would not give me his chromosome match file. It wasn’t that he wanted to keep his information private. Au contraire, Meir is a world expert at Y-DNA research, specializing in the Levite line, and he receives hundreds of DNA files from people willing to help. I know he’d be more than willing to help me.

But Meir’s reason was very interesting. He said to me:

The “autosomal soup” is not science. far from it.
It is pseudo science on verge of charlatanism.
Leave me out of this fiasco.

I pressed him further on this, and he told me he’d do an exception for me if I could meet a challenge. Putting aside the known relations, If I could show him how a mere 7 unknowns out of the 7,600 are related to Harry, he’d be willing to participate. So sort of like Sodom and Gomorrah which needed 10, I’ve got to find 7 good people who I can match to. 

I told Meir this is a fair offer. I said I don’t know if I will succeed in identifying 7 relationship paths just using the triangulation information, but I shall try. The rewards of succeeding are just too great to ignore.

It’s going to be fun!

Obviously, 3,306 pairs of test files is enough for me for now. But if you check your FamilyTreeDNA matches and notice that Harry Braunstein is listed as one of your matches, contact me, and I’ll try to include you in my tests. 

Extreme Autosomal Segment Triangulation (EAST) - Part 1
EAST Part 2 - Double Match Triangulation

FamilyTreeDNA’s Chromosome Match File - Sat, 25 Jun 2016

My DMT (Double Match Triangulation) program that will make use of FamilyTreeDNA chromosome segment match data is nearing completion. As I was looking through some of the results it was producing, I got an unexpected surprise.

I found there were some people my uncle Harry matched to in his Chromosome Browser match file that were not among his Family Finder matches. I found out that:

The FamilyTreeDNA Family Finder information is different from their
Chromosome Match information

To make sure, I re-downloaded both the Family Finder matches and the Chromosome Match file for my uncle. My uncle had 7,777 people listed as matches in his Family Finder Match download (FFMD). In his Chromosome Match download (CMD), my uncle had 176,436 segment matches which were from 8,017 people.

There were 26 people whose names were listed twice in the FFMD. Many of them were two different people with the same name, but a few were the same people with two different tests done. But they were merged together in the CMD and their matches were combined into 26 single people. This is a mistake by FamilyTreeDNA that they should fix. Since my program uses the Chromosome Match download, the two kits will be treated as one for matching until this is fixed. (See my post: Misleading Double Entries in FamilyTreeDNA Data, which gives more information about this problem)

Also, the FFMD file downloads Unicode characters correctly, but the CMD file does not. So the CMD file does not display names that are written with accents or in a different script correctly and sometimes does not include the person at all. FamilyTreeDNA should fix this as well. There are 31 people in the CMD whose names do not display properly or who are not included in the FFMD file.

Then the people’s names in the CMD file have two spaces between their first name and last name. There should be only one space between their names, as in the FFMD file. Fix please.

The bigger question is why does the FFMD has 7,777 people versus the 8,017 in the CMD? That’s 240 people who are in the Chromosome Match file that are not listed in the Family Finder. These people all have significant cM matches. I don’t know why they are in the Chromosome Match file but don’t show up in Family Finder. My suspicion is that there is some criteria that is filtering them out of the Family Finder matches. I don’t know what that is. Maybe someone from FamilyTreeDNA should explain. Whatever the reason, the people listed in the Family Finder as matches should be the same as the people listed in the Chromosome Browser download, and FamilyTreeDNA should fix this.

Next problem. For the 7,777 matching people, the total cM for 660 of them in the CMD was at least 5 cM higher than what was given as the Shared cM in the FFMD.

So I picked a person at random who was among these 660. In FamilyFinder, my uncle has this match to Carole who shows up as having 47.70 Shared cM with Harry.

image

I then went to the Chromosome Browser, found Carole, and did the Download to Excel (CSV Format) that is the 1st Optional View at the top of the page. It gave the following:

image

Ah, I see. The difference is that the Family Finder shared cM don’t include the X chromosome cM. This is okay. Since the X chromosome is included in the Chromosome Match Download file, my program will be able to find and triangulate X matches as well.

So there are a few glitches in FamilyTreeDNA’s creation of the Chromosome Match Download file.  I have the above concerns (shown in red). If they get addressed by FamilyTreeDNA, it would help people like myself who want to make use of the file. If anyone reading this knows any of the technical people at FamilyTreeDNA, please let them know about this blog post. They can contact me if they want more explanation.

None-the-less, those aren’t show-stopping problems. My DMT program should still do a pretty good job analyzing the Chromosome Match Download file, despite its minor flaws.