Louis Kessler’s Behold Blog

There’s been a lot happening the past few years. As I’ve developed Behold, I’ve tried to stay aware of the trends in the genealogy field and the expectations of genealogists of their genealogy software.

I was very inspired by James Tanner’s blog post yesterday titled: Expanding Our View of What is Possible in Genealogical Research. James correctly says that the old way of doing genealogy that we all did 30 years ago is gone. In other words, we no longer have to travel to the library, vital statistics office, or archives and laboriously track down all the bits of information we need to put our ancestors together piece by piece. No. Technology has fallen upon us. It allows us to sit comfortably in our house on our computers and search and find more records and connect with more people and more relatives that we ever could have imagined possible.

The world has changed. Here is my expectation of what is coming:

1. More Interest in Genealogy

   Companies such as Ancestry, FamilySearch and MyHeritage have been claiming tens of millions of subscribers. I’ve heard that MyHeritage is adding thousands of new users each day. Over 20,000 people were at RootsTech in Salt Lake City this year. Who Do You Think You Are and Genealogy Roadshow are now regular programming on major networks.

   Why is this? Because technology has turned genealogy from a niche hobby for only the most studious meticulous researchers to one that can be done by anyone with an internet connection.

2. Everything Digital

   One of the most tedious tasks 30 years ago was paper, and writing up your family information, and organizing it, and storing it.

   It’s becoming a digital world. Everything is getting scanned. It can be saved online, or shared in the cloud, and organized in folders and every word can be indexed so anything can be found.

   Genealogy software developers are learning as well that people want/need to record their assumptions and reasoning so programs are starting to make that possible and incorporate these features. The data is digitally transferred to your smartphone so you can take it with you. Your camera, scanner, social network, online browser, cloud data and genealogy tool is becoming one device that you carry around with you wherever you go.

3. Online Data and Online Trees Ad Infinitum

   There are so many online repositories and so many online records, it is getting to the point that no one person has enough time in their lifetime to research all there is about their family.

   The online services now give you smart matches or similarly-named tools that match your data to potential family trees or records that may or may not be pertinent to you. You can easily get 10,000 of these “hints” thrust upon you. If you take only 10 minutes to thoroughly review, assess and if necessary incorporate the results of each smart link into your research, that will only take 2,000 hours of your time. By then, you’ll likely have 20,000 new links to check.

   This is obviously unmanageable and cannot persist. It means that new tools will be coming to identify and make the dissemination of this information easier. (I’m thinking deeply about this)

4. Down with Standards. Up with APIs

   I’ve been a supporter for years of both the BetterGEDCOM and FHISO initiatives for a new genealogy data communication standard. But I’m now feeling the effort will not get anywhere unless it completely changes its emphasis.

   We don’t want to transfer just data anymore. We want to connect the information available at the online repositories and online services to what we have and make corrections, add conclusions and connect the conclusions to their evidence. In other words, we want our data AND our reasoning AND the evidence behind our reasoning to transfer and connect seamlessly with the online resources.

   I really think AncestorSync had the right idea. Connect to everything. Use the Application Programming Interfaces (APIs) of each company to connect your data to theirs. Hide the details of the API from the user and make it seemless to the user. It should just work like magic. Unfortunately, the implementation of that idea was much harder than the even the very smart people at AncestorSync thought, and the effort was abandoned.

   But it’s starting again. RootsMagic is connecting to MyHeritage and FamilySearch. FamilySearch has partner sites who interact with its data. And other sites are building public APIs as well.

   Once there is a company big enough that connects to everywhere by linking to all these APIs, it will becomes hugely popular, and the genealogical world will take another giant leap.

5. My Data / My Research

   The concept of one world tree is fine. The concept of individual linked trees is also fine. Each has advantages and disadvantages.

   But everyone wants/needs to separate out the data that they “know” is true from all the other stuff/junk/conjecture/miscellanea that Joe Blow has put up on some online site. We want to know exactly what we have personally examined and verified and concluded.

   So there is still an extreme need for personal genealogy data. The best place for that is still and will always be on your own personal computing device so you can ensure that no one else will update or tamper and destroy what you worked so hard to produce. So desktop software is not dead and will never be (at least until the desktop itself dies).

   All you’ll need is that magic API program from #4 and you’ll be set.

6. Genetics and DNA

   The elephant is in the room. The technological advances that made DNA testing affordable to the masses in the past 10 years has taken the world by storm. Millions of people have been tested at several different testing companies and a whole new science of genealogy has been born.

   It is really unbelievable what you can do with DNA results when a company has a million other tested people you can compare with. Genealogists are in the “still trying to figure this out” phase, but it’s really simple when you think about it.

   You have two genealogies. There’s your traditional genealogy of whom you, your relatives and the records think your ancestors were. Then there’s your genetic genealogy that says who your genetic ancestors were. These two genealogies are not the same. They may not even be close. The rate of genetic NPE (Not the Parent Expected) has been estimated at between 1% and 3% or higher. By the 6th generation, half the ancestors in your genetic tree might not be who you thought they should be.

   Genealogy will, by necessity, evolve so that people realize they have multiple ancestries, and will want to trace both their traditional family and their genetic family. People have to smarten up first and realize that there’s a reason why your grandfather does not have a DNA match with you. So don’t promote DNA research through your family until you are absolutely sure no one will get hurt by it!

   But this DNA thing is phenomenal. Take it. Embrace it. Use it if you dare.

It’s that unexpected exceptional case that you (of course) never expected, but then happens.

I’ve been working hard the last couple of months to finish Version 1.1 of Behold. I’m so excited about it and I’ve effectively finished putting in everything I want to put in. I’ve been running through my test cases to ensure it all works it should.

My counter on my phone says that today is the 19th consecutive day in a row that I’ve worked on Behold since back from holidays. For 3 weekends now, including one before I left on holidays, I was expecting to announce that this version was ready. That’s 3 weekends now that one or more of these unexpected exceptions has occurred that has prevented the release. This latest one, which took me 5 days to resolve, is an excellent example. With it you’ll see some of the great information that Behold’s Everything Report will give you in Version 1.1.

While testing my own GEDCOM file, I came across this:

This is my father’s stepfather (who I’m named after) who’s first wife died and, as was traditional in those days in the farming communities, widowers needed a wife, and he was quickly matched up to my father’s mother Goldie whose husband passed away 4 years earlier. They married less than 5 months after Louis’ wife passed away.

So what’s wrong here. The first thing I saw above that was wrong was that at Louis’ death, it is saying that he was widowed 11 years. That is not correct. He had married his 2nd wife and was married 11 years, not widowed, when he died.

The second thing I see wrong here is that Sarah is marked as Wife 2 and Goldie as Wife 1. They are shown in the correct order but are numbered wrong.

The third thing wrong was that Sarah’s death was not shown.

It took 3 days inspecting and debugging my code before I thought of looking in the GEDCOM file that contained the data. Sure enough, Louis’ two FAMS records that represent his marriages are listed in the incorrect order of Goldie first and Sarah second. I already had included a check in Behold to ensure that marriages in the GEDCOM file are in correct marriage date order. Most genealogy programs do output these records correctly, but Behold will issue a warning if they don’t. And then Behold will fix the order.

I had a puzzle.

Why didn’t the order get fixed? Well, the marriage date with Sarah was unknown. So Behold couldn’t fix the order.

Why then was this still a problem? It was because the spouse of the last marriage had died, so it displayed Louis as a widower.

Why was wife 2 listed first? Because there was no marriage date and I had a slight bug that didn’t order the marriages correctly in that case.

Why didn’t Sarah’s death show up. That’s because if Sarah was married second, then it was after Goldie’s marriage. This means that Sarah’s death was before Goldie’s marriage which was before Sarah’s marriage and I put smarts into Behold so it wouldn’t display your spouse’s events prior to your marriage with your spouse – since he/she wasn’t in your life yet.

And the fact that Louis was widowed for 3 months and that Goldie was widowed for 4 years when they married is important. This needed to be added.

Once I understood the problems (that was the hard part) the rest was easy to fix (if you think that 8 hours of programming work is easy). While sorting each person’s marriages, if a marriage date was not given, then I had to check the spouse’s death date and ensure that it wasn’t before the date of any of the previous marriages. If so, then this marriage must have happened before the previous marriage and the order will need to be switched.

I’m not sure anyone will have been able to follow all this, because my head’s spinning just from writing it.

With the work done, final results were pleasing. I got this:

Now this is no longer just your standard birth/marriage/death information. It tells a real story and puts everything in context:  Louis was born in 1878 in Russia. He first married Sarah (date of marriage unknown or it would be shown). Sarah died when Louis was 50 to 51 and when Sarah was 51. Louis then married when he was still 50 to 51 after being widowed for 3 months. His 2nd wife was Goldie who was 36 and was herself widowed 4 years. Louis died at the age of 61 to 62 after being married to Goldie for 11 years.

I have designed this information to be extremely useful for anyone trying to do family research and understand the lives of their family members. Context is everything, and I’m working to produce an Everything Report that will supply you all the context you will want and need.

To the FHISO Board, TSC Coordinators and Membership,

Richard Smith said on Feb 21: "Earlier this year, the FHISO Board, TSC and other stakeholders decided that designing a new conclusion-transfer format /ab initio/ was not currently a priority … and unless and until we have the resources to develop a new technology we’d like to focus on incremental changes …"

As a paying individual member of FHISO, I am interested in seeing FHISO moving forward and not having to sluff anything off, especially due to lack of resources, which in this case I think means people and expertise. Richard mentions the FHISO Board. Have they been meeting 4 times a year as required in the Bylaws? Do we have minutes of the Board being recorded to document the decisions the Board is making and the actions it is taking? Is the general membership allowed access to the minutes to see what is being decided?

I am concerned because I did see (and retweeted) the really-nice-to-see picture of the meeting of the four FHISO Board members at RootsTech 2015 this month and that was great. But from what people tell me about RootsTech this year, FHISO was almost invisible and never mentioned there. There was no FHISO information available, there were no lectures centralizing on a new standard, and nobody organized any discussions or even informal meetings for interested people about it. This was, for goodness sake, Roots "Tech" and there were all sorts of developers there who could have an interest and maybe participate to help develop the new standard. If nothing else, Drew Smith should have been advertising himself as the Chair of FHISO and promoting FHISO, but he only seemed to be referred to as one of the two Genealogy Guys. Is FHISO not at all interested in looking for people with skills and interest in helping the effort?

What of our Founding Members? There are some great companies in FHISO’s Founding Member list. As an individual member, I’m disappointed to see that not more than 20 unaffiliated individuals have taken enough of an interest in the past year to participate in the TSC mailing list. And of us, very few were and are willing to volunteer for the Technical Standing Committees when so many are needed. There’s a lot to do and it will never happen at the current pace.

I’d like to know if the Board has considered asking their Founding Members to send some people to participate in the discussions and help FHISO create a momentum that will get this new standard creation going. If the Board has not, then it would be my suggestion that they do. If FHISO can get one or two people from each Founding Member to actively and interestedly help with FHISOs goals, then that could really get the ball rolling. Plus there would be the double advantage of giving those members a stake in this, to make it happen, and it will.

My biggest worry is that there’s nothing happening in the background. No Board Meetings. No decisions. No initiative. I as a member am completely in the dark as to what’s going on. Please reassure me.

Louis

This article has also been posted on the FHISO TSC-public mailing list.