Login to participate
  
Register   Lost ID/password?
The Behold User Forum » Topic           prev Prev   Next next

Importing Two Ged Files - Categorized in: Feature SuggestionsFeature Suggestions

19 posts. Started 8 Aug 2005 by sjm1010. Latest reply 24 Feb 2013 by lkessler. RSS 2.0 feed for this topic RSS
1. Stephen McCarthy (sjm1010)
United States flag
Joined: Mon, 8 Aug 2005
1 blog comment, 17 forum posts
Posted: Mon, 8 Aug 2005 Permalink
Lou,

What about the ability to load two separate GED files at once for a side by side comparison, this would allow the program to compare and search out duplicate files or for the user to manually edit and remove files from either GED. When you complete the editing you can merge the new file with your master or just save it as a separate file. I suppose this would be a great way to merge one or more files that contain your family rather than performing a merge for each individual GED that you come across.
2. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Tue, 9 Aug 2005 Permalink
Now you've reached into something that I feel very strongly about.

I am and always have been dead set against merging other people's data into your own. I am shocked at how much importance is seen in being able to do this. This has caused a horrible problem: a proliferation of "junk genealogy" where little is proven or even questioned. The important thing seems to be adding 2000 more people to your family, whether or not they are truly connected - and then no work is done to verify that anything in that data is true.

But even if we forget about that, then there is the second problem: Let's say you get a program that has amazing side-by-side merge capabilities. It only takes 1 minute to figure out what you want to merge about each person. The 500 people you need to merge only take you 10 hours. Then, 2 months later you get an updated version of the file you had merged. Do you now want to go through it all again to get the updates? When doing this, will you be able to tell what was from your original data and what was from the earlier version of the file and the decisions you made about which data is better and why?

It's all wrong and a waste of time, I say. Instead, I think it is important to keep separate files that are received from different people or places. It is this concept that I am building into Behold. Behold can merge multiple files and you can select the families from each file that you want to display. The information about which files and which families can be saved in a Behold file. By opening the Behold file, it will reload all the files and families as you have specified them. That way all files can remain separate, but useable together.

If you want to then save this as one GEDCOM file, Behold will allow it (in Version 1.5). But I wouldn't recommend doing that.

Now there are two related ideas. One is to be able to identify which people in different files are the same. Then the data can be displayed together in the Everything Report. Doing that will allow the direct comparison that you are interested in, but will alleviate the burden and problems of merging. I want to allow the identification of identical people to be stored in the Behold file so that this info can retained and reloaded when needed. Hmmm. It's not in my Future Plans, so I'll add it.

The other idea is to simply use Behold to load multiple files and look for possible people matches. I've had that concept for a long time, and have some ideas on how to do it, but the details need to be worked out. Possibly color highlighting or something like that.

I've rambled on a bit here and I'm not sure if I completely addressed your suggestion. Assuming you have a good tool that allows you to look at the data combined from separate files, do you still have a real need to merge them that I may be overlooking?

Louis
3. Stephen McCarthy (sjm1010)
United States flag
Joined: Mon, 8 Aug 2005
1 blog comment, 17 forum posts
Posted: Tue, 9 Aug 2005 Permalink
Okay first things first, I want a REFUND! I
4. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Tue, 9 Aug 2005 Permalink
Whew. You had me worried. First about merging for the sake of merging, and second about wanting a refund ... also just kidding.

My problem with side-by-side merge/display programs is that they need to have selected fields to compare: Name, Birth date, Birth place, Wife, Children, etc. There is a lot they can't show, such as notes, source references, multiple names, all the pictures, etc. In other words, they pick and choose what they show and cannot show everything. They do this in yet another modal box that takes over your screen. That is not how Behold is set up to work.

Behold's comparison would be to take the "Everything" about one person in one file and displaying it with the Everything about that same person in another file. The data will not be physically merged but will display together, which is I guess what you meant all along.


I could get Behold to put all the data from the first file just before the data from the 2nd file, e.g.

F1-14
John Smith
b. 14 Nov 1900 in New York, NY. [S23]; 17 Nov 1900 in Long Island, NY. [S42][S16-3][N5]

F2-42
John H. Smith
b. Approx 1902. Estimated from marriage date.


Or I could get Behold to display them with each specific type of info shown together, if I can figure a way to do it without it being too messy, e.g.:

F1-14; F2-42
John Smith; John H. Smith
b. 14 Nov 1900 in New York, NY. [S23]; 17 Nov 1900 in Long Island, NY. [S42][S16-3][N5]; Approx 1902. Estimated from marriage date.


I haven't put too much thought into this and nothing is final yet, but your comments are helping me to evolve this idea into something that will work well.

The goal, of course, is to have a format to make all the info available so as to help solve those nagging mysteries and inconsistencies. Then your conclusions and best guesses can finally be properly documented.

Louis
5. Stephen McCarthy (sjm1010)
United States flag
Joined: Mon, 8 Aug 2005
1 blog comment, 17 forum posts
Posted: Wed, 10 Aug 2005 Permalink
What would be the answer when you do find a comparison that actually matches from the two everything files? Would you allow Behold to integrate the information from either file or would you just manually enter only the data that interests you into your main, or none of the above.

Steve
6. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Wed, 10 Aug 2005 Permalink
Despite my big disclaimer in my first message: Yes, I see that I must add this capability to Behold. I agree that there is a need for such a function for many valid reasons.

I'm not sure how I will do it yet. Maybe by simply selecting one or more people then using a right mouse menu function (or other ways) to get Behold to combine them. Items from the 2nd (or 3rd or 4th) files will be tagged with a Source that will indicate what file it originally came from.

(I'm using the word "combine", but yes, it is a merge).

But it will, like most things in Behold, be very different than other programs. Because all the info is visible in the Everything report, seeing it, selecting it, and using this merge function will be unbelievably easy to do. It will combine notes, sources, links and not miss anything. The merged data will look identical to the unmerged data (with the change of some reference numbers and addition of source references). Then manual edits (which will also be incredibly easy to do) can polish it off.

It is because this can be made so easy to do that I am worried.

Louis
7. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Sun, 14 Aug 2005 Permalink
Regarding ranting about junk genealogies, this is interesting:
randomgenealogy.com/2005/08/13/lost-fortunes-in-the-family-tree-slight-rant/


8. Stephen McCarthy (sjm1010)
United States flag
Joined: Mon, 8 Aug 2005
1 blog comment, 17 forum posts
Posted: Sun, 14 Aug 2005 Permalink
Excellent article Louis, very good reading, and it's very sad indeed to see that the quick and easy cure-all has made its way to the point where people ignore their true place in history.

I personally never cared if I was the son of Kings King. I am completely content just finding out about my ancestors every day lives, what they accomplished and what they had endured. This is why I am very particular with facts in my daily research.

Unfortunately, as the article reflects, disturbing amounts of people are making it a part time business venture rather than a full time personal adventure.


Steve
9. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Wed, 23 Nov 2005 Permalink

Further thinking on this: "Linking individuals from different databases ... has to be the way of the future".

See my thoughts in My Behold Blog posting for Wednesday, November 23, 2005.

Louis

10. David Bryant (DavidB)
New Zealand (Aotearoa) flag
Joined: Tue, 15 May 2007
0 blog comments, 1 forum post
Posted: Tue, 15 May 2007 Permalink
This is an important concept deserving careful attention.

When teaching I always advise users to open received GEDCOM files separately rather than a straight addition. Two key reasons:
1. When exchanging data with other family researchers there are (usually) a great many duplicates and the automatic merge function is rarely good enough to accurately distinguish between 'probable matches' and 'possibles' - such as twins Jean and Joan. Manual merges can be very time consuming.
2. Some of the information is not reliable and needs independent verification before adding to one's main family tree. For this reason it is important to retain the submitters information as an over-arching source record.

I like the concept of side-by-side Gedcoms with cross links to COMBINE data without a full MERGE. But I would like to be able to copy and paste from one to the other when I am satisfied about the quality of the data. (I'm a new user so haven't explored enough to know whether this feature is already available.)
11. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Tue, 15 May 2007 Permalink

Thank you David for your ideas.

I think your advice is the correct advice. I have very strong feelings AGAINST merging other people's data into one's own. I'd go even farther and say that you should NOT add other people's data to your own.

Behold will be introducing a concept I call "Virtual Merging". I discuss it in my Behold Blog post of August 28, 2006.

My idea is that (with Behold's help), you specify which people in the files are the same, and Behold will produce one report for you, merging the information for those identical people from the files.

Does that make sense to you?

Louis

12. seecan05 (seecan05)
United States flag
Joined: Mon, 2 Jan 2012
0 blog comments, 2 forum posts
Posted: Mon, 2 Jan 2012 Permalink

Hello Louis,
I am considering purchasing a membership to Behold. First however, I have a couple of questions and one led me to the subject heading of this particular post "Importing two GED files".
My files are very large, not nearly as large as some, but there are more than 2000 names in my family file. And this is mainly my paternal grandmother's side. But I also have a maternal grandmother's GED files.
I would like to keep these seperate, and so can I import my Mother side, and then a 2nd GED file for my father with one membership?

My second question, and I apologize because it is unrelated to the topic: In reading your bio stating that you are age 55. If my membership would be forever, is that my forever or your forever. What do you have in place that will allow Behold out live us both?

13. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Mon, 2 Jan 2012 Permalink

Seecan:

Currently, Behold will import two (or more) GEDCOMs and display them together. The current weakness is that it currently has no way for you to indicate the people who you know are the same in the two families, so every duplicated person will be shown twice.

In the case of your paternal grandfather and paternal grandmother, it is likely that your parents and all their descendants will be in both files, as well as possibly some cousins who may have married between the families and their descendants.

That will be fixed when I implement Virtual Merging, which will give you ways to simply mark the people that are the same so that their data and their descendents' data can be displayed combined by person. Currently, that is on my future plans page, but after Version 2.0 and editing. Although every time someone asks about something, like you just did, it moves up in my internal priority list.

In the meantime, you should still find the current merging very useful. The people will be shown separately, but the Name Index, Place Details, Source Details and everything else should prove very useful and allow you to find data problems in both your files.

Hmmm. Now you're getting me thinking of a way I can implement Virtual Merging much more quickly. I can on the Organize Families page, allow you to add the person from the Second GEDCOM file, and allow you to specify that they are the same as a previous person on that page. You'll only need to do that for a few of them, being the beginning of the common lines. It should work out quite well. I may try to do that prior to going on. Thanks for the push.

Your second question is a very good one and I'm happy to answer. Although no one can predict the future and I may get run over by a bus tomorrow. But barring something like that, I'm planning to continue work on Behold for the foreseeable future which is likely a minimum of 10 years. I'm absolutely positive it will be successful because there is no other program like it, and it's the program I will want to use to record my own genealogy (which will extensively test it as well as I start to enter my masses of data).

Currently, I have nothing in place should that bus be lurking. But I have been thinking about it. If I grow large enough, I'll hire others to help me, including programmers. (I'd love to get big enough to sponsor my own Genealogy Cruise!) If not, and Behold remains of interest, I may sell it to someone willing to keep it going, or donate it to the Open Source effort. But Behold's my baby. I won't let it die.

Besides, $40 is less than you'll spend on a single upgrade of another program. So after 10 years of use, I'm sure you won't be lamenting the loss of the $40. What will be more important to you will be if you will not lose all your hours of work and be able to continue your research with another program. Make sure your program will export compliant GEDCOM. Behold will. Many programs don't

Louis

14. seecan05 (seecan05)
United States flag
Joined: Mon, 2 Jan 2012
0 blog comments, 2 forum posts
Posted: Tue, 3 Jan 2012 Permalink

Louis,
Thank you for the quick response. You are right in that the cost of $40 is much less than other programs. That is appreciated. It almost seems worth it to purchase 2 memberships, one for files for each side of my family..... notice I said almost. Then when you figure out a way to not have one person duplicated (what you suggested above seems a bit tedious), I could possibly merge the files from the other membership so that as my immediate family grows I won't have to add new members of my family in both membership files. I'm sure that you have already thought of that scenerio and worked it into your plans.

I will want to make sure I have included all of my information before I create yet another GED file to upload to Behold.

Thanks again.

15. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Tue, 3 Jan 2012 Permalink

Well, my idea for Virtual Merging is extremely necessary. Too often, people are merging other people's GEDCOMs into their own, trusting that other people did theirs correctly and proliferating what are called "junk genealogies".

Not only that, you WANT to keep your work separate. If relatives send something to you, why go through days or weeks of work to merge theirs into yours, only to have them send you an updated GEDCOM six months later. That time could be much better expended doing your own research, rather than on some mechanical merging that could be done virtually.

I'll make the virtual merging painless. The most difficult thing for me will be figuring out how to help identify who might be potential merge candidates, since names and places might be spelled a little differently, dates might be different, and spouses and children can be different or missing. Once I figure out something for that, the rest is a piece of cake.

Louis

16. trolleydave (trolleydave)
United States flag
Joined: Sat, 8 Sep 2012
3 blog comments, 10 forum posts
Posted: Mon, 29 Oct 2012 Permalink

Hi
Browsing this Forum on linking vs merging, I came across Louis' post from Nov 2005 with a comment on the program GenP and it's ability to link mulitiple databases. Since I'm totally convinced that small databases with linking are the way to go (one of the many things promised in Behold that I can hardly stand to wait for), I decided to take at look at it. I downloaded the demo version and generated a couple of minimal databases, based on two grandparents. Grandpa's file included both his ancestors and descendents, but no spouse. Grandma's file had only some ancestors. (The demo version won't import GEDCOMs).

For me the pgm had a rather steep learning curve, but after much fiddling around, I did manage to create a working link between them, specifically between one of Grandpa's children in his file and my grandmother in hers. It works both ways it seems, the child's pedigree includes his parents and grandparents on both sides, and a descendents list for grandma's father includes her son from grandpa's file.

What it didn't do is make a connection between grandma and grandpa, so that grandpa's family page has no spouse. GenP seems to be resolutely agnostic on relationships; if I'm understanding it correctly, every one needs to be specified. Accordingly, I guess that one would also need a marriage link between grandma and grandpa, and I don't know if that is allowed. And each of the other 7 children in grandpa's family would need separate links to their mother?

The instructions are less than clear and the video tutorial on linking goes by so fast that even after many re-runs, I'm still not clear on what is important or not or why certain sequences are as they are.

Working with GenP did point out one problem with being dogmatic about relationships. If there is no "child" category with implicit relationship to a set of "parents", entering large families becomes a lot more work. You certainly want a way of entering non-conventional relationships, but even in today's societies, some specific block of children biologically associated with one male and one female parent are in an overwhelming majority and a shortcut to that assumption is a good place to start.

I guess I don't have the time or the $50 to invest further in this program, but it was an interesting exercise, for what it's worth.
Dave

17. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Mon, 29 Oct 2012 Permalink

Thanks Dave for the review of GenP - which I actually haven't tried myself. Unlike GenP, Behold maintains the two-way relationships. When one way is created, the other is as well (and Behold will identify one-ways in the GEDCOM that should be two-ways).

Hopefully, when I add virtual merging to Behold, I'll be able to make the identification of identical people very simple, so that it shouldn't take more than a couple of clicks. Then, somehow it will have to cascade to spouses, children, parents etc and allow some sort of checklist confirmation of the ones to combine. But that will be worked out when the time comes.

Louis

18. serenity2006 (serenity2006)
United States flag
Joined: Sat, 9 Feb 2013
0 blog comments, 2 forum posts
Posted: Sat, 9 Feb 2013 Permalink

I completely agree with the need to keep GEDCOM files from other users separate. When I first started doing genealogy research online (decades ago), I fell into the newbie trap of merging GEDCOMs into my own and ended up with a huge mess.

I do need the ability to view GEDCOM files side-by-side and merge individuals into a single family tree. I have experimented with multiple genealogy software products over the years and now have about 20 versions of my family tree spread out all over the place. I am looking for an easy way to merge all of the various trees into one family tree so I can identify the gaps are in my research. Once I get all the pieces into one tree, I will be able to delete all the old versions and clean up the space on my computer.

I am also doing a one-place study on a town for their 150 year celebration in 2018. I hadn't really thought through my approach when I started the project 35 years ago or when I started moving it all to computer files about 20 years ago. Frankly, I never thought that I would be doing the research for the town celebration at the time the project started. So now I have a separate tree for each census year for the town. There are numerous overlaps of people from one census year to the next and I need to merge all of these trees into one tree for the town history project. You are correct in your statement that the merge functions in the software products on the market are limited and it has been a grueling task trying to clean up the mess I made when I created separate trees for each census.

I look forward to the addition of the side-by-side and combine feature in Behold.

Theresa

19. Louis Kessler (lkessler)
Canada flag
Joined: Sun, 9 Mar 2003
148 blog comments, 200 forum posts
Posted: Sun, 24 Feb 2013 Permalink

Theresa,

I doubt I will be implementing a side-by-side feature. My current thoughts are that I'll probably be mixing the data together, with events for each person ordered by date, and allowing you to highlight the data from the different GEDCOMs with a different background color.

You would not want to use any genealogy program's merge utility to merge people in different censuses at one place. All program have people-based merges, that don't work for sources. You need a source-based data entry system, one that will suggest likely people that the source refers to. There isn't one out there that I know of (yet).

Louis

Leave your Reply

You must login to post your reply.

Login to participate
  
Register   Lost ID/password?