How Good are GenSoftReviews Ratings? - Sat, 4 Jan 2014
It’s been over 5 years since I set up GenSoftReviews. My goal was to have a website, sort of like TripAdvisor for travelers, that would allow comparisons of Genealogy software from reviews and ratings by actual users. During that time, users have provided over 1900 reviews for hundreds of different programs.
A few interesting trends have emerged over that time:
First, the majority of the reviews are either from people who really like the program and give it 4 or 5 stars, or who have some major complaint with the program and gave it only 1 or 2 stars. Complaints include the inability to get the program to work on their machine, the program crashes, loses data, poor customer service, or some other problem that may or may not actually be the fault of the software or the company.
Geni was one program that had several policy changes that caused a long series of user complaints and bad reviews. First they eliminated their free account. Then former free account holders could not access their own data unless they paid for a subscription. And they could not delete their account or data. These problems and the number of people who encountered them have resulted in Geni being unable to raise its overall rating above 2 out of 5.
It is really difficult for any program to get up to an overall rating of 4.5 or more out of 5 if they have very many reviews with any of these sort of complaints. As a result, the programs that rise to the top of GenSoftReviews in ranking tend to be stable programs with a dedicated user base who like the program, that are less prone to user problems or major failures. This includes programs like Ancestral Quest, Ahnenblatt, PAF, The Next Generation and Brother’s Keeper. Their users know what they’re getting, have used that program with confidence for many years, and are treated to a stable design with few updates.
Second, it is difficult for a technically advanced program to get up to an overall rating of 4 out of 5. Those people who understand and appreciate what this type of program provides them will give the program a very high rating. But there will be just as many people who are either not willing or not interested in learning how to use the advanced features who will dismiss the program as being complicated. Those people will argue that the program is not easy to use.
The Master Genealogist is an excellent example of such a program. It gets many glowing reviews from people who love the program. It gets just as many from people who can’t stand it. A program like this will consistently sit in the 3.6 rating level as every two great reviews are offset by a poor review. Some of the interaction in the reviews between the two different sides gets a little heated from time to time, and on occasion I have had to go in and moderate some of the more colorful dialogues going on there.
Another technically advanced program is Gramps, which got good reviews from its Unix users in its native environment, but not from Windows and Mac users who it throws for a loop. This past year I separated Gramps on GenSoftReviews into the three platforms, and sure enough by placing the reviews of the Unix version with those who use it natively and understand Unix, the rating of Gramps went up to 4.15 allowing it to attain a Users Choice Award. But Gramps is a rare case where the program could be segmented with its primary user base. That’s not possible for programs like The Master Genealogist who will unfortunately have to suffer with the naysayers.
Third, it is difficult for any of the very popular and feature filled mainstream programs from attaining a rating very much above 4 out of 5. These programs get lots of votes and appeal to many people. But they include so much and do yearly updates adding many new features. When you have such an all-encompassing program, it will never be perfect. You will usually like most of what it does, but there’s those one or two features that you don’t like, or are different than what you need. As a result, few people give these programs 5 out of 5, since few people will think of them as perfect. Most will therefore consider the few faults and give the program a very good rating of 4 out of 5. The fact that the program does so much actually works against itself in the ratings here.
In this group you have programs like RootsMagic which you’d think should have a higher rating than 4.11 and not be rated lower than programs like Ahnenblatt and PAF. Or the program Legacy, which just slipped to 3.95 in 2013 thus not qualifying for a Users Choice Award for the first time in 5 years. It will be interesting to see what effect the newly released version 8.0 of Legacy will have on its ratings during 2014.
The original FamilyTreeMaker (up to Version 16) has won User Choice Awards and was another popular and feature filled program in the same mould as RootsMagic and Legacy. The current rewritten FamilyTreeMaker should have been expected to do the same, but it has been plagued by releases having performance issues and bugs, as well as unfavorable comparisons by users to the original version. It is almost surprising that these complaints have taken the current FamilyTreeMaker’s rating down to nearly 2 out of 5.
So then back to the question of how good are GenSoftReviews ratings?
I don’t think the ratings should be used to directly compare one program to say which is best or which you should try first.
Instead, what the ratings give is a benchmark that the individual program should strive to maintain or improve upon.
It’s my hope that developers will see and use the ratings and reviews of their products on the GenSoftReviews site to address applicable user concerns and improve upon their software. Doing that will raise the overall level and quality of genealogy software and will benefit all genealogists everywhere.