Louis Kessler’s Behold Blog

For the last few months, my Behold site has had hiccups. There were times when pages loaded very slowly or were inaccessible. I was in chat sessions with IXWebhosting’s technical support a half a dozen times in the last two weeks to say my websites: beholdgenealogy.com and gensoftreviews.com were not responding.

I knew this couldn’t continue this way if I wanted Behold to be successful. So I started my look for a service that I could rely on for the next several years while I concentrate on Behold.

Looking for a webhost is tough. Shared hosting is inexpensive and there are many options, but you are at the mercy of the other hundreds or thousands of sites that are sharing your web server and database server. Dedicated hosting, where you rent your own machine, is very expensive and has a number of downsides you wouldn’t expect. Virtual hosting, where you get a slice of a dedicated machine is possible, but performance sometimes is not as good as even shared hosting.

There is a new technology out by some webhosts that uses Clustered Servers. In theory it has advantages over even dedicated servers, because it can load-balance, give both Windows and Unix functionality at once and servers can fail or be added or removed without any downtime. I wanted to go back to a Canadian host (just because) and one of the biggest, Netfirms.ca, happens to use Clustered Servers in their Enterprise Plan.

Now if you look at various host reviews, you’ll find lots of comments about various Hosts. Over the last 5 years, I must have viewed thousands of these reviews. Netfirms.ca is in Toronto, but they have Netfirms.com that has many more servers in the USA and elsewhere. They are a very large company, and their reviews have been much less than exemplary. In fact they’re rated quite poorly.

But I think there is a difference in basic shared hosting versus the next level up. Probably 99% of people use the basic hosting, and there are fundamental problems with shared hosting. Most users of shared hosting do not understand the technical issues, and when something goes wrong, are at the mercy of the webhosts - the same way I am when my car’s transmission fails. Simply said (and I don’t quite remember where I heard this) if a player doesn’t know how to play golf, it’s not the fault of the golf course that they had a bad score.

Well, I took the plunge. I’ve spent the last 3 days solid (nice Christmas break, eh?) transferring this site, my GenSoftReviews site, my Heritage Centre site and my lkessler site to Netfirms. This includes the domain registrations and the email accounts. Some is still in progress.

There’s been a lot of work that has been overwhelming me, but I’m getting there. This is a Unix based service, so file names are case sensitive. Server Side Includes only work with .shtml files here, and I was using .htm at IXWebhosting. I’ve decided to change them all to PHP since it will give me more potential to add functionality in the future. Then I had to copy all my MySQL databases to the new host. I also have to manually make changes that are made at the old host since the copy.

Also the SMTP-based emailing from PHP which I was using at IXWebhosting is not supported at Netfirms. So the top priority has been to change my Behold Trial routine so that it would work again, and to make sure the registration of Behold from Plimus works.

So a little pain for hopefully future gain.

I really like the new control panel at Netfirms. Their support seems pretty good. I like the way I access all my sites together there and I’ve reorganized my copy of my sites on my own machine to match. And they’re big enough that I would expect they’ll be around for awhile.

My website basically works again. There are glitches that I can see (comments not showing up on posts, Forum views not working, and more I expect) that I will try to fix tonight and tomorrow.

It’s nice to be recognized. Tamura Jones has awarded his Geneablog Awards and he named my Behold Blog as the “Deepest Genealogy Development blog”. Mine was one out of 9 Blog Awards that he presented. I was familiar with 5 of the others and have to agree with his selections.

I started this Blog back on November 7, 2002. I wanted to document my work and make it available to everyone. My goal has always been to try to improve the way genealogy software works to help the genealogist. If other software developers want to take up my ideas, that would be great. I’m sad to see that over these eight years, all genealogy software, including the new Web 2.0 online sites, are all still forms-based data entry. No innovation there at all, and that’s what I’m trying to pound into everyone’s heads that it is needed and necessary to make data entry a breeze. I guess it’s still version 2.0 of Behold that will need to introduce this.

Well that got me off topic a bit.

The first important thing that I knew was missing from my blog was an RSS feed. I was reading RSS feeds from other blogs, but because mine was not really a blog, but just a webpage, I had no automated way to do this. So I spent some time, learned the RSS specification, and wrote a program to parse my Blog page and create an RSS feed. I added my feed On July 18, 2004.

For 6 years the blog was a simple webpage, where I added each new entry to the top. The page just grew and grew. By 2005, the blog had grown to quite a size, and I restarted it on March 22, 2005 to begin the Alpha testing phase of Behold.

I entered 2008 with almost 600 blog entries. I had decided then to redo Behold’s website design and thought it a good idea to switch to WordPress to be the blogging tool. I added all my old entries into the new blog to start it off and continued from there. The new blog went live on April 21, 2008. The most important aspect of the conversion to WordPress was to enable interaction. You now can comment on Posts directly without having to go to the Forum to do so.

Well that’s the history of this blog from a technical nature. But what I’m moreso proud of is the content that I have included. It is the personal diary of my program Behold, and it reflects my innermost thinking about how software for genealogists should be written.

But as I write this, not all is well. My website is with a host that has everything I need technically, but it is a shared service, and the mySQL server running the requests for my Blog, my Forum and for the GenSoftReviews site is very heavily used. It averages over 800 SQL requests every second!!! - That’s a lot! Because of this, I’ve noticed a lot of times where response times of my site slow down to 5, 10, 30 seconds or more. This is unacceptable and I’m trying to see what can be done. I’ve got everything customized so perfectly and exactly the way I want right now that it would be a shame to have to switch webhosts - which is always a crapshoot at best anyway, with no guarantee that things will be any better where you end up. Also it would involve a significant amount of work (probably a few weeks) to get everything reconfigured because every host is a bit different.

None-the-less, I’ve rambled on with this post. I am very proud that my blog was recognized. Thank you Tamura. I’ll try to keep up the standards I’ve set for myself.

Unicode files and files in many other characters sets often start off with a few special characters call the BOM (Byte Order Mark). This makes it easy for Windows or other programs, like Behold, to figure out the character set and display the text correctly.

GEDCOM files are files like any other. If they are have UTF-8 or Unicode character sets, they should have a BOM preceeding the file to indicate this. But unfortunately, many GEDCOMs that I’ve seen in this format are missing the BOM. What that means is that a text reader, and Behold for that matter, by default will assume that the file is standard ANSI text and it will come out a garbled mess.

Because of this, Behold needs a way to check any file without a BOM, to see what character set it is from before it tries to process it. But I had no idea of the best way to do that.

So I went to my good friend Stackoverflow and asked the Question: How Can I Best Guess the Encoding when the BOM (Byte Order Mark) is Missing?.

I am just in shock at how great a tool Stackoverflow is for me as a programmer. It wasn’t long before I had four very helpful answers and then closed the question because I had obtained the solution I needed:

This was the conclusion:

ShreevatsaR’s answer led me to search on Google for “universal encoding detector delphi” which surprised me in having this post listed in #1 position after being alive for only about 45 minutes! That is fast googlebotting!! And also amazing that Stackoverflow gets into 1st place so quickly.

The 2nd entry in Google was a blog entry by Fred Eaker on Character encoding detection that listed algorithms in various languages.

I found the mention of Delphi on that page, and it led me straight to the Free OpenSource ChsDet Charset Detector at SourceForge written in Delphi and based on Mozilla’s i18n component.

Fantastic! Thank you all those who answered (all +1), thank you ShreevatsaR, and thank you again Stackoverflow, for helping me find my answer in less than an hour!

Once I add this to Behold, it will mean that Behold will be able read GEDCOM files in unidentified character sets, and eliminate the missing BOM problem.

But I will have to custom ANSEL into it. Many early genealogy programs produced GEDCOMs in ANSEL, and that character set is no longer used in current-day Windows Operating Systems.