Login to participate
  
Register   Lost ID/password?

Louis Kessler's Behold Blog

Seven Years of Improvements - Sun, 12 Jul 2015

I know my last two posts were not written in English for the normal person. But I felt I had to document a bit of the work involved in just upgrading a development environment. It is complex process full of pits and valleys. Getting through it needs a machete and a torch. We programmers struggle with our programming tools as much as our users struggle with the programs we create.

Now it is time to reap the rewards. Delphi and TRichView and ElPack and EurekaLog each have gone through seven years of improvements. I can run through the change logs of each version, and see what changes are notable and will provide benefits to Behold users.

 

Delphi improvements

Lots has been added since Delphi 2009.

The most important improvement for the future of Behold is FireDAC with SQLite support. I will be using this for Behold’s database.

Another improvement is 64-bit capability. Now I will be able to make Behold available both as a 32-bit program and as a 64-bit program.

There are new online data access protocols now available including REST web services, and libraries I may be needing. The newer versions of Delphi include improved JSON low level processing and an improved XML engine.

There were supposedly style improvements that take on the Theme of the O/S (Vista, Windows 7, WIndows 8), etc., but other than it adding a blue background to indicate that a toolbar item was selected, I haven’t noticed too much difference. It’s also touch-enabled. XE8 added support for Windows 10 and Windows 10 styles, so it will be ready when that comes.

Delphi has really enhanced its Generics library, which is a suite of data structures I can use in Behold. They are well-written, generalized and fast. They should speed my coding time, and over time I’ll replace my older data structures with these. Delphi 2009 included an early version of Generics, but there were some bugs so I only made limited use of them.

A Parallel Computing library has been added that will allow me to optimize Behold with background processes and to keep it running as fast as possible.

They also made lots of fixes and improvement. One item that really pleased me was a fix that helped Behold’s Find box. When you bring it up, it stays on your screen, but it is called a non-modal box because it does not take control of your screen. So you can find something and then go to the Everything Report and work on it, and then go back to the Find box. The default action in the Find box is of course to find the text. In Behold 1.1 and earlier, going back and forth between the Find Box and the main program would somehow lose the default action. I would press Ctrl-F to go back to the Find box and then press enter to get the next search, but it wouldn’t work. I tried in the past to find an easy fix to this, but it was a Delphi/Operating System thing. What pleased me is apparently, whatever was causing this loss of the default was fixed sometime between Delphi 2009 and XE8. Hopefully more subtle bugs like this were also fixed by going to XE8, without many new ones creeping in.

 

RichView improvements

The Everything Report is built on RichView, so any new features and improvements to RichView can benefit Behold.

I was using RichView 10.1.4 from 2008 and have now installed version 15.7.

Now it can export as a word document (.docx files). I’ve added DocX as an export option in Behold and it will be available in the next release.

Some fixes/improvements include improved rendering of bidirected text, optimizations in calculating text width (which is probably the main reason why Behold loads 30% faster when compiiled with Delphi XE8 and RichView 15.7), touch screen support, e.g. handles when selecting with a touchscreen:image

Also improvement to the line breaking algorithm, 64-bit compilation support, style templates (which I may or may not use). Hidden items were added, but I already had my own customized implementation of hidden items which I liked better, so I kept my own.

 

ElPack improvements

I use ElPack in Behold for the TreeView, the grids in the Organize pages, and a few other components. I had the 2009 Version and I upgraded to the 2015 Version to get a version that would work with Delphi XE8.

ElPack is not LMD Innovative’s major product. They acquired it quite a while ago from ELDOS who I initially purchased the package from because I needed a treeview and grid with Unicode support. So LMD is maintaining it, and is slowly merging it with their core LMD products, but they are not doing much to enhance it.

They’ve added 64-bit and touch-based features. They fixed a number of bugs. And they’ve included a few new packages with ElPack like a DialogPack that I could make use of in the future.

 

EurekaLog improvements

EurekaLog traps errors that would otherwise crash Behold. It lets me display a message to inform the user there is a problem, and provides some information that might allow me to find and fix the problem. I also use it when I’m developing since it can help me find and fix memory leaks.

I first purchased EurekaLog Version 6 back in 2009. I initially used EurekaLog’s default dialogs and this allowed the users to email the problem to me. It caused me problems. Users complained. So I changed it:

I came up with my own form after careful thinking. I did not want a generic box. I wanted something that the user would know it was my program self-detecting the problem, and that it was not Windows telling them that my program had a problem. EurekaLog’s generic dialogs look too much like Windows. I attempted to fix the email problems with a "mailto" email. But even with that, the emailing was problematic, so I changed it to my current form, where I instead ask the user to drop off the bug information on the Behold feedback page:

image

EurekaLog’s improvements going from Version 6.x to 7.x involved a complete rewrite of their product. My old code wouldn’t work. I placed a support ticket with EurekaLog and Kevin of their support team worked with me and provided me a framework to continue to trap errors and provide my own form the way I had been doing.

Upgrading to Version 7 will provide 64-bit error trapping and better information for me for debugging purposes.

 

Overall

Upgrading to my new development environment that includes the latest versions of Delphi, TRichView, ElPack and EurekaLog took 14 days and involved about 400 recompilations of Behold.

Now I’ve got the tools available I’ll need to go forward and create a native Behold database to store your data. Then, with a place to store your edits, I’ll be able to add editing to Behold.

Getting Behold Running in Delphi XE8 - Mon, 6 Jul 2015

Last post, I updated my development tool and installed new versions of my three 3rd party applications. Now it was time to get Behold working again in the new Delphi XE8 environment.

Theoretically, if all updates made to an environment are backwards compatible, then old code would just run unchanged in the new environment. But they seldom are. Work needs to be done to convert.

To get started, I included the 3rd party packages without my customizations to see if what was left would still run. Then I compiled, and knowing that there was little likelihood of it working, I got:

2 Errors, 5 Warnings, 1 Hint

One error was an incompatible type between Integer and TRVTag. This was a change I remember reading that RichView had made a few years ago to use strings instead of integers for their tags. The second was the error aborting the compile because it couldn’t compile the unit. For now, I just changed the declarations to eliminate the error and proceed.

image  image

That doesn’t sound so bad for a first attempt, to have only 1 error. But I know how these compilers work. You fix one error and the compiler can then look at more code. Sure enough, after I fixed it, I now had:

16 Errors, 5 Warnings, 2 Hints

EurekaLog was now the culprit. The errors indicated major changes to that package, and indeed when I checked out their site, I found that Version 7 was almost completely rewritten.

image

I wasn’t trying to get EurekaLog to do much. It was passing information from unexpected errors to a special form I had set up for that purpose. I looked to see what changed, and it wasn’t simple. I always give myself a couple of hours to see if I can figure out any problem. I go through their documentation and try various things. If stumped I might check StackOverflow and maybe ask a question there about how to do it. But I could not make any headway on this, so I went to the next step and submitted a support ticket on the EurekaLog site with my problem and requested a method to fix it. I got a response quickly, within a couple of hours (on a Saturday!) and that answer led me to some good examples, but they weren’t quite enough to help me. So I added to my ticket and while waiting went back to fix the rest of what wasn’t working.

I commented out the offending EurekaLog statements and that allowed Behold to compile and run. It was nice to see Behold running from XE8. It ran fine, looked fine (except for a minor problem with the toolbar icons), and did not have any obvious problems.

There were still lots of warnings and hints to fix. These included fixing declarations, replacing a deprecated symbol or routine with another, and new messages about possibly uninitialized variables that the XE8 compiler was picking up that 2009 didn’t.

I worked my way through the most important compiler warnings and hints. In doing so, I must have recompiled the program 200 times over 2 days, finding and fixing things that worked under the old environment but not in the new one. I finally managed to get through them, leaving only a few hints about variables that are only valid on some operating systems.

With everything finally working with 32-bit processing, it was time to try 64-bit.


On my first 64-bit compile, I was quite surprised that there were new errors. The 64-bit compiler must have different checks in it. For example, one warning that came up was for code that I’ve had in place for as long as I can remember. It had never given me a message before. It was:

N := MaxN;    { Execute the loop at least once }
while N = MaxN do begin
   …
   N := …;
   …
end;

The warning message on the first assignment statement was that the value assigned to N was not being used. 32-bit compilation never gave me this. 64-bit must optimize differently. So to eliminate the warning, I changed this to:

repeat
   …
   N := …;
   …
until N <> MaxN;

There were a bunch of little fixes like this I could make. But then I was getting errors again. The 64-bit compilation wanted me to address the RichView change of their tags from integer to string values. I was going to try to use a setting in RichView that allowed using the old style, but then I found the 64-bit compiler was not accessing those settings. I posted a query on the RichView forum. While waiting for a reply, I found that using the old style was not recommended, so I decided to go through the work and change it. It took me a few hours to update what was needed here. Once I did, Behold was able to compile and run in 64-bit.

The last thing to do was to add my customizations back into the RichView and ElPack routines. I had customized a 3 RichView units and 4 ElPack units to enhance them to provide some functionality I needed for Behold and to fix some bugs in them. There were changes to 7 different units. They represent dozens (maybe even hundreds) of hours of work I spent many years ago to isolate problems or limitations in these packages and figure out how to get them to work the way I needed.

I had delineated all my changes clearly with comments in my customized units, so they were easy to find. One of my typical changes looked like this:

{ LK - Jul 15, 2012 : Start of code to allow invisible text }
const
  rvInvisible = 21; {= rvInvisible in _links.pas}
{ LK - Jul 15, 2012 : End of code to allow invisible text }

Now the question was, how much did these units change in 7 years? Fortunately, these were mostly low-level units with core functionality. The new version of these units did not change much from the old version, and I was able to simply slot my customized code back in, occasionally with minor modifications. I had the RichView units working in just 15 minutes.

But then it took me 2 hours to get the 64-bit version compiling. Some sort of strange dependency in the use of one unit by another was only revealing itself one unit at a time. And this was just in the 64-bit compile, not the 32-bit compile. It was very strange. I never did figure out what this was, but I thankfully reached the end of the loop after going through only 60 of the 200 units.

I stumbled a bit while updating the ElPack units with my custom code. In the ElIni unit, which handles reading and writing the Behold Organize file, the ElPack unit did not read ASCII files properly. I had the customized code to do this correctly. But it did not seem to be working. I installed, tested and removed that code 3 or 4 times. Behold was not reading any of the information from the customized Behold file at all. Then then it hit me. My code wasn’t what was wrong. The customized Behold file I was trying to read was empty. There just wasn’t anything to read. I kicked myself about this oversight. Only cost me 2 hours.

So that was it. Behold is now all updated and is now pretty well working under my new Delphi XE8 environment. I’m still waiting for the EurekaLog support message and I’ll get back to that later.

At this point, I couldn’t help myself. I just had to do run some comparisons of Delphi 2009 performance versus Delphi XE, and then of 32-bit performance versus 64-bit performance. The results were interesting:

Compared to the current Behold Version 1.1 release (which was compiled with Delphi 2009), the new 32-bit compilation under XE8 is about 30% faster and uses about the same memory.

Then I found that the 64-bit compilation is about 12% faster than Version 1.1, but is about 18% slower than 32-bit speed. And 64-bit uses almost 50% more memory.

This surprised me. My computer has a Windows 64-bit operating system. I would have thought 64-bit processing should have run faster than 32-bit processing on my machine, but I guess the manipulation of all that extra memory the 64-bit program takes also adds a time penalty.

Once I convert Behold’s in-memory data structures to a true disk-based database, that will eliminate much of its memory use. Then time will be more dependent on 32-bit versus 64-bit database access. Once that’s done, I’ll recheck these results and won’t be surprised if 64-bit happens to emerge as the winner.image

Updating My Development Tool - Fri, 3 Jul 2015

I’ve been working with Delphi as my development tool for Behold since about 1996. I’m fortunate that the language has continued to be maintained and upgraded during that time, even though it changed ownership several times.

The version of Delphi that I have been using to develop Behold for the past 7 years has been Delphi 2009, which I upgraded to from Delphi 4 in October 2008. That was a necessary upgrade because that added Unicode capabilities which is an essential for any genealogy program.

Now that I’ll be adding editing, I must again upgrade my development tool. In March, I purchased Delphi XE7 with a free upgrade to Delphi XE8 which came out in April. Yes, that’s a full 8 versions (about one per year) after Delphi 2009. They started using the XE moniker to represent the 2010’s.

image

The number one essential that XE8 provides that 2009 didn’t is a new database management environment called FireDAC with interfaces to the SQLite database, which I’ll be using for Behold’s file format. I selected SQLite because its a multi-platform, speedy, single file database with a small footprint that’s embeddable within the program. This also happens to be the same database system that RootsMagic uses.

A second thing XE2 and after provides that 2009 doesn’t is 64-bit support. So finally, I’ll be able to create a 64-bit version of Behold which should provide advantages on 64-bit machines. Once I get to this, I will be interested to compare the performance, memory use and file size limits of 32-bit versus 64-bit processing, although in reality.

The installation of Delphi XE8 was pretty straightforward and seemed to take about an hour. It upgrades to a new directory, so I am able to keep my 2009 Development active in case I have to go back to check for differences between what 2009 and XE8 does. 

I need to upgrade all my 3rd party packages to new versions that will work with XE8.  I want to get my new XE8 environment working, while also keeping my 2009 environment working at least for a while. That way, if something is not working the same as it used to, I could test it in 2009 and determine if it was because of the upgrade, or if there was some other reason. Once I’m satisfied that Behold is working reliably in the XE8 environment, then I will be able to dump the 2009 environment.

I don’t update my development tool very often. The steps are often problematic. This is what happened updating my 3rd party packages this time around:

  1. TRichView, which I use for the Everything Report, is now up to Version 15.7.1. I had previously been using version 10.1.4 from 2008. I downloaded the new version, which I get free upgrades for. I love the idea of free upgrades in a product, and TRichView’s policy prompted me to do the same with Behold. When I tried to install the new version, the 32 bit-version installed fine, but the 64-bit version did not because the directory for it was not set up yet in my installation of Delphi XE8. I found that I had to load a sample program and rebuild it in 64-bit for Delphi to create the needed directories. A nasty thing this installation did was to delete the TRichView binary from the 2009 version. I had to recreate that binary. Doing so was tricky since Delphi coughs badly when a package is missing and doesn’t let you in easily. I had some fiddling around to do to get it working. This is not something I do every day so I had to look for help or clues and figure out what the proper directories were. But I was able to get this all going in a couple of hours. I added a message on the TRIchView private forum to let the author know about these glitches in the installation so he can prevent them in future versions.
     
  2. LMD Elpack, is what I use for the TreeView, the grids in the Organize pages, and a few lesser components. I was using their 2009 Version. I needed the 2015 upgrade which includes support for XE8. So I ordered the upgrade. It’s nice that TRichView users get 20% off of any LMD purchase. Installing the ElPack upgrade was easier than TRichView was. The files were loaded into their own 2015 directory that was separate from the previous 2009 directory. And it knew that it was only supposed to be installed into Delphi XE8 and not Delphi 2009. After going through hoops to get the installation package from the LMD protected area, the installation was straightforward and worked reasonably well.
     
  3. EurekaLog, is what I use to trap errors and show an a error reporting box to the user, rather than just letting the program crash. It also helps me find and fix memory leaks in Behold. It had a special Cyber Sunday deal of 70% off for one day only last November which I took advantage of. That was for Version 7.x.  So now I downloaded and installed the latest 7.2.5 version which supports XE8. I was previously using version 6.0.21 from 2009. Installing EurekaLog had a few problems. First, It wanted to delete the previous version of the program before it would allow the new install. I found a EurekaLog support document on how to install two versions of EurekaLog. I had to go into RegEdit and change Eureka log to EurekaLogOld, then install the new version, and then go back into RegEdit and rename EurekaLogOld back to EurekaLog. Thats not a very nice way to have to do it. But then, it still didn’t work. When installing the new version, it gave two system errors. It couldn’t start InstallDiag.exe or Viewer.exe because VCRUNTIME140.dll was missing from my computer. After researching, I found that happens to be a library that comes with Visual Studio 2015, which of course I don’t have installed. EurekaLog’s installation should not depend on this. I downloaded the file (found from an answer in StackOverflow) and reran the installation of EurekaLog 7. Again I had to do the RegEdit thing for Version 6. Now the installation of 7 into Delphi XE8 worked. But Delphi 2009 still wasn’t working. I went back and found my original 6.0.29 installation and reinstalled it. Behold in Delphi 2009 recompiles and runs again. Yay!  By the way, EurekaLog’s install program is unsigned in both its version 6.x and 7.x – meaning Windows gives big bad warnings about running that “untrusted” program.

Now that was easy wasn’t it? Only about 2 days elapsed time. Behold now again compiles and runs fine in Delphi 2009. Delphi XE8 is also set up with the new versions of the needed packages installed into it. The next step is to get Behold to compile and run in the new environment.