Thoughts on Extended Depth of Field With Don Williams' Microscopy Sequence

Rik Littlefield (email rj.littlefield at computer.org)
Page last modified May 8, 2005

Introduction

At http://groups.yahoo.com/group/Microscope/message/24573, Don Williams posted links to a sequence of microscope images that had proved troublesome to extended-depth-of-field software.  Those links were http://www.science-info.org/pages/edfwill/499/ (for the images from CombineZ5 and Helicon Focus) and http://www.science-info.org/pages/edfwill/499/IMGS/ (for the series of 48 source images).  Several followups were posted, notably at http://groups.yahoo.com/group/Microscope/message/24579 where Rik Littlefield suggested that uncompressed images might work better, and at  http://groups.yahoo.com/group/Microscope/message/24585 where Alan Hadley suggested an alternative processing sequence with CombineZ5.

The purpose of this page is to further analyze the source sequence and suggest additional improvements to the processing sequence of CombineZ5 and the parameter settings of Helicon Focus 3.10.

Note: this page contains large images, totalling around 5 MB.  Depending on the speed of your Internet connection, you may wish to go do something else while all of the images download.  Once they are cached by your browser, re-visiting the page will be significantly faster.

The Source Sequence


Probably the best way to describe the source sequence is to illustrate it.  Here is an animated GIF showing all 48 frames of the source sequence, reduced to 25% resolution, and played continuously at 5 frames per second.



Notice that the imagery consists of numerous stationary dark objects that come into and leave focus, combined with an assortment of mobile objects that are apparently freely swimming around in the field of view.

The stationary appearance of many objects, combined with the obvious movement of others, suggests that automatic image alignment (registration) may not be a good idea, as it may easily attempt to align the moving objects, to the detriment of the stationary ones.

We also see, despite the reduction to 25%, that nothing is rendered really sharply in any of these images.  This suggests that the extended depth-of-field software might benefit from using larger radii, or more smoothing, or both, depending on the software's processing model.

Results With Various Parameters and Processing Steps

(Any of the non-animated images below can be clicked, to see the image at higher resolution.)

Helicon Focus

The image below is the result of processing the source image sequence with Helicon Focus 3.10 using default parameters R=8, S=4.  This image shows an obvious mesh-like pattern around certain objects.


The image below here is also Helicon Focus, with default parameters except that automatic alignment has been turned off.  Notice that the meshlike pattern has essentially disappeared.  That pattern was apparently due to enhancement of JPEG compression artifacts that became misaligned as a result of jitter in the registration process, perhaps caused by the moving objects.  Notice in both of these images, that the random collection of dark spots at lower left is due to selection of only a couple of different organisms, in different places at different times, as they moved around the frame.  (See the animation.)
 

The Helicon Focus image can be improved further by judicious adjustment of its processing parameters.  Here is what happens when one increases the radius of the focus and smoothing windows by selecting R=10, S=6 instead of the default R=8, S=4.  Notice that pixellation and noise are further reduced, with no significant loss of image detail.
 

CombineZ5

Here are results of CombineZ5's default "Do Stack" macro.  This image exhibits significant "swirliness" in some areas, particularly around some of the major stationary objects.  This effect is often reduced by increased smoothing of the depth map.  Detailed examination of the time tag, in the full-resolution image, shows that image registration has also perhaps done a few pixels of inappropriate shifting, but in this case (for unknown reasons) this did not cause enhancement of the jpeg compression artifacts.


The image below is based on Alan Hadley's comments in http://groups.yahoo.com/group/Microscope/message/24585, where he suggested to reduce to monochrome, average groups of 2 frames, and then standard Do Stack.


I have played around further with CombineZ5, but I have not been able to significantly improve its output beyond Alan's suggestion.  For the record, here is what results from the following sequence:
Balance Colour and Brightness
Find Detail(50)
Remove Islands(5)
Fill Gaps
Create a Lowpass Filter(100,0)
Filter Depth Map
Copy Last Filter Output to Depthmap
Interpolated Output
Create a Highpass Filter(1000,750)
Filter Active Frame
This seems roughly comparable to Alan's suggestion, particularly given the difference between any of these composite images and the reality suggested by the animation of the source imagery.


Direct Comparison

Following is a two-image animated comparison of the best results (so far) from Helicon Focus (R=10, S=6, no autoadjustment) and CombineZ5 (Alan's sequence, as I understand it).


Summary and Interpretation

With tuning, both Helicon Focus and CombineZ5 are able to produce a composite image that captures more content than any single source frame.  Neither of them works very well with default parameters.  For this particular stack, I would be inclined to give the nod to Helicon Focus, after tuning (and based only on visual appearance, not in-depth understanding of the source material).  However, neither Helicon Focus nor CombineZ5 (in my opinion) produces a composite image that conveys as much information as the low resolution animation.  Both of them are prone to artifacts.  Clearly much further development is required.  It would be a significant improvement for the stacking programs to understand at least a little bit about diffraction, instead of blindly assuming that sharp gradients imply focused detail.

Claims and Disclaims

The work presented here is unreviewed, unpublished documentation of capabilities and techniques developed by the author.  It is intended for online discussion and is subject to future edits in place.  Please do not reference in publications; contact the author instead.  Text copyright Rik Littlefield, 2005.