Known Image and Image Processing Issues: Overview

We have documented a few errors in the image processing that are due to a moving piece of dust, illumination variation in Phase Contrast layer, debris in a culture, a fluctuating colony area around minimum area threshold, interactions of colonies with fiduciary marks, presence of a highly fluorescent object and an undetected scratch on a plate before seeding colonies.

A Piece of Dust

Location: Replicate 1, Time Frame 69, Colony ID 14, Phase contrast layer.
Observation: There are disconnected bright regions as shown in Figure 1.
Explanation: moving piece of dust
There is a piece of dust that was overlapping the colony when it was imaged, and then had moved slightly when the image row above it was taken. This is an example of a moving object within the time it takes to acquire a single row of images. A user can remove the piece of dust by filtering in the database view.
Moving Dust screenshot
Figure 1. A screen capture of a moving piece of dust

Illumination variation in Phase Contrast Layer

Location: Replicate 1, Time Frame 159, Colony ID 972 through 1054.
Observation: Phase contrast layer does not have any signal and GFP layer does not have any signal.
Explanation: Adaptive segmentation threshold
This appears to be an artifact of segmentation because of the scratches that appear in the phase channel. When the phase contrast and GFP layers are overlaid semi-transparently then there is foreground where no cell exists - see Figure 2. There is more foreground during time point 159 than either 158 or 160. The automated foreground segmentation (EGT) selected a slightly lower threshold based on the histogram of the phase channel and therefore more area is considered as foreground. These objects exist for a single time frame. For consistency reasons, we did not tweak the gradient threshold of automated segmentation.
segmentation of frame 159 screenshot
Figure 2. A screen capture of low-contrast segmented regions at the time point 159
segmentation of frame 158
Time frame 158
segmentation of frame 159
Time frame 159
segmentation of frame 160
Time frame 160
Figure 3. A comparison of segmentation results in three consecutive frames 158, 159 and 160.

Debris identified as a colony

Location: Replicate 1, Time Frame 1, Colony ID 120 (see Figure 4).
Observation: The colony ID 120 merges with a colony ID 118 at time frame 4 to become a colony ID 316. It then splits at time frame 6. In the meantime, an other part of the debris is detected from time to time (t=5,6 as colony ID 322, then at time frames 36 to 60 as colony ID 480). Afterwards, the colony merges with colony ID 316 mentioned above and keeps the colony ID 480.
Explanation: Colony interactions with a scratch over time
At the time frame 1, the colony ID 120 is segmented in the phase contrast channel using the gradient based technique to find the colony edges. The edges of this scratch region are sharp enough to be detected as foreground. There will always be some false positives in the segmentation especially if the scratch mimics colony foreground like this. So then this becomes a question of having functionality to remove a colony, or labelling it as invalid and propagating this through the other metadata based on a user declaring this object as not being colony. A user can remove these "segments" by filtering in the database view.
Debris screenshot
Figure 4. A screen capture of a debris that is detected as a colony

Lineage discontinuity

Location: Replicate 1, Time Frame 6, Colony ID 230 (see Figure 5).
Observation: This colony received at least 7 labels, making it very hard to track its behaviour over time. It appears as 7 different colonies with 7 disconnected trees. Neither mergers nor splits occurred.
Colony ID 230 from time frame t=1 to t=6
Colony ID 368 from time frame t=14 to t=15
Colony ID 388 at time frame t=19
Colony ID 396 from time frame t=21 to 25
Colony ID 426 from time frame t=27 to 57
Colony ID 546 from time frame t=59 to 64
Colony ID 571 from time frame t=66 to 122
Explanation: fluctuating colony size over time around the minimum segment size threshold
This colony is coming in and out of detection because it straddles the minimum colony area threshold. At some frames it is large enough to be detected, in other frames it is not. The cell tracker only looks at the single previous frame, there is no extended search through time to try and connect lineages with gaps in them. Adding that functionality would fall under future work. There are several possible stop gaps to fix this situation. The easiest one might be to ignore and remove any colonies without a lineage length longer than a defined threshold (say 10 frames) discounting any merging events. Doing so would mask this problem in the data by removing the colony. Another methods of working around this include manually adding in segmented masks for the time points where the colony was too small and then tracking once there are no gaps in the lineage.
Lineage discontinuity screenshot
Figure 5. A screen capture of a colony with fluctuating size over time around the minimum segment size threshold

Colonies touching fiduciary marks (two X marks on a plate)

Location: Replicate 1, all frames, Phase contrast layer.
Observation: There are colonies touching the X marks on a plate as shown in Figure 6. The segmentation mask excludes parts of these colonies.
Explanation: interaction of colonies with the X marks

The two X marks on a plate were manually created before seeding the culture in replicate 1 in order to investigate a possible use of fiduciary marks for frame-to-frame registration purposes. The need for registration comes from the fact that the cells have to be fed every day over the course of five days. During the feeding time, the plate is removed, the media is exchanged and the plate is put back under a microscope. This process introduces a frame-to-frame registration problem between feeding times.

While fiduciary marks can assist with registration, colonies might behave differently depending on the surface morphology. In order to exclude the colonies whose behaviour might be governed by a different biological phenomena than the growth in a culture, we created a mask manually that covers the fiduciary marks and expanded the mask by morphological image processing operations. However, the manual mask might cut through growing colonies in a proximity of the fiduciary marks.

Fiduciary mark screenshot
Figure 6. A screen capture of colonies touching one of the two fiduciary marks

Colonies moving in and out of field of view

Location: Replicate 2, Time frames 76-78, Phase contrast layer.
Observation: There are some colonies that have oscillating colony area values (see Figure 7).
Explanation: colonies moving in and out of field of view

Although the frames are stitched from many fields of view, they cover about 17% of the culture dish. In addition, as observed in replicate 2, the higher seeding density of colonies in the upper right corner of the overall field of view causes more colonies to be at the border of what is being measured. Thus, there will be colonies moving in and out of the overall field of view which should be considered in the studies focusing on colony growth rate.

Area size oscillation screenshot
Figure 7. A screen capture of a colony area that oscillates due to the limited overall field of view

Highly fluorescent object

Location: Replicate 2, Time frame 82, GFP layer.
Observation: There is a highly fluorescent object in the field of view that is not a colony (see Figure 8).
Explanation: This artifact must have been introduced in the bottom side of the plate during observation or feeding - there is a hot air blower that helps regulate air temperature in the microscope incubation chamber by drawing in unfiltered ambient air. It is possible that a piece of dust was drawn into the chamber during the imaging and stuck to the bottom of the plate. This artifact is undesirable and has to be corrected digitally. We created a rectangular shape mask manually around the artifact and eliminated from segmentation results all pixels within the region. The rectangular mask can be seen when looking at the segmentation layer (see Figure 9).

This artifact is undesirable and has to be corrected digitally. We created a rectangular shape mask manually around the artifact and eliminated from segmentation results all pixels within the region. The rectangular mask can be seen when looking at the segmentation layer (see Figure 9).

highly fluorescent object screenshot
Figure 8. A screen capture of a highly fluorescent object in GFP layer
highly fluorescent object mask screenshot
Figure 9. A screen capture of a segmentation layer that includes the rectangular region eliminating the highly fluorescent object in GFP layer

Scratch on a plate

Location: Replicate 3, All time frames, Phase contrast layer.
Observation: There was a scratch on a plate in replicate 3. The scratch caused high intensity values in Phase contrast images (see Figure 10). These high contrast intensities are detected as foreground during segmentation (see Figure 11) and connect multiple standalone colonies into one foreground object.
Explanation: This artifact was part of the cell culture plate and was not detected before seeding the culture on the plate.

In order to eliminate the artifact and its undesirable impact on connecting standalone colonies, we edited the segmentation results manually and removed the scratch pixels from the final mask.

scratch across colonies screenshot
Figure 10. A screen capture of a scratch cutting across multiple colonies in Phase contrast layer
segmentation of a scratch across colonies screenshot
Figure 11. A screen capture of a scratch cutting across multiple colonies detected by a segmentation method and corrected manually