Neuron Segmentation from Noisy Fluorescence Microscopy Images Using Deep Learning with Local and Global Scale Fusion

Neuron segmentation from 3D fluorescence microscopy (FM) images is often required to study the structure and activity of neurons in the brain. While Convolutional Neural Networks (CNNs) have been used for many 3D biomedical imaging analysis tasks, they struggle to analyze 3D FM images of neurons for three reasons. First, partial volume effects and extreme class imbalance make it challenging for CNNs to segment thin neural dendrites in large 3D volumes. Second, FM images contain significant amounts of voxel noise that further obscures the signal needing to be segmented. Lastly, limited training data make it challenging for CNNs to learn optimal features. In this chapter, we overcome these limitations by introducing a novel 3-stage CNN that fuses local and global scales which is pre-trained on GAN-generated synthetic data. Noise-robust local representations are first learned using a shallow 3D CNN. Then, a 3D U-Net is employed to learn global representations that amplify the filtered signal. A final 3D CNN integrates both scales to achieve state-of-the-art neural segmentation from low Signal-to-Noise Ratio FM images. We demonstrate our deep learning architecture on a 3D FM dataset of retinal neurons so that readers can apply deep learning to their own FM analysis problems.