Neutron time-of-flight imaging can provide a unique contrast mechanism of crystalline properties. Recently, computed tomography scans using time-of-flight instruments have been used to study structural and spectral characteristics of samples in 3D. To address the challenge of long measurement times associated with hyperspectral neutron computed tomography, the Oak Ridge National Laboratory neutron imaging team has recently demonstrated an autonomous system which can significantly reduce the measurement time by enabling high quality reconstructions from a sparse set of measurements. Some of the core components of such systems are the novel tomographic reconstruction algorithms including those based on artificial intelligence methods. In this work, a new training method is proposed to improve the performance of the artificially intelligent CT reconstruction algorithms. This training method can improve the quality of the reconstruction from very sparse time-of-flight scans. Our method helps hyperspectral tomography systems to obtain high-quality reconstructions with sparse scanning, which can potentially enable hyperspectral neutron computed tomography with reasonable acquisition times and particularly impact research projects which need to scan multiple similar sample beamlines such as the newly constructed VENUS beamline at the Spallation Neutron Source.