Training natural language understanding (NLU) models for dialog systems requires collecting user utterances as training data. Unlike general classification or sequence labeling tasks, NLU data instances are inherently tied to their dialog context. This paper explored the use of a novel active utterance collection approach based on multi-armed bandits to improve the efficiency of data collection for training NLU models. By selecting the most informative dialog contexts (scenes) to collect data from using a multi-armed bandit algorithm, the proposed framework aims to maximize the improvement in NLU model performance with limited data annotation. An experiment on English dialog datasets demonstrated that the active utterance collection approach outperformed random data selection when a substantial labeled validation dataset was available. However, its effectiveness diminished in scenarios with limited or incrementally growing validation data. The results highlight the importance of validation data availability in NLU model training and suggest potential for hybrid approaches in practical applications.