In the field of supervised feature selection, two factors are commonly considered when designing practical algorithms: class relevance and feature count. Class relevance represents the extent to which selected features determine class labels, directly impacting the prediction performance of post-feature-selection learning algorithms. Conversely, selecting a small number of features while maintaining class relevance is crucial for accurate data interpretation. This paper emphasizes a third index, nuisance, which captures class-uncorrelated information within selected features. While class relevance can be quantitatively evaluated by mutual information, nuisance can be assessed by, for example, the conditional entropy of selected features given class labels. Nuisance is considered to impact both the performance of post-feature-selection learning algorithms and the quality of data interpretation. We present a practical feature selection algorithm named Balance-Optimized Relevance and Nuisance Feature Selection (BornFS), which aims to achieve an optimal balance among these three indices. We demonstrate that BornFS exhibits excellent scalability and accuracy. In our experiments, BornFS drastically reduces the feature count from 10 million to under 200 while maintaining high accuracy in post-feature-selection classification.