Lexicase selectionLexicase selection is a successful parent selectionParent selection method in genetic programming that has outperformed other methods across multiple benchmark suitesBenchmark suite. Unlike other selection methods that require explicit parameters to function, such as tournament size in tournament selection, lexicase selectionLexicase selection does not. However, if evolutionary parameters like population size and number of generations affect the effectiveness of a selection method, then lexicase’s performance may also be impacted by these ‘hidden’ parameters. Here, we study how these hidden parameters affect lexicase’s ability to exploit gradients and maintain specialists using diagnostic metrics from an existing benchmark suiteBenchmark suite that measures a selection scheme’sSelection scheme ability to exploit and explore handcrafted search spaces. By varying the population size with a fixed evaluation budget, we show that smaller populations tend to have greater exploitationExploitation capabilities, whereas larger populations tend to maintain more specialists. We also consider the effect redundant test cases have on specialist maintenance, and find that high redundancy may hinder the ability to optimize and maintain specialists, even for larger populations. Ultimately, we highlight that the interaction between population size, evaluation budget, and test cases must be carefully considered for the characteristics of the problemExploration beingLexicase selection solvedAlgorithm configuration.