<p>In visual search, observers often miss rare targets. This low prevalence effect (LPE) is resistant to many cognitive interventions. However, a recent study showed that having participants identify the item that was most similar to the target (similarity search) eliminated the LPE. As real-world searches often require binary decisions (e.g., is there a threat in this bag?) we tested whether the benefits of similarity search generalize to binary decisions and to more naturalistic stimuli. Participants searched for <i>T</i> shapes amongst near-<i>T</i> distractors and the prevalence of true <i>T</i>s was manipulated. In the similarity-search-only condition, participants clicked on the “most <i>T</i>-like object.” In the similarity search &amp; binary decision condition, participants additionally reported whether the chosen item was a true <i>T</i> (yes/no). We found that in some circumstances, similarity search can be used to attenuate the LPE. However, there was an LPE for the binary decision task. Participants were less likely to classify the target as a true <i>T</i> during low prevalence compared with high. We replicated this result in an additional experiment using more naturalistic stimuli. Participants watched clips of road videos and clicked on the “most hazardous location” in the video, followed by a binary decision (“would you need to respond to that hazard? yes/no”). Though participants located the hazards regardless of prevalence, there was an LPE for the binary decision task. Together, these results indicate potential limitations in applying similarity search outside the laboratory; the LPE is still seen in these searches if a binary decision is involved.</p>