<p>Establishing a reliable and valid measure is crucial for ensuring the accuracy and replicability of research findings on risk and uncertainty preference. However, few studies have assessed the reliability and validity of behavioral measures of uncertainty preference. This study examined the convergent validity and test–retest reliability of three commonly used uncertainty preference measures: forced binary choice, certainty equivalent, and matching probability tasks. Experiments <InternalRef RefID="Sec6">1</InternalRef> (<i>N</i> = 302) and <InternalRef RefID="Sec16">2</InternalRef> (<i>N</i> = 366) tested the convergent validity and test–retest reliability of one-off assessment of these measures and found that the three measures did not demonstrate satisfactory convergent validity and test–retest reliability for the one-off assessment. Experiment <InternalRef RefID="Sec23">3</InternalRef> (<i>N</i> = 311) increased the number of repeats to explore whether repeated measurements could enhance convergent validity and test–retest reliability by addressing the attenuation effect of lack of reliability. The convergent validity between certainty equivalent and matching probability improved in the repeated measurement condition. However, the test–retest reliability of the three measures was still not satisfactory in repeated measurement conditions. These findings highlight the measurement issues in the behavioral measures of uncertainty preferences. The potential causes of this low validity and reliability of behavioral measures of uncertainty preference are discussed.</p>