Cognitive flexibility is critical for individuals to adapt to rapidly changing environments. This study investigated the effects of modality, preparation time, and their interaction on cognitive flexibility through the task-switching paradigm. This study also explored the relationships among cognitive flexibility, field dependence, and context sensitivity. This study used a 2 (modality: visual modality, auditory modality) × 2 (block type: mixed task, single task) × 2 (cue-stimulus interval: short, long) within-subject design, and recorded the response times and error rates to investigate cognitive flexibility. The results indicated that: (1) participants had longer response times and higher error rates in mixed tasks, suggesting a general switching cost; (2) modality affected participants’ reaction time in single tasks and switching costs; (3) longer preparation time reduced switching costs; (4) participants’ cognitive flexibility did not have direct relationships with their field dependence and context sensitivity. The findings enriched the study of cognitive flexibility.