Verification and validation of vehicles is a complex yet critical process, particularly for ensuring safety and coverage through simulations. However, achieving realistic and useful simulations comes with significant challenges. To explore these challenges, we conducted a workshop with experts in the field, allowing them to brainstorm key obstacles. Following this, we distributed a survey to consolidate findings and gain further insights into potential solutions. The experts identified 17 key challenges, along with proposed solutions, an assessment of whether they represent next steps for research, and the roadblocks to their implementation. While a lack of resources was not initially highlighted as a major challenge, utilizing more resources emerged as a critical necessity when experts discussed solutions. Interestingly, we expected some of these challenges to have already been addressed or to have systematic solutions readily available, given the collective expertise in the field. Many of the identified problems already have known solutions, allowing us to shift focus towards unresolved challenges and share the next steps with the broader community.