Collective perception (CP) enables connected and automated vehicles (CAVs) to exchange driving environment perception data in order to improve their on-board perception, essentially creating a redundant ‘sensor’ for the CAV and extending its (on-board) Field-of-View (FoV). The use of CP for mixed traffic environments that include CAVs requires a thorough evaluation and validation. Due to the CP large-scale field testing infeasibility and based on the ETSI work, most of the previous work on CP has considered large scale-simulations with a focus on connectivity/network aspects. More recently, large-scale collaborative perception synthetic datasets and open source benchmarks have appeared allowing the perception engineers familiar with CARLA to study CP from a perception point of view missing so far. This work reviews recent achievements in this direction to bridge this gap and motivate future research. As a result of this critical state-of-the-art review, we also produce a set of high-level safety validation requirements for CP testing in simulation, by focusing on urban environments, where non-light-of-sight scenarios hinder the traditional on-board perception task.