Multi-view 3D object detection in driving scenarios heavily relies on large amounts of precisely annotated bounding boxes. Due to the substantial difficulty of annotation under 3D settings, inaccurate annotations will be inevitable as the amount of data increases, which challenges the practical implementation of 3D object detection for new scenes. To address this issue, we define a more practical hybrid supervised multi-view 3D object detection task, which requires algorithms to learn from a few clean annotations and massive inaccurate ones. In this paper, we propose a Dual Discrepancy-based Continuation Learning (DDCL) approach for the proposed task, which introduces a continuation optimization process consisting of noise learning, noise correction, and noise elimination stages. In the noise learning stage, training samples are used to establish three subsets with different noise proportions, which are then used to initialize three detectors with different noise-sensitivity, respectively. In the noise correction stage, the dual discrepancy among the detectors’ predictions are designed and utilized to dynamically correct the noisy annotations. In the noise elimination stage, a transition process that gradually removes all noisy samples is employed to further reduce the negative impact of uncorrected noisy samples. Extensive experiments on the nuScenes dataset with hybrid annotations show the effectiveness of our method DDCL, outperforming the comparison method performance by 4.63% mAP and 3.05% NDS.