This paper investigates the challenges and solutions associated with modality bias in LiDAR-Camera-based 3D Object Detection (LC-3DOD) systems. Modality bias, where models disproportionately rely on the dominant modality, poses significant risks, particularly in safety-critical applications like autonomous driving. Our research aims to enhance the robustness of multimodal systems by addressing this bias and the associated robustness to sensor failures. We begin by defining and quantifying modality bias within LC-3DOD systems, demonstrating its impact on system robustness under corrupted conditions. We identify a bias towards the LiDAR signal, which has a stronger correlation to the predictions compared to the camera inputs. We show how a well known modality dropout technique is useful in mitigating this bias, however, we encounter the phenomena of accuracy-robustness trade off for robustness to sensor failures as a limiting practical constraint on the possible robustness enhancement. We propose an end-to-end adaptive inference architecture that leverages multiple model heads, each optimized for a specific regime of LiDAR input quality called Corruption-Aware-Fusion, which incorporates a LiDAR Corruption Estimation Module to dynamically assess the LiDAR signal quality and select the optimal head variant based on those conditions. CAF robustly balances clean accuracy with enhanced performance under severe corruptions. Our CAF framework achieves state of the art accuracy on nuScenes-C LiDAR failures—with performance gains increasing as corruption severity rises—thereby safeguarding overall accuracy by mitigating the collapse that typically occurs under severe LiDAR degradations. Central to CAF is a LiDAR Corruption Estimation Module, a critical component that dynamically assesses corruption type and severity to drive adaptive model selection.