<p>Chip-scale quantum magnetometers featuring both ultra-high sensitivity and uniform spin polarization are highly desired for practical applications and have been diligently pursued. However, the fulfillment of such capabilities for quantum magnetometers typically necessitates a separate heating unit, bulky reflector, and beyond, severely impeding on-chip integration and batch fabrication of these quantum devices. Herein, we present a novel paradigm for the wafer-level fabrication of ultra-sensitive chip-scale quantum magnetometer, which is enabled by integrating a highly reflective mirror and a temperature-controlled component on the optically transparent windows of the MEMS atomic vapor cell, thereby providing a genuinely all-in-one atomic vapor cell with a temperature stability better than ±5 mK at up to 200°C as well as a reflectivity of 95% at Rb D1 transition wavelength. With the as-developed on-chip atomic vapor cell with internal dimensions of Φ 3×1.5 mm³, we configured a chip-scale single-beam atomic magnetometer with a sensitivity floor of about 15 fT/Hz^1/2, along with a theoretically more homogeneous spin polarization distribution. We envision that the proposed chip-scale integration solution paves a concrete route for batch manufacturing and widespread application of quantum magnetometers.</p>