We have constructed and characterized an affordable medium-finesse optical cavity \(({\mathcal {F}}\sim {1500})\) for the stabilization of tunable diode lasers at two different wavelengths (780, and \({960\,\textrm{nm}}\) , respectively). Its main element is an ultra-low expansion glass spacer, whose temperature was stabilized using thermoelectric cooler elements inside a vacuum chamber. By combining the dual sideband technique and the classical Pound–Drever–Hall technique, we were able to lock the lasers at any frequency within the cavity free spectral range. The cavity presents a long-term drift of resonant frequency of \({1.2\,\mathrm{MHz/day}}\) , which can be compensated for by temperature variation. Finally, we demonstrate the cavity use in an electromagnetically induced transparency microwave spectrum experiment in a thermal atomic sample.