<p>Speech emotion recognition (SER) remains challenging because emotion-relevant cues are often unevenly distributed over time and are closely entangled with speaker-related, prosodic, and linguistic variation. Although self-supervised speech encoders provide strong acoustic representations, direct utterance-level aggregation and single-task optimization may still limit effective emotion modeling. To address this issue, MLTESER, a multi-task SER framework built on a pre-trained wav2vec 2.0 backbone and equipped with lightweight temporal enhancement and complementary auxiliary supervision, is developed. Wav2vec 2.0 is adopted as the shared acoustic backbone because it provides strong contextualized speech representations while preserving a suitable balance between representational strength, computational cost, and experimental comparability. Within this framework, lightweight temporal enhancement is used to construct an enhanced auxiliary path for prosodic and semantic supervision, while speaker supervision and CTC supervision are imposed directly on the shared acoustic path. In addition, prosodic and semantic cues are fused before final emotion prediction. In this way, lightweight temporal refinement and coordinated multi-view supervision are incorporated within a unified representation learning framework. Experiments on IEMOCAP show that MLTESER achieves 82.20% UA and 81.53% WA under the 10-fold speaker-dependent setting, while remaining competitive under the stricter 10-fold speaker-independent protocol with 77.03% UA and 76.78% WA. In addition, under the standard 5-fold leave-one-session-out protocol, MLTESER achieves 74.50% UA and 74.12% WA, providing a standard session-level independent reference. Additional results on ShEMO and under additive noise further indicate that the proposed framework provides a favorable balance among recognition accuracy, complexity, generalization, and robustness.</p>