Automatic depression detection from semi-structured interviews has gained increasing relevance as a scalable and non-invasive complement to conventional clinical assessment, particularly in digital and remote-care contexts. However, modeling long-form interviews remains challenging due to their weakly supervised nature: a single subject-level label must be inferred from multiple heterogeneous conversational segments. This setting naturally fits the Multiple Instance Learning (MIL) paradigm, where each interview is represented as a bag of instances associated with a global label. Nevertheless, existing MIL-based approaches typically rely on implicit attention or aggregation mechanisms that compress local evidence into a final decision without explicitly grounding segment importance in structured clinical knowledge. Here, we propose a text-conditioned attention-based MIL framework, termed TCAMIL, for audio-based depression detection that incorporates symptom-level semantic alignment prior to aggregation. A semantic relevance filtering stage is implemented through a cross-encoder architecture to match interview text segments with Patient Health Questionnaire-8 (PHQ-8) symptom items, preserving only clinically aligned instances. This filtering enables the construction of noise-reduced MIL bags composed of symptom-relevant audio fragments, reducing the impact of semantically diverse and weakly informative content. The selected instances are then processed using a domain-adapted encoder and aggregated via trainable attention pooling to produce subject-level predictions under weak supervision. Experiments conducted on the DAIC-WOZ dataset demonstrate improved performance. Additionally, qualitative analyzes indicate enhanced interpretability, as the model assigns higher relevance to segments aligned with structured symptom descriptors.