Distributed stream processing systems are widely used for handling real-time data across many industries. These systems face several challenges while managing continuous queries, especially when working with large, dynamic, and multi-source datasets. Traditional query optimizers often struggle with fluctuating workloads, changing data arrival rates, and resource constraints. To address these issues, this paper introduces a hybrid and intelligent query optimization framework using Reinforcement Learning (RL)-agents with Graph Neural Networks (RL-GNNQO). The proposed method learns optimal query plans by interacting with the streaming environment in real-time. It uses graph-based query representations to understand the structure and flow of queries, and RL agents to select efficient execution paths. A Temporal Attention (TA) unit is also added to better capture time-based changes in query workload patterns. This combination helps reduce latency, improves throughput, and adapts well to changes in the stream. The RL-GNNQO framework is tested across several simulated and real-world streaming workloads. Results show that the proposed model achieves better response times and resource usage compared to existing optimization methods. The hybrid design also supports continuous learning, which helps improve performance over time.