The Dragonfly interconnect is widely adopted by extreme-scale systems, yet its sharing nature often results in traffic from various applications competing for network resources, causing workload interference and leading to variable application runtime. We aim to leverage deep neural network methods to forecast application iteration times, using network features collected at the router port level. We address the problem by employing a graph neural network-based dynamic model that is trained on an ad-hoc graph structure that reflects the physical characteristics of the system, and can capture its temporal and structural dynamics. Results show that our methodology is able to outperform the baselines for one and two future steps ahead.