Magnetic Resonance Imaging (MRI)-derived brain networks are significant for detecting neurological disorders. Nevertheless, traditional node-centric methods usually focus excessively on capturing features on individual nodes (i.e., brain regions) but ignore the critical information of dynamic evolution between the edges (i.e., connections between brain regions), resulting in suboptimal identification performance. Regarding this, two key challenges are identified: 1) oversimplified edge dynamics cannot adequately capture neural information transmission, and 2) neglecting higher-order edge causal effects and structural asymmetry leads to insufficient analysis of the collaborative patterns across multiple brain regions. To overcome these challenges, this paper introduces a novel edge-centric fusion network (EdgeViewDet) for neurological disorder detection. First, based on conditional Granger causality analysis, an edge-centric effective connectivity network (EC-ECN) is designed to investigate high-order time-varying causal effects within sliding windows. Then, an edge-centric structural connectivity network (EC-SCN) based on graph diffusion is employed to capture structural lateralization abnormalities. Additionally, to efficiently fuse multiple structural-causal features, a directed spatiotemporal feature extraction module (DST-FE) is designed, which helps improve the feature discrimination by considering the underlying relations among structural-causal features in different edges. The superior disease detection performance of EdgeViewDet is validated through extensive experiments on two public datasets and one private dataset. The code of our work will be released upon acceptance.