Recent advances in cross-view multi-object tracking have demonstrated promising results. These methods jointly model single-view and cross-view object associations as an undirected graph optimization problem. However, existing methods for calculating the feature distance of the same object from the same and different perspectives may lead to significant gaps in correlation scores, resulting in unfairness in modeling undirected graph edges. To address this issue, we introduce a novel dual-head feature extractor for edge alignment. The single-view motion branch, inspired by single-object tracking (SOT), focuses on single-view motion by establishing the temporal information heatmap. For the cross-view appearance branch, we introduce Cross-view Consistency Loss (CC Loss) to improve cross-view association. These two branches, integrated as detection head branches, are trained with distinct strategies to strengthen their respective feature representations without additional computational overhead. Extensive experiments demonstrate that our approach achieves state-of-the-art performance. Such a structure can adapt to different types of cross-view datasets, which means that the tracker can be deployed in more open scenarios.