Tracklet Association is an effective post-processing method for maintaining trajectory identity consistency in Multi-Object Tracking (MOT). Existing methods usually adopt two separate models to first split trajectories and then connect tracklets. However, they overlook the intrinsic relationship between the two tasks. In this work, we propose a graph neural network (GNN)-based framework to unify splitting and connection tasks in a single model that only needs to be trained once. Specifically, we summarize the commonalities of the two tasks by comparing the consistency of neighboring features and transform them into the edge prediction problems in GNN, i.e., where to split and whether to connect. We design two modules for global and local graph perception during message passing to effectively capture identity-consistent information of trajectories. Experiments conducted on the MOT17 and DanceTrack benchmarks demonstrate that our approach significantly improves IDF1 and HOTA metrics, showcasing its effectiveness in tracking consistently.