In rapidly advancing technology contexts like artificial intelligence and autonomous driving, heterogeneous vehicles including connected and automated vehicles (CAVs), connected vehicles (CVs), and traditionally human-driven vehicles (TVs), coexist within the urban traffic. Achieving collaborative path planning among heterogeneous vehicles is essential to enhance the overall traffic efficiency in this mixed traffic scenario. However, most of the existing methods are to enhance traffic efficiency by promoting collaboration among homogeneous agents, and can not solve the collaborative path planning problem of heterogeneous vehicles in mixed traffic. To tackle this, we formulate collaborative path planning for heterogeneous vehicles as a heterogeneous multi-agent Markov game, modeling CAVs and CVs as heterogeneous agents that make different decisions in mixed traffic. Then, we propose a heterogeneous multi-agent reinforcement learning framework (HMA) to solve the Markov game. In this framework, CAVs make path-planning decisions to effectively guide traffic. CVs selectively follow CAVs with the same destination to avoid new traffic congestion due to mass following. Moreover, the advantage function decomposition enables heterogeneous vehicles to independently learn policies, eliminating the influence of different decision types of heterogeneous vehicles. Besides, to promote collaborative path planning of agents, we adopt the sequential update strategy to enable each agent to consider the latest policies of other agents when updating, eliminating possible traffic congestion. This algorithm generates a more coherent joint policy that promotes collaboration between heterogeneous agents. We establish a simulator based on dynamic urban environments and conduct extensive experiments. The results demonstrate that our approach significantly outperforms all baselines.