Platooning control of connected autonomous vehicles (CAVs) is emerging as a promising strategy to enhance fuel efficiency, traffic flow, and overall safety at road intersections. However, coordinating multiple self-driving vehicles while ensuring traffic safety and efficiency is challenging due to inherent nonlinearities in vehicle dynamics. Towards addressing this challenge, an optimal distributed leader-tracking control for heterogeneous vehicle platoons with nonlinear dynamics at an unsignalized intersection is proposed. The distributed control system is designed based on reinforcement learning (RL) theory that learns the optimal control actions from the vehicle system data without requiring any prior knowledge of vehicle dynamics. The performance of the developed control strategy is validated through a novel virtual testing platform, called Mixed Traffic Simulator (MiTraS), combining MATLAB/Simulink and SUMO microscopic traffic simulator. The design and structure of MiTraS is explained and various cooperative driving maneuvers are simulated using this platform to demonstrate its potential and effectiveness in validity the theoretical findings of the work.