Low Earth Orbit (LEO) satellite network is a crucial component of the Space-Air-Ground Integrated Network (SAGIN) architecture and Non-Terrestrial Networks (NTN). It can provide global, real-time, and seamless broadband data services to many users, playing a critical role in emergencies such as disaster relief. This paper proposes an LEO satellite networks satellite-ground link handover strategy based on Advantage Actor-Critic (A2C) to tackle challenges associated with high handover frequency among ground users, data transmission blocking, and access selection post-handover due to the high-speed cyclic motion of satellites. The strategy considers three handover factors: signal reception strength, satellite potential service time, and service quality. At first, the handover problem is modeled as a combinatorial optimization problem, subsequently transformed into a Markovian Action-Value Function (MAVF) based Multi-Agent Markov Decision Process (MAMDP), and solved in the framework of the A2C algorithm proposed in this paper. The results of simulations illustrate the superior performance of the proposed handover strategy compared to three alternative strategies, namely, Random Handover (RH), Maximum number of Free Channels (MFC), and Maximum Service Time (MST), in terms of both convergence and practicality. The average number of handover occurrences can be reduced by up to 58%, consequently lowering the probability of blocking data transmission. Moreover, signal quality exhibits more excellent stability and can be enhanced by up to 39%, thereby ensuring user communication quality.