With the development of connected and automated vehicles (CAVs), traffic flow gradually transitions to a mixed state with increasing penetration rates. The introduction of CAVs is expected to positively impact traffic performance in mixed traffic flow. However, research indicates that the decision-making and behavioral preferences of CAVs are influenced by Social Value Orientation (SVO), which can lead to selfish driving behaviors that can obstruct traffic at lower SVO levels. Therefore, this paper models the impact of SVO on CAV decision-making as a partially observable Markov decision process (POMDP) and proposes a multi-agent reinforcement learning (MARL) algorithm based on dynamic SVO. This algorithm optimizes safety, efficiency, and comfort by controlling the driving behavior of CAVs to achieve optimal performance in mixed traffic. In a ramp merging driving scenario, we simulate mixed traffic states with different penetration rates and traffic flows using the Highway-env platform. Simulation results demonstrate that when the SVO is within the range of 55 \(^{\circ }\) -72 \(^{\circ }\) , various indicators of mixed traffic flow achieve optimal performance. We identify this as the optimal SVO range. Additionally, compared with the latest baseline algorithms, it is found that prosocial SVO settings significantly improve the algorithm’s performance, proving the effectiveness of the proposed algorithm.