This study presents a comprehensive investigation into the phase behavior of shale oil-CO2-water systems within nanoporous reservoirs with a focus on the thermodynamic and phase equilibrium characteristics influenced by nanopore confinement. Shale oil reservoirs typically exhibit complex nanopore structures where pore radii range between 10 to 30 nm. The strong interactions between the solid pore walls and the confined fluids lead to significant deviations in fluid phase behavior compared to conventional reservoirs. The presence of water, often introduced by hydraulic fracturing fluids, further complicates the fluid distribution and mass transfer processes in these nanopores. A multiphase phase equilibrium model was developed, incorporating fluid adsorption, capillary pressure effects, and interfacial tension adjustments. Two fluid injection scenarios were examined: one where CO2 is injected into the nanopores before water and the other where water precedes CO2 injection. The study revealed that the nanopore confinement effect reduces the gas phase fraction while increasing the oil and water phase fractions as the pore size decreases. This confinement leads to enhanced transfer of CO2 from the gas phase into the oil and water phases, forming layered fluid structures due to the restricted molecular movement within the nanopores. Additionally, the study found that when water enters the pores before CO2, a “water shielding” effect occurs, hindering CO2 transfer into the oil phase. This phenomenon significantly reduces the efficiency of CO2 extraction of light hydrocarbons, limiting its effectiveness in enhanced oil recovery. In contrast, when CO2 enters the nanopores first, efficient phase transfer between CO2 and shale oil occurs, regardless of the rock’s wettability.