The development of Large Language Model chatbots has made access to general knowledge through natural conversation readily available. Such conversational agents, conversing through speech and embodied with an animated body, may be perceived as social actors with a distinct social presence. Consequently, interacting with them may be perceived as social interaction, be interdependent from the user’s social characteristics, and provide further benefits beyond accessing general knowledge. Additionally, in Virtual Reality, it is possible to share the same environment as a virtual agent, potentially further enhancing face-to-face communication (F2F) and increasing social presence with said agent. In this study, we investigate how display immersiveness (2D desktop monitor and a Head-Mounted Display), along with user social characteristics (such as shyness and self-consciousness), influence a conversational agent’s social presence and trustworthiness when conversing about general knowledge topics. Our findings reveal that immersion levels do not significantly alter the agent’s social presence and trustworthiness. However, user social characteristics, specifically regarding their communication apprehension, impact the affective traits of the social presence of the conversational agent. These results underscore the importance of considering user social characteristics when designing conversational agents, as well as the distinction between sharing an environment with others and effectively feeling social presence toward others.