Real-time embedded systems (RTES) require the efficient distribution of complex functionalities across multiple processors, making robust deployment models crucial to ensure real-time constraints are met while optimizing overall system performance. The challenge of task placement and scheduling in RTES is compounded by the NP-hard nature of the problem, especially when multiple objectives must be considered, where traditional methods often fall short. This paper presents MPSQL (Multi-Objective Placement Scheduling based on Q-learning), a novel approach utilizing reinforcement learning to optimize RTES deployment in a multi-objective framework on heterogeneous architectures. MPSQL operates in two phases: the first phase models task placement as a Markov Decision Process (MDP) and applies Pareto Q-learning to allocate tasks to processors, focusing on system extensibility and energy consumption. The second phase addresses task scheduling, also modeled as an MDP, and employs Q-learning to determine the optimal scheduling for each Pareto placement model with the goal of minimizing response time. This structured, two-phase approach results in a set of Pareto deployment models that balance and optimize the three key objectives. A case study validates the effectiveness of MPSQL in generating high-quality deployment models for multi-objective RTES.