Microservice Systems (MSS) are one of the current promising approaches for realizing modular evolving systems for dynamic and complex environments. Each comprises small autonomous components (microservices) capable of collaborating to conduct comprehensive tasks. However, these promising systems face many challenges inherent in their distributed nature, such as the increased complexity and the uncertainties of the dynamic environment. Self-Adaptive System (SAS) is a prominent solution that can mitigate these problems. However, current MSS self-adaptation approaches depend heavily on utilizing infrastructure-level supported actions like auto-scaling. These actions have several limitations, including, their inefficiency in providing solutions to complex scenarios, such as the cases of high workload and the conflict in Quality of Service (QoS). Consequently, recent research is orienting toward application-level adaptation solutions. The effort done in MSS application-level adaptation has, however, been very limited, with proposed solutions being incomplete, reference architecture-driven, or programming languages dependent. In this PhD proposal, we introduce an application-level solution that aims to address these identified limitations. It proposes a dynamic software product line (DSPL) approach that can drive a new architecture for the MSS at runtime. The approach is augmented with a reinforcement learning (RL) agent that reasons about architecture decisions. This proposed solution offers promising benefits in terms of maintaining business continuity, optimizing resource usage by preventing over-provisioning, and contributing to, the overall, efficiency and effectiveness of the MSS operations.