Context: Securing consistent user feedback is a critical yet resource-intensive challenge in new product development (NPD). Active Personas (APs), user archetypes powered by Generative AI, offer a novel approach to generate realistic user feedback on demand, enabling internal product experimentation. Objective: This study evaluates the effectiveness of AP in generating “user” feedback by evaluating the alignment between AP-generated feedback and actual (human) users and investigates how persona and Large Language Model (LLM) choice influence the feedback. Method: Adopting a Design Science Research methodology, we designed and developed AP as a novel artifact. We demonstrated and evaluated the artifact in a case of a mobile transport app, creating eight AP instances from two personas and four LLMs. The AP-generated feedback was compared against human feedback from interviews and Google Play reviews using a triangulated analysis based on Nielsen’s usability heuristics. Results: Our findings reveal a strong alignment between AP and human feedback, with APs effectively identifying various usability and accessibility issues, confirming what the actual users also found. The persona definition significantly dictated the evaluation’s criticality, and the choice of LLM further influenced the evaluative stance. Conclusion: APs have the potential to augment early-stage usability evaluations and are a viable option for human users for internal experimentation, providing rapid, low-cost feedback. While they supplement, not replace, direct human interaction, this work validates the transformation of personas from static artifacts into dynamic, generative agents for NPD.