This paper explores the utilization of Large Language Models (LLMs) in Multi-Agent Systems (MAS) in scenarios where the agents are expected to collaborate and negotiate their preferences, creating temporary alliances to achieve a common goal (complex task). MAS have been acknowledged for their potential in facilitating collaboration to solve complex problems. However, widespread adoption of MAS is impeded by challenges related to defining communication languages and developing frameworks that balance specificity for complex use cases with general applicability across different domains. The emergence of LLMs, such as GPT-4, presents a novel approach to MAS, offering advanced natural language processing capabilities that (potentially) circumventing the need for explicit communication language definitions. This paper proposes an MAS implementation utilizing LLMs within the Autogen framework, emphasizing collaboration and negotiation among agents, through a case study involving a product manufacturing scenario where agents are tasked with intricate decision-making. Results from three test scenarios demonstrate the efficacy of this approach, that can be used to enhance further developments in MAS scenarios of application. However, despite the promise, challenges remain, including the cost of running LLMs and the need for further exploration of their capabilities.