The effectiveness of modern AI technologies is supported by the steadily increasing complexity of calculations and – as a consequence – is accompanied by an increasing opacity of the results. The lack of explanation in decision-making undermines users’ trust in digital products and services. One of the promising approaches to creating explainable AI is through the use of argumentation theories for decision-making both in general-purpose systems and in applied areas such as medical informatics, law informatics, robotics, security, control systems, and others. Argumentation combined with XAI is a wide research field. The paper proposes an approach that formalizes the idea of argumented (not reducible to deductive reasoning) decision-making by using argumentation logic. The proposed formal tools are used to analyze the correspondence of the decisions made to objective knowledge based on logical consistency. The dependence of the outcome of such an analysis on the argumentative semantics used indicates the need for selecting semantics that are adequate to the nature of the task in order to be used in control and decision-making systems.