Evaluating the Influence of Argumentation Markers on the Identification of Reasoning Models

The article focuses on evaluating the influence of argumentation markers on the identification of specific reasoning models with machine learning methods. The evaluation process consists of a sequence of classification experiments with different feature sets. The experiments cover the identification of arguments with three specific reasoning models: “Expert Opinion”, “Example”, and “Practical Reasoning”. These models are characterized by 1) an active use in scientific articles (as evidenced by their high frequency in the employed corpus) and 2) reliance of their textual expression on typical words and phrases (markers). Each model corresponds to a separate subset of the overall dataset: 680 arguments for classifying the “Example” model, 386 for “Practical Reasoning”, 172 for “Expert Opinion” (in each case, a half of the arguments employs the corresponding model, while the other half relies on any other model except for these three). The overall dataset contains 1975 arguments from 45 scientific articles in Russian language (on linguistics and computational technologies). The argumentation in these articles is annotated with the ArgNetBank Studio platform. Classification experiments employ machine learning methods of different types: multinomial naive Bayes, support vector machine, and multilayer perceptron. The feature sets differ by the inclusion or exclusion of discourse markers and persuasion modes indicators (expressions characterizing three argumentation aspects: logos, pathos, and ethos). The experiments show that the best improvement of identification scores (on average across all schemes and classifiers) corresponds to the representation of arguments with discourse markers (plus 10% for precision and 7% for F-measure over the lemmas baseline).