A Comparison of Text Representation Techniques and Encoder-Decoder Implementations in a Deep Neural Network for Converting Natural Language into Formal Logic Formulas

Semantic parsing is the task of extracting a structured machine-interpretable representation from natural language utterance. This representation can be used for various applications such as question answering, information extraction, and dialogue systems. However, semantic parsing is a challenging problem that requires dealing with the ambiguity, variability, and complexity of natural language. This paper investigates neural parsing of natural language (NL) sentences to first-order logic (FOL) formulas. FOL is a widely used formal language for expressing logical statements and reasoning. FOL formulas can capture the meaning and structure of NL sentences in a precise and unambiguous way. FOL parsing is approached as a sequence-to-sequence mapping task using both long short-term memory (LSTM) and transformer encoder-decoder architectures for character-, subword-, and word-level text tokenisation. These models are trained on NL-FOL datasets with supervised learning and evaluated using various metrics. Previous solutions to neural FOL parsing differ dramatically in training approaches and scale. As such, there is no comprehensive comparison of models for different methods of text representations or encoder-decoder architectures. The main contributions of this paper are: the formation of a complex NL-FOL benchmark that includes algorithmically generated and human-annotated FOL formulas, evaluation of 15 sequence-to-sequence models on the task of neural FOL parsing for different text representations and encoder-decoder architectures, and an in-depth analysis of the strengths and weaknesses of these models.