Output Structure Simplification to Enhance Transformer-Based Text-To-Workflow Translation
摘要
Workflows are structured sequences of tasks that define how a process is executed. Manually creating a workflow from a given natural language description is challenging and time-consuming. One potential solution is to automate this process using a Transformer model. However, translating text into a particular workflow format with Transformers could be challenging due to the structural complexity of the target format and a diverse vocabulary, which can reduce accuracy. A data-centric method (Text-to-SSF-Workflow) is proposed to improve Transformer-based Text-to-Workflow translation by simplifying the output structure and reducing vocabulary size through abstraction without modifying the Transformer's architecture. In the absence of publicly available Text-to-Workflow datasets, this study utilizes a Text-to-Source-Code dataset, where the source code is converted into workflow represented by a JSON format and a Simplified Structured Format (SSF). The conversion from source code is achieved through Abstract Syntax Tree (AST) parsing, followed by node-level abstraction using FAISS K-Means Clustering. A domain-adapted tokenizer, based on BERT, is also introduced to better handle code-like syntax in workflow formats for Transformer model training. Preliminary experiments of this research show that training a standard Transformer model with a simple-format dataset can significantly improve translation accuracy.