A method for analyzing handwritten program flowchart based on detection transformer and logic rules

Handwritten program flowcharts can describe program design ideas and algorithms, which is helpful for users to understand and analyze the execution process of programs. Hence, accurately recognizing and analyzing handwritten program flowcharts is essential. However, there are still some challenges in the process of recognizing and analyzing the handwritten program flowcharts, such as accurately recognizing the borders, avoiding potential confusion between the borders and the text, and understanding the logical structure information of the flowchart. To address these issues, this paper introduces a method for recognizing and analyzing handwritten program flowcharts. Specifically, we propose integrating the learnable reference boxes mechanism and semantic alignment mechanism into the Detection Transformer model (LS-DETR) to enhance its performance in recognizing borders in a flowchart. In addition, a strategy for analyzing the logic structure of flowcharts based on logic rules has been proposed, which can check the execution logical structure of program flowcharts and examines the number of loops and the execution times of loop bodies within them. The logic rules are derived from knowledge related to flowchart construction principles, program execution principles, and other relevant information. Experimental results demonstrate that the LS-DETR exhibits a 2.39% increase in Average Precision (AP) compared to the DETR, while also achieving nearly 8.3 times faster convergence speed. The example demonstrates that the execution process of a flowchart can be accurately analyzed and judged based on logic rules.