This chapter proposes a multielement attention (MEA) mechanism, which is a generalized form of the self-attention mechanism. MEA models the spatial dependencies of input images by modeling elements in feature maps as nodes of a graph and using adjacency matrices to aggregate information from adjacent nodes when calculating attention weights. By designing different adjacency matrices, this chapter proposes three types of MEAs to learn local, neighboring, and global spatial information. An approach to integrate primitive representation learning and a multielement attention mechanism is also proposed by using visual text representations to provide global visual guidance for the decoding process. The experimental results show that MEAs that learn neighboring and global information can achieve better results than the self-attention mechanism can and that incorporating primitive representation learning can further improve model performance.