Document entity recognition is a critical task in the field of document analysis. While many large language models have demonstrated success in this domain, they often fail to incorporate the spatial layout relationships between entities within a document. In this work, we introduce VisHubGAT, a novel graph-based model that integrates visible connectivity and hub nodes to enhance entity classification. Our approach constructs a document graph representation where hub nodes aggregate label-specific features, leveraging pre-trained models such as BERT and LayoutLMv3. Additionally, we introduce multi-type edges to encode both spatial and semantic relationships, and a Graph Attention Network (GAT) with edge-aware attention, which incorporates edge features into the attention mechanism for improved relational modeling. An edge classification layer further refines entity connectivity. Our approach effectively captures inter-entity relationships within the document, thereby enhancing the entity classification capabilities of large language models on public datasets (FUNSD, CORD, DocILE), showing state-of-the-art results by using different modalities. Ablation studies highlight the importance of our structured graph design for document entity recognition.