Softwarization technologies of the Internet network functions made it feasible to control a huge-scale network topology. The optimal scaling strategy of virtualized network functions (VNFs), that processes network traffic to satisfy user’s request, becomes challenging due to the high complexity. To handle this challenge, reinforcement learning and graph neural network-based approaches have been proposed and showed remarkable performance with less computation cost compared to the dynamic programming method. However, in the previous work, the utilized graph neural network model is limited to encode useful information for the VNF scaling task with the fixed adjacency matrix. Furthermore, the previous work assumed the less realistic network setting that does not distinguish the node’s type. In this paper, we propose an advanced VNF scaling approach to solve such limitations. Specifically, we utilize the graph attention network that has a learnable adjacency matrix. In addition, we propose to input node types and spatial information to make our VNF scaling agent adaptable to a realistic network setting which leads to performance improvements. Moreover, we propose to apply the phasic policy gradient algorithm that can be advantageous to the task. Experiment results conducted on various datasets demonstrate that the proposed method generally achieves a better quality of service with less resource utilization than the previous work. Finally, the best configuration of the proposed VNF scaling agent achieves a similar level of performance to the dynamic programming method with a 30 times faster execution time.