The Graph Transformer Network (GTN) is a state-of-the-art solution for graph representation learning, which utilizes the transformer architecture to capture attention and long-range dependencies in a non-euclidean dataset such as a graph. However, even with GTN in a downstream learning task, there are usually a few data points that are harder to classify than others and are at risk of being wrongly classified. Metric-based meta-learning, which encompasses many meta-learning algorithms, proves promising in such scenarios. Through this work, we propose a novel 2-stage pipeline combining a Graph Transformer Network with metric-based meta-learning methodologies to make downstream classification tasks easier. The work described in this article investigates and evaluates this pipeline in two specific learning setups using two Metric Learning frameworks, i.e., a Siamese Network for binary classification of graphs and a Prototypical Network for multi-class classification of nodes, both appended to a representation learning block of Graph Transformer Network. The experiments described in this article conclusively prove the efficacy of such a pipeline both in full-shot and few-shot learning scenarios.