Tumor microenvironment (TME) comprises immune and stromal cells alongside tumor cells, impacting tumor progression and patient prognosis. Digital pathology has revolutionized histopathological analysis, enabling the extraction of rich information from whole slide images (WSIs). Despite the high performance of Multiple Instance Learning (MIL) in predicting outcomes from WSIs, its interpretability and clinical relevance remain limited. We propose a hierarchical graph neural network approach integrating cell- and tissue-level features to improve interpretability and predictive performance for gastric cancer survival prognosis. Our model utilizes the segmentation model trained with a dataset for precise cell classification into eight major cell types in TME, and extracts detailed cellular features. Extensive experiments demonstrate that our model outperforms traditional MIL methods, as well as providing biologically meaningful insights. Our approach promises enhanced interpretability and reliability in clinical workflows, with potential applications across cancer types.