With the rise of service globalization and the advent of LLMs, users are becoming increasingly active on the internet to discover services and engage in social interaction. Instead of browsing through vast amounts of information, users prefer to interact directly with smart devices for decision-making and recommendations. However, there are two main challenges in this process: firstly, user needs are often ambiguous, with different functionalities potentially being described in similar terms. Secondly, the internet hosts a large number of services and requirements, complicating the process of service composition. To address the first challenge, this paper proposes the Graph Self-Attention Transformer (GSAT) model, which enhances representation from both semantic and topological perspective. From topological perspective, it integrates local features by walking through the historical records of mashups, uses graph self-attention module on this records, and employs an attention mechanism on all mashups to capture global features. From semantic perspective, it enhances mashup and API descriptions with the help of LLMs. To verify the effectiveness in solving the second challenge, This paper partitions the ProgrammableWeb dataset under and evaluates the GSAT performance under the cold-start setting. This paper compares GSAT with traditional methods and several LLMs, including BERT, T5, LLaMA and ChatGPT. The experiments show that GSAT effectively distinguishes between mashups and achieves state-of-the-art (SOTA) performance.