Federated learning (FL) is widely adopted on data mining for decentralized data analysis and decision-making process. However, FL still faces privacy challenges. The existing schemes to solve the privacy concerns of FL are mainly based on secure multi-party computing, differential privacy, etc. These methods hide the raw gradients against the server, but the raw gradients is essential in tasks such as robustness improvement, anomaly detection and noise data elimination. While unmodified FL can collect the raw gradients, it also exposes the privacy of participants. We identify cutting off the link between the raw gradients and the participant can effectively protect the privacy of users. Therefore, to provide the raw gradients to FL servers and protect the privacy of participants, we propose an unlinkable federated learning (UFL) scheme. In this scheme, the unlinkability is well guaranteed with shuffle based on a position exchange protocol. In addition, secret sharing mechanism is used to ensure the rawness of gradient data and robustness of participant drop-outs. Finally, we provide the security proof to demonstrate the rawness, unlinkability as well as fault tolerance of the scheme, and conduct extensive experiments and the results demonstrate the efficiency of the scheme.