The objective of offline reinforcement learning is to derive an optimal policy using a static dataset. Due to the estimation errors of out-of-distribution (OOD) action value, most offline reinforcement learning algorithms constrain the policy within the actions in the dataset. TD3 + BC is one of these methods to alleviate OOD problem. In TD3 + BC, the agent employs behavior cloning to align its policy closely with that found in the dataset. Although TD3 + BC performances well in some environments, it is too conservative. TD3 + BC simply imitates the existing policy in the dataset. It disregards the quality of the state-action pairs within the dataset. This leads to a failure to learn good policies. Therefore, in this paper, a sample quality based imitation approach for offline reinforcement learning (SQIRL) is proposed. In the proposed method, the quality of each training sample in the dataset is evaluated, and then the high-quality samples are imitated. Therefore, the proposed method not only alleviates the OOD problem as original TD3 + BC, but also learns a better policy. This algorithm is compared with the existing methods, such as imitation learning methods, policy regularization methods, uncertainty penalty methods, and policy constraints methods. It is found that this algorithm outperforms the comparison methods in the MuJoCo environment. It is important to highlight that this approach is especially effective in scenarios where the overall data quality is low. For example, the average normalized rewards of the proposed algorithm are 200%, 181% higher than those of TD3 + BC in the Random dataset of halfcheetah and hopper environment. Meanwhile, in the Full-Replay dataset of halfcheetah, hopper and walker2d environments, compared with the TD3 + BC, the average normalized rewards obtained by the proposed method increase by 81.4%,157%, and 21.8%, respectively.