Finding point-level correspondences is a fundamental problem in ultrasound (US), enabling US landmark tracking for intraoperative image guidance and motion estimation. Most US tracking methods are based on optical flow or feature matching, initially designed for RGB images. Therefore domain shift can impact their performance. Ground-truth correspondences could supervise training, but these are expensive to acquire. To solve these problems, we propose a self-supervised point-tracking model called PIPsUS. Our model can track an arbitrary number of points at pixel-level in one forward pass and exploits temporal information by considering multiple, instead of just consecutive, frames. We developed a new self-supervised training strategy that utilizes a long-term point-tracking model trained for RGB images as a teacher to guide the model to learn realistic motions and use data augmentation to enforce tracking from US appearance. We evaluate our method on neck and oral US and echocardiography, showing higher point tracking accuracy when compared with fast normalized cross-correlation and tuned optical flow. Codes are available at https://github.com/aliciachenw/PIPsUS .