One of the prospective domains in remote healthcare is monitoring home physical rehabilitation using mobile phones and providing patients with real-time feedback on their exercise performance. Assessing such performance involves analyzing the similarity of spatio-temporal features extracted from human motion data. State-of-the-art research provides multiple tools for estimating human motion from mobile camera video streams. However, their applicability to physical therapy monitoring is not sufficiently explored. To address this problem, we introduce a new rehabilitation dataset (REHAB24-6), which provides untrimmed RGB videos, 2D and 3D skeletal ground truth of human motion, and temporal segmentation for six rehabilitation exercises. We also propose a novel pose transformation technique to evaluate existing 2D and 3D pose estimation methods trained on different datasets with distinct body models. Our experiments explore the current limitations of the state-of-the-art, particularly the depth estimation, and offer recommendations for selecting appropriate models. Finally, we propose similarity-based techniques to assess the ability of estimated pose sequences to discern exercise performance and report promising results of current pose detectors for rehabilitation assistance.