Background <p>Patient-reported outcome measures (PROMs) are often challenging to analyze and interpret. Indeed, patients may give different answers to PROMs over time, not only because their level of the target construct (e.g. anxiety) has changed but also because their interpretation of the items aiming at measuring the construct has changed. For instance, cancer treatment may trigger changes in the patients’ internal standard of measurement (i.e., recalibration response shift, RS), resulting in a lack of measurement invariance over time. In addition, interpretation of PROMs items may differ according to cancer type (differential item functioning, DIF). If ignored, DIF and RS may impact inferences made from PROMs; they are also crucial to investigate as they may be related to patients’ adaptation after a salient health event, e.g. cancer diagnosis. Our objectives were to show how cross-sectional and longitudinal Rasch models can be used to detect, interpret, and account for DIF and RS, where appropriate, when measuring anxiety and depression in breast cancer and melanoma patients.</p> Methods <p>Anxiety and depression were assessed in breast cancer (<i>n</i> = 337) and melanoma patients (<i>n</i> = 110) using the Hospital Anxiety and Depression Scale (HADS) at 1- (T1) and 6-month (T2) post-diagnosis. DIF and RS analyses were performed using the RespOnse Shift ALgorithm at the Item level (ROSALI) based on Rasch models, i.e. cross-sectional and longitudinal Partial Credit Models (PCM).</p> Results <p>DIF and RS were identified in the anxiety (DIF and RS) and depression (RS only) subscales of the HADS. DIF and RS had a moderate (anxiety) or significant impact (depression) on the results, providing different conclusions depending on whether or not they were considered. More specifically, wrongly presuming (longitudinal) measurement invariance would have resulted in overestimating the increase in depression over time among breast cancer patients.</p> Conclusion <p>The ROSALI procedure based on Rasch models is freely available (<a href="https://pro-online.net/">https://pro-online.net/</a>). ROSALI enabled to investigate measurement invariance at item-level, providing insight into cancer patients’ experience, possibly revealing psychological distress but also psychological adaptation to challenging health events. Although some methods are available, DIF and RS are still too often ignored which can lead biased measurements of constructs and to suboptimal healthcare decision making.</p> Trial registration number <p>n°CT.gov: NCT02893774, registration date: 2016-08-25. Of note, this was a prospective study which was retrospectively registered.</p>