Modeling and Optimizing Dynamic Treatment Regimens in Continuous Time

Modeling and optimizing sequential clinical actions (dynamic treatment regimes, DTRs) in continuous time are crucial in many biomedical applications, especially in patient care and treatment for chronic diseases. Traditional statistical and machine learning methods usually focus on estimating a sequence of optimal treatments at given stages of intervention. However, optimizing clinical actions in continuous time has rarely been studied. In this chapter, we develop a model-based decision framework to optimize the personalized clinical decisions in continuous time including scheduling follow-up visits and assigning an optimal treatment at each visit. Specifically, the proposed framework has two steps. The first step is a Bayesian method that jointly models follow-up visits and treatments using a marked temporal point process (MTPP), which is then linked to clinical measurements through parameter sharing. The second step is to optimize the sequential clinical decisions while accounting for uncertainties in the clinical measurements using policy optimization methods. We demonstrate the usefulness of the proposed framework by applying it to a dataset from electronic medical records of patients after kidney transplantation, yielding interpretable and clinically useful results.