Designing Personalized Treatment: An Application to Anticoagulation Therapy

In this study, we develop an analytical framework for personalizing the anticoagulation therapy of patients who are taking warfarin. Consistent with medical practice, our treatment design consists of two stages: (i) the initiation stage, modeled using a partially‐observable Markov decision process, during which the physician learns through systematic belief updates about the unobservable patient sensitivity to warfarin, and (ii) the maintenance stage, modeled using a Markov decision process, during which the physician relies on his formed belief about patient sensitivity to determine the stable, patient‐specific, warfarin dose to prescribe. We develop an expression for belief updates in the POMDP, establish the optimality of the myopic policy for the MDP, and derive conditions for the existence and uniqueness of a myopically optimal dose. We validate our models using a real‐life patient data set gathered at the Hematology Clinic of the Jewish General Hospital in Montreal. The proposed analytical framework and case study enable us to develop useful clinical insights, for example, concerning the length of the initiation period and the importance of correctly assessing patient sensitivity.