Decision support tools in healthcare require a strong confidence in the developed Machine Learning (ML) models both in terms of performances and in their ability to provide users a deeper understanding of the underlying situation. This study presents a novel method to construct a risk stratification based on ML and local explanations. An open-source dataset was used to demonstrate the efficiency of this method that well identified the main subgroups of patients. Therefore, this method could help practitioners adjust and build protocols to improve care deliveries that would better reflect patient’s risk level and profile.