Robustness evaluation of the rainbow framework for self-adaptation

Self-adaptive (or autonomic) systems incorporate complex software components that act as controllers of a target system by executing actions through effectors, based on information monitored by probes. Despite the growing importance and criticality of controllers in many application domains, a central concern about them is the difficulty in assessing their robustness when architecting self-adaptive systems. In previous work, we proposed an approach for evaluating the robustness of controllers in self-adaptive systems. In this practical experience report, we describe a comprehensive evaluation of the robustness of a particular controller, in our case Rainbow, in the context of two case studies: a benchmark case study that reproduces the typical infrastructure for a news website, and an industrial middleware for monitoring populated networks of devices. The aim of this work is to assess to what extent the use of a different target system has an impact on the robustness of the controller, which has to be customized in different ways, and may need to resort to the activation of different features, depending on the particular target system. Our analysis concludes that the customization of Rainbow (the controller) has little impact on its robustness because of the way the controller was designed and built, and this modularization of non-functional requirements is indeed encouraging when architecting self-adaptive systems.