Abstract
Cloud computing gave rise to a Cloud-native[1] approach for operating application software in the cloud, whereby applications are segmented into micro-services that can be designed and deployed independently of each other. This significantly increases application maintainability, reduces time to market, and helps leveraging cloud computing model. On the other hand, this approach increases the system level complexity of the application and poses new challenges, such as how services discover each other, and how application handles individual service upgrades. To support cloud-native paradigm, new development, deployment, and orchestration tools are created. One of such tools is Istio [2] service mesh, built to connect, secure, control, and observe services. While immensely useful to application developers, Istio is an additional layer in cloud compute platform software stack and is thus prone to failure or misuse.
In this work, we address the question of how to explore and troubleshoot software systems managed by Istio, focusing on micro-services upgrades and versioning. Cloud native applications are not upgraded at once like monolithic applications are. Instead, individual micro-services are gradually upgraded over time, so that older service instances live side by side with newer ones, as part of a single application. Istio supports application upgrades by splitting traffic as defined in configurable resources named virtualservices. When everything is good, Istio relieves operators from overheads related to system upgrades. The question is what happens when something goes wrong, e.g. when Istio is misconfigured. Such errors are inherently hard to detect, especially as overall systems complexity and scale grow.
We have extended a real-time network topology and protocol analyzer, Skydive [4], developed to explore and troubleshoot the physical, the virtual, and the application connectivity in the cloud. To support cloud-native environments, we have developed k8s [3] and Istio probes and contributed them to the community. These probes enrich Skydive object model with k8s objects, e.g. pods and services, as well as with Istio objects, e.g. virtualservices and destinationrules.
We use Bookinfo, an online book store application, to demonstrate how Skydive helps exploring misconfiguration in Istio versioning. One of Bookinfo microservices, called Reviews, has two functionally different versions, and the user expects Istio to split traffic between them according to weights configured in the related virtualservice. Figure 1 presents Skydive's representation of the actual configuration, discovered at run-time by our probes. Using this representation, aided by querying and filtering capabilities of Skydive, one can explore Istio and k8s objects and relationships between them, to debug an undesired versioning behavior.
In the future, we aim at supporting additional use cases, e.g security policies, circuit breakers, timeouts, and retries. We expect that the most value will be derived from multilayer exploration combined with Skydive's capability to capture/inject traffic.
