The PetShop Dataset -- Finding Causes of Performance Issues across Microservices

Overview

• Identifying root causes for unexpected or undesirable behavior in complex systems is a significant challenge, especially in modern cloud applications with numerous microservices.
• Existing research has proposed various techniques, but a lack of standardized datasets for quantitative benchmarking has led research groups to create their own datasets.
• This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications, including latency, requests, and availability metrics, as well as 68 injected performance issues.

Plain English Explanation

When complex systems, like cloud-based applications with many small services (known as microservices), start behaving unexpectedly or in an undesirable way, it can be very challenging to figure out the root cause of the problem. The research community has proposed various techniques to tackle this issue, but the lack of standardized datasets has meant that researchers have had to create their own datasets for testing and experimentation.

This paper presents a new dataset that is specifically designed to help evaluate methods for identifying the root causes of performance issues in microservice-based applications. The dataset includes metrics like latency (how long it takes for a request to be processed), the number of requests, and the availability (how often the system is able to respond) of the application, measured in 5-minute intervals. Importantly, the dataset also includes 68 deliberately introduced performance problems, which increase latency and reduce availability in different parts of the system.

By providing this dataset, the researchers hope to enable further development and testing of techniques for root cause analysis in these complex, microservice-based applications. Having a standardized dataset will make it easier for researchers to compare and improve their methods.

Technical Explanation

The paper presents a dataset designed to enable the evaluation of root cause analysis techniques in the context of microservice-based applications. The dataset includes time-series metrics such as latency, request counts, and availability, collected at 5-minute intervals from a distributed application. Crucially, the dataset also includes 68 injected performance issues that increase latency and reduce availability in various parts of the system.

The researchers showcase how this dataset can be used to evaluate the accuracy of a variety of root cause analysis methods, spanning different causal and non-causal approaches to the problem. By providing a standardized dataset, the authors aim to enable more systematic benchmarking and further development of techniques in this important area of cloud incident management and fault recovery.

Critical Analysis

The dataset presented in this paper addresses an important gap in the research landscape by providing a standardized resource for evaluating root cause analysis techniques in microservice-based applications. This is a valuable contribution, as the lack of such datasets has historically hindered the systematic comparison and improvement of these methods.

However, it is worth noting that the dataset is limited to a specific distributed application scenario and may not capture the full complexity of real-world cloud environments. Additionally, the introduced performance issues, while diverse, may not fully reflect the range of unexpected behaviors that can occur in production systems. Further research may be needed to assess the representativeness of this dataset and its applicability to a broader set of microservice architectures and failure modes.

Conclusion

This paper presents a valuable dataset for evaluating root cause analysis techniques in the context of microservice-based applications. By providing a standardized resource with both normal and anomalous performance data, the researchers aim to enable more systematic benchmarking and development of methods for identifying the underlying causes of issues in these complex, distributed systems. The dataset represents an important step forward in addressing the challenge of root cause analysis, which is crucial for the reliable operation of modern cloud-native applications.