How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions
0
Sign in to get full access
Overview
- This paper presents a systematic mapping study on the current state of requirements engineering (RE) practices for AI-based systems.
- The researchers examined published literature to identify the key practices, challenges, and future research directions in this domain.
- The study provides insights into the maturity and development of RE for AI-based systems, which is crucial as these systems become more prevalent.
Plain English Explanation
Requirements engineering (RE) is the process of identifying, documenting, and managing the requirements for a software system. When it comes to AI-based systems, the requirements can be more complex and challenging to capture compared to traditional software systems.
This paper investigates the current state of RE for AI-based systems. The researchers conducted a systematic mapping study, which involves reviewing and categorizing a large body of published literature on this topic. Their goal was to understand the key practices, challenges, and future research directions in this field.
The findings suggest that RE for AI-based systems is still a relatively young and evolving area. While some practices and techniques have been developed, there are still many open challenges and opportunities for further research and improvement. For example, the paper highlights the need for better methods to elicit and specify requirements for the complex and often uncertain behaviors of AI systems.
By providing a comprehensive overview of the current state of RE for AI-based systems, this study can help researchers and practitioners identify areas that require more attention and investment. As AI technologies continue to become more pervasive, addressing these RE challenges will be crucial for ensuring the development of reliable and trustworthy AI-based systems.
Technical Explanation
The researchers performed a systematic mapping study to investigate the current state of requirements engineering (RE) for AI-based systems. They searched various databases and identified 142 relevant publications, which they then categorized and analyzed to answer the following research questions:
- What are the key RE practices and techniques being used for AI-based systems?
- What are the main challenges and barriers in applying RE to AI-based systems?
- What are the future research directions and opportunities in this domain?
The analysis revealed several key findings:
- RE Practices: The most commonly reported RE practices include requirements elicitation, specification, and validation techniques, as well as the use of modeling and simulation for AI systems.
- Challenges: Challenges include the inherent uncertainty and dynamism of AI systems, the difficulty in specifying high-level goals and non-functional requirements, and the need for novel approaches to traceability and change management.
- Future Directions: Promising research areas include developing better methods for requirements elicitation and specification, improving the integration of RE with AI development processes, and exploring the role of human-AI collaboration in RE.
The researchers also identified several limitations in the current body of literature, such as the lack of empirical studies and the dominance of conceptual and experience-based papers. They suggest that more empirical research is needed to further validate and refine the findings of this mapping study.
Critical Analysis
The systematic mapping study provides a comprehensive overview of the current state of requirements engineering (RE) for AI-based systems, highlighting both the progress made and the significant challenges that remain. The researchers have done a commendable job in synthesizing the available literature and identifying the key themes and trends.
One strength of the study is its breadth, covering a wide range of RE practices, challenges, and future research directions. This broad scope gives readers a good understanding of the overall landscape and the multifaceted nature of the problem.
However, the study also reveals the relative immaturity of this field. The dominance of conceptual and experience-based papers, rather than empirical studies, suggests that the community has not yet developed a robust set of best practices and well-validated techniques for RE in the context of AI-based systems.
Additionally, the study does not delve deeply into the specific technical and organizational challenges that may arise when applying traditional RE approaches to AI-based systems. For example, the paper could have explored in more detail the difficulties in specifying and verifying the complex, context-dependent behaviors of AI systems, or the challenges of integrating RE with agile and iterative AI development processes.
Despite these limitations, this systematic mapping study serves as an important starting point for researchers and practitioners interested in advancing the state of RE for AI-based systems. By highlighting the key issues and research directions, the paper provides a solid foundation for future work in this critical and rapidly evolving field.
Conclusion
This systematic mapping study offers a comprehensive overview of the current state of requirements engineering (RE) for AI-based systems. The findings suggest that while some progress has been made in developing RE practices and techniques for these systems, the field is still relatively young and faces significant challenges.
The study identifies several key RE practices, such as requirements elicitation, specification, and validation, as well as the use of modeling and simulation. However, it also highlights major challenges, including the inherent uncertainty and dynamism of AI systems, the difficulty in specifying high-level goals and non-functional requirements, and the need for novel approaches to traceability and change management.
Looking to the future, the paper outlines promising research directions, such as improving requirements elicitation and specification methods, better integrating RE with AI development processes, and exploring the role of human-AI collaboration in RE. Addressing these issues will be crucial as AI-based systems become increasingly prevalent in various domains.
By providing a comprehensive overview of the current state of RE for AI-based systems, this systematic mapping study serves as an important foundation for researchers and practitioners working to advance the field and ensure the successful development of reliable and trustworthy AI-powered applications.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions
Umm-e- Habiba, Markus Haug, Justus Bogner, Stefan Wagner
Artificial intelligence (AI) permeates all fields of life, which resulted in new challenges in requirements engineering for artificial intelligence (RE4AI), e.g., the difficulty in specifying and validating requirements for AI or considering new quality requirements due to emerging ethical implications. It is currently unclear if existing RE methods are sufficient or if new ones are needed to address these challenges. Therefore, our goal is to provide a comprehensive overview of RE4AI to researchers and practitioners. What has been achieved so far, i.e., what practices are available, and what research gaps and challenges still need to be addressed? To achieve this, we conducted a systematic mapping study combining query string search and extensive snowballing. The extracted data was aggregated, and results were synthesized using thematic analysis. Our selection process led to the inclusion of 126 primary studies. Existing RE4AI research focuses mainly on requirements analysis and elicitation, with most practices applied in these areas. Furthermore, we identified requirements specification, explainability, and the gap between machine learning engineers and end-users as the most prevalent challenges, along with a few others. Additionally, we proposed seven potential research directions to address these challenges. Practitioners can use our results to identify and select suitable RE methods for working on their AI-based systems, while researchers can build on the identified gaps and research directions to push the field forward.
Read more9/12/2024
0
Generative AI for Requirements Engineering: A Systematic Literature Review
Haowei Cheng, Jati H. Husen, Sien Reeve Peralta, Bowen Jiang, Nobukazu Yoshioka, Naoyasu Ubayashi, Hironori Washizaki
Context: Generative AI (GenAI) has emerged as a transformative tool in software engineering, with requirements engineering (RE) actively exploring its potential to revolutionize processes and outcomes. The integration of GenAI into RE presents both promising opportunities and significant challenges that necessitate systematic analysis and evaluation. Objective: This paper presents a comprehensive systematic literature review (SLR) analyzing state-of-the-art applications and innovative proposals leveraging GenAI in RE. It surveys studies focusing on the utilization of GenAI to enhance RE processes while identifying key challenges and opportunities in this rapidly evolving field. Method: A rigorous SLR methodology was used to analyze 27 carefully selected primary studies in-depth. The review examined research questions pertaining to the application of GenAI across various RE phases, the models and techniques used, and the challenges encountered in implementation and adoption. Results: The most salient findings include i) a predominant focus on the early stages of RE, particularly the elicitation and analysis of requirements, indicating potential for expansion into later phases; ii) the dominance of large language models, especially the GPT series, highlighting the need for diverse AI approaches; and iii) persistent challenges in domain-specific applications and the interpretability of AI-generated outputs, underscoring areas requiring further research and development. Conclusions: The results highlight the critical need for comprehensive evaluation frameworks, improved human-AI collaboration models, and thorough consideration of ethical implications in GenAI-assisted RE. Future research should prioritize extending GenAI applications across the entire RE lifecycle, enhancing domain-specific capabilities, and developing strategies for responsible AI integration in RE practices.
Read more9/12/2024
0
Automated User Story Generation with Test Case Specification Using Large Language Model
Tajmilur Rahman, Yuecai Zhu
Modern Software Engineering era is moving fast with the assistance of artificial intelligence (AI), especially Large Language Models (LLM). Researchers have already started automating many parts of the software development workflow. Requirements Engineering (RE) is a crucial phase that begins the software development cycle through multiple discussions on a proposed scope of work documented in different forms. RE phase ends with a list of user-stories for each unit task identified through discussions and usually these are created and tracked on a project management tool such as Jira, AzurDev etc. In this research we developed a tool GeneUS using GPT-4.0 to automatically create user stories from requirements document which is the outcome of the RE phase. The output is provided in JSON format leaving the possibilities open for downstream integration to the popular project management tools. Analyzing requirements documents takes significant effort and multiple meetings with stakeholders. We believe, automating this process will certainly reduce additional load off the software engineers, and increase the productivity since they will be able to utilize their time on other prioritized tasks.
Read more4/3/2024
0
Regulatory Requirements Engineering in Large Enterprises: An Interview Study on the European Accessibility Act
Oleksandr Kosenkov, Michael Unterkalmsteiner, Daniel Mendez, Jannik Fischbach
Context: Regulations, such as the European Accessibility Act (EAA), impact the engineering of software products and services. Managing that impact while providing meaningful inputs to development teams is one of the emerging requirements engineering (RE) challenges. Problem: Enterprises conduct Regulatory Impact Analysis (RIA) to consider the effects of regulations on software products offered and formulate requirements at an enterprise level. Despite its practical relevance, we are unaware of any studies on this large-scale regulatory RE process. Methodology: We conducted an exploratory interview study of RIA in three large enterprises. We focused on how they conduct RIA, emphasizing cross-functional interactions, and using the EAA as an example. Results: RIA, as a regulatory RE process, is conducted to address the needs of executive management and central functions. It involves coordination between different functions and levels of enterprise hierarchy. Enterprises use artifacts to support interpretation and communication of the results of RIA. Challenges to RIA are mainly related to the execution of such coordination and managing the knowledge involved. Conclusion: RIA in large enterprises demands close coordination of multiple stakeholders and roles. Applying interpretation and compliance artifacts is one approach to support such coordination. However, there are no established practices for creating and managing such artifacts.
Read more9/12/2024