LLM4PM: A case study on using Large Language Models for Process Modeling in Enterprise Organizations

Overview

• This paper investigates the potential of using Large Language Models (LLMs) to assist in creating process models in organizational settings.
• The researchers conduct a case study where they develop and test an LLM-based chatbot called PRODIGY (PROcess moDellIng Guidance for You) within the Hilti Group, a multinational company.
• The focus is on understanding how LLMs can aid human process modellers in creating process flow diagrams.

Plain English Explanation

The researchers wanted to explore how Large Language Models could be used to help people create process models, which are visual diagrams that show the steps in a business process. They did this by building a chatbot called PRODIGY and testing it with process modellers at a large company called the Hilti Group.

First, the researchers talked to 10 professional process modellers at Hilti to understand the challenges they face in their daily work. Then, they used that information to design and build the PRODIGY chatbot. Finally, they had the original 10 modellers use PRODIGY and give feedback on the pros and cons.

The key idea is to see how LLMs can be incorporated into production systems to enhance business process management activities. The researchers wanted to show a real-world example of how industries can leverage LLMs to improve their operations.

Technical Explanation

The researchers conducted a case study to explore the use of Large Language Models (LLMs) in process modelling. They first carried out a preliminary user study with 10 professional process modellers from the Hilti Group to understand the pain points they encounter in their daily work. Based on the insights from this study, they designed and implemented the PRODIGY chatbot, which leverages LLM technology to assist human modellers in creating process flow diagrams.

To evaluate PRODIGY, the researchers had the original 10 user study participants use the chatbot and then provide feedback on its pros and cons. The goal was to assess how well PRODIGY could support process modelling tasks and to gather actionable insights on how LLMs can benefit business processes.

Through this case study, the researchers showcase the first practical application of LLM technology for process modelling in a real-world organizational setting. The findings shed light on the potential for using LLMs as research assistants to enhance Business Process Management activities.

Critical Analysis

The paper provides a promising initial exploration of using LLMs to support process modelling, but it also acknowledges several limitations and areas for further research. For example, the user study was relatively small, and the researchers note that larger-scale evaluations would be needed to generalize the findings.

Additionally, the paper does not delve deeply into potential ethical or privacy concerns that may arise from using LLM-based systems in organizational contexts, such as the risk of biased outputs or the handling of sensitive business data. These are important considerations that future work should address.

Overall, the researchers have taken a thoughtful and systematic approach to investigating this application of LLM technology. However, more extensive testing and analysis will be necessary to fully understand the strengths, weaknesses, and broader implications of using LLMs for process modelling in real-world organizational settings.

Conclusion

This paper presents the first practical application of using Large Language Models to assist in process modelling within a multinational company. The researchers developed and tested an LLM-based chatbot called PRODIGY, which was designed to help human process modellers create workflow diagrams more efficiently.

The case study provides insights into the potential benefits of leveraging LLMs to enhance Business Process Management activities, as well as some of the challenges and limitations that need to be addressed. While further research is required, this work represents an important step forward in exploring how emerging AI technologies can be integrated into organizational processes to improve productivity and efficiency.