Learning from Litigation: Graphs and LLMs for Retrieval and Reasoning in eDiscovery

Overview

• Electronic Discovery (eDiscovery) is the process of identifying relevant documents from a large collection for legal proceedings
• Traditional approaches like BM25 or fine-tuned pre-trained models face performance, computational, and interpretability challenges
• Large Language Model (LLM)-based methods prioritize interpretability but sacrifice performance and throughput
• This paper introduces DISCOvery Graph (DISCOG), a hybrid approach that combines the strengths of graph-based methods and LLMs

Plain English Explanation

When legal teams need to review a large amount of documents for a case, they use a process called electronic discovery (eDiscovery) to find the most relevant ones. Traditional methods for this can be slow, expensive, and hard to understand.

This paper presents a new approach called DISCOvery Graph (DISCOG) that combines the benefits of two different techniques. First, it uses a graph-based method to quickly identify the most relevant documents. Then, it uses large language models (LLMs) to provide clear explanations for why those documents were chosen.

Compared to other methods, DISCOG is more accurate, efficient, and interpretable. It can handle datasets with both balanced and imbalanced distributions, and it reduces document review costs by up to 99.9% compared to manual processes.

Technical Explanation

The paper introduces DISCOvery Graph (DISCOG), a hybrid approach that combines the strengths of graph-based methods and large language models (LLMs).

The graph-based component generates embeddings and predicts links, ranking the corpus for a given request. This provides accurate document relevance prediction. The LLM-driven component then provides reasoning for the document relevance, addressing the interpretability challenges of traditional approaches.

The authors evaluate DISCOG on datasets with both balanced and imbalanced distributions, and it outperforms baseline methods in F1-score, precision, and recall by an average of 12%, 3%, and 16%, respectively. In an enterprise context, DISCOG reduces document review costs by 99.9% compared to manual processes and by 95% compared to LLM-based classification methods alone.

Critical Analysis

The paper acknowledges that while DISCOG addresses many of the limitations of traditional eDiscovery approaches, there are still some potential areas for improvement. For example, the authors note that the graph-based component may struggle with very large or highly dynamic document collections.

Additionally, the authors do not provide a detailed comparison of DISCOG's performance to other state-of-the-art graph-based or LLM-augmented retrieval methods in the eDiscovery domain. Further research could explore how DISCOG compares to these alternative approaches.

Overall, the paper presents a promising hybrid solution that combines the strengths of graph-based and LLM-driven methods for eDiscovery. However, there may be opportunities to further refine and benchmark the approach against the latest developments in retrieval and graph-based techniques.

Conclusion

The DISCOvery Graph (DISCOG) approach introduced in this paper represents a significant advancement in the field of eDiscovery. By leveraging the strengths of both graph-based methods and large language models, DISCOG can accurately identify relevant documents, provide clear explanations for its decisions, and drastically reduce the cost and time required for document review.

This hybrid approach has the potential to transform the eDiscovery process, making it more efficient, effective, and transparent for legal teams. As the use of AI and natural language processing continues to grow in the legal industry, innovations like DISCOG will likely play an increasingly important role in helping organizations navigate the complexities of electronic discovery.