Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong

Overview

• Large language models (LLMs) are increasingly used to access information online, but their truthfulness and accuracy are important considerations.
• Researchers compared LLMs and search engines in helping users fact-check claims.
• They found that users reading LLM explanations were more efficient but tended to over-rely on them, even when the explanations were wrong.
• Providing contrastive explanations from LLMs (both why a claim is true and false) helped mitigate over-reliance, but did not significantly outperform search engines.
• Combining search engine results and LLM explanations offered no additional benefits compared to search engines alone.

Plain English Explanation

Large language models (LLMs) are powerful AI systems that can generate human-like text. As more people use LLMs to find information online, it's important to understand how reliable and accurate they are. This study looked at whether LLMs or traditional search engines are better at helping people fact-check claims.

The researchers had 80 people try to verify claims using either LLM explanations or search engine results. They found that people using the LLM explanations were able to do it more quickly, but they often believed the LLM even when the explanation was wrong. This over-reliance on the LLM could be dangerous, especially in important situations where inaccurate information could lead to serious consequences.

To address this, the researchers tried having the LLM provide both sides of the story - explaining why the claim was true and why it was false. This "contrastive" approach helped reduce people's over-trust in the LLM, but it still didn't outperform regular search engines. And combining the search results with the LLM explanations didn't provide any extra benefits compared to just using the search engine alone.

Overall, the study suggests that while LLMs can be efficient at providing information, we shouldn't blindly trust their explanations, especially for important facts. We still need to carefully evaluate the information from multiple sources, just as we would with a regular web search.

Technical Explanation

The researchers conducted experiments with 80 crowdworkers to compare the effectiveness of large language models (LLMs) and traditional search engines (information retrieval systems) in helping users fact-check claims. They prompted the LLMs to validate a given claim and provide corresponding explanations.

The results showed that users reading the LLM explanations were significantly more efficient at fact-checking than those using search engines, while achieving similar accuracy. However, the users tended to over-rely on the LLM explanations, even when they were incorrect.

To address this over-reliance, the researchers asked the LLMs to provide "contrastive" explanations - explaining both why the claim was true and why it was false. Presenting both sides of the explanation to users helped mitigate their over-trust in the LLM. However, this contrastive approach still could not significantly outperform search engines.

Furthermore, the study found that providing both search engine results and LLM explanations did not offer any complementary benefits compared to using search engines alone.

Critical Analysis

The study highlights an important challenge with using LLMs for high-stakes information access: over-reliance on their explanations, even when they are inaccurate. This could lead to critical consequences in real-world settings where factual information is crucial.

While the contrastive explanations helped reduce this over-reliance, they did not ultimately outperform traditional search engines. This suggests that natural language explanations from LLMs may not be a reliable replacement for directly reading the source material retrieved through web searches.

The study acknowledges that further research is needed to address the limitations of LLMs in fact-checking tasks. Potential areas for improvement could include enhancing LLM capabilities to better assess the credibility of information, or developing more sophisticated techniques to combine LLM outputs with search engine results.

Additionally, the study's focus on crowdworkers may limit the generalizability of the findings. Evaluating the performance of LLMs and search engines with a more diverse set of users, including those with varying levels of digital literacy, could provide additional insights.

Conclusion

This study highlights the importance of critically evaluating the information provided by large language models, even when they appear to offer efficient and natural language-based explanations. While LLMs can be useful tools for accessing online information, users should not blindly trust their outputs, especially in high-stakes situations where factual accuracy is crucial.

The findings suggest that traditional search engines may still be a more reliable approach for fact-checking, as users are less likely to over-rely on the information they retrieve. As LLM technology continues to advance, further research will be needed to ensure these powerful AI systems can be leveraged safely and responsibly for information access and verification.