AI Transparency in Academic Search Systems: An Initial Exploration

Read original: arXiv:2408.10229 - Published 8/21/2024 by Yifan Liu, Peter Sullivan, Luanne Sinnamon

🤖

Overview

AI-enhanced academic search systems are becoming more popular among researchers.
Investigating the transparency of these systems is crucial to ensure trust and integrity in scholarly work.
This study examines the websites of 10 AI-enhanced academic search systems to assess their level of transparency.

Plain English Explanation

The study looks at AI-enhanced academic search systems, which are tools that use artificial intelligence to help researchers find relevant academic papers. As these systems become more common, it's important to understand how transparent they are about how they work.

The researchers analyzed the websites of 10 different AI-enhanced academic search systems that were recommended by university libraries. They found varying levels of transparency:

5 of the systems provided detailed information about their inner workings
3 systems offered partial information
2 systems gave little to no information

This is concerning because it means the academic community is using tools with unclear functionalities. This raises issues around research integrity, such as whether the search results are reliable and reproducible, and whether researchers can be held accountable for how they use these AI-powered tools.

Technical Explanation

The study employed a qualitative content analysis approach to examine the websites of 10 AI-enhanced academic search systems identified through university library guides. The researchers assessed the level of transparency provided by each system regarding their underlying mechanisms and functionality.

Their analysis found that the transparency levels varied across the sample:

5 of the systems provided detailed information about their search algorithms, data sources, and other technical details.
3 systems offered partial information, revealing some but not all aspects of their inner workings.
2 systems gave little to no information about how they operate, essentially functioning as "black boxes" to users.

These findings indicate that the academic community is recommending and using AI-powered search tools with opaque functionalities. This raises concerns about research integrity, including issues of reproducibility and researcher responsibility.

Critical Analysis

The paper acknowledges that the sample size of 10 systems is limited and may not be representative of the entire landscape of AI-enhanced academic search tools. Additionally, the researchers only analyzed the information publicly available on the systems' websites, which may not reflect the full transparency efforts of the providers.

One potential limitation is that the study did not assess the actual performance or accuracy of the search results produced by the various systems. It is possible that some less transparent systems could still provide reliable and useful results to researchers.

Further research could delve deeper into the specific transparency practices of these systems, such as whether they provide clear disclosure of known biases or limitations. Investigating user perceptions and trust in these tools would also be valuable in understanding the broader implications for academic research and publishing.

Conclusion

This study highlights the importance of transparency in AI-enhanced academic search systems as they become more widely adopted by researchers. The findings suggest that the academic community is using tools with varying degrees of openness about their inner workings, which could undermine trust in the search outcomes and the overall integrity of scholarly work.

Addressing these transparency issues is crucial to ensure that researchers can make informed decisions about the tools they use and be held accountable for the reliability and reproducibility of their findings. As AI continues to play a growing role in academic research, maintaining high standards of transparency and integrity will be essential for the credibility of the scientific process.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🤖

AI Transparency in Academic Search Systems: An Initial Exploration

Yifan Liu, Peter Sullivan, Luanne Sinnamon

As AI-enhanced academic search systems become increasingly popular among researchers, investigating their AI transparency is crucial to ensure trust in the search outcomes, as well as the reliability and integrity of scholarly work. This study employs a qualitative content analysis approach to examine the websites of a sample of 10 AI-enhanced academic search systems identified through university library guides. The assessed level of transparency varies across these systems: five provide detailed information about their mechanisms, three offer partial information, and two provide little to no information. These findings indicate that the academic community is recommending and using tools with opaque functionalities, raising concerns about research integrity, including issues of reproducibility and researcher responsibility.

8/21/2024

AI data transparency: an exploration through the lens of AI incidents

Sophia Worth, Ben Snaith, Arunav Das, Gefion Thuermer, Elena Simperl

Knowing more about the data used to build AI systems is critical for allowing different stakeholders to play their part in ensuring responsible and appropriate deployment and use. Meanwhile, a 2023 report shows that data transparency lags significantly behind other areas of AI transparency in popular foundation models. In this research, we sought to build on these findings, exploring the status of public documentation about data practices within AI systems generating public concern. Our findings demonstrate that low data transparency persists across a wide range of systems, and further that issues of transparency and explainability at model- and system- level create barriers for investigating data transparency information to address public concerns about AI systems. We highlight a need to develop systematic ways of monitoring AI data transparency that account for the diversity of AI system types, and for such efforts to build on further understanding of the needs of those both supplying and using data transparency information.

9/6/2024

Transparency, Privacy, and Fairness in Recommender Systems

Dominik Kowald

Recommender systems have become a pervasive part of our daily online experience, and are one of the most widely used applications of artificial intelligence and machine learning. Therefore, regulations and requirements for trustworthy artificial intelligence, for example, the European AI Act, which includes notions such as transparency, privacy, and fairness are also highly relevant for the design of recommender systems in practice. This habilitation elaborates on aspects related to these three notions in the light of recommender systems, namely: (i) transparency and cognitive models, (ii) privacy and limited preference information, and (iii) fairness and popularity bias in recommender systems. Specifically, with respect to aspect (i), we highlight the usefulness of incorporating psychological theories for a transparent design process of recommender systems. We term this type of systems psychology-informed recommender systems. In aspect (ii), we study and address the trade-off between accuracy and privacy in differentially-private recommendations. We design a novel recommendation approach for collaborative filtering based on an efficient neighborhood reuse concept, which reduces the number of users that need to be protected with differential privacy. Furthermore, we address the related issue of limited availability of user preference information, e.g., click data, in the settings of session-based and cold-start recommendations. With respect to aspect (iii), we analyze popularity bias in recommender systems. We find that the recommendation frequency of an item is positively correlated with this item's popularity. This also leads to the unfair treatment of users with little interest in popular content. Finally, we study long-term fairness dynamics in algorithmic decision support in the labor market using agent-based modeling techniques.

7/1/2024

📊

Integrating measures of replicability into scholarly search: Challenges and opportunities

Chuhao Wu, Tatiana Chakravorti, John Carroll, Sarah Rajtmajer

Challenges to reproducibility and replicability have gained widespread attention, driven by large replication projects with lukewarm success rates. A nascent work has emerged developing algorithms to estimate the replicability of published findings. The current study explores ways in which AI-enabled signals of confidence in research might be integrated into the literature search. We interview 17 PhD researchers about their current processes for literature search and ask them to provide feedback on a replicability estimation tool. Our findings suggest that participants tend to confuse replicability with generalizability and related concepts. Information about replicability can support researchers throughout the research design processes. However, the use of AI estimation is debatable due to the lack of explainability and transparency. The ethical implications of AI-enabled confidence assessment must be further studied before such tools could be widely accepted. We discuss implications for the design of technological tools to support scholarly activities and advance replicability.

5/6/2024