Measuring publication relatedness using controlled vocabularies

Overview

• Measuring the relatedness between scientific publications is important for various areas like bibliometrics and science policy.
• Controlled vocabularies can be a promising basis for measuring relatedness, addressing issues with citation-based or text-based similarity.
• Several controlled-vocabulary-based relatedness measures have been developed, but their accuracy and suitability for different research questions have not been comprehensively tested.

Plain English Explanation

Researchers often need to understand how different scientific publications are related to one another. This can be useful for things like evaluating the impact of research or organizing scientific knowledge. One way to measure this "relatedness" is by looking at the keywords or terms used to describe the publications, rather than just citing other papers or analyzing the text itself. This can help address some of the issues that come up when using those other methods.

While there have been a few different ways suggested for measuring relatedness using keywords, the authors of this paper wanted to see how well these different methods actually work and which ones are best suited for different research questions. They reviewed the existing measures, developed a new one, and then tested all of them using real data about topics in the biomedical field.

Technical Explanation

The paper reviews several existing controlled-vocabulary-based relatedness measures and develops a new measure, then benchmarks the performance of these measures using the TREC Genomics dataset as ground truth.

The new measure they developed is based on the semantic similarity between the keywords or terms used to describe the publications. It takes into account both the direct matching of terms as well as the semantic relationships between them.

The benchmark tests showed that the new measure and the one proposed by Ahlgren et al. (2020) each had different strengths and weaknesses. These results can help researchers decide which method to use when studying topics like interdisciplinarity, information retrieval, clustering of science, and how researchers switch between different research topics.

Critical Analysis

The paper provides a comprehensive evaluation of several controlled-vocabulary-based relatedness measures, which is a valuable contribution to the field. However, the authors acknowledge that the benchmark test using the TREC Genomics dataset may have limitations in terms of representing the full scope of possible research questions and publication types.

Additionally, while the new measure developed in the paper shows promising performance, its effectiveness may depend on the specific domain and controlled vocabulary being used. Further research could explore how the measures perform in other scientific fields or with different controlled vocabularies.

The paper also does not delve into potential biases or limitations of controlled vocabularies themselves, which could impact the accuracy and applicability of the relatedness measures. Exploring these issues could be an interesting area for future work.

Conclusion

This paper provides an important evaluation of different methods for measuring the relatedness between scientific publications using controlled vocabularies. The results can help researchers choose the appropriate technique for their particular research needs, whether that's studying interdisciplinary connections, improving information retrieval, or understanding how researchers navigate different research topics over time. While the methods have some limitations, this work represents a significant step forward in developing more robust and reliable ways to analyze the relationships between scientific publications.