We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields

Overview

• Natural Language Processing (NLP) is a rapidly advancing field with the potential to significantly impact the world
• However, this progress comes with substantial risks that require broad engagement across various fields of study
• This paper quantifies the degree of influence between NLP and 23 other fields, analyzing NLP publications, citations, and cross-field engagement over time

Plain English Explanation

Natural Language Processing (NLP) is a branch of artificial intelligence that focuses on enabling computers to understand, interpret, and generate human language. As NLP has advanced, it has become poised to substantially influence many aspects of our lives, from improved voice assistants to automated language translation.

However, this rapid progress also comes with significant risks and challenges that require input and collaboration from a wide range of academic disciplines. To better understand the state of this cross-disciplinary engagement, the researchers in this paper analyzed a large dataset of NLP publications, citations, and the connections between NLP and 23 other fields of study.

Their analysis revealed that, unlike most areas of research, the diversity of fields engaged with NLP has actually declined over time. In the 1980s, NLP papers cited a wide range of fields, but by 2022, this cross-field engagement had reached an all-time low. NLP has also become more insular, with researchers citing other NLP papers at an increasing rate and fewer "bridge" papers connecting NLP to other disciplines.

Furthermore, the researchers found that NLP is heavily dominated by computer science, with less than 8% of NLP citations coming from linguistics and less than 3% from math and psychology. This suggests that NLP may be overlooking important perspectives and insights from these other fields.

Overall, this study highlights the urgent need for NLP researchers to reflect on and actively address the declining cross-disciplinary engagement in their field. Maintaining a diverse range of influences is crucial for ensuring that NLP develops in a responsible and well-rounded manner.

Technical Explanation

The researchers in this paper sought to quantify the degree of influence between NLP and 23 other fields of study. They analyzed a dataset of approximately 77,000 NLP papers, 3.1 million citations from NLP papers to other publications, and 1.8 million citations from other papers to NLP work.

To measure cross-field engagement, the team developed a metric called the Citation Field Diversity Index (CFDI). This index ranges from 0 to 1, with higher values indicating a more diverse set of cited fields. The researchers tracked the CFDI of NLP over time, finding that it had declined from 0.58 in 1980 to 0.31 in 2022 - an all-time low.

In addition, the analysis showed that NLP has become increasingly insular, with researchers citing other NLP papers at a higher rate and fewer "bridge" papers connecting NLP to other disciplines. The distribution of NLP citations was also highly skewed, with over 80% going to computer science and less than 8% to linguistics, and less than 3% to math and psychology.

Critical Analysis

While this study provides valuable insights into the state of cross-disciplinary engagement in NLP, there are a few potential limitations and areas for further research:

• The analysis is limited to the 23 fields explicitly included in the study, and there may be other relevant disciplines that were not considered.
• The paper does not delve into the potential reasons behind the declining cross-field engagement, such as institutional or funding pressures, cultural factors, or the inherent technical nature of NLP research.
• The findings may not be generalizable to all subfields of NLP, as the researchers did not examine potential differences in cross-disciplinary engagement across various NLP applications or techniques.

Additionally, one could argue that the decline in cross-field engagement is not necessarily a negative outcome if NLP is becoming more focused and efficient within its core domains. However, the authors make a compelling case that maintaining diverse influences is crucial for the responsible development of NLP technologies, which have the potential to significantly impact society.

Conclusion

This study provides a sobering look at the state of cross-disciplinary engagement in the field of Natural Language Processing. The researchers found that, unlike most academic fields, NLP's engagement with a diverse range of disciplines has steadily declined over the past few decades, reaching an all-time low in 2022.

This trend towards insularity is concerning, as it suggests that NLP may be overlooking important perspectives and insights from fields like linguistics, mathematics, and psychology. Maintaining a broad range of influences is crucial for ensuring that NLP develops in a responsible and well-rounded manner, especially given its potential to substantially impact the world.

The findings of this paper suggest that NLP researchers and the broader academic community must actively work to address this trend and foster greater cross-disciplinary collaboration. Only by doing so can we unlock the full potential of Natural Language Processing while mitigating its inherent risks and challenges.