An Embedded Diachronic Sense Change Model with a Case Study from Ancient Greek

Overview

• Word meanings change over time, and word senses evolve, emerge, or die out in the process
• Modeling such changes accurately is challenging, especially for ancient languages with sparse corpora
• Quantifying uncertainty in sense-change estimates is important in these cases
• GASC (Genre-Aware Semantic Change) and DiSC (Diachronic Sense Change) are existing generative models used to analyze sense change in an ancient Greek text corpus
• These models represent word senses as distributions over context words and sense prevalence as a distribution over senses
• The models are fitted using Markov Chain Monte Carlo (MCMC) methods to measure temporal changes

Plain English Explanation

Words can change in meaning over time, and new word senses can emerge while others disappear. This process of sense change is particularly challenging to model accurately for ancient languages, where the available texts are often limited and scattered.

Accurately quantifying the uncertainty in these sense-change estimates is crucial, as it helps us understand how reliable the findings are. Two existing models, GASC and DiSC, have been used to analyze sense changes in an ancient Greek text corpus. These models represent the different senses of a word, such as "kosmos" (which can mean "decoration," "order," or "world"), as distributions over the words that commonly appear in that sense. They also model the prevalence of each sense as a distribution.

The researchers fit these models using a statistical technique called Markov Chain Monte Carlo (MCMC), which allows them to measure how these representations of word senses change over time.

Technical Explanation

The paper introduces EDiSC, an Embedded DiSC model, which combines word embeddings with the DiSC model to provide improved performance. The authors show that EDiSC offers better predictive accuracy, better recovery of ground truth, and better quantification of uncertainty compared to the original DiSC model. EDiSC also demonstrates better sampling efficiency and scalability when using MCMC methods.

The key challenge in fitting these models is discussed, as the authors explore the difficulties in accurately modeling sense changes, especially for ancient languages with limited text corpora.

Critical Analysis

The paper provides a valuable contribution to the field of lexical semantic change analysis, particularly for ancient languages. The authors acknowledge the inherent challenges in modeling sense changes with sparse data and the importance of quantifying uncertainty in the results.

While the EDiSC model shows promising improvements over the previous DiSC model, the authors could have delved deeper into the limitations and potential issues with their approach. For example, they could have discussed the impact of the specific word embedding technique used, or the sensitivity of the results to the choice of hyperparameters in the MCMC sampling.

Additionally, the authors could have explored the broader implications of their findings, such as how the improved sense-change modeling could inform historical linguistics or enhance our understanding of language evolution.

Conclusion

This paper presents an important advancement in the field of lexical semantic change analysis, particularly for ancient languages with limited text corpora. The introduction of the EDiSC model, which combines word embeddings with the DiSC framework, demonstrates improved performance in terms of predictive accuracy, ground-truth recovery, and uncertainty quantification.

The challenges of fitting these models and the inherent difficulties in modeling sense changes for ancient languages are also discussed, highlighting the significance of this research and the need for continued advancements in this area. The findings of this paper could have far-reaching implications for historical linguistics, language evolution studies, and our understanding of how word meanings change over time.